grahamc/hydra - Change ZH6B56XRZON7AGKUPGKABU2SWOIQSYBTXVOMR66X6P6MG25Y4WZAC

Try harder to find build logs

Due to the fixed-output derivation hashing scheme, there can be multiple derivations of the same output path. But build logs are indexed by derivation path. Thus, we may not be able to find the log of a build or build step using its derivation. So as a fallback, Hydra now looks for other derivations with the same output paths.

Created by Eelco Dolstra on August 30, 2013

ZH6B56XRZON7AGKUPGKABU2SWOIQSYBTXVOMR66X6P6MG25Y4WZAC

Dependencies

In channels

main

upstream

Change contents

Replacement in src/lib/Hydra/Controller/Build.pm at line 38 [6.1]
B:BD[5.1641] → [5.1641:1682]
```
    my ($self, $c, $path, $status) = @_;
```
[5.1641]
[5.1682]
```
    my ($self, $c, $path) = @_;
```

Replacement in src/lib/Hydra/Controller/Build.pm at line 40 [6.1]

B:BD[5.1730] → [5.1730:1876]

        { path => $path, busy => 0, status => $status },
        { join => ["buildstepoutputs"], order_by => ["stopTime"], limit => 1 })->single;

[5.1730]

[6.682]

        { path => $path, busy => 0 },
        { join => ["buildstepoutputs"], order_by => ["status", "stopTime"], rows => 1 })->single;

Replacement in src/lib/Hydra/Controller/Build.pm at line 46 [6.1]
B:BD[5.1906] → [5.1906:1950]
```
    my ($self, $c, $drvPath, $status) = @_;
```
[5.1906]
[5.1950]
```
    my ($self, $c, $drvPath) = @_;
```

Replacement in src/lib/Hydra/Controller/Build.pm at line 48 [6.1]

B:BD[5.1998] → [5.1998:2120]

        { drvpath => $drvPath, busy => 0, status => $status },
        { order_by => ["stopTime"], limit => 1 })->single;

[5.1998]

[5.2120]

        { drvpath => $drvPath, busy => 0 },
        { order_by => ["status", "stopTime"], rows => 1 })->single;

Replacement in src/lib/Hydra/Controller/Build.pm at line 71 [6.1]

B:BD[5.2271] → [5.2271:2417]

        my $cachedBuildStep = findBuildStepByOutPath($self, $c, $path,
            $build->buildstatus == 0 || $build->buildstatus == 6 ? 0 : 1);

[5.2271]

[7.0]

        my $cachedBuildStep = findBuildStepByOutPath($self, $c, $path);

Replacement in src/lib/Hydra/Controller/Build.pm at line 97 [6.1]

∅:D[8.100] → [9.57:124]

B:BD[9.57] → [9.57:124]

    ($c->stash->{eval}) = $c->stash->{build}->jobsetevals->search(

[8.100]

[9.124]

    $c->stash->{eval} = $c->stash->{build}->jobsetevals->search(

Replacement in src/lib/Hydra/Controller/Build.pm at line 99 [6.1]

B:BD[9.154] → [9.154:199]

        { limit => 1, order_by => ["id"] });

[9.154]

[3.4281]

        { rows => 1, order_by => ["id"] })->single;

Replacement in src/lib/Hydra/Controller/Build.pm at line 134 [6.1]

B:BD[6.1443] → [10.211:251]

    showLog($c, $step->drvpath, $mode);

[6.1443]

[6.1554]

    showLog($c, $mode, $step->drvpath, map { $_->path } $step->buildstepoutputs->all);

Replacement in src/lib/Hydra/Controller/Build.pm at line 140 [6.1]

B:BD[11.221] → [10.252:305]

    showLog($c, $c->stash->{build}->drvpath, $mode);

[11.221]

[6.1923]

    showLog($c, $mode, $c->stash->{build}->drvpath, map { $_->path } $c->stash->{build}->buildoutputs->all);

Replacement in src/lib/Hydra/Controller/Build.pm at line 145 [6.1]
B:BD[11.302] → [10.306:341]
```
    my ($c, $drvPath, $mode) = @_;
```
[11.302]
[12.0]
```
    my ($c, $mode, $drvPath, @outPaths) = @_;
```

Replacement in src/lib/Hydra/Controller/Build.pm at line 147 [6.1]

B:BD[12.1] → [10.342:385]

    my $logPath = getDrvLogPath($drvPath);

[12.1]

[6.1997]

    my $logPath = findLog($c, $drvPath, @outPaths);
    print STDERR "log = $logPath\n";

Replacement in src/lib/Hydra/Controller/Build.pm at line 154 [6.1]

B:BD[11.384] → [10.493:540]

        my $pipeline = "nix-store -l $drvPath"

[11.384]

[11.465]

        my $pipeline = ($logPath =~ /.bz2$/ ? "bzip2 -d < $logPath" : "cat $logPath")

Replacement in src/lib/Hydra/Controller/Build.pm at line 162 [6.1]

B:BD[11.759] → [10.541:623]

        $c->stash->{'plain'} = { data => (scalar logContents($drvPath)) || " " };

[11.759]

[12.353]

        $c->stash->{'plain'} = { data => (scalar logContents($logPath)) || " " };

Replacement in src/lib/Hydra/Controller/Build.pm at line 172 [6.1]

B:BD[13.307] → [10.624:699]

        $c->stash->{contents} = (scalar logContents($drvPath, 50)) || " ";

[13.307]

[13.382]

        $c->stash->{contents} = (scalar logContents($logPath, 50)) || " ";

Replacement in src/lib/Hydra/Controller/Build.pm at line 177 [6.1]

B:BD[14.49] → [10.700:786]

        $c->stash->{'plain'} = { data => (scalar logContents($drvPath, 50)) || " " };

[14.49]

[14.119]

        $c->stash->{'plain'} = { data => (scalar logContents($logPath, 50)) || " " };

Replacement in src/lib/Hydra/Controller/Build.pm at line 364 [6.1]

B:BD[15.340] → [15.340:464]

                ? findBuildStepByOutPath($self, $c, $path, 0)
                : findBuildStepByDrvPath($self, $c, $path, 0)

[15.340]

[15.464]

                ? findBuildStepByOutPath($self, $c, $path)
                : findBuildStepByDrvPath($self, $c, $path)

Replacement in src/lib/Hydra/Helper/Nix.pm at line 19 [17.436]

B:BD[16.48] → [5.9433:9497]

    jobsetOverview removeAsciiEscapes getDrvLogPath logContents

[16.48]

[18.844]

    jobsetOverview removeAsciiEscapes getDrvLogPath findLog logContents

Insertion in src/lib/Hydra/Helper/Nix.pm at line 263 [17.436]

[10.1312]

[19.3643]

    return undef;
}
# Find the log of the derivation denoted by $drvPath.  It it doesn't
# exist, try other derivations that produced its outputs (@outPaths).
sub findLog {
    my ($c, $drvPath, @outPaths) = @_;
    if (defined $drvPath) {
	my $logPath = getDrvLogPath($drvPath);
	return $logPath if defined $logPath;
    }
    
    return undef if scalar @outPaths == 0;
    my @steps = $c->model('DB::BuildSteps')->search(
	{ path => { -in => [@outPaths] } },
	{ select => ["drvpath"]
        , distinct => 1
	, join => "buildstepoutputs"
	});
    foreach my $step (@steps) {
	my $logPath = getDrvLogPath($step->drvpath);
	return $logPath if defined $logPath;
    }

Replacement in src/lib/Hydra/Helper/Nix.pm at line 296 [17.436]

B:BD[20.459] → [10.1315:1422]

    my ($drvPath, $tail) = @_;
    my $logPath = getDrvLogPath($drvPath);
    die unless defined $logPath;

[20.459]

[20.487]

    my ($logPath, $tail) = @_;

Replacement in src/lib/Hydra/View/TT.pm at line 11 [23.6789]

B:BD[21.41] → [22.0:51]

    expose_methods => [qw/log_exists ellipsize/]);

[21.41]

[23.6867]

    expose_methods => [qw/log_exists buildLogExists buildStepLogExists/]);

Insertion in src/lib/Hydra/View/TT.pm at line 17 [23.6789]

[10.9874]

}
sub buildLogExists {
    my ($self, $c, $build) = @_;
    my @outPaths = map { $_->path } $build->buildoutputs->all;
    return defined findLog($c, $build->drvpath, @outPaths);

Replacement in src/lib/Hydra/View/TT.pm at line 25 [23.6789]

B:BD[23.6919] → [22.52:166]

sub ellipsize {
    my ($self, $c, $s, $n) = @_;
    return length $s <= $n ? $s : substr($s, 0, $n - 3) . "...";

[23.6919]

[22.166]

sub buildStepLogExists {
    my ($self, $c, $step) = @_;
    my @outPaths = map { $_->path } $step->buildstepoutputs->all;
    return defined findLog($c, $step->drvpath, @outPaths);

Replacement in src/root/build.tt at line 26 [23.7350]

B:BD[5.11941] → [5.11941:11990]

          [% has_log = log_exists(step.drvpath);

[5.11941]

[24.0]

          [% has_log = buildStepLogExists(step);

Replacement in src/root/build.tt at line 180 [23.7350]

B:BD[25.59] → [4.1236:1299]

            [% IF !isAggregate && log_exists(build.drvpath) %]

[25.59]

[26.6492]

            [% IF !isAggregate && buildLogExists(build) %]

Replacement in src/root/deps.tt at line 15 [28.1684]

B:BD[27.462] → [27.462:513]

            IF log_exists(node.buildStep.drvpath);

[27.462]

[2.91]

            IF buildStepLogExists(node.buildStep);