* Log evaluation errors etc. in the DB.

Dependencies

• [2] D5Z5M4MA
• [3] SHBLLAVH * More global substitution.
• [4] JK2QWPH6
• [5] B72GLND4
• [6] GCHNNFZP
• [7] BD3GRK4B * Get rid of "positive failures" and separate log phases.
• [8] 67P45PY4
• [9] ZEHSSVFG
• [10] YTZOC7C5 * Editing of jobset inputs.
• [11] 2GK5DOU7 * Downloading closures.
• [12] K5BEBWKM
• [13] KOTB7BKV
• [14] M552HLIA * Support variant builds.
• [15] L2E6EVE2 * Merged the Build and Job tables.
• [16] J5UVLXOK * Start of a basic Catalyst web interface.
• [17] WYN733ST * Store build duration, handle cached builds.
• [18] HJLYC753 * Adding input value alternatives.
• [19] 5QJP6JHS * Get dependencies from the database.
• [20] PHX2HIVG * Store info about the build inputs in the build record.
• [21] TLZ2SPBR
• [22] TQKGQ5R3
• [23] CLJQCY2X * Store info about all the build actions and allow them to be
• [24] X27GNHDV * Basic job info in the database.
• [25] VCOSLZRP
• [26] IMY5UQE3
• [27] FDE3BJAP * Refactoring.
• [28] TWURROKC
• [29] DVNWJXWW * Generic declaration of build products.
• [30] 7YBYT2LQ
• [31] 2AUODJBT
• [32] BVOPAMLS
• [33] UVMFS73T * Some jQuery / CSS hackery.
• [34] WHAFVCEI
• [35] IK53RV4V
• [36] 7LKUAIGC
• [37] GWCV3TQV * BuildInputs table: link to dependencies, include store paths.
• [38] 4JPNFWRB * Use jquery for the logfile manipulation.
• [*] N22GPKYT * Put info about logs / build products in the DB.
• [*] LQNBKF3D
• [*] BA46C5LN * Pretty-print the logs.

Change contents

• replacement in src/Hydra/lib/Hydra/Controller/Root.pm at line 59
  [5.406]→[5.0:161](∅→∅),[5.161]→[5.549:591](∅→∅),[5.549]→[5.549:591](∅→∅)
  − , where => "finished != 0 and timestamp = (select max(timestamp) from Builds where project == me.project and attrName == me.attrName and finished != 0)"

  − , order_by => "project, attrname"
  [5.406]

  [5.591]
  + , where => "finished != 0 and timestamp = (select max(timestamp) from Builds " .

  + "where project == me.project and attrName == me.attrName and finished != 0 and system == me.system)"

  + , order_by => "project, attrname, system"
• replacement in src/Hydra/lib/Hydra/Schema/Buildinputs.pm at line 39
  [5.782]→[3.733:875](∅→∅)
  − # Created by DBIx::Class::Schema::Loader v0.04005 @ 2008-11-25 11:59:19

  − # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:NSUTUwG2qbvbFkZxezeSEA
  [5.782]

  [5.924]
  + # Created by DBIx::Class::Schema::Loader v0.04005 @ 2008-11-25 12:58:40

  + # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:H8JAWhQt1ehZTEEPH2KGBw
• replacement in src/Hydra/lib/Hydra/Schema/Buildproducts.pm at line 36
  [5.4922]→[3.994:1136](∅→∅)
  − # Created by DBIx::Class::Schema::Loader v0.04005 @ 2008-11-25 11:59:19

  − # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:9b3ZNy73Cjat0gr5nQHIpA
  [5.4922]

  [5.5064]
  + # Created by DBIx::Class::Schema::Loader v0.04005 @ 2008-11-25 12:58:40

  + # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:EAo0rZSn+5Q5OY7Y1xjuyA
• replacement in src/Hydra/lib/Hydra/Schema/Buildresultinfo.pm at line 30
  [5.2382]→[3.1251:1393](∅→∅)
  − # Created by DBIx::Class::Schema::Loader v0.04005 @ 2008-11-25 11:59:19

  − # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:uTiPHxrs6S19Laa/qeb7CA
  [5.2382]

  [5.2524]
  + # Created by DBIx::Class::Schema::Loader v0.04005 @ 2008-11-25 12:58:40

  + # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:5R03Kdy5ujkr7s9OfCNbTA
• replacement in src/Hydra/lib/Hydra/Schema/Builds.pm at line 73
  [5.6194]→[3.1753:1895](∅→∅)
  − # Created by DBIx::Class::Schema::Loader v0.04005 @ 2008-11-25 11:59:19

  − # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:Ky0EF25kaZIaY+ofkP0ohw
  [5.6194]

  [5.3377]
  + # Created by DBIx::Class::Schema::Loader v0.04005 @ 2008-11-25 12:58:40

  + # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:G1KBKGvEgnTcrgDkUnG1UQ
• replacement in src/Hydra/lib/Hydra/Schema/Buildschedulinginfo.pm at line 26
  [5.4473]→[3.2247:2389](∅→∅)
  − # Created by DBIx::Class::Schema::Loader v0.04005 @ 2008-11-25 11:59:19

  − # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:kt5gBt14ay7/DiIaQpKopA
  [5.4473]

  [5.4615]
  + # Created by DBIx::Class::Schema::Loader v0.04005 @ 2008-11-25 12:58:40

  + # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:MkMR6pE6mrN2BQL18ROYfA
• replacement in src/Hydra/lib/Hydra/Schema/Buildsteps.pm at line 38
  [5.2816]→[3.2499:2641](∅→∅)
  − # Created by DBIx::Class::Schema::Loader v0.04005 @ 2008-11-25 11:59:19

  − # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:ljGW5oOaJQ/uUKQx80W5SA
  [5.2816]

  [5.2958]
  + # Created by DBIx::Class::Schema::Loader v0.04005 @ 2008-11-25 12:58:40

  + # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:JbawGso3rboFeQMX6XfgXQ
• replacement in src/Hydra/lib/Hydra/Schema/Jobsetinputalts.pm at line 34
  [5.1831]→[3.2717:2859](∅→∅)
  − # Created by DBIx::Class::Schema::Loader v0.04005 @ 2008-11-25 11:59:19

  − # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:mG0ka8XxC3ZEs/KgGKe5Hg
  [5.1831]

  [5.1973]
  + # Created by DBIx::Class::Schema::Loader v0.04005 @ 2008-11-25 12:58:40

  + # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:7+oJOQegjLhp5JeuCS0KPw
• replacement in src/Hydra/lib/Hydra/Schema/Jobsetinputs.pm at line 46
  [5.1681]→[3.2993:3135](∅→∅)
  − # Created by DBIx::Class::Schema::Loader v0.04005 @ 2008-11-25 11:59:19

  − # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:trcxVA3lLfgTC0TCHL3mVw
  [5.1681]

  [5.1823]
  + # Created by DBIx::Class::Schema::Loader v0.04005 @ 2008-11-25 12:58:40

  + # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:BCzeQkhJYieHoL4ppjdJvw
• edit in src/Hydra/lib/Hydra/Schema/Jobsets.pm at line 20
  [5.2464]

  [5.2464]
  + { data_type => "text", is_nullable => 0, size => undef },

  + "errormsg",
• edit in src/Hydra/lib/Hydra/Schema/Jobsets.pm at line 23
  [5.2524]

  [5.2524]
  + "errortime",

  + { data_type => "integer", is_nullable => 0, size => undef },
• replacement in src/Hydra/lib/Hydra/Schema/Jobsets.pm at line 51
  [5.2992]→[3.3352:3494](∅→∅)
  − # Created by DBIx::Class::Schema::Loader v0.04005 @ 2008-11-25 11:59:19

  − # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:MYl8lWfWLCIAGSulR3m5zw
  [5.2992]

  [5.3134]
  + # Created by DBIx::Class::Schema::Loader v0.04005 @ 2008-11-25 12:58:40

  + # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:kEcdBc9XKVhz6zO/XEkAxg
• replacement in src/Hydra/lib/Hydra/Schema/Projects.pm at line 33
  [5.3685]→[3.3586:3728](∅→∅)
  − # Created by DBIx::Class::Schema::Loader v0.04005 @ 2008-11-25 11:59:19

  − # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:tLlYQAkX5CeUR2GJir8ifg
  [5.3685]

  [5.3827]
  + # Created by DBIx::Class::Schema::Loader v0.04005 @ 2008-11-25 12:58:40

  + # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:rRYzLbd9BlZAQdOjT/GQcw
• replacement in src/Hydra/lib/Hydra/Schema.pm at line 11
  [5.6498]→[3.3753:3895](∅→∅)
  − # Created by DBIx::Class::Schema::Loader v0.04005 @ 2008-11-25 11:59:19

  − # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:8AQc7Q1I5DrtSjFp722iMA
  [5.6498]

  [5.6640]
  + # Created by DBIx::Class::Schema::Loader v0.04005 @ 2008-11-25 12:58:40

  + # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:iYwU4nHj9C2HkHwpYlVygQ
• edit in src/Hydra/programs/Scheduler.pl at line 6
  [4.492]

  [40.1392]
  + use IPC::Run;
• edit in src/Hydra/programs/Scheduler.pl at line 19
  [5.6684]

  [5.6684]
  +

  + sub captureStdoutStderr {

  + my $stdin = ""; my $stdout; my $stderr;

  + my $res = IPC::Run::run(\@_, \$stdin, \$stdout, \$stderr);

  + return ($res, $stdout, $stderr);

  + }
• edit in src/Hydra/programs/Scheduler.pl at line 26
  [5.6685]

  [5.1826]
  +
• edit in src/Hydra/programs/Scheduler.pl at line 47
  [5.2738]

  [5.6820]
  + print "copying input ", $input->name, " from $uri\n";

  +
• edit in src/Hydra/programs/Scheduler.pl at line 52
  [5.6956]→[5.0:50](∅→∅)
  − print " copied to $storePath\n";
• replacement in src/Hydra/programs/Scheduler.pl at line 76
  [5.198]→[5.198:362](∅→∅)
  − my $drvPath = `nix-instantiate $nixExprPath --attr $jobName $extraArgs`

  − or die "cannot evaluate the Nix expression containing the job definitions: $?";
  [5.198]

  [5.362]
  + (my $res, my $drvPath, my $stderr) = captureStdoutStderr(

  + "nix-instantiate", $nixExprPath, "--attr", $jobName, @{$extraArgs});

  + die "cannot evaluate the Nix expression for job `$jobName':\n$stderr" unless $res;
• replacement in src/Hydra/programs/Scheduler.pl at line 81
  [5.383]→[5.383:698](∅→∅)
  − # Call nix-env --xml to get info about this job (drvPath, outPath, meta attributes, ...).

  − my $infoXml = `nix-env -f $nixExprPath --query --available "*" --attr-path --out-path --drv-path --meta --xml --system-filter "*" --attr $jobName $extraArgs`

  − or die "cannot get information about the job: $?";
  [5.383]

  [5.698]
  + # Call nix-env --xml to get info about this job (drvPath, outPath, meta attributes, ...).

  + ($res, my $infoXml, $stderr) = captureStdoutStderr(

  + qw(nix-env --query --available * --attr-path --out-path --drv-path --meta --xml --system-filter *),

  + "-f", $nixExprPath, "--attr", $jobName, @{$extraArgs});

  + die "cannot get information about the job `$jobName':\n$stderr" unless $res;
• replacement in src/Hydra/programs/Scheduler.pl at line 102
  [5.9604]→[5.12130:12182](∅→∅)
  − print " already scheduled/done\n";
  [5.9604]

  [5.9646]
  + print "already scheduled/done\n";
• replacement in src/Hydra/programs/Scheduler.pl at line 106
  [5.9677]→[5.12183:12224](∅→∅)
  − print " adding to queue\n";
  [5.9677]

  [5.9950]
  + print "adding to queue\n";
• edit in src/Hydra/programs/Scheduler.pl at line 145
  [5.1179]

  [5.1179]
  +

  +

  + sub setJobsetError {

  + my ($jobset, $errorMsg) = @_;

  + eval {

  + $db->txn_do(sub {

  + $jobset->errormsg($errorMsg);

  + $jobset->errortime(time);

  + $jobset->update;

  + });

  + };

  + }
• replacement in src/Hydra/programs/Scheduler.pl at line 166
  [5.6226]→[5.6226:6249](∅→∅)
  − warn $@ if $@;
  [5.6226]

  [5.1451]
  + if ($@) {

  + print "error evaluating job `", $jobName, "': $@";

  + setJobsetError($jobset, $@);

  + }
• replacement in src/Hydra/programs/Scheduler.pl at line 174
  [5.1512]→[5.1512:1576](∅→∅)
  − print " finding alternatives for argument $argName\n";
  [5.1512]

  [5.1576]
  + #print "finding alternatives for argument $argName\n";
• replacement in src/Hydra/programs/Scheduler.pl at line 183
  [5.1734]→[5.1734:1838](∅→∅)
  − print " INPUT ", $input->name, " (type ", $input->type, ") alt ", $alt->altnr, "\n";
  [5.1734]

  [5.1838]
  + #print "input ", $input->name, " (type ", $input->type, ") alt ", $alt->altnr, "\n";
• replacement in src/Hydra/programs/Scheduler.pl at line 185
  [5.1902]→[5.1902:1932](∅→∅)
  − my $newArgs = "";
  [5.1902]

  [5.1932]
  + my @newArgs = @{$extraArgs};
• replacement in src/Hydra/programs/Scheduler.pl at line 187
  [5.1995]→[5.1995:2128](∅→∅)
  − # !!! escaping

  − $newArgs = " --arg $argName '{path = " . $inputInfo->{$argName}->{storePath} . ";}'";
  [5.1995]

  [5.2128]
  + push @newArgs, "--arg", $argName, "{path = " . $inputInfo->{$argName}->{storePath} . ";}";
• replacement in src/Hydra/programs/Scheduler.pl at line 189
  [5.2192]→[5.2192:2283](∅→∅)
  − $newArgs = " --argstr $argName '" . $inputInfo->{$argName}->{value} . "'";
  [5.2192]

  [5.2283]
  + push @newArgs, "--argstr", $argName, $inputInfo->{$argName}->{value};
• replacement in src/Hydra/programs/Scheduler.pl at line 193
  [5.2392]→[5.2392:2473](∅→∅)
  − $jobName, $jobExpr, $extraArgs . $newArgs, $argsNeeded, $n + 1);
  [5.2392]

  [5.2473]
  + $jobName, $jobExpr, \@newArgs, $argsNeeded, $n + 1);
• edit in src/Hydra/programs/Scheduler.pl at line 221
  [5.3366]

  [5.3366]
  + my @newArgs = @{$extraArgs};

  + push @newArgs, "--arg", $argName, "{path = " . $prevBuild->outpath . ";}";

  +
• replacement in src/Hydra/programs/Scheduler.pl at line 226
  [5.3453]→[5.3453:3603](∅→∅)
  − $jobName, $jobExpr,

  − $extraArgs . " --arg $argName '{path = " . $prevBuild->outpath . ";}'",

  − $argsNeeded, $n + 1);
  [5.3453]

  [5.3603]
  + $jobName, $jobExpr, \@newArgs, $argsNeeded, $n + 1);
• replacement in src/Hydra/programs/Scheduler.pl at line 230
  [5.7481]→[5.3612:3617](∅→∅)
  −
  [5.7481]

  [5.3617]
  +
• edit in src/Hydra/programs/Scheduler.pl at line 245
  [5.7582]

  [5.7582]
  +

  + print "evaluating $nixExprPath\n";
• replacement in src/Hydra/programs/Scheduler.pl at line 248
  [5.7583]→[5.7583:7781](∅→∅)
  − print " EVALUATING $nixExprPath\n";

  −

  − my $jobsXml = `nix-instantiate $nixExprPath --eval-only --strict --xml`

  − or die "cannot evaluate the Nix expression containing the jobs: $?";
  [5.7583]

  [5.7781]
  + (my $res, my $jobsXml, my $stderr) = captureStdoutStderr(

  + "nix-instantiate", $nixExprPath, "--eval-only", "--strict", "--xml");

  + die "cannot evaluate the Nix expression containing the jobs:\n$stderr" unless $res;
• replacement in src/Hydra/programs/Scheduler.pl at line 266
  [5.8170]→[5.8170:8206](∅→∅)
  − print " JOB $jobName\n";
  [5.8170]

  [5.8206]
  + print "considering job $jobName\n";
• replacement in src/Hydra/programs/Scheduler.pl at line 275
  [5.8555]→[5.8555:8609](∅→∅)
  − print " needs input $argName\n";
  [5.8555]

  [5.4438]
  + #print "needs input $argName\n";
• replacement in src/Hydra/programs/Scheduler.pl at line 283
  [5.6352]→[5.6352:6410](∅→∅)
  − $jobName, $jobExpr, "", \@argsNeeded, 0);
  [5.6352]

  [5.6410]
  + $jobName, $jobExpr, [], \@argsNeeded, 0);
• replacement in src/Hydra/programs/Scheduler.pl at line 293
  [5.1831]→[5.9961:10009](∅→∅),[5.9961]→[5.9961:10009](∅→∅)
  − print "PROJECT ", $project->name, "\n";
  [5.1831]

  [5.10009]
  + print "considering project ", $project->name, "\n";
• replacement in src/Hydra/programs/Scheduler.pl at line 295
  [5.10063]→[5.10063:10115](∅→∅)
  − print " JOBSET ", $jobset->name, "\n";
  [5.10063]

  [5.1832]
  + print "considering jobset ", $jobset->name, " in ", $project->name, "\n";
• replacement in src/Hydra/programs/Scheduler.pl at line 298
  [5.1899]→[5.1899:1951](∅→∅)
  − };

  − warn $@ if $@;

  − }
  [5.1899]

  [5.10169]
  + };

  + if ($@) {

  + print "error evaluating jobset ", $jobset->name, ": $@";

  + setProjectError($jobset, $@);

  + }

  + }
• edit in src/Hydra/root/build.tt at line 4
  [5.1440]→[5.7398:7413](∅→∅),[5.2304]→[5.7398:7413](∅→∅),[5.7398]→[5.7398:7413](∅→∅)
  − [% USE date %]
• replacement in src/Hydra/root/build.tt at line 27
  [5.5783]→[5.5783:5852](∅→∅)
  − <td>[% date.format(build.timestamp, '%Y-%m-%d %H:%M:%S') %]</td>
  [5.5783]

  [5.285]
  + <td>[% PROCESS renderDateTime timestamp = build.timestamp %]</td>
• replacement in src/Hydra/root/build.tt at line 89
  [5.28]→[5.6140:6297](∅→∅)
  − <td>[% IF build.resultInfo.starttime %][% date.format(build.resultInfo.starttime, '%Y-%m-%d %H:%M:%S') %][% ELSE %]<em>(cached build)</em>[% END %]</td>
  [5.28]

  [5.97]
  + <td>[% IF build.resultInfo.starttime %][% PROCESS renderDateTime timestamp = build.resultInfo.starttime %][% ELSE %]<em>(cached build)</em>[% END %]</td>
• replacement in src/Hydra/root/build.tt at line 93
  [5.141]→[5.6298:6453](∅→∅)
  − <td>[% IF build.resultInfo.stoptime %][% date.format(build.resultInfo.stoptime, '%Y-%m-%d %H:%M:%S') %][% ELSE %]<em>(cached build)</em>[% END %]</td>
  [5.141]

  [5.209]
  + <td>[% IF build.resultInfo.stoptime %][% PROCESS renderDateTime timestamp = build.resultInfo.stoptime %][% ELSE %]<em>(cached build)</em>[% END %]</td>
• replacement in src/Hydra/root/build.tt at line 350
  [5.4474]→[5.4474:4553](∅→∅)
  − <td>[% date.format(input.build.timestamp, '%Y-%m-%d %H:%M:%S') %]</td>
  [5.4474]

  [5.714]
  + <td>[% PROCESS renderDateTime timestamp = input.build.timestamp %]</td>
• edit in src/Hydra/root/common.tt at line 10
  [5.3027]
  +

  + [% USE date %]

  +

  + [% BLOCK renderDateTime %]

  + [% date.format(timestamp, '%Y-%m-%d %H:%M:%S') -%]

  + [% END %]
• edit in src/Hydra/root/index.tt at line 68
  [5.11173]→[5.4554:4555](∅→∅)
  −
• edit in src/Hydra/root/index.tt at line 69
  [5.1415]→[5.3609:3627](∅→∅)
  − <h2>Projects</h2>
• edit in src/Hydra/root/index.tt at line 70
  [5.4574]→[5.4574:4731](∅→∅),[5.4731]→[5.11173:11174](∅→∅),[5.11173]→[5.11173:11174](∅→∅),[5.11174]→[5.1416:1417](∅→∅)
  − <ul>

  − [% FOREACH project IN projects -%]

  − <li><a href="[% c.uri_for('/project' project.name) %]"><tt>[% project.name %]</tt></a></li>

  − [% END -%]

  − </ul>

  −

  −
• replacement in src/Hydra/root/layout.tt at line 122
  [5.11740]→[5.11740:11747](∅→∅)
  − </html>
  [5.11740]
  + </html>
• edit in src/Hydra/root/project.tt at line 91
  [41.1864]

  [41.1864]
  + [% IF !edit && jobset.errormsg %]

  + <tr>

  + <th>Last evaluation error:</th>

  + <td>

  + <em>On [% PROCESS renderDateTime timestamp = jobset.errortime %]:</em>

  + <pre class="multiLineMsg error">[% HTML.escape(jobset.errormsg) %]</pre>

  + </td>

  + </tr>

  + [% END %]
• edit in src/Hydra/root/static/css/hydra.css at line 47
  [5.9142]

  [5.9142]
  + vertical-align: top;
• edit in src/Hydra/root/static/css/hydra.css at line 173
  [42.609]

  [5.1971]
  + .multiLineMsg {

  + padding: 0em;

  + margin: 0em;

  + white-space: pre-wrap;

  + }

  +
• replacement in src/Hydra/root/static/css/logfile.css at line 89
  [2.35]→[2.35:36](∅→∅)
  − }
  [2.35]
  + }
• edit in src/hydra.sql at line 173
  [5.4981]

  [5.4981]
  + errorMsg text, -- used to signal the last evaluation error etc. for this jobset

  + errorTime integer, -- timestamp associated with errorMsg