Cache flake-based jobset evaluations

[?]

May 10, 2019, 10:41 PM

SW7STLQ76HI564FLEJTVUGFMZ2DNMHPJFYFRWY5KYP2OQC42VIVAC

Dependencies

[2] QUMWPGCU Add flake configuration to the web interface
[3] ZPHREC7X hydra-eval-jobset: Support flakes
[4] WVD3YYON hydra-evaluator: Add some debug code
[5] MPFSVI5X Pedantry: CLOCK_REALTIME -> CLOCK_MONOTONIC
[6] AZCCANUB Missing part of aaacf9eda36f5cdcc77cec20d6f49fbb6f925105
[7] RXVJFQ5A Evaluator cleanups
[8] 5SHCWE7X * Prevent repeated evaluation of a jobset with the same inputs. This
[9] PDZD5QOR hydra-evaluator: handle the case where there is no previous jobset eval
[10] IMQRX4MP hydra-eval-jobs: Use JSON instead of XML
[11] Y6AHH4TH Remove the logfile and logSize columns from the database
[12] LZVO64YG Merge in the first bits of the API work
[13] KPZNJ33U * In views, support selecting a job that doesn't depend on the
[14] G2T4WAHI Store the inputs of each evaluation in the database
[15] POPU2ATH * hydra_scheduler: use eval-jobs.
[16] SB2V735V Keep track of the database schema version
[17] CCIORPJ5 When manually scheduling an eval, force re-instantiation of store derivations
[*] N22GPKYT * Put info about logs / build products in the DB.
[*] NS7SND6R hydra-evaluator: Send statistics to statsd
[*] 4JE45C3C Force creation of a new uncached eval if jobs have been removed
[*] RFE6T5LG * Store jobset evaluations in the database explicitly. This includes

Change contents

edit in src/lib/Hydra/Schema/JobsetEvals.pm at line 89

[4.614]

[4.614]

is_nullable: 1

=head2 flake

data_type: 'text'
edit in src/lib/Hydra/Schema/JobsetEvals.pm at line 119

[4.759]

[4.867]

"flake",
{ data_type => "text", is_nullable => 1 },
replacement in src/lib/Hydra/Schema/JobsetEvals.pm at line 198

[4.18354]→[4.31017:31159](∅→∅)

# Created by DBIx::Class::Schema::Loader v0.07033 @ 2013-06-13 01:54:50
# DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:SlEiF8oN6FBK262uSiMKiw

[4.18354]

[4.1674]

# Created by DBIx::Class::Schema::Loader v0.07049 @ 2019-05-11 00:16:10
# DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:XwlJFCJiS0LHsLg2fFqfUg
edit in src/script/hydra-eval-jobset at line 610

[20.125]

[20.125]

my $flakeRef = $jobset->flake;
if (defined $flakeRef) {
(my $res, my $json, my $stderr) = captureStdoutStderr(
600, "nix", "flake", "info", "--tarball-ttl", 0, "--json", "--", $flakeRef);
die "'nix flake info' returned " . ($res & 127 ? "signal $res" : "exit code " . ($res >> 8))
. ":\n" . ($stderr ? decode("utf-8", $stderr) : "(no output)\n")
if $res;
$flakeRef = decode_json($json)->{'uri'};
}
replacement in src/script/hydra-eval-jobset at line 639

[4.55]→[4.585:681](∅→∅)

if (defined $prevEval && $prevEval->hash eq $argsHash && !$dryRun && !$jobset->forceeval) {

[4.55]

[4.1961]

if (defined $prevEval && $prevEval->hash eq $argsHash && !$dryRun && !$jobset->forceeval && $prevEval->flake eq $flakeRef) {
replacement in src/script/hydra-eval-jobset at line 650

[4.217]→[3.1003:1112](∅→∅)

my $jobs = evalJobs($inputInfo, $exprType, $jobset->nixexprinput, $jobset->nixexprpath, $jobset->flake);

[4.217]

[4.218]

my $jobs = evalJobs($inputInfo, $exprType, $jobset->nixexprinput, $jobset->nixexprpath, $flakeRef);
edit in src/script/hydra-eval-jobset at line 710

[21.385]

[22.6334]

, flake => $flakeRef
edit in src/sql/hydra.sql at line 529

[4.1113]

[22.7667]

flake text, -- immutable flake reference
edit in src/sql/upgrade-57.sql at line 7

[2.4556]

alter table JobsetEvals add column flake text;