* Prevent repeated evaluation of a jobset with the same inputs. This

Dependencies

• [2] 3XTHEUMP * Implemented the clone feature.
• [3] TWVSALRL * Allow the maximum number of concurrent builds per platform to be
• [4] S66BOMVU * Added authentication.
• [5] X27GNHDV * Basic job info in the database.
• [6] D3DIBMOK * For products that are directories (like manuals), allow a default
• [7] B72GLND4
• [8] BD3GRK4B * Get rid of "positive failures" and separate log phases.
• [9] GNIEG2GC * Disambiguate jobs by jobset name. I.e. jobs with the same name in
• [10] AHTEIK7G * Added a maintainers field to the Builds table.
• [11] EYNG4EL4 * Regenerate the bindings from a clean sqlite database.
• [12] ZVTSOVHN * Support Subversion checkouts.
• [13] L2E6EVE2 * Merged the Build and Job tables.
• [14] 37R34XJO * Negative caching: don't perform a build if a dependency already
• [15] M552HLIA * Support variant builds.
• [16] H7CNGK4O * Log evaluation errors etc. in the DB.
• [17] 7YBYT2LQ
• [18] S5PV6IIM * Represent jobs explicitly in the DB.
• [19] P5X4P6VK * Renaming "release sets" to "views" (not finished yet). Having
• [20] YAPITGB3 * Boolean inputs.
• [21] 4D4U5IPY * Allow jobsets to be disabled.
• [22] KN3VYE5P * Cleaned up the foreign key constraints.
• [23] FHF6IZJQ * Basic release management: releases are now dynamically computed as
• [24] VJHIHMEH * Store the meta.longDescription and meta.license attributes in the
• [25] KOTB7BKV
• [26] V4RNHJNR * Add a link to each project's homepage. Suggested by karltk.
• [27] 67P45PY4
• [28] GWCV3TQV * BuildInputs table: link to dependencies, include store paths.
• [29] S6OISBQ3 * Mark the "current" builds in a jobset, i.e. those corresponding to
• [30] ZEHSSVFG
• [31] IK53RV4V
• [32] BVOPAMLS
• [33] 3ZCEPLNO
• [34] PHX2HIVG * Store info about the build inputs in the build record.
• [35] ECBA3GQO * Make the schema class names match the case of the SQL table names.
• [36] TQKGQ5R3
• [37] OOQ2D3KC * Refactoring: move fetchInput out of hydra_scheduler into a separate
• [38] R5D7DZPE
• [39] CLJQCY2X * Store info about all the build actions and allow them to be
• [40] 6BLUKEQ2 * Caching of "path" inputs, and fake a revision number for those.
• [41] DVNWJXWW * Generic declaration of build products.
• [42] RU7AQO7U * Role-based access control. Only admins can create projects. Only
• [43] YTZOC7C5 * Editing of jobset inputs.
• [44] CMU3YKOU * Store the release name.
• [45] TLZ2SPBR
• [46] BHZXGT2H * Channels: provide an index page that lists all the packages in the
• [47] RBNQKATL * Adding persistant releases. A release is a named set of builds.
• [48] SHBLLAVH * More global substitution.
• [49] LCKWLQW3 * In Sqlite "release" is now a keyword, so use "release_" instead.
• [50] 3HCBU2FA
• [51] 3E6IP3R3 * Add the name of the jobset to ReleaseSetJobs, otherwise we can't
• [*] D5QIOJGP * Move everything up one directory.
• [*] N22GPKYT * Put info about logs / build products in the DB.
• [*] RWIBJ5L4 * Autoflush stdout.
• [*] FDE3BJAP * Refactoring.
• [*] POPU2ATH * hydra_scheduler: use eval-jobs.

Change contents

• replacement in src/lib/Hydra/Helper/AddBuilds.pm at line 11
  [3.2785]→[2.1305:1355](∅→∅)
  − our @EXPORT = qw(fetchInput evalJobs checkBuild);
  [3.2785]

  [3.2815]
  + our @EXPORT = qw(fetchInput evalJobs checkBuild inputsToArgs);
• file addition: JobsetInputHashes.pm (----------)
  [53.477]
  + package Hydra::Schema::JobsetInputHashes;

  +

  + # Created by DBIx::Class::Schema::Loader

  + # DO NOT MODIFY THE FIRST PART OF THIS FILE

  +

  + use strict;

  + use warnings;

  +

  + use base 'DBIx::Class';

  +

  + __PACKAGE__->load_components("Core");

  + __PACKAGE__->table("JobsetInputHashes");

  + __PACKAGE__->add_columns(

  + "project",

  + {

  + data_type => "text",

  + default_value => undef,

  + is_foreign_key => 1,

  + is_nullable => 0,

  + size => undef,

  + },

  + "jobset",

  + {

  + data_type => "text",

  + default_value => undef,

  + is_foreign_key => 1,

  + is_nullable => 0,

  + size => undef,

  + },

  + "hash",

  + {

  + data_type => "text",

  + default_value => undef,

  + is_nullable => 0,

  + size => undef,

  + },

  + "timestamp",

  + {

  + data_type => "integer",

  + default_value => undef,

  + is_nullable => 0,

  + size => undef,

  + },

  + );

  + __PACKAGE__->set_primary_key("project", "jobset", "hash");

  + __PACKAGE__->belongs_to("project", "Hydra::Schema::Projects", { name => "project" });

  + __PACKAGE__->belongs_to(

  + "jobset",

  + "Hydra::Schema::Jobsets",

  + { name => "jobset", project => "project" },

  + );

  +

  +

  + # Created by DBIx::Class::Schema::Loader v0.04999_09 @ 2009-11-17 14:04:55

  + # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:f8/4vTSQJbmAh/0PZHeFDg

  +

  +

  + # You can replace this text with custom content, and it will be preserved on regeneration

  + 1;
• edit in src/lib/Hydra/Schema/Jobsets.pm at line 102
  [3.2021]

  [3.2594]
  + {

  + "foreign.jobset" => "self.name",

  + "foreign.project" => "self.project",

  + },

  + );

  + __PACKAGE__->has_many(

  + "jobsetinputhashes",

  + "Hydra::Schema::JobsetInputHashes",
• replacement in src/lib/Hydra/Schema/Jobsets.pm at line 117
  [3.2992]→[3.2764:2909](∅→∅)
  − # Created by DBIx::Class::Schema::Loader v0.04999_09 @ 2009-10-23 16:56:03

  − # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:M+eetraKtSfF8q3cqJhEPw
  [3.2992]

  [3.3134]
  + # Created by DBIx::Class::Schema::Loader v0.04999_09 @ 2009-11-17 14:04:55

  + # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:xWsqXneZw90uEw/vcEXc4w
• edit in src/lib/Hydra/Schema/Projects.pm at line 90
  [3.2384]

  [3.3683]
  + __PACKAGE__->has_many(

  + "jobsetinputhashes",

  + "Hydra::Schema::JobsetInputHashes",

  + { "foreign.project" => "self.name" },

  + );
• replacement in src/lib/Hydra/Schema/Projects.pm at line 97
  [3.3685]→[3.2997:3142](∅→∅)
  − # Created by DBIx::Class::Schema::Loader v0.04999_09 @ 2009-10-23 16:56:03

  − # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:zhL+ArisX2ZFU0NPIuDLdw
  [3.3685]

  [3.3827]
  + # Created by DBIx::Class::Schema::Loader v0.04999_09 @ 2009-11-17 14:04:55

  + # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:dWe2DEsuZuOjVj4IA8TwQg
• edit in src/script/hydra_scheduler.pl at line 8
  [3.10946]

  [55.47]
  + use Digest::SHA qw(sha256_hex);
• edit in src/script/hydra_scheduler.pl at line 54
  [56.4019]

  [57.2191]
  + # Hash the arguments to hydra_eval_jobs and check the

  + # JobsetInputHashes to see if we've already evaluated this set of

  + # inputs. If so, bail out.

  + my @args = ($jobset->nixexprinput, $jobset->nixexprpath, inputsToArgs($inputInfo));

  + my $argsHash = sha256_hex("@args");

  +

  + if ($jobset->jobsetinputhashes->find({hash => $argsHash})) {

  + print " already evaluated, skipping\n";

  + txn_do($db, sub {

  + $jobset->update({lastcheckedtime => time});

  + });

  + return;

  + }

  +
• edit in src/script/hydra_scheduler.pl at line 101
  [3.5849]

  [3.6621]
  +

  + $jobset->jobsetinputhashes->create({hash => $argsHash, timestamp => time});
• edit in src/sql/hydra.sql at line 379
  [3.5020]

  [3.5020]
  + );

  +

  +

  + -- This table is used to prevent repeated Nix expression evaluation

  + -- for the same set of inputs for a jobset. In the scheduler, after

  + -- obtaining the current inputs for a jobset, we hash the inputs

  + -- together, and if the resulting hash already appears in this table,

  + -- we can skip the jobset. Otherwise it's added to the table, and the

  + -- Nix expression for the jobset is evaluated. The hash is computed

  + -- over the command-line arguments to hydra_eval_jobs.

  + create table JobsetInputHashes (

  + project text not null,

  + jobset text not null,

  + hash text not null,

  + timestamp integer not null,

  + primary key (project, jobset, hash),

  + foreign key (project) references Projects(name) on delete cascade on update cascade,

  + foreign key (project, jobset) references Jobsets(project, name) on delete cascade on update cascade