* Negative caching: don't perform a build if a dependency already

[?]
Mar 9, 2009, 5:21 PM
37R34XJOGMP3E3DD4RGGRX5LBW2LGB6J3E6XOXLWSU2QEIWBEM4QC

Dependencies

  • [2] 2T42QGZD * Register builds as GC roots so they don't get deleted.
  • [3] NDL67SQT * XHTML validity.
  • [4] NLJJZVHO * Use ->update({...}) properly.
  • [5] ECBA3GQO * Make the schema class names match the case of the SQL table names.
  • [6] 7YBYT2LQ
  • [7] ZEHSSVFG
  • [8] V4RNHJNR * Add a link to each project's homepage. Suggested by karltk.
  • [9] 6BLUKEQ2 * Caching of "path" inputs, and fake a revision number for those.
  • [10] IK53RV4V
  • [11] YAPITGB3 * Boolean inputs.
  • [12] EYNG4EL4 * Regenerate the bindings from a clean sqlite database.
  • [13] VJHIHMEH * Store the meta.longDescription and meta.license attributes in the
  • [14] BHZXGT2H * Channels: provide an index page that lists all the packages in the
  • [15] FHF6IZJQ * Basic release management: releases are now dynamically computed as
  • [16] TWVSALRL * Allow the maximum number of concurrent builds per platform to be
  • [17] M552HLIA * Support variant builds.
  • [18] KOTB7BKV
  • [19] WYN733ST * Store build duration, handle cached builds.
  • [20] CMU3YKOU * Store the release name.
  • [21] B72GLND4
  • [22] D3DIBMOK * For products that are directories (like manuals), allow a default
  • [23] VCOSLZRP
  • [24] S66BOMVU * Added authentication.
  • [25] TQKGQ5R3
  • [26] GWCV3TQV * BuildInputs table: link to dependencies, include store paths.
  • [27] J5UVLXOK * Start of a basic Catalyst web interface.
  • [28] X27GNHDV * Basic job info in the database.
  • [29] WHAFVCEI
  • [30] CLJQCY2X * Store info about all the build actions and allow them to be
  • [31] PHX2HIVG * Store info about the build inputs in the build record.
  • [32] BD3GRK4B * Get rid of "positive failures" and separate log phases.
  • [33] 67P45PY4
  • [34] YTZOC7C5 * Editing of jobset inputs.
  • [35] SHBLLAVH * More global substitution.
  • [36] H7CNGK4O * Log evaluation errors etc. in the DB.
  • [37] R5D7DZPE
  • [38] RU7AQO7U * Role-based access control. Only admins can create projects. Only
  • [39] BVOPAMLS
  • [40] DVNWJXWW * Generic declaration of build products.
  • [41] ZVTSOVHN * Support Subversion checkouts.
  • [42] A52HEFHQ * Allow builds to be restarted (if they failed with a transient error,
  • [43] L2E6EVE2 * Merged the Build and Job tables.
  • [44] 3ZCEPLNO
  • [45] TLZ2SPBR
  • [*] ZILILXXK * Allow scheduled builds to be cancelled. They're not removed from
  • [*] N22GPKYT * Put info about logs / build products in the DB.

Change contents

  • replacement in src/lib/Hydra/Schema/BuildInputs.pm at line 39
    [5.782][5.295:437]()
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-06 14:20:12
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:H25BhHe9wbU6nj6fSKjZnw
    [5.782]
    [5.924]
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-09 18:05:06
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:L6NP/+9zhMg4TRw3w911rg
  • replacement in src/lib/Hydra/Schema/BuildProducts.pm at line 38
    [5.4922][5.516:658]()
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-06 14:20:12
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:rfkj/A+Li7Q0hWydqtJHAw
    [5.4922]
    [5.5064]
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-09 18:05:06
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:yTJRsclWIpHApRSLqyR06Q
  • edit in src/lib/Hydra/Schema/BuildResultInfo.pm at line 28
    [5.883]
    [5.883]
    { data_type => "integer", is_nullable => 0, size => undef },
    "faileddepbuild",
    { data_type => "integer", is_nullable => 0, size => undef },
    "faileddepstepnr",
  • replacement in src/lib/Hydra/Schema/BuildResultInfo.pm at line 38
    [5.2382][5.659:801]()
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-06 14:20:12
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:wmL9881G+dZrgHKM83dHXw
    [5.2382]
    [5.2524]
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-09 18:05:06
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:o1XAUgKd15pN76Rs8aX+IA
  • edit in src/lib/Hydra/Schema/BuildResultInfo.pm at line 41
    [5.2525]
    [5.2525]
    __PACKAGE__->belongs_to(
    "failedDep",
    "Hydra::Schema::BuildSteps",
    { id => "faileddepbuild", stepnr => "faileddepstepnr" },
    );
  • edit in src/lib/Hydra/Schema/BuildResultInfo.pm at line 47
    [5.2526][5.2526:2616]()
    # You can replace this text with custom content, and it will be preserved on regeneration
  • replacement in src/lib/Hydra/Schema/BuildSchedulingInfo.pm at line 30
    [5.4473][5.802:944]()
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-06 14:20:12
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:6cTj5JovtTmtemvQjWtucQ
    [5.4473]
    [5.4615]
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-09 18:05:06
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:MeS9DLCs3kuhS4pTqfl4Lg
  • edit in src/lib/Hydra/Schema/BuildSteps.pm at line 36
    [5.2498][5.2814:2815](),[5.2814][5.2814:2815]()
  • edit in src/lib/Hydra/Schema/BuildSteps.pm at line 37
    [5.2816][5.945:1087]()
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-06 14:20:12
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:LyNlkn5XjBnLp7M4ipB/ZQ
  • edit in src/lib/Hydra/Schema/BuildSteps.pm at line 38
    [5.2959]
    [5.2959]
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-09 18:05:06
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:nFSksz60oxcM1+nYqAdjzw
  • edit in src/lib/Hydra/Schema/BuildSteps.pm at line 41
    [5.2960][5.2960:3050]()
    # You can replace this text with custom content, and it will be preserved on regeneration
  • replacement in src/lib/Hydra/Schema/Builds.pm at line 79
    [5.6194][5.1088:1230]()
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-06 14:20:12
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:8U9CmcfeowLJVViKiR3n1g
    [5.6194]
    [5.3377]
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-09 18:05:06
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:p9SIzdyW9p4+eqwKwXzAdQ
  • replacement in src/lib/Hydra/Schema/CachedPathInputs.pm at line 25
    [5.1497][5.1231:1373]()
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-06 14:20:12
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:+F3EiYsvXuOjnGDrkhLxng
    [5.1497]
    [5.1639]
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-09 18:05:06
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:zvTGb3qhlacSWzc0H+7dfg
  • replacement in src/lib/Hydra/Schema/CachedSubversionInputs.pm at line 23
    [5.2342][5.1374:1516]()
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-06 14:20:12
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:O0QvXjOulMVTjhW4rRHkmQ
    [5.2342]
    [5.2484]
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-09 18:05:06
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:TY8SHfWiBibwiG3WPkP8Mg
  • replacement in src/lib/Hydra/Schema/JobsetInputAlts.pm at line 34
    [5.1831][5.1517:1659]()
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-06 14:20:12
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:qXh+0QbMUHIDmQCG9T5qdA
    [5.1831]
    [5.1973]
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-09 18:05:06
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:4iKXGIdrm56xZeHFxa/K4A
  • replacement in src/lib/Hydra/Schema/JobsetInputs.pm at line 46
    [5.1681][5.1660:1802]()
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-06 14:20:12
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:TUTdwfhSsFMKBXa/wKenOQ
    [5.1681]
    [5.1823]
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-09 18:05:06
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:rh03/olFXpMiTA+zhRYltg
  • replacement in src/lib/Hydra/Schema/Jobsets.pm at line 53
    [5.2992][5.1803:1945]()
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-06 14:20:12
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:dVevQ8lPI2/IRpYoJgzLBA
    [5.2992]
    [5.3134]
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-09 18:05:06
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:IfDpZfiD9haRHLXGdkapGg
  • replacement in src/lib/Hydra/Schema/Projects.pm at line 48
    [5.3685][5.1946:2088]()
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-06 14:20:12
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:Q5mSKzMxB9px2ja8NjK/9Q
    [5.3685]
    [5.3827]
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-09 18:05:06
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:7VRmssxrhgzfySJ6OGuhGg
  • replacement in src/lib/Hydra/Schema/ReleaseSetJobs.pm at line 35
    [5.6070][5.2089:2231]()
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-06 14:20:12
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:+Ky8V3sZIgT22hgF27Y0cw
    [5.6070]
    [5.6212]
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-09 18:05:06
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:GdOtS3nLs7dwpArSdGob0w
  • replacement in src/lib/Hydra/Schema/ReleaseSets.pm at line 32
    [5.7142][5.2232:2374]()
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-06 14:20:12
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:f+4AnWTJsi4RDfxoJxECgw
    [5.7142]
    [5.7284]
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-09 18:05:06
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:lQymTkBv/Av2y6iYjvP5PQ
  • replacement in src/lib/Hydra/Schema/SystemTypes.pm at line 19
    [5.2105][5.2375:2517]()
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-06 14:20:12
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:Af/LU15/hpXngfrBrDMI/A
    [5.2105]
    [5.2247]
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-09 18:05:06
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:KZzTedcMG8IxkLgEEkdn9A
  • replacement in src/lib/Hydra/Schema/UserRoles.pm at line 20
    [5.3736][5.2518:2660]()
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-06 14:20:12
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:/yDlbFhRYDzf+0VHzygrhA
    [5.3736]
    [5.3878]
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-09 18:05:06
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:9YGxnZD9hLoJJ8b1aejBuA
  • replacement in src/lib/Hydra/Schema/Users.pm at line 33
    [5.3782][5.2661:2803]()
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-06 14:20:12
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:d7M/Q6OucU9NUCSB5zZK7Q
    [5.3782]
    [5.3924]
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-09 18:05:06
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:HwKK4N8V5fzvR3XrKnrDEA
  • replacement in src/lib/Hydra/Schema.pm at line 11
    [5.6498][5.2804:2946]()
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-06 14:20:12
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:hRwus0A1vxnFCzQxghFgTw
    [5.6498]
    [5.6640]
    # Created by DBIx::Class::Schema::Loader v0.04005 @ 2009-03-09 18:05:06
    # DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:DQAmauNODAxmM2mF/AE2aQ
  • replacement in src/root/build.tt at line 39
    [3.132][5.1697:1780](),[5.602][5.1697:1780]()
    <span class="error">Build returned a non-zero exit code</span>
    [3.132]
    [5.2176]
    <span class="error">Build returned a non-zero exit code</span>
  • edit in src/root/build.tt at line 46
    [47.1499]
    [5.665]
    [% ELSIF build.resultInfo.buildstatus == 5 %]
    <img src="/static/images/failure.gif" alt="Failed" />
    <span class="error">Build inhibited because a dependency previously failed to build</span>
    [% failedDep = build.resultInfo.failedDep %]
    (namely, <a href="[% c.uri_for('/build' failedDep.id.id 'nixlog' failedDep.stepnr) %]"><tt>[% failedDep.outpath %]</tt></a>)
  • replacement in src/root/build.tt at line 264
    [5.1108][5.3131:3166]()
    [% IF build.resultInfo.errormsg %]
    [5.1108]
    [5.1108]
    [% IF build.resultInfo.errormsg && build.resultInfo.buildstatus != 5 %]
  • edit in src/script/hydra_build.pl at line 27
    [5.3622]
    [2.495]
    my $failedDepBuild;
    my $failedDepStepNr;
  • edit in src/script/hydra_build.pl at line 36
    [5.3533]
    [4.443]
    # Check whether a dependency of this build has previously
    # failed. If so, don't even bother to build, since it will
    # fail anyway. !!! Once Nix has negative caching, this code
    # can go.
    my @drvDeps = split '\n', `nix-store --query --requisites --include-outputs $drvPath`;
    die "cannot query dependencies of `$drvPath': $?" if $? != 0;
    @drvDeps = grep { $_ =~ /.drv$/ } @drvDeps;
    my @drvOutputs = split '\n', `nix-store --query --outputs @drvDeps`;
    die "cannot query outputs of the dependencies of `$drvPath': $?" if $? != 0;
    foreach my $dep (@drvOutputs) {
    # !!! This checks more than it has to, namely
    # build-time-only dependencies of dependencies (which
    # don't need to be built). However, it shouldn't matter:
    # if the dependency was built, then presumably *its*
    # dependencies were built as well.
    # !!! should disregard fixed-output derivations (?)
    if (!isValidPath($dep)) {
    my ($step) = $db->resultset('BuildSteps')->search(
    {outPath => $dep}, {rows => 1, order_by => "stopTime DESC"});
    if (defined $step && $step->status != 0) {
    $buildStatus = 5;
    $failedDepBuild = $step->id->id;
    $failedDepStepNr = $step->stepnr;
    goto done;
    }
    }
    }
  • edit in src/script/hydra_build.pl at line 198
    [5.4020]
    [5.4020]
    done:
  • replacement in src/script/hydra_build.pl at line 202
    [5.4043][4.817:871]()
    $build->({finished => 1, timestamp => time});
    [5.4043]
    [5.7526]
    $build->update({finished => 1, timestamp => time});
  • edit in src/script/hydra_build.pl at line 224
    [5.1954]
    [5.4574]
    , faileddepbuild => $failedDepBuild
    , faileddepstepnr => $failedDepStepNr
  • edit in src/sql/hydra.sql at line 65
    [47.3171]
    [5.4847]
    -- 5 = build not done because a dependency failed previously
  • edit in src/sql/hydra.sql at line 78
    [5.2508]
    [5.4926]
    -- If buildStatus == 5, the primary key of the failed build step.
    failedDepBuild integer,
    failedDepStepNr integer,
  • replacement in src/sql/hydra.sql at line 88
    [5.7150][5.7150:7186]()
    id integer not null,
    [5.7150]
    [5.7186]
    id integer not null, -- !!! rename to "build"
  • replacement in src/sql/hydra.sql at line 93
    [5.7291][5.7291:7316]()
    drvPath text,
    [5.7291]
    [5.7316]
    drvPath text,