grahamc/hydra2 - Change ACFFJUAN5R2AFJVVJ5OBX6BQHT3M4SHO7QXILF5LURVIFPDORLIQC

hydra-update-gc-roots: Keep more evals

We now keep all unfinished evaluations of a jobset, in addition to the <keepnr> most recent finished evaluations.

The main motivation is to ensure that mirror-{nixos,nixpkgs} work properly: if building an evaluation takes too long, some of its builds may already have been garbage-collected by the time the others finish.

Created by Eelco Dolstra on September 18, 2013

ACFFJUAN5R2AFJVVJ5OBX6BQHT3M4SHO7QXILF5LURVIFPDORLIQC

Dependencies

In channels

main

Change contents

Insertion in src/script/hydra-update-gc-roots at line 59 [5.472]

[3.302]

[4.720]

# For scheduled builds, we register the derivation as a GC root.
print STDERR "*** looking for scheduled builds\n";
keepBuild $_ foreach $db->resultset('Builds')->search({ finished => 0 }, { columns => [ @columns ] });

Deletion in src/script/hydra-update-gc-roots at line 89 [5.472]

B:BD[6.306] → [6.306:334]

B:BD[6.334] → [7.920:921]

∅:D[7.921] → [6.343:475]

B:BD[6.343] → [6.343:475]

            next;
        }
        if ($keepnr <= 0 ) {
            print STDERR "*** jobset ", $project->name, ":", $jobset->name, " set to keep 0 builds\n";

Replacement in src/script/hydra-update-gc-roots at line 92 [5.472]

B:BD[7.923] → [8.123:224]

        print STDERR "*** looking for all builds in the $keepnr most recent evaluations of jobset ",

[7.923]

[9.18998]

        print STDERR "*** looking for all builds in the unfinished and $keepnr most recent finished evaluations of jobset ",

Insertion in src/script/hydra-update-gc-roots at line 94 [5.472]

[9.19052]

[10.797]


        my @evals;
        # Get the unfinished evals.
        push @evals, $_->get_column("eval") foreach $jobset->builds->search(
            { finished => 0 },
            { join => "jobsetevalmembers", select => "jobsetevalmembers.eval", as => "eval", distinct => 1 });
        # Get the N most recent finished evals.
        if ($keepnr) {
            push @evals, $_->get_column("id") foreach $jobset->jobsetevals->search(
                { hasNewBuilds => 1 },
                { where => \ "not exists (select 1 from builds b join jobsetevalmembers m on b.id = m.build where m.eval = me.id and b.finished = 0)"
                , order_by => "id desc", rows => $keepnr });
        }

Replacement in src/script/hydra-update-gc-roots at line 112 [5.472]

B:BD[2.116] → [2.116:145]

∅:D[2.145] → [8.349:617]

B:BD[8.349] → [8.349:617]

            , id => { -in =>
                \ [ "select build from JobsetEvalMembers where eval in (select id from JobsetEvals where project = ? and jobset = ? and hasNewBuilds = 1 order by id desc limit ?)",
                    [ '', $project->name ], [ '', $jobset->name ], [ '', $keepnr ] ] }

[2.116]

[2.146]

            , id => { -in => $db->resultset('JobsetEvalMembers')->search({ eval => { -in => [@evals] } }, { select => "build" })->as_query }

Deletion in src/script/hydra-update-gc-roots at line 135 [5.472]
B:BD[10.1500] → [11.1304:1369]
∅:D[11.1369] → [12.2315:2366]
B:BD[13.769] → [12.2315:2366]
B:BD[12.2366] → [4.2620:2723]
∅:D[4.2723] → [5.1968:1970]
B:BD[5.1968] → [5.1968:1970]
```
# For scheduled builds, we register the derivation as a GC root.
print STDERR "*** looking for scheduled builds\n";
keepBuild $_ foreach $db->resultset('Builds')->search({ finished => 0 }, { columns => [ @columns ] });
```