Distinguish build step states

Dependencies

• [2] YNO7CQ6P hydra-queue-runner: More accurate memory accounting
• [3] EPWEMRI2 Allow determinism checking for entire jobsets
• [4] EYAA7MJY hydra-server: Support logs in S3
• [5] LSUX6IQR Update to latest nixUnstable
• [6] V6S6OYIA Formatting tweaks
• [7] UUGBVEGY * Development notes.
• [8] UNVMKJV5 Unify build and step status codes
• [9] LVQXQIYA Kill active build steps when builds are cancelled
• [10] LENA4O2V Periodically close RemoteStore connections
• [11] K42RSSSI
• [12] CQTN62OH Die tabs die
• [13] YF3HTIPF Use partial indexes
• [14] LZVO64YG Merge in the first bits of the API work
• [15] OTNJLJHA Sort build steps
• [16] TPNHTE5V Remove obsolete Builds columns and provide accurate "Running builds"
• [17] PMNWRTGJ Add multiple output support
• [18] NQ2X3Y4K Don't render machine name if not applicable to step
• [19] 7LWB2J2Z Periodically clear orphaned build steps
• [20] PLOZBRTR Add command ‘hydra-queue-runner --status’ to show current status
• [21] HTL6HIBM machine-status: Read /etc/nix.machines instead of using the BuildMachines table
• [22] 24BMQDZA Start of single-process hydra-queue-runner
• [23] R2PON6R7 Allow non-admin users to see the machine status page
• [24] 6QRHXIM3 * Speed up the jobset index page. Especially the query to get the
• [25] IE2PRAQU hydra-queue-runner: Send build notifications
• [26] KDBWLKRP Index BuildSteps on propagatedFrom
• [27] 62MQPRXC Pass null values to libpqxx properly
• [28] BAFICF73 Support hydra-build-products on binary cache stores
• [29] TFVGX4JS Add an index to speed up the /project/<name> pages
• [30] 2GUAKGTB Fix indentation of build.tt
• [31] 7LFMSF4K Don't show "localhost" as machine for cached failed build steps
• [32] DKJFD6JN Process Nix API changes
• [33] 6K5PBUUN Use buildEnv to combine Hydra's Perl dependencies
• [34] BG6PEOB2 Make the output size limit configurable
• [35] T5BIOVJE Add support for tracking custom metrics
• [36] O5ZANJMR Make "Running builds" and "Machine status" pages faster
• [37] RAKTHYAI * In the job status and error pages, show when the status of a job
• [38] 73YR46NJ hydra-queue-runner: Write directly to a binary cache
• [39] 2DNPZFPN Step cancellation: Don't use pthread_cancel()
• [40] N4IROACV Move buildRemote() into State
• [41] PH3DFCNU Render machine correctly if it doesn't contain @
• [42] B7ENVLRS hydra-queue-runner: Make build notification more reliable
• [43] 5AIYUMTB Basic remote building
• [44] YZAI5GQU Implement a database connection pool
• [45] UNVLTCV4 Fix showing machine name for aborted build steps
• [46] E46TNJK6 Use faster query to determine number of running builds
• [47] TX7Q4RAS Add page showing latest build steps
• [48] 5JB5DKQL Don't repeat links to build step logs
• [49] OG7BEM57
• [50] MHVIT4JY Split hydra-queue-runner.cc more
• [51] UVNTWTWG Prevent download of NARs we just uploaded
• [52] EHEQ4AY3 Fix retry of transient failures
• [*] BYVRA54Q Temporarily disable machines on any exception, not just connection failures
• [*] NAYQT2GT hydra-queue-runner: Use cmdBuildDerivation
• [*] HJOEIMLR Refactor
• [*] J5UVLXOK * Start of a basic Catalyst web interface.
• [*] ZWCTAZGL added newsitems, added some admin options to clear various caches.
• [*] N22GPKYT * Put info about logs / build products in the DB.
• [*] CLJQCY2X * Store info about all the build actions and allow them to be
• [*] D5QIOJGP * Move everything up one directory.

Change contents

• replacement in doc/dev-notes.txt at line 98
  [6.5964]→[6.5964:6094](∅→∅)
  − delete from builds where finished = 0 and not exists (select 1 from buildschedulinginfo s where s.id = builds.id and busy = 1);
  [6.5964]

  [6.94]
  + delete from builds where finished = 0 and not exists (select 1 from buildschedulinginfo s where s.id = builds.id and busy != 0);
• replacement in src/hydra-queue-runner/build-remote.cc at line 125
  [3.81]→[6.0:67](∅→∅),[6.274]→[6.0:67](∅→∅)
  − RemoteResult & result, std::shared_ptr<ActiveStep> activeStep)
  [3.81]

  [6.2925]
  + RemoteResult & result, std::shared_ptr<ActiveStep> activeStep,

  + std::function<void(StepState)> updateStep)
• edit in src/hydra-queue-runner/build-remote.cc at line 143
  [54.10]

  [6.3315]
  +

  + updateStep(ssConnecting);
• edit in src/hydra-queue-runner/build-remote.cc at line 210
  [54.1633]

  [54.1633]
  + updateStep(ssSendingInputs);

  +
• edit in src/hydra-queue-runner/build-remote.cc at line 268
  [55.895]

  [54.3446]
  + updateStep(ssBuilding);

  +
• edit in src/hydra-queue-runner/build-remote.cc at line 381
  [54.7058]

  [5.198]
  + updateStep(ssReceivingOutputs);

  +
• edit in src/hydra-queue-runner/builder.cc at line 197
  [6.4260]

  [6.4260]
  +

  + auto updateStep = [&](StepState stepState) {

  + pqxx::work txn(*conn);

  + updateBuildStep(txn, buildId, stepNr, stepState);

  + txn.commit();

  + };
• replacement in src/hydra-queue-runner/builder.cc at line 207
  [6.4376]→[3.934:1043](∅→∅)
  − buildRemote(destStore, machine, step, maxSilentTime, buildTimeout, repeats, result, activeStep);
  [6.4376]

  [6.2415]
  + buildRemote(destStore, machine, step, maxSilentTime, buildTimeout, repeats, result, activeStep, updateStep);
• replacement in src/hydra-queue-runner/builder.cc at line 222
  [6.4608]→[6.2549:2593](∅→∅)
  − if (result.stepStatus == bsSuccess)
  [6.4608]

  [6.351]
  + if (result.stepStatus == bsSuccess) {

  + updateStep(ssPostProcessing);
• edit in src/hydra-queue-runner/builder.cc at line 225
  [6.441]

  [2.669]
  + }
• replacement in src/hydra-queue-runner/hydra-queue-runner.cc at line 274
  [6.22]→[6.7878:7964](∅→∅),[6.7878]→[6.7878:7964](∅→∅)
  − ("update BuildSteps set busy = 0, status = $1, stopTime = $2 where busy = 1")
  [6.22]

  [6.425]
  + ("update BuildSteps set busy = 0, status = $1, stopTime = $2 where busy != 0")
• edit in src/hydra-queue-runner/hydra-queue-runner.cc at line 317
  [6.9258]

  [6.9258]
  + }

  +

  +

  + void State::updateBuildStep(pqxx::work & txn, BuildID buildId, unsigned int stepNr, StepState stepState)

  + {

  + if (txn.parameterized

  + ("update BuildSteps set busy = $1 where build = $2 and stepnr = $3 and busy != 0 and status is null")

  + ((int) stepState)

  + (buildId)

  + (stepNr)

  + .exec().affected_rows() != 1)

  + throw Error("step %d of build %d is in an unexpected state", stepNr, buildId);
• replacement in src/hydra-queue-runner/hydra-queue-runner.cc at line 907
  [6.1102]→[6.1102:1220](∅→∅)
  − ("update BuildSteps set busy = 0, status = $1 where build = $2 and stepnr = $3 and busy = 1")
  [6.1102]

  [6.1220]
  + ("update BuildSteps set busy = 0, status = $1 where build = $2 and stepnr = $3 and busy != 0")
• edit in src/hydra-queue-runner/state.hh at line 44
  [56.1921]

  [6.3974]
  + typedef enum {

  + ssPreparing = 1,

  + ssConnecting = 10,

  + ssSendingInputs = 20,

  + ssBuilding = 30,

  + ssReceivingOutputs = 40,

  + ssPostProcessing = 50,

  + } StepState;

  +

  +
• edit in src/hydra-queue-runner/state.hh at line 476
  [56.7254]

  [56.7254]
  +

  + void updateBuildStep(pqxx::work & txn, BuildID buildId, unsigned int stepNr, StepState stepState);
• replacement in src/hydra-queue-runner/state.hh at line 533
  [3.2846]→[6.2341:2413](∅→∅),[6.1984]→[6.2341:2413](∅→∅)
  − RemoteResult & result, std::shared_ptr<ActiveStep> activeStep);
  [3.2846]

  [6.2016]
  + RemoteResult & result, std::shared_ptr<ActiveStep> activeStep,

  + std::function<void(StepState)> updateStep);
• replacement in src/lib/Hydra/Controller/Root.pm at line 56
  [6.65]→[6.95:171](∅→∅),[6.95]→[6.95:171](∅→∅)
  − "select count(distinct build) from buildsteps where busy = 1");
  [6.65]

  [6.2622]
  + "select count(distinct build) from buildsteps where busy != 0");
• replacement in src/lib/Hydra/Controller/Root.pm at line 148
  [6.1314]→[6.122:162](∅→∅)
  − { "buildsteps.busy" => 1 },
  [6.1314]

  [6.1369]
  + { "buildsteps.busy" => { '!=', 0 } },
• replacement in src/lib/Hydra/Controller/Root.pm at line 196
  [6.413]→[6.413:464](∅→∅)
  − "where busy = 1 order by machine, stepnr",
  [6.413]

  [6.464]
  + "where busy != 0 order by machine, stepnr",
• replacement in src/root/build.tt at line 32
  [6.85]→[6.5096:5236](∅→∅)
  − [% IF ( type == "All" ) || ( type == "Failed" && step.busy == 0 && step.status != 0 ) || ( type == "Running" && step.busy == 1 ) %]
  [6.85]

  [6.54]
  + [% IF ( type == "All" ) || ( type == "Failed" && step.busy == 0 && step.status != 0 ) || ( type == "Running" && step.busy != 0 ) %]
• replacement in src/root/build.tt at line 58
  [6.932]→[6.5237:5497](∅→∅)
  − <td>[% IF step.busy == 1 || ((step.machine || step.starttime) && (step.status == 0 || step.status == 1 || step.status == 3 || step.status == 4 || step.status == 7)); INCLUDE renderMachineName machine=step.machine; ELSE; "<em>n/a</em>"; END %]</td>
  [6.932]

  [3.3256]
  + <td>[% IF step.busy != 0 || ((step.machine || step.starttime) && (step.status == 0 || step.status == 1 || step.status == 3 || step.status == 4 || step.status == 7)); INCLUDE renderMachineName machine=step.machine; ELSE; "<em>n/a</em>"; END %]</td>
• replacement in src/root/build.tt at line 60
  [3.3293]→[6.1002:1082](∅→∅),[6.1002]→[6.1002:1082](∅→∅)
  − [% IF step.busy == 1 %]

  − <strong>Building</strong>
  [3.3293]

  [6.1082]
  + [% IF step.busy != 0 %]

  + [% IF step.busy == 1 %]

  + <strong>Preparing</strong>

  + [% ELSIF step.busy == 10 %]

  + <strong>Connecting</strong>

  + [% ELSIF step.busy == 20 %]

  + <strong>Sending inputs</strong>

  + [% ELSIF step.busy == 30 %]

  + <strong>Building</strong>

  + [% ELSIF step.busy == 40 %]

  + <strong>Receiving outputs</strong>

  + [% ELSIF step.busy == 50 %]

  + <strong>Post-processing</strong>

  + [% ELSE %]

  + <strong>Unknown state</strong>

  + [% END %]
• replacement in src/root/machine-status.tt at line 45
  [6.788]→[4.3805:3940](∅→∅)
  − <td><a class="row-link" href="[% c.uri_for('/build' step.build 'nixlog' step.stepnr 'tail') %]">[% step.stepnr %]</a></td>
  [6.788]

  [6.1149]
  + <td>[% IF step.busy >= 30 %]<a class="row-link" href="[% c.uri_for('/build' step.build 'nixlog' step.stepnr 'tail') %]">[% step.stepnr %]</a>[% ELSE; step.stepnr; END %]</td>
• edit in src/sql/hydra.sql at line 275
  [60.7341]

  [60.7366]
  + -- 0 = not busy

  + -- 1 = building

  + -- 2 = preparing to build

  + -- 3 = connecting

  + -- 4 = sending inputs

  + -- 5 = receiving outputs

  + -- 6 = analysing build result
• replacement in src/sql/hydra.sql at line 658
  [6.2384]→[6.0:71](∅→∅)
  − create index IndexBuildStepsOnBusy on BuildSteps(busy) where busy = 1;
  [6.2384]

  [6.71]
  + create index IndexBuildStepsOnBusy on BuildSteps(busy) where busy != 0;
• file addition: upgrade-56.sql (----------)
  [61.3004]
  + drop index IndexBuildStepsOnBusy;

  + create index IndexBuildStepsOnBusy on BuildSteps(busy) where busy != 0;