Keep track of wait time per system type

Dependencies

• [2] 5N7LVAJN Keep track of requiredSystemFeatures in the machine stats
• [3] RQUAATWB Add status dump facility
• [4] HJOEIMLR Refactor
• [5] 4I2HF4L3 Unindent
• [6] EYR3EW6J Keep stats for the Hydra auto scaler
• [7] MHVIT4JY Split hydra-queue-runner.cc more
• [8] WE5Q2NVI Allow build to be bumped to the front of the queue via the web interface
• [9] 46ADBTMQ Start steps in order of ascending build ID
• [10] UPNGFCXG For completeness, re-implement meta.schedulingPriority
• [11] IK2UBDAU Revive jobset scheduling
• [*] 24BMQDZA Start of single-process hydra-queue-runner

Change contents

• edit in src/hydra-queue-runner/dispatcher.cc at line 19
  [3.14988]

  [3.14988]
  + step_->runnableSince = std::chrono::system_clock::now();
• replacement in src/hydra-queue-runner/dispatcher.cc at line 147
  [3.292]→[3.317:388](∅→∅)
  − std::unordered_map<std::string, unsigned int> runnablePerType;
  [3.292]

  [3.292]
  + struct RunnablePerType

  + {

  + unsigned int count{0};

  + std::chrono::seconds waitTime{0};

  + };

  + std::unordered_map<std::string, RunnablePerType> runnablePerType;
• replacement in src/hydra-queue-runner/dispatcher.cc at line 167
  [3.390]→[2.0:53](∅→∅)
  − runnablePerType[step->systemType]++;
  [3.390]

  [3.18020]
  + auto & r = runnablePerType[step->systemType];

  + r.count++;
• edit in src/hydra-queue-runner/dispatcher.cc at line 174
  [3.3929]

  [3.3929]
  + r.waitTime += std::chrono::duration_cast<std::chrono::seconds>(now - step_->runnableSince);
• replacement in src/hydra-queue-runner/dispatcher.cc at line 230
  [3.1899]→[2.54:174](∅→∅)
  − assert(runnablePerType[step->systemType]);

  − runnablePerType[step->systemType]--;
  [3.1899]

  [3.4106]
  + auto & r = runnablePerType[step->systemType];

  + assert(r.count);

  + r.count--;
• replacement in src/hydra-queue-runner/dispatcher.cc at line 255
  [3.939]→[3.939:1047](∅→∅)
  − for (auto & i : runnablePerType)

  − (*machineTypes_)[i.first].runnable = i.second;
  [3.939]

  [3.4615]
  + for (auto & i : runnablePerType) {

  + auto & j = (*machineTypes_)[i.first];

  + j.runnable = i.second.count;

  + j.waitTime = i.second.waitTime;

  + }
• edit in src/hydra-queue-runner/dispatcher.cc at line 262
  [3.19309]

  [3.4626]
  + lastDispatcherCheck = std::chrono::system_clock::to_time_t(now);

  +
• replacement in src/hydra-queue-runner/dispatcher.cc at line 326
  [3.1786]→[3.1786:1832](∅→∅)
  − machineType.lastActive = time(0);
  [3.1786]

  [3.1721]
  + machineType.lastActive = std::chrono::system_clock::now();
• edit in src/hydra-queue-runner/hydra-queue-runner.cc at line 487
  [3.2319]

  [3.2319]
  + if (i.second.runnable > 0)

  + nested2.attr("waitTime", i.second.waitTime.count() +

  + i.second.runnable * (time(0) - lastDispatcherCheck));
• replacement in src/hydra-queue-runner/hydra-queue-runner.cc at line 491
  [3.2362]→[3.2362:2431](∅→∅)
  − nested2.attr("lastActive", i.second.lastActive);
  [3.2362]

  [3.2431]
  + nested2.attr("lastActive", std::chrono::system_clock::to_time_t(i.second.lastActive));
• edit in src/hydra-queue-runner/state.hh at line 178
  [3.2893]

  [3.3082]
  +

  + /* The time at which this step became runnable. */

  + system_time runnableSince;
• replacement in src/hydra-queue-runner/state.hh at line 330
  [3.2640]→[3.2640:2670](∅→∅)
  − time_t lastActive{0};
  [3.2640]

  [3.2670]
  + system_time lastActive;

  + std::chrono::seconds waitTime; // time runnable steps have been waiting
• edit in src/hydra-queue-runner/state.hh at line 345
  [3.3027]

  [3.6777]
  +

  + std::atomic<time_t> lastDispatcherCheck{0};