Update to latest nixUnstable

[?]
Sep 14, 2017, 3:22 PM
LSUX6IQR7TBTTCDQAVD3KDAVARLM7SUD7PPVTR5IOKA3BNZDYA7AC

Dependencies

  • [2] BYVRA54Q Temporarily disable machines on any exception, not just connection failures
  • [3] JPHDKOMJ hydra-queue-runner: Keep some notification statistics
  • [4] IE2PRAQU hydra-queue-runner: Send build notifications
  • [5] IKJBYIGY Update to reflect BinaryCacheStore changes
  • [6] N4IROACV Move buildRemote() into State
  • [7] FJCKDJKB Acquire the send lock only while actually sending
  • [8] BRAESISH Warn if PostgreSQL appears stalled
  • [9] SL3WSRAC hydra-queue-runner: Limit memory usage
  • [10] V6H6BWMK Sync with Nix
  • [11] N2NKSKHS Refactor local binary cache code into a subclass
  • [12] B2L4T3X6 Sync with Nix
  • [13] 4YCF3KBG Concurrent hydra-evaluator
  • [14] UYDRSHYR Don't wait forever to acquire the send lock
  • [15] YZAI5GQU Implement a database connection pool
  • [16] HJOEIMLR Refactor
  • [17] BG6PEOB2 Make the output size limit configurable
  • [18] 73YR46NJ hydra-queue-runner: Write directly to a binary cache
  • [19] MB3TISH2 Rate-limit the number of threads copying closures at the same time
  • [20] 5AIYUMTB Basic remote building
  • [21] 3YSJ3LYK Remove finally.hh
  • [22] EYR3EW6J Keep stats for the Hydra auto scaler
  • [23] GH4S4AWM Rename file
  • [24] DIEY5USN Keep better bytesReceived/bytesSent stats
  • [25] A4A5KO5Q Add error/warnings for deprecated store specification
  • [26] A2GL5FOZ Moar stats
  • [27] 24BMQDZA Start of single-process hydra-queue-runner
  • [28] 7LB6QBXY Keep track of the number of build steps that are being built
  • [29] GTUZLZRH Add an S3-backed binary cache store
  • [30] FITVNQ2S Keep track of the time we spend copying to/from build machines
  • [31] XLYHZUHT Cache .narinfo lookups
  • [32] NAYQT2GT hydra-queue-runner: Use cmdBuildDerivation
  • [33] MS676RZW Remove s3binarystore (moved to nix in d155d80)
  • [34] ENXUSMSV Make concurrency more robust
  • [35] MHVIT4JY Split hydra-queue-runner.cc more

Change contents

  • file deletion: counter.hh (----------)
    [4.187][4.433:467](),[4.467][4.234:234]()
    #pragma once
    #include <atomic>
    typedef std::atomic<unsigned long> counter;
    struct MaintainCount
    {
    counter & c;
    MaintainCount(counter & c) : c(c) { c++; }
    ~MaintainCount() { auto prev = c--; assert(prev); }
    };
    MaintainCount(counter & c, std::function<void(unsigned long)> warn) : c(c)
    {
    warn(++c);
    }
    #include <functional>
  • replacement in src/hydra-queue-runner/build-remote.cc at line 234
    [2.2639][2.2639:2711]()
    auto mc1 = std::make_shared<MaintainCount>(nrStepsWaiting);
    [2.2639]
    [2.2787]
    auto mc1 = std::make_shared<MaintainCount<counter>>(nrStepsWaiting);
  • replacement in src/hydra-queue-runner/build-remote.cc at line 236
    [2.2812][2.2812:2861]()
    MaintainCount mc2(nrStepsCopyingTo);
    [2.2812]
    [2.2861]
    MaintainCount<counter> mc2(nrStepsCopyingTo);
  • replacement in src/hydra-queue-runner/build-remote.cc at line 279
    [2.3849][2.3849:3896]()
    MaintainCount mc(nrStepsBuilding);
    [2.3849]
    [2.3896]
    MaintainCount<counter> mc(nrStepsBuilding);
  • replacement in src/hydra-queue-runner/build-remote.cc at line 374
    [2.7058][2.7058:7108]()
    MaintainCount mc(nrStepsCopyingFrom);
    [2.7058]
    [4.4791]
    MaintainCount<counter> mc(nrStepsCopyingFrom);
  • replacement in src/hydra-queue-runner/hydra-queue-runner.cc at line 108
    [4.308][4.308:345]()
    MaintainCount State::startDbUpdate()
    [4.308]
    [4.345]
    nix::MaintainCount<counter> State::startDbUpdate()
  • replacement in src/hydra-queue-runner/hydra-queue-runner.cc at line 110
    [4.347][4.347:566]()
    return MaintainCount(nrActiveDbUpdates, [](unsigned long c) {
    if (c > 6) {
    printMsg(lvlError, format("warning: %d concurrent database updates; PostgreSQL may be stalled") % c);
    }
    });
    [4.347]
    [4.1621]
    if (nrActiveDbUpdates > 6)
    printError("warning: %d concurrent database updates; PostgreSQL may be stalled", nrActiveDbUpdates.load());
    return MaintainCount<counter>(nrActiveDbUpdates);
  • replacement in src/hydra-queue-runner/hydra-queue-runner.cc at line 486
    [3.1][3.1:58]()
    MaintainCount mc(nrNotificationsInProgress);
    [3.1]
    [4.1801]
    MaintainCount<counter> mc(nrNotificationsInProgress);
  • edit in src/hydra-queue-runner/state.hh at line 11
    [4.1277][4.1277:1299]()
    #include "counter.hh"
  • edit in src/hydra-queue-runner/state.hh at line 24
    [4.1567]
    [4.1567]
    typedef std::atomic<unsigned long> counter;
  • edit in src/hydra-queue-runner/state.hh at line 26
    [4.1568]
    [4.1568]
  • replacement in src/hydra-queue-runner/state.hh at line 443
    [4.13176][4.911:946]()
    MaintainCount startDbUpdate();
    [4.13176]
    [4.13314]
    nix::MaintainCount<counter> startDbUpdate();