Updates for negative .narinfo caching

Dependencies

• [2] RX6UB7YW Better AWS error messages
• [3] DXFMYCNW Fix build
• [4] W4W4ZTNV Some more logging
• [5] SOB276BA Keep some statistics for the binary cache stores
• [6] ZM34T2NW Typo
• [7] QSBS2ISO S3BinaryCacheStore::isValidPath(): Do a GET instead of HEAD
• [8] GTUZLZRH Add an S3-backed binary cache store
• [9] QDEGRYZV Reduce severity level of some message
• [*] 24BMQDZA Start of single-process hydra-queue-runner
• [*] XLYHZUHT Cache .narinfo lookups

Change contents

• edit in src/hydra-queue-runner/hydra-queue-runner.cc at line 642
  [12.917]

  [3.2053]
  + nested.attr("narInfoMissing", stats.narInfoMissing);
• replacement in src/hydra-queue-runner/s3-binary-cache-store.cc at line 90
  [3.743]→[3.743:853](∅→∅)
  − } catch (S3Error & e) {

  − if (e.err == Aws::S3::S3Errors::NO_SUCH_KEY) return false;

  − throw;
  [3.743]

  [3.853]
  + } catch (InvalidPath & e) {

  + return false;
• replacement in src/hydra-queue-runner/s3-binary-cache-store.cc at line 144
  [3.4695]→[3.4695:4761](∅→∅)
  − std::string S3BinaryCacheStore::getFile(const std::string & path)
  [3.4695]

  [3.4761]
  + std::shared_ptr<std::string> S3BinaryCacheStore::getFile(const std::string & path)
• replacement in src/hydra-queue-runner/s3-binary-cache-store.cc at line 159
  [3.5268]→[3.4996:5046](∅→∅),[3.4996]→[3.4996:5046](∅→∅)
  − auto now1 = std::chrono::steady_clock::now();
  [3.5268]

  [3.5046]
  + try {
• replacement in src/hydra-queue-runner/s3-binary-cache-store.cc at line 161
  [3.5047]→[2.903:1013](∅→∅)
  − auto result = checkAws(format("AWS error fetching ‘%s’") % path,

  − client->GetObject(request));
  [3.5047]

  [3.5103]
  + auto now1 = std::chrono::steady_clock::now();
• replacement in src/hydra-queue-runner/s3-binary-cache-store.cc at line 163
  [3.5104]→[3.5104:5154](∅→∅)
  − auto now2 = std::chrono::steady_clock::now();
  [3.5104]

  [3.5154]
  + auto result = checkAws(format("AWS error fetching ‘%s’") % path,

  + client->GetObject(request));
• replacement in src/hydra-queue-runner/s3-binary-cache-store.cc at line 166
  [3.5155]→[3.5155:5229](∅→∅)
  − auto res = dynamic_cast<std::stringstream &>(result.GetBody()).str();
  [3.5155]

  [3.5269]
  + auto now2 = std::chrono::steady_clock::now();
• replacement in src/hydra-queue-runner/s3-binary-cache-store.cc at line 168
  [3.5270]→[3.5270:5366](∅→∅)
  − auto duration = std::chrono::duration_cast<std::chrono::milliseconds>(now2 - now1).count();
  [3.5270]

  [3.5229]
  + auto res = dynamic_cast<std::stringstream &>(result.GetBody()).str();
• replacement in src/hydra-queue-runner/s3-binary-cache-store.cc at line 170
  [3.5230]→[2.1014:1103](∅→∅),[3.167]→[3.5367:5421](∅→∅),[2.1103]→[3.5367:5421](∅→∅),[3.85]→[3.5367:5421](∅→∅)
  − printMsg(lvlTalkative, format("downloaded ‘s3://%1%/%2%’ (%3% bytes) in %4% ms")

  − % bucketName % path % res.size() % duration);
  [3.5230]

  [3.5421]
  + auto duration = std::chrono::duration_cast<std::chrono::milliseconds>(now2 - now1).count();
• replacement in src/hydra-queue-runner/s3-binary-cache-store.cc at line 172
  [3.5422]→[3.5422:5489](∅→∅)
  − stats.getBytes += res.size();

  − stats.getTimeMs += duration;
  [3.5422]

  [3.5445]
  + printMsg(lvlTalkative, format("downloaded ‘s3://%1%/%2%’ (%3% bytes) in %4% ms")

  + % bucketName % path % res.size() % duration);
• replacement in src/hydra-queue-runner/s3-binary-cache-store.cc at line 175
  [3.5446]→[3.5446:5462](∅→∅)
  − return res;
  [3.5446]

  [3.5462]
  + stats.getBytes += res.size();

  + stats.getTimeMs += duration;

  +

  + return std::make_shared<std::string>(res);

  +

  + } catch (S3Error & e) {

  + if (e.err == Aws::S3::S3Errors::NO_SUCH_KEY) return 0;

  + throw;

  + }
• replacement in src/hydra-queue-runner/s3-binary-cache-store.hh at line 55
  [3.6288]→[3.6288:6348](∅→∅)
  − std::string getFile(const std::string & path) override;
  [3.6288]

  [3.6348]
  + std::shared_ptr<std::string> getFile(const std::string & path) override;