grahamc/hydra - Change YNO7CQ6P56YQP7UMGQFL4AZED7DBXE32U7TI6OMOKT6GMWZDMWWQC

hydra-queue-runner: More accurate memory accounting

We now take into account the memory necessary for compressing the NAR being exported to the binary cache, plus xz compression overhead.

Also, we now release the memory tokens for the NAR accessor after releasing the NAR accessor. Previously the memory for the NAR accessor might still be in use while another thread does an allocation, causing the maximum to be exceeded temporarily.

Also, use notify_all instead of notify_one to wake up memory token waiters. This is not very nice, but not every waiter is requesting the same number of tokens, so some might be able to proceed.

Created by Eelco Dolstra on November 16, 2016

YNO7CQ6P56YQP7UMGQFL4AZED7DBXE32U7TI6OMOKT6GMWZDMWWQC

Dependencies

In channels

main

upstream

Change contents

Replacement in src/hydra-queue-runner/build-remote.cc at line 376 [4.85]

B:BD[3.8141] → [3.8141:8244]

               available. FIXME: only need this for binary cache
               destination stores. */

[3.8141]

[3.8244]

               available, which is twice the NAR size (namely the
               uncompressed and worst-case compressed NAR), plus 150
               MB for xz compression overhead. (The xz manpage claims
               ~94 MiB, but that's not was I'm seeing.) */

Replacement in src/hydra-queue-runner/build-remote.cc at line 381 [4.85]

B:BD[3.8306] → [3.8306:8374]

            auto memoryReservation(memoryTokens.get(totalNarSize));

[3.8306]

[3.8374]

            size_t compressionCost = totalNarSize + 150 * 1024 * 1024;
            result.tokens = std::make_unique<nix::TokenServer::Token>(memoryTokens.get(totalNarSize + compressionCost));

Insertion in src/hydra-queue-runner/build-remote.cc at line 396 [4.85]

[5.232]

[3.8937]

            /* Release the tokens pertaining to NAR
               compression. After this we only have the uncompressed
               NAR in memory. */
            result.tokens->give_back(compressionCost);

Insertion in src/hydra-queue-runner/builder.cc at line 182 [6.133]
[5.441]
[6.4678]
```
        result.accessor = 0;
        result.tokens = 0;
```
Insertion in src/hydra-queue-runner/state.hh at line 53 [8.1128]
[7.1810]
[5.442]
```
    std::unique_ptr<nix::TokenServer::Token> tokens;
```
Insertion in src/hydra-queue-runner/token-server.hh at line 43 [10.1640]
[2.112]
[9.2573]
```
            debug("acquiring %d tokens", tokens);
```

Replacement in src/hydra-queue-runner/token-server.hh at line 58 [10.1640]

B:BD[10.2807] → [10.2807:2858]

        Token(Token && t) : ts(t.ts) { t.ts = 0; }

[10.2807]

[10.2858]

        Token(Token && t) : ts(t.ts), tokens(t.tokens), acquired(t.acquired)
        {
            t.ts = 0;
            t.acquired = false;
        }

Insertion in src/hydra-queue-runner/token-server.hh at line 68 [10.1640]

[10.2969]

            give_back(tokens);
        }
        bool operator ()() { return acquired; }
        void give_back(size_t t)
        {
            debug("returning %d tokens", t);
            if (!t) return;
            assert(acquired);
            assert(t <= tokens);

Replacement in src/hydra-queue-runner/token-server.hh at line 81 [10.1640]

B:BD[9.2954] → [9.2954:3030]

                assert(*inUse >= tokens);
                *inUse -= tokens;

[9.2954]

[10.3105]

                assert(*inUse >= t);
                *inUse -= t;
                tokens -= t;

Replacement in src/hydra-queue-runner/token-server.hh at line 85 [10.1640]

B:BD[10.3119] → [10.3119:3156]

            ts->wakeup.notify_one();

[10.3119]

[10.3156]

            // FIXME: inefficient. Should wake up waiters that can
            // proceed now.
            ts->wakeup.notify_all();

Deletion in src/hydra-queue-runner/token-server.hh at line 90 [10.1640]
B:BD[10.3167] → [10.3167:3215]
```
        bool operator ()() { return acquired; }
```