I’ve been maintaining a pipeline to build pijul against musl libc daily (for alpine and glibc-less distros) for some time now, and I’ve noticed that Pijul builds stopped working on May 5, and it looks like it was in a change between ZJWCPRMH and NAHGUOC2, and you can see in the logs the hash of the change of the previous successful build and of the unsuccessful build, and the error that happened here. Also, it still happening after the last change, as you can see in the last failed pipeline, although I have some caching enabled, the repository contents are not cached (the pipeline is mostly stateless).

I’ve tried to find this sequence at line 811 in the libpijul/record.rs, but there is nothing there, so I suspected that NEST was hiding it from me, since I never pull the repo in the pipeline, just cloned to a new directory. Then I decided to clone pijul locally and look at record.rs:

➜  pijul_repo $ cat libpijul/src/record.rs | rg -n '>>>>>>>'
811:>>>>>>> 0 [PDTUHOMV]

And the line was there, then I assumed that since there were some conflict problems in libpijul, because of the comment @pmeunier did in this discussion (but maybe I understood it wrong), the produced binary of previous successful pipeline could be faulty/buggy, so I fixed those lines and recompiled pijul and cloned the project again, but it was of no help, the line still there even with a fresh binary compiled with the latest change (O67AB66O).

I don’t know how that happened, I looked at all the changes from the last time it succeeded and I cannot find those lines through the NEST interface (so it is probably hiding it), but I’m sure this will cause builds from source to fail.

If you need any additional information, just tell me.

Have a nice day.

Sincerely, Jonathan

I can’t reproduce from a fresh clone, but if you want to email a tarball of your faulty repo to pe@pijul.org, that could be helpful.

I ran into this problem today, and I’ll email the tarball as requested.

Here were my steps to repro:

1. I’m running macOS 12.4. My cargo version is cargo 1.62.0 (a748cf5a3 2022-06-08)
2. brew install openssl@3 xxhash zstd
3. cargo install pijul --version "~1.0.0-beta"
4. pijul clone https://nest.pijul.com/pijul/pijul. That retrieved version O67AB66OLFEZMDX6YNPUN5TP4LLJYBCMXBTX62WSKVKUQFPDA32QC
5. Open libpijul/src/record.rs. At line 810, I got this:

            if end > start {
>>>>>>> 0 [PDTUHOMV]
<<<<<<< 0
                (

It seems to be triggered in a very specific environment, when I compile pijul with glibc and perform a clone, the file is correct, but when I compile against musl-libc and do a clone using this binary, it comes with a broken record.rs. I was guessing it had something to do with xxHash library, I tried to update it and still didn’t solved the problem.

As @evan-dickinson pointed out, compiling on macOS and then using the binary to clone pijul repository leads to the same problem, I will be sending the faulty repo shortly.

Edit: After hours of debugging (and getting gdb to work with musl compiled pijul), I’ve found that it doesn’t matter if it is musl libc or glibc, if you clone the pijul repository and build it, then run the produced pijul binary, the error will be there on record.rs. I’ve thinking that my glibc pijul was the latest, but it actually isn’t:

➜  ~  which pijul
/home/jonathan/.cargo/bin/pijul
➜  ~  pijul --version
pijul 1.0.0-beta

The error is not present on 1.0.0-beta, but is present somehow on the latest version.

Here is a Dockerfile to reproduce this in a completely clean environment:

FROM rust:1.62.0
RUN apt update && apt install -y libxxhash-dev libzstd-dev
RUN cargo search pijul
RUN cargo install pijul --version=1.0.0-beta.2
RUN pijul clone https://nest.pijul.com/pijul/pijul
RUN grep -r '>>>>>>> 0 \[PDTUHOMV\]' pijul

You will see this output before the last intermediate container is removed:

pijul/libpijul/src/record.rs:>>>>>>> 0 [PDTUHOMV]

Which means that the problem is there.

My debugging results showed that the code seems to be reaching those points:

gdb> br remote/mod.rs:1405
gdb> br alive/output.rs:219

Hi! This conflict is gone now, and I’ve improved the algorithm around conflicts recently. I’ll close for now, let’s see if that issue arises again.