Hi, I wanted to try pijul on a large-ish private repository I am working on. It contains 30k commits, 500 files on master, and the .git folder is 350 MB large.

After running pijul git for several minutes, when approx. 1/4 of the history is converted according to git status, I get the following error:

Error: Block error: Position { change: ChangeId(DCC73FQ7RAEWA), pos: ChangePosition(L64(174)) }

There is no other output.

Is there anything I can do to debug the issue, given the repository is not public? I can try to replicate on some public repositories.

Tried again on master. There is some console output now for pijul git, but otherwise the error is the same.

Hi @potocpav, you can get some more log information with environment variables.

(For eg in fish shell: env RUST_LOG="pijul=debug,libpijul=debug" pijul)

Thanks @fabian. This is the output:

[2021-08-12T13:05:32Z DEBUG libpijul::apply] Applying SolveOrderConflict { change: NewVertex(NewVertex { up_context: [Position { change: Some(DCC73FQ7RAEWBQST2EZIYYWGYQQOLGXJSYMA2ACJ6X3UZISGYTXAC), pos: ChangePosition(L64(174)) }], down_context: [], flag: (empty), start: ChangePosition(L64(271)), end: ChangePosition(L64(271)), inode: Position { change: Some(JGFTASFVPG4DPQFVIK2JBT7HXAYDRFXCJF2RFLKGA3IOZNWGHUVAC), pos: ChangePosition(L64(798)) } }), local: Local { path: "src/Administration/AdministrationControl.hs", line: 746 } } (1)
[2021-08-12T13:05:32Z DEBUG libpijul::pristine::sanakirja] find_block_end, loop, k = V(ChangeId(MUGIEC3PYRAGY)[0:63]), p = Position { change: ChangeId(SP6HYYHILE7GY), pos: ChangePosition(L64(271)) }
[2021-08-12T13:05:32Z DEBUG libpijul::pristine::sanakirja] find_block_end, loop, k = V(ChangeId(SP6HYYHILE7GY)[269:269]), p = Position { change: ChangeId(SP6HYYHILE7GY), pos: ChangePosition(L64(271)) }
[2021-08-12T13:05:32Z DEBUG libpijul::pristine::sanakirja] find_block_end, V(ChangeId(SP6HYYHILE7GY)[269:269]) Position { change: ChangeId(SP6HYYHILE7GY), pos: ChangePosition(L64(271)) }
[2021-08-12T13:05:32Z DEBUG libpijul::pristine::sanakirja] find_block_end, V(ChangeId(MUGIEC3PYRAGY)[0:63]) Position { change: ChangeId(SP6HYYHILE7GY), pos: ChangePosition(L64(271)) }
[2021-08-12T13:05:32Z DEBUG libpijul::pristine::sanakirja] find_block_end: BLOCK ERROR
[2021-08-12T13:05:32Z DEBUG libpijul::pristine::sanakirja] find_block: BLOCK ERROR
[2021-08-12T13:05:32Z DEBUG libpijul::apply::vertex] put_newvertex V(ChangeId(SP6HYYHILE7GY)[271:271]) [Position { change: Some(DCC73FQ7RAEWBQST2EZIYYWGYQQOLGXJSYMA2ACJ6X3UZISGYTXAC), pos: ChangePosition(L64(174)) }] [] (empty) ChangeId(SP6HYYHILE7GY)
[2021-08-12T13:05:32Z DEBUG libpijul::apply::vertex] put_up_context Position { change: ChangeId(DCC73FQ7RAEWA), pos: ChangePosition(L64(174)) }
[2021-08-12T13:05:32Z DEBUG libpijul::pristine::sanakirja] find_block_end, loop, k = V(ChangeId(DCC73FQ7RAEWA)[174:208]), p = Position { change: ChangeId(DCC73FQ7RAEWA), pos: ChangePosition(L64(174)) }
[2021-08-12T13:05:32Z DEBUG libpijul::pristine::sanakirja] find_block_end, loop, k = V(ChangeId(DCC73FQ7RAEWA)[35:173]), p = Position { change: ChangeId(DCC73FQ7RAEWA), pos: ChangePosition(L64(174)) }
[2021-08-12T13:05:32Z DEBUG libpijul::pristine::sanakirja] find_block_end, V(ChangeId(DCC73FQ7RAEWA)[35:173]) Position { change: ChangeId(DCC73FQ7RAEWA), pos: ChangePosition(L64(174)) }
[2021-08-12T13:05:32Z DEBUG libpijul::pristine::sanakirja] find_block_end, V(ChangeId(DCC73FQ7RAEWA)[174:208]) Position { change: ChangeId(DCC73FQ7RAEWA), pos: ChangePosition(L64(174)) }
[2021-08-12T13:05:32Z DEBUG libpijul::pristine::sanakirja] find_block_end: BLOCK ERROR
[2021-08-12T13:05:32Z DEBUG pijul] Block error: Position { change: ChangeId(DCC73FQ7RAEWA), pos: ChangePosition(L64(174)) }
Error: Block error: Position { change: ChangeId(DCC73FQ7RAEWA), pos: ChangePosition(L64(174)) }

There are many BLOCK ERROR messages sprinkled thoughout the log, but they don’t seem to be fatal (?)

I’m really interested in this issue. @potocpav, several things have changed both in Sanakirja and in Pijul since you reported this bug. Is this still happening?

@pmeunier No, it still doesn’t work but the issue is different. I will close this issue, and open another one if I can replicate it on a public repo.

I don’t know whether to consider that progress or not :(

Can you describe the issue?

I don’t know whether to consider that progress or not :(

Me neither. Now the conversion just hangs on this issue, which is pretty easy to reproduce.

Alright, now that the issue is fixed, I can’t wait to know what the next one is.

Quite some amazing improvements today! Got much further this time, but still failed after several thousand commits. Seems the failure is on a large merge commit (merging 1000 commits). Output:

(...)
INFO Importing commit 01968e468cb757c7851d514181fc0b58f54f6f5e: Merge remote branch 'origin/production' into staging

ERROR While adding "curl.exe": Record(PathNotInRepo("curl.exe"))
ERROR While adding "libcurl.dll": Record(PathNotInRepo("libcurl.dll"))
ERROR While adding "libeay32.dll": Record(PathNotInRepo("libeay32.dll"))
ERROR While adding "libssl32.dll": Record(PathNotInRepo("libssl32.dll"))
ERROR While adding "r.bat": Record(PathNotInRepo("r.bat"))
Error: Block error: Position { change: ChangeId(3MGHZJ6TNSZ3Y), pos: ChangePosition(L64(174)) }

I will try to replicate it on some public repository.

Are some of these files symbolic links? I’m a bit worried about the Block error, is 3MGH… a valid patch in the repo?

If you want, you won’t give me much information about your files by sending me the output of pijul debug after that error.

(send that to pe@pijul.org)

As a matter of fact, 3MGHZJ6TNSZ3Y is not a valid change. I also couldn’t find any of the mentioned paths (“curl.exe”, etc.) in the repo.

$ pj change 3MGHZJ6TNSZ3Y
Error: Change not found: 3MGHZJ6TNSZ3Y`

I sent the logfile via the e-mail.

Thanks for the log, I’ll check it out. I was actually asking for a different thing, what happens if you just run the following command?

pijul debug > graph

And send me the graph file?

Ah sorry, sent the correct thing now.

I spotted some bugs today while fixing the tests, I don’t think these cases would ever happen in real-world repositories, but who knows? Maybe they did in yours. If you care to try again, I’m interested.

I’m removing the beta tag. I’m expecting more people to try Pijul out after the beta release, so I believe they might catch the same bug on a public repo.