On current main (QBTRVXILORX5) with patch from #544:

❯ touch A; git add A; git commit -m "Add A"
[master (root-commit) 5dc0f12] Add A
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 A

❯ git rm A
rm 'A'

❯ git commit -m "Remove A"
[master 0b7556a] Remove A
 1 file changed, 0 insertions(+), 0 deletions(-)
 delete mode 100644 A

❯ git log
commit 0b7556aaf673bad51aa8f687510ebf11ae28f52e (HEAD -> master)
Author: Henrik Tjäder <henrik@tjaders.com>
Date:   Fri Oct 8 23:35:09 2021 +0200

    Remove A

commit 5dc0f1201078201871215619edffaee9aa58516d
Author: Henrik Tjäder <henrik@tjaders.com>
Date:   Fri Oct 8 23:34:50 2021 +0200

    Add A

❯ pijul git
INFO Loading Git history…
INFO Importing commit 5dc0f1201078201871215619edffaee9aa58516d: Add A

INFO Importing commit 0b7556aaf673bad51aa8f687510ebf11ae28f52e: Remove A

❯ pijul log
Change SZRTTHY4VSRH73KRO3MHVCAUBT7GDB6WOQOF5MJDQAIJJV5DZEPQC
Author: Henrik Tjäder
Date: 2021-10-08 21:34:50 UTC

    Add A

<MISSING>
Change xxxxx

  Remove A
<MISSING>

❯ ls -l

Expected output would be two changes, one adding and one removing A.

With debug printouts:

❯ env RUST_LOG=info,libpijul::record=warn,libpijul::git=debug,libpijul::output=off,sanakirja=off pijul git
INFO Loading Git history…
INFO Importing commit 5dc0f1201078201871215619edffaee9aa58516d: Add A

[2021-10-08T22:01:06Z INFO libpijul::apply] repairing missing contexts
[2021-10-08T22:01:06Z INFO libpijul::apply] done applying change
[2021-10-08T22:01:06Z INFO libpijul::apply] updating Add { pos: ChangePosition(L64(1)), inode: Inode(4UB5Q4D7PJPA4) }
INFO Importing commit 0b7556aaf673bad51aa8f687510ebf11ae28f52e: Remove A

Commenting out line 752 of pijul/src/commands/git.rs:

            //Err(Error::Add(AddError::Io(e))) if e.kind() == std::io::ErrorKind::NotFound => {}

yields this error:

❯ rm -Rf .pijul

❯ env RUST_LOG=info,libpijul::record=warn,libpijul::git=debug,libpijul::output=off,sanakirja=off pijul git
INFO Loading Git history…
INFO Importing commit 5dc0f1201078201871215619edffaee9aa58516d: Add A

[2021-10-08T22:06:41Z INFO libpijul::apply] repairing missing contexts
[2021-10-08T22:06:41Z INFO libpijul::apply] done applying change
[2021-10-08T22:06:41Z INFO libpijul::apply] updating Add { pos: ChangePosition(L64(1)), inode: Inode(4UB5Q4D7PJPA4) }
INFO Importing commit 0b7556aaf673bad51aa8f687510ebf11ae28f52e: Remove A

ERROR While adding "A": Add(Io(Os { code: 2, kind: NotFound, message: "No such file or directory" }))

Which makes sense, the file does not exist and it is not possible to read it and add it.

Curiously, by removing the file altogether from the prefixes BTreeSet:

                    git2::Delta::Deleted => {
                        //let old_path = old_path.to_path_buf();
                        //prefixes.insert(old_path);
                    }

lets Pijul pick up the file deletion:

❯ rm -Rf .pijul

❯ env RUST_LOG=info,libpijul::record=warn,libpijul::git=debug,libpijul::output=off,sanakirja=off pijul git
INFO Loading Git history…
INFO Importing commit 5dc0f1201078201871215619edffaee9aa58516d: Add A

[2021-10-08T22:11:42Z INFO libpijul::apply] repairing missing contexts
[2021-10-08T22:11:42Z INFO libpijul::apply] done applying change
[2021-10-08T22:11:42Z INFO libpijul::apply] updating Add { pos: ChangePosition(L64(1)), inode: Inode(4UB5Q4D7PJPA4) }
INFO Importing commit 0b7556aaf673bad51aa8f687510ebf11ae28f52e: Remove A

[2021-10-08T22:11:42Z INFO libpijul::apply] repairing missing contexts
[2021-10-08T22:11:42Z INFO libpijul::apply] done applying change
[2021-10-08T22:11:42Z INFO libpijul::apply] updating Deleted { inode: Inode(4UB5Q4D7PJPA4) }

❯ pijul log
Change 5EHCCC5WLAYLTMLFAXFAIUOZFAGNL4SYGJQLUZ6PPV3ZNAS675IAC
Author: Henrik Tjäder
Date: 2021-10-08 21:35:09 UTC

    Remove A

Change SZRTTHY4VSRH73KRO3MHVCAUBT7GDB6WOQOF5MJDQAIJJV5DZEPQC
Author: Henrik Tjäder
Date: 2021-10-08 21:34:50 UTC

    Add A

Which is the expected output. However, I am not sure if this is a valid assumption. In order to follow the general pattern done for added files I separated deleted files from prefixes into prefixes_deleted which then uses the explicit remove_file() instead.

With this change:

❯ rm -Rf .pijul/

❯ env RUST_LOG=info,libpijul::record=warn,libpijul::git=debug,libpijul::output=off,sanakirja=off pijul git
INFO Loading Git history…
INFO Importing commit 5dc0f1201078201871215619edffaee9aa58516d: Add A

[2021-10-08T22:24:32Z INFO libpijul::apply] repairing missing contexts
[2021-10-08T22:24:32Z INFO libpijul::apply] done applying change
[2021-10-08T22:24:32Z INFO libpijul::apply] updating Add { pos: ChangePosition(L64(1)), inode: Inode(4UB5Q4D7PJPA4) }
INFO Importing commit 0b7556aaf673bad51aa8f687510ebf11ae28f52e: Remove A

[2021-10-08T22:24:32Z INFO libpijul::apply] repairing missing contexts
[2021-10-08T22:24:32Z INFO libpijul::apply] done applying change
[2021-10-08T22:24:32Z INFO libpijul::apply] updating Deleted { inode: Inode(4UB5Q4D7PJPA4) }

❯ pijul log
Change 5EHCCC5WLAYLTMLFAXFAIUOZFAGNL4SYGJQLUZ6PPV3ZNAS675IAC
Author: Henrik Tjäder
Date: 2021-10-08 21:35:09 UTC

    Remove A

Change SZRTTHY4VSRH73KRO3MHVCAUBT7GDB6WOQOF5MJDQAIJJV5DZEPQC
Author: Henrik Tjäder
Date: 2021-10-08 21:34:50 UTC

    Add A

I’ll add both changes to the discussion.

Does this explicit remove_file() makes sense at all?

Hmm, there’s still something odd

Cloning this repo: git@github.com:rtic-rs/cortex-m-rtic.git and then running pijul git succeeds but leaves the following:

❯ git status
On branch master
Your branch is up to date with 'origin/master'.

Untracked files:
  (use "git add <file>..." to include in what will be committed)
        .ignore
        .pijul/
        ci/after-success.sh
        ci/install.sh
        ci/script.sh
        examples/t-init-main.rs
        examples/t-resource.rs
        examples/t-stask-main.rs
        examples/task_named_main.rs
        examples/type-usage.rs
        macros/src/codegen/spawn_body.rs
        tests/multi.rs

nothing added to commit but untracked files present (use "git add" to track)

Looking at which commit deleted macros/src/codegen/spawn_body.rs

❯ git show 524273c96a978299b64e51a9cdcc007585a0f170 --stat
commit 524273c96a978299b64e51a9cdcc007585a0f170
Author: Emil Fresk <emil.fresk@...>
Date:   Sun Oct 11 18:38:38 2020 +0200

    Now with spawn/schedule from anywhere

 examples/schedule.rs                 |   6 +++---
 examples/spawn.rs                    |   4 +++-
 examples/spawn2.rs                   |  10 ++++++----
 macros/Cargo.toml                    |   2 +-
 macros/src/analyze.rs                |   1 -
 macros/src/check.rs                  |  62 +++++++++++++++++++++++++++-----------------------------------
 macros/src/codegen.rs                |  16 ++++++++--------
 macros/src/codegen/dispatchers.rs    |  20 +-------------------
 macros/src/codegen/hardware_tasks.rs |   2 +-
 macros/src/codegen/module.rs         | 212 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--------------------------------------------------------------------------------------------------------------------------------------------------------
 macros/src/codegen/pre_init.rs       |   8 ++++----
 macros/src/codegen/schedule.rs       |  90 ------------------------------------------------------------------------------------------
 macros/src/codegen/schedule_body.rs  |  59 -----------------------------------------------------------
 macros/src/codegen/software_tasks.rs |  89 ++++++++++++++++++++++++++++++++++-------------------------------------------------------
 macros/src/codegen/spawn.rs          | 121 -------------------------------------------------------------------------------------------------------------------------
 macros/src/codegen/spawn_body.rs     |  76 ----------------------------------------------------------------------------
 macros/src/codegen/timer_queue.rs    |  69 ++++++++++++++++++++++-----------------------------------------------
 macros/src/codegen/util.rs           |  16 +---------------
 macros/src/lib.rs                    |   1 -
 19 files changed, 171 insertions(+), 693 deletions(-)

Any insight/pointers welcome :)

First, thanks for the report and the attempts! I started trying things out, and the problem was much simpler than I expected: libpijul::working_copy::filesystem::get_prefix was returning an error when the prefix wasn’t found.

We do want deleted files to be in the prefixes, in case a commit deletes something at the root of a huge repo.

I tested that hypothesis, which fixed that bug immediately.