I have been making quite a bit of progress on reducing the time taken to run cargo build. The changes so far have been focused on 2 main areas:

1. Removing unnecessary dependencies
   • Some crates have overlapping functionality
   • The default feature list can include unused features
2. Splitting the pijul crate into smaller sub-crates
   • As the crate is Rust’s unit of compilation, this should help increase rate of iteration through faster build times
   • This also allows for functionality to be used by other crates in the ecosystem (eg @laumann ’s pijul-conflicts)

A future area to improve, as mentioned by @pmeunier on this Zulip thread, would be to feature-gate features such as ssh (similar to how git is already feature-gated). I would also like to investigate using a faster linker (preferably mold) and even cranelift as a codegen backed for debug builds.

For now my current plan is to keep writing patches that focus on splitting the project down into smaller crates where appropriate. If there is anything I can do to help make the review process easier, please let me know.

Nice to get some unnecessary dependencies removed.

Do you have any numbers for the compilation time before and after these changes?

As for the crate splitting, the only thing pijul-conflicts would really need from the pijul crate is the Repository struct, and maybe a few other odds and ends. I wouldn’t necessarily explode the pijul crate into many small crates, instead I’d probably have a libpijul-cli (or something in that vein) that indicates it’s useful utilities for creating CLI extensions that hook into pijul and libpijul.

Excellent work! Thanks a lot @finchie.

As described in #807, this appears to have caused a bug in change recording.

Yes, and it also triggered two other bugs:

• When unrecording them on my machine, I noticed a conflict had appeared. This was due to a bug in unrecord, now fixed by #XNY6VDZSCNHSAS53HRVBT5AO7HBMAWPOQX3ORCYPWVWRKA4EXMFQC. The bug was in a part that isn’t tested very often, which is when we un-delete a vertex. Currently only unrecord does this, and most projects obviously unrecord much less than they apply.

• The Nest deleted the patches when I deleted them from main. This is a serious bug, but since I’m rewriting the Nest I don’t think I will fix it.

I am very sorry to have caused these issues, looking at this with a fresh pair of eyes I realized that it was not at all obvious that I didn’t intend for these patches to be merged so soon. I’ll get back to you once this is resolved.

These patches have been completely re-written, with a more direct focus on splitting the repository into logical parts instead of improving compile times. That being said, these changes will probably help with such a goal. I’ve listed all the crates that these new patches add to the workspace below, and of course if there’s anything I can do to help or improve these patches please let me know.

pijul-interaction

This is a completely new crate, with a focus on centralizing methods of interacting with the user. This includes functionality such as:

• Indicators:
  • Progress bars
  • Spinners
• Prompts
  • Selection
  • Password
  • Text
  • Confirmation

The crate currently supports 2 contexts for interactivity, InteractiveContext::NotInteractive and InteractiveContext::Terminal. Terminal interactivity is handled by indicatif and dialoguer, and the non-interactive side stores all interaction state and returns an erorr if user input is required. In the future this crate could be extended to handle other contexts (such as web browsers or IDEs), and include proper localization support. It should also be possible to add a testing backend that makes it possible to write expressive tests for how we interact with the user.

I suspect that the NotInteractive context will need a little fine-tuning before it is production-ready, but I think the best way to solve this would be comprehensive testing of the command-line interface. I think Rust’s CLI testing ecosystem still needs some maturation and perhaps this will motivate me to drive it slightly forward :)

pijul-config

This is the old pijul::config module refactored into its own crate. Not much has changed here, but it probably needs a design review at some point as it seems to have grown mostly organically over time.

pijul-repository

To me, this is the most important crate to come out of this refactoring. Most people looking to use pijul programmatically will probably want this level of abstraction, rather than libpijul itself. As mentioned by @stellarpower on Zulip, this would probably need a lot of documentation and polish so users can interact with the library without knowing too much of the internals.

pijul-identity

Not much to say here, there are definitely known areas for improvement, mostly around syncing identities. Mostly unchanged.

pijul-remote

Same here, it has inherited the prove functionality that previously was in pijul::identity, as this created a circular dependency.

I’ve updated the title once again, from “[DRAFT]: Improve compile times” to “Split Pijul CLI into multiple sub-crates”. Not much has changed from the user’s perspective but should make things a bit cleaner under the hood. I am not aware of any regressions introduced but there probably are some, please ping me on Zulip if there’s anything I can fix!

I finally found the time to apply these, sorry for the delay. One minor issue: I like how the time of pijul pull for example is shown, however they wrap to the next line on my terminal, is this intended?