The sound distributed version control system

#925 [PERFORMANCE] are diff and-or record too heavy?

Opened by tankf33der on April 4, 2024
tankf33der on April 4, 2024

Lets insert the whole installed FreeBSD14 distribution and diff or record. They will eat all memory.

Can you make them lighter ?

cd /
piju init 
pijul add -r .
# FAST
pijul ls | wc -l
187017
### any of these command would be killed by OOM:
# pijul record -am.
# pijul diff
# pijul diff -s
# pijul diff -su
tankf33der on April 4, 2024

FreeBSD14 host.

Linux kernel 5.10.214 on 71k sources file recorded fast on 2GB RAM machine.

Kernel 6.6.24 on 87k files recorded without swap.

Kernel 6.8.3 swapped on 83k files. https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.8.3.tar.xz

tankf33der on April 4, 2024

Alpine Linux 3.19 host:

Kernel 6.8.3 recorded fast but swapped several seconds.

tankf33der on April 4, 2024

Alpine Linux can not record whole distribution on 276k files on 2GB RAM. Killed by OOM.

tankf33der on April 4, 2024

I can imagine why pijul needs huge (?) memory for recording, but why it needs the same amount of RAM for diff -su ?

Easier way how to get huge (5+GB) memory run:

cd
mkdir p1
cd p1
cp -Rp ../pijul .
rm pijul/.ignore
pijul add -r .
pijul ls | wc -l
15753

### 
# pijul rec -am.
# pijul diff 
# pijul diff -su
tankf33der on April 4, 2024

If i checked correctly where is no memory leak.

tankf33der on April 5, 2024

I imagine how big user with big mono repo can not work.