The sound distributed version control system

#425 [CLONE] cant clone linux3x

Closed on July 7, 2021
tankf33der on May 9, 2021

Recloned-rebuilt latest pijul to support tags.

Now cant clone linux3x repo over ssh and https.

$ pi clone tankf33der@nest.pijul.com:tankf33der/linux3x
Downloading changes [==================================================] 31/31 
Downloading changes [==================================================] 31/31                                           
           Applying [==================================================] 31/31                                           
 Completing changes [============================================>     ] 8/9                                             
$ pi clone https://nest.pijul.com/tankf33der/linux3x linnnn
Downloading changes [======================================>           ] 24/31                                           
           Applying [======================================>           ] 24/31                                           
Downloading changes [======================================>           ] 24/31                                           
           Applying [======================================>           ] 24/31                                           
Downloading changes [==================================================] 31/31                                           
           Applying [========================================>         ] 25/31                                           
Error: Changestore error: ZSTD("Unknown frame descriptor")
$
$ pi clone https://nest.pijul.com/tankf33der/linux3x linnnn
Downloading changes [======================================>           ] 24/31 
           Applying [======>                                           ] 4/31  
[2021-05-09T06:09:04Z ERROR pijul::remote::http] Error while downloading "https://nest.pijul.com/tankf33der/linux3x/.pijul", retrying
Downloading changes [=========================================>        ] 26/31 
           Applying [========================================>         ] 25/31 
Error: Changestore error: ZSTD("Unknown frame descriptor")
$
tankf33der on May 11, 2021

Over ssh it hang and do nothing on “Completing changes”.

Now i got repo copy over https and at least can run tests.

pmeunier on July 5, 2021

This was due to a bug in my SSH library: SSH can re-exchange keys after a number of bytes have been exchanged, in order to prevent dictionary or replay attacks. This used to work in Thrussh, but I introduced a bug in the latest update to async/await, where packets sent “during” a key exchange could end up in a queue of pending packets forever.

This is fixed in Thrussh 0.33.2.

pmeunier closed this discussion on July 5, 2021
tankf33der on July 5, 2021

over https:

$ pijul clone https://nest.pijul.com/tankf33der/linux3x
Downloading changes [>                                                 ] 0/31
           Applying [>                                                 ] 0/31
[2021-07-05T09:53:22Z ERROR pijul::remote::http] Error while downloading "https://nest.pijul.com/tankf33der/linux3x/.pijul", retrying
[2021-07-05T09:53:22Z ERROR pijul::remote::http] Error while downloading "https://nest.pijul.com/tankf33der/linux3x/.pijul", retrying
Downloading changes [>                                                 ] 0/31                                     
           Applying [>                                                 ] 0/31                                     
Downloading changes [>                                                 ] 0/31                                     
Downloading changes [>                                                 ] 0/31                                     
           Applying [>                                                 ] 0/31                                     
Downloading changes [>                                                 ] 0/31                                     
Downloading changes [>                                                 ] 0/31                                     
           Applying [>                                                 ] 0/31                                     
Downloading changes [>                                                 ] 0/31                                     
Downloading changes [>                                                 ] 0/31                                     
           Applying [>                                                 ] 0/31                                     
Downloading changes [>                                                 ] 0/31                                     
Downloading changes [>                                                 ] 0/31                                     
           Applying [>                                                 ] 0/31                                     
tankf33der reopened this discussion on July 5, 2021
tankf33der on July 5, 2021

over ssh i seen this picture before, i will wait as long as I can. It hanged and do nothing, laptop 100% idle.

pijul clone tankf33der@ssh.pijul.com:tankf33der/linux3x
Downloading changes [==================================================] 31/31                                                      
Downloading changes [==================================================] 31/31                                                      
           Applying [==================================================] 31/31                                                      
 Completing changes [============================================>     ] 8/9                                                        
pmeunier on July 5, 2021

I see what’s going on, at least over HTTP: I used to do this by chunks, but for some reason, Tokio refused to send some of my files this way, so I started reading the entire file. And apparently, this is saturating the memory. I’ll think about it.

Any news from your SSH attempt?

tankf33der on July 5, 2021

It hang, i stopped it. I cant clone since May and tried hard.

pmeunier on July 5, 2021

Alright, it hangs on my side too, I just tried. I just realised I hadn’t updated the SSH version in the deployed Nest, I’ll redeploy now (and then sleep ;-)).

tankf33der on July 6, 2021

The same behavior for ssh and https. Nothing changed. Even tried disable firewall :).

pmeunier on July 6, 2021

SSH fixed, I just tested. Working on HTTPS now.

tankf33der on July 7, 2021

Confirmed - over SSH works and running linux3x on vps now.

pmeunier on July 7, 2021

Finally nailed it for HTTPS: the connections have a timeout in order to protect the servers from attackers. But since there are more servers now, and they’re behind a proxy, we can relax the timeout a little bit, which I’ve done. I can now clone your large patches.

pmeunier closed this discussion on July 7, 2021
tankf33der on July 7, 2021

Confirmed clone over ssh and https works now.