Retry HTTP downloads if we don't get a full patch

pmeunier

Dec 7, 2021, 10:35 AM

7Z3KZV6GY6IKLOWWRM6MVLOPAMP5MSJGQ3TVEZMMCSBVIP4NWRIQC

Dependencies

• [2] OIOMXESD Better error handling in HTTP
• [3] BVVMTOYW Proper renaming of changes downloaded over HTTP
• [4] IQ4FCHPZ HTTP connections: pooling + retry on error
• [5] JRENVH5D Reqwest 0.11
• [6] AI73GKAO Adding a UserAgent header to the http downloader
• [7] TPEH2XNB 1.0.0-alpha.28, with Tokio 1.0
• [8] VBMXB443 Retrying if the HTTP connection drops while reading the body
• [*] FBXYP7QM Forgot to add remote::http

Change contents

• edit in pijul/src/remote/http.rs at line 36
  [2.213]

  [2.213]
  + use tokio::io::AsyncWriteExt;
• edit in pijul/src/remote/http.rs at line 41
  [2.381]→[2.381:431](∅→∅)
  − use tokio::io::AsyncWriteExt;
• edit in pijul/src/remote/http.rs at line 48
  [2.606]

  [2.606]
  + f.flush().await?;
• edit in pijul/src/remote/http.rs at line 76
  [3.1193]

  [2.897]
  + let mut size = res

  + .headers()

  + .get(reqwest::header::CONTENT_LENGTH)

  + .and_then(|x| x.to_str().ok())

  + .unwrap_or("0")

  + .parse::<usize>()

  + .ok();
• replacement in pijul/src/remote/http.rs at line 85
  [3.137]→[2.920:1027](∅→∅)
  − Ok(Some(chunk)) => send.send(Some(chunk)).await?,

  − Ok(None) => done = true,
  [3.137]

  [2.1027]
  + Ok(Some(chunk)) => {

  + if let Some(ref mut s) = size {

  + *s -= chunk.len();

  + }

  + send.send(Some(chunk)).await?;

  + }

  + Ok(None) => match size {

  + Some(0) | None => done = true,

  + _ => break,

  + },