Detect binary files by NUL byte before encoding guess

DzmingLi

Feb 9, 2026, 8:12 AM

XWJJMW2WCG3P4AKAMJQQ3ZNFP3FP3TA36EG5US665N6SQA3Z4V5QC

Dependencies

• [2] HWYGVLP5 Replacing the temporary copy of chardetng with the published version
• [3] 6CZYYOG7 Faster guessing of encoding
• [4] 5FI6SBEZ Re-implement change printing and parsing
• [5] SGXOEWHU Adding a patched chardetng (temporarily)
• [6] NYOF5766 track file encoding in the record, including change text for file adds
• [*] SXEYMYF7 Fixing the bad changes in history (unfortunately, by rebooting).
• [*] 7KNPYIDU Splitting the WorkingCopy trait into a read-only record and a read/write output

Change contents

• edit in libpijul/src/working_copy/mod.rs at line 29
  [3.73]

  [3.73]
  + let slice = &buffer[init..];

  + if looks_binary(slice) {

  + return Ok(None);

  + }
• replacement in libpijul/src/working_copy/mod.rs at line 34
  [3.125]→[3.125:171](∅→∅),[3.171]→[2.34:127](∅→∅)
  − detector.feed(&buffer[init..], true);

  − if let Some(e) = crate::get_valid_encoding(&detector, None, true, &buffer[init..]) {
  [3.125]

  [3.75]
  + detector.feed(slice, true);

  + if let Some(e) = crate::get_valid_encoding(&detector, None, true, slice) {
• edit in libpijul/src/working_copy/mod.rs at line 43
  [9.30]

  [9.30]
  + fn looks_binary(buf: &[u8]) -> bool {

  + if buf.is_empty() {

  + return false;

  + }

  + buf.iter().any(|&b| b == 0)

  + }

  +