#89 hmac-sha2-256-etm@openssh.com

Opened by mixis on Aug 31, 2026, 8:39 PM
Approve the patches, then land them on main with:
// mixis on Aug 31, 2026, 8:39 PM

My NixOS machines only enable Encrypt-then-MAC modes, which prevented me from using pijul.

So I vibe-coded it using https://cvsweb.openbsd.org/annotate/src/usr.bin/ssh/PROTOCOL,v?rev=HEAD section 1.5 and the openssh source as a reference. Take that with a huge dose of salt. This is more useful for an overview of what a solution may look like. That being said, it works for me and I prefer that over enabling MtE.

AAAAC3NzaC1lZDI1NTE5AAAAIOVbqBaMBnSYBqonIjsEySBrgA33Huyf/QPbj7Iav+5z
~ AAAAC3NzaC1lZDI1NTE5AAAAIOVbqBaMBnSYBqonIjsEySBrgA33Huyf/QPbj7Iav+5z authored a change · pushed by mixis on Aug 31, 2026, 8:44 PM
· AAAAC3NzaC1lZDI1NTE5AAAAIOVbqBaMBnSYBqonIjsEySBrgA33Huyf/QPbj7Iav+5z · on Aug 31, 2026, 7:38 PM
W7F3ME3NYLLMMUFB44OB5O2H3EI27EMN7S2C3D44LIQT7XSVRWYQC
// mixis on Sep 1, 2026, 7:01 AM

To ease review, I’ll add a summary of what it did. Note that it also claims to have found a bug through testing, that it fixed for aes256-ctr, but not for untouched aes256-gcm.

Encrypt-then-MAC (hmac-sha2-256-etm@openssh.com) in thrussh

1. My understanding of ETM mode

What the protocol says

The OpenSSH PROTOCOL document (section “1.5 transport: Protocol 2 Encrypt-then-MAC MAC algorithms”) defines -etm variants as doing the MAC and encryption in the opposite order from RFC 4253. Concretely:

mac = MAC(key, sequence_number || packet_length || encrypted_packet)

where packet_length is encoded as a uint32 in the clear, and encrypted_packet is the encryption of

padding_length || payload || random_padding

(n1 = packet_length - padding_length - 1, n2 = padding_length).

The motivation stated in the spec is security: in classic SSH the receiver decrypts unauthenticated ciphertext before checking the MAC, which (combined with a cipher flaw) acts as a “decryption oracle” that can reveal plaintext. ETM moves authentication before decryption so that unauthenticated ciphertext is never decrypted.

Exact wire format (verified against openssh-portable source)

I read the actual OpenSSH implementation (packet.c, cipher.c, cipher-aesctr.c, mac.c) to pin down the details. For aes256-ctr + hmac-sha2-256-etm@openssh.com:

Send (ssh_packet_send2_wrapped)

  1. aadlen = 4 for ETM (the length field is “additional authenticated data” that is copied, not encrypted).
  2. Padding is computed after subtracting aadlen, so packet_length itself (the encrypted body) is a multiple of the cipher block size (16), with a minimum of 4 padding bytes.
  3. The length field is written unencrypted; only the body (padding_length || payload || padding) is passed to the cipher.
  4. The MAC is computed over seqn(4) || length(4, clear) || ciphertext, and appended in the clear.

Receive (ssh_packet_read_poll2)

  1. aadlen = 4; the length is read directly (cipher_get_length just does PEEK_U32(cp) — no decryption).
  2. The body length (need = packet_length) must be a multiple of block_size (16).
  3. The MAC is checked before cipher_crypt — this is the whole point: verify-then-decrypt.
  4. Then the body is decrypted.

mac.c::mac_compute confirms the HMAC input is seqno(4 bytes big-endian) || data — identical to thrussh’s existing hmac_sha256::authenticate/verify, which already prepend the sequence number the same way.

Contrast with the current (non-ETM) aes256-ctr path

thrussh’s current (and OpenSSH’s) non-ETM CTR ordering is:

  • Send: MAC(key, seqn || length || padding_length || payload || padding) over the plaintext, then CTR-encrypt length || padding_length || payload || padding (the length is encrypted), then append the MAC in the clear.
  • Receive: read a full 16-byte block, decrypt it to get the length, read the rest, decrypt the whole plaintext, then verify the MAC over the plaintext.

So the three essential differences for ETM are: (a) the length is no longer encrypted, (b) the MAC covers ciphertext plus the clear length, and (c) verification happens before decryption.


2. Summary of changes

All changes are in thrussh/ssh/src.

mac/mod.rs

  • Added Name::is_etm() (checks the "-etm@openssh.com" suffix).
  • Made hmac_sha256::make_key pub(crate) (so the ETM MAC descriptor and the new test can reuse the identical key material).
  • Added hmac_sha256_etm module with NAME = "hmac-sha2-256-etm@openssh.com" and a MAC descriptor that reuses hmac_sha256::make_key. No new key type or HMAC code is needed — the computation is identical; only what data is fed to it (and when) changes.

negotiation.rs

  • Added mac::hmac_sha256_etm::NAME to the mac!() preference list, ahead of hmac-sha2-256 and none.

kex.rs (compute_keys)

  • Added a match arm mapping hmac_sha256_etm::NAME → the ETM MAC descriptor.
  • Cipher selection now branches on mac.is_etm(): aes256-ctr + ETM MAC selects the new CIPHER_ETM, otherwise the normal CIPHER. AEAD ciphers (chacha20-poly1305, aes256-gcm) are unaffected (they ignore the MAC entirely).

cipher/aes256_ctr.rs (the core)

  • Added an etm: bool field to Key and a second static CIPHER_ETM whose make_* functions set it.
  • Branched the four relevant trait methods:
    • padding_length: for ETM, pad so packet_length (body) is a multiple of 16 (payload.len() + 1 base), rather than 4 + packet_length (payload.len() + 5 base). Random extra padding is a multiple of 32, which keeps the block alignment in both modes.
    • length_block_size: 4 for ETM (clear length), 16 for non-ETM (first cipher block).
    • decrypt_packet_length: identity read of the clear uint32 for ETM.
    • seal: for ETM, encrypt only the body and MAC over seqn || length(clear) || ciphertext; non-ETM branch is unchanged.
    • open: for ETM, verify the MAC before decrypting, then decrypt the body and return it; non-ETM branch unchanged apart from the return-value fix below.
  • Added a round-trip test module (ctr_roundtrip, etm_roundtrip) that exercises CipherPair::writecipher::read over a tokio::io::duplex for both modes.

Also fixed in the same file (see §3)

  • The non-ETM open now returns Ok(&payload[4..]) (the body) instead of Ok(payload) (the full [length || body] buffer).

No changes needed in cipher/mod.rs

cipher::read and CipherPair::write already abstract the length field behind length_block_size()/decrypt_packet_length() and the MAC area behind tag_len()/seal()/open(), so the ETM cipher drops in without touching that orchestration.

Validation performed

  • cargo build -p thrussh --features openssl — passes.
  • cargo build -p thrussh (no openssl) — passes.
  • cargo test -p thrussh --features openssl — 3 lib tests + 2 doc-tests pass.
  • cargo test -p thrussh — passes.

3. The potential bug I was concerned about

The OpeningKey::open return-value contract

cipher::read (in cipher/mod.rs) relies on this contract: after open returns plaintext, it does

let padding_length = plaintext[0] as usize;   // expects the padding_length byte
let plaintext_end = plaintext.len() - padding_length;
buffer.buffer.resize(plaintext_end + 4);      // keeps [len(4)][padding_length(1)][payload]

and the caller (client/mod.rs/server) consumes buffer.buffer[5..] as the payload. Therefore open must return the body (padding_length || payload || padding), i.e. with the 4-byte packet length stripped.

Two of the four ciphers did this correctly (clear::open and chacha20poly1305::open return &buf[4..]), but two returned the full buffer:

  • aes256_ctr::open returned Ok(payload)[length || body].
  • aes256_gcm::open returned Ok(payload)[length || body].

Consequence

When open returns [length || body], plaintext[0] is the packet length’s most-significant byte, which is always 0 for sub-16 MiB packets. So padding_length is misread as 0, padding is never stripped, and cipher::read returns the wrong byte count. My new ctr_roundtrip test caught this concretely: it returned 228 where 27 was expected.

Why it was latent

The default cipher is chacha20-poly1305 (which returns the body correctly), so the aes256-ctr/aes256-gcm read paths are rarely exercised. This bug only surfaces when those ciphers are actually negotiated.

What I did

I fixed aes256_ctr::open’s non-ETM branch to Ok(&payload[4..]) (matching the contract, and what my ETM branch already did). The new ctr_roundtrip test now passes. The ETM branch always returned the body correctly from the start.

What remains for deeper investigation

  • aes256_gcm::open still has the same bug (Ok(payload) instead of Ok(&payload[4..])). I left it alone because it is a separate file and outside the ETM scope, but it is the same one-line, clearly-correct fix and should be addressed.
  • Worth verifying whether any other cipher’s open/seal deviates from the body-returning contract (chacha and clear are correct; gcm is not).
  • A gcm round-trip test analogous to ctr_roundtrip/etm_roundtrip would lock this in and immediately fail on the current gcm code, making the bug undeniable.

4. Live sshd interop test

Added thrussh/ssh/examples/etm_interop.rs, which forces ETM from the client and connects to a real OpenSSH sshd. I ran it against the system sshd (OpenSSH 10.5p1) and it passed — the client negotiated aes256-ctr + hmac-sha2-256-etm@openssh.com, authenticated, and ran echo interop-ok successfully, validating thrussh’s ETM seal and open against a real OpenSSH peer.