Modern AEAD transport encryption (X25519 + ChaCha20-Poly1305 handshake)

[Segfaultd]

Summary

Replace the legacy secure connection layer with a modern AEAD based handshake: X25519 key exchange plus ChaCha20-Poly1305 (or AES-GCM) per packet encryption, negotiated as a new handshake version.

Motivation

The current SecureHandshake layer predates modern transport security practice. State of the art for game transports is an authenticated key exchange over a small pinned public key, followed by per packet AEAD with nonce management, enabled by default. OpenSSL 3 is already a hard dependency, so all required primitives (X25519, ChaCha20-Poly1305, AES-GCM, HKDF) are available without adding a new dependency.

Design sketch

• Server holds a static X25519 keypair; clients pin the public key out of band (same trust model the existing security layer uses)
• Handshake: client ephemeral X25519 -> HKDF -> per direction keys, bound to the connection GUIDs to prevent replay across connections
• Data phase: AEAD over the full datagram payload with a per direction 64 bit counter nonce; the reliability layer header stays inside the encrypted envelope
• Negotiate via a new offline protocol version so unencrypted and legacy peers are either supported or cleanly rejected by policy

Non goals

• Certificate chains / PKI
• Encrypting truly offline messages such as unconnected pings

Acceptance criteria

• New handshake selectable per peer via InitializeSecurity style API, on by default for new code paths
• Interop test matrix: new/new, new/legacy, new/plaintext, covering both accept and reject policies
• Throughput benchmark showing overhead within acceptable bounds versus plaintext on loopback