Multipath Draft-21

[sleepybishop]

WIP

Hello Kazuho, I hope you are well.

Following up on #559, this pr implements draft-21 of the quic multipath spec. I know you are hesitant about merging drafts, so happy to leave pr as draft until it becomes more useful.

I've done interop testing against noq. Which was forked from quinn to add multipath support.

I've tried to isolate the per-path data with new accessors to avoid impacting the path 0 fast path when multipath is not used.

picoquic landed support for draft21 yesterday but I have not done interop testing against it yet.

[kazuho]

Thanks for working on this. I know it’s still a draft, but let me suggest a design:

MPQUIC is two nested layers, and this PR flattens them into one.

The draft specification has an outer notion of "path" (identified by Path ID) and, within each such path, the ordinary RFC 9000 path machinery:

• Outer (the MPQUIC path, keyed by Path ID) owns a packet number space in both directions, its own loss recovery / CC / pacer, and its
  own CID sequence space ("Each path ID has its own connection ID sequence number space"). Its Path ID is monotonic and must never be
  reused — precisely because the 32-bit Path ID is part of the AEAD nonce (PathID(32) | zeroes(2) | PN(62) XOR IV).
• Inner (RFC 9000 path handling), living inside one outer path — candidate 4-tuples, PATH_CHALLENGE/PATH_RESPONSE, address validation,
  NAT rebinding. Here reusing address slots is fine and expected; the network path can change while the Path ID and its PN space stay
  fixed.

quicly today is essentially one outer path already: egress.{cc,loss,pacer,packet_number} + the application ingress space are the PN space + recovery, and conn->paths[] (st_quicly_conn_path_t, with path_challenge/probe_only/dcid/promote_path) is the inner address/migration layer. So the natural extension is to name that grouping and let a connection hold N of them (N=1 under RFC 9000, so v1 needs no special-casing):

  quicly_conn_t
    └─ path_spaces[]                 // length 1 for RFC 9000
         ├─ path_id                  // monotonic, never reused
         ├─ egress PN + ingress PN space (next_expected_pn + ack_queue → PATH_ACK)
         ├─ loss / cc / pacer
         ├─ local+remote CID sequence space
         ├─ status (available/backup)
         └─ addrs[]                  // today's st_quicly_conn_path_t: 4-tuples, PATH_CHALLENGE, validation

This PR instead extends the inner array element with per-path cc/loss/pn and special-cases slot 0 to alias conn->egress via the get_*() accessors. By adjusting the data structure as stated above, I think we can avoid lots of conditional statements and simplify the code.

[sleepybishop]

Thank you for your feedback Kazuho, much appreciated. I've refactored to address it.

Some of the design decisions might look puzzling, my intent is to use quicly within a process that adapts to use all available links and bandwidth and provides userspace with simple uds or tun hooks to send data through so I've made the scheduler pluggable.

Lastly, I don't want to litter the repo will pull requests that are not useful to you, but if you do have interest, I've implemented rfc9287 and rfc9368. I'de like to add support for rfc8899 but that depends on how this pr shakes out as it would touch much of the same surface area.

Thanks again,

[kazuho]

@sleepybishop Thank you, and understood. I really appreciate your work here.

Re 9287 and 8368, I will take a look at the branches and see if we can pick them upstream.