Skip to content

Reproducible transport benchmark suite with competitor harnesses #51

Description

@Segfaultd

Summary

Add a reproducible benchmark suite measuring throughput, latency under loss, and CPU per connection, with harnesses for comparing against ENet and GameNetworkingSockets.

Motivation

The README already documents syscall counts for the batched I/O path with a reproducible command line. Extending that rigor to end to end transport metrics would guide performance work (congestion control, batching, allocator changes) with hard numbers, catch regressions, and give prospective users an honest basis for evaluation.

Proposal

  • A Benchmarks/ directory with self contained scenario binaries: bulk reliable ordered throughput, small message rate (packets per second), RTT/jitter under tc netem loss and delay profiles, N clients against one server measuring CPU and memory per connection
  • Docker based runner so results are reproducible on any Linux host, following the container pattern already used for Linux verification
  • Optional comparison harnesses implementing the same scenarios on ENet and GameNetworkingSockets, fetched only when a BENCHMARK_COMPETITORS style option is on, never linked into the shipping library
  • Results published as a docs page with the exact commands, hardware notes, and library versions; a scheduled CI job tracks trend lines on a fixed runner class

Non goals

  • Microbenchmarks of internal containers (separate concern)
  • Claims about competitor performance without published, runnable scenarios

Acceptance criteria

  • docker run one liner reproduces every headline number
  • At least the throughput and loss profile scenarios wired into a scheduled CI workflow with archived results

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions