Skip to content

refactor: simplify the init-concurrency limiter to a single shared budget - #785

Open
kinyoklion wants to merge 1 commit into
feat/concurrency-init-limitsfrom
rlamb/relay-init-concurrency-limiter
Open

refactor: simplify the init-concurrency limiter to a single shared budget#785
kinyoklion wants to merge 1 commit into
feat/concurrency-init-limitsfrom
rlamb/relay-init-concurrency-limiter

Conversation

@kinyoklion

@kinyoklion kinyoklion commented Jul 31, 2026

Copy link
Copy Markdown
Member

Refines the init-concurrency limiter primitive and configuration that #780 landed on the feature branch, in preparation for wiring it into the poll and stream endpoints.

What changes

  • Drop per-environment fairness. The limiter had an optional per-environment gate (PerEnvMax) and a matching INIT_PER_ENV_MAX_PERCENT config option. Both are removed. The limiter now bounds concurrent admissions with a single shared budget: a fixed number of held slots plus a bounded FIFO queue of waiters. Acquire no longer takes an environment key.
  • INIT_SEND_TIMEOUT documentation and default now describe it as an absolute cap on how long one gated delivery may hold a slot.

Why one shared budget

The budget we need to protect is Relay's overall memory and egress during a reconnect herd; that is a global resource, not a per-environment one. A per-environment gate rejects an environment that is under the global budget while slots sit free, and it adds bookkeeping (a sync.Map of gates) for a fairness property we do not need here. A single shared budget is simpler and matches the resource it guards.

No behavior change

Nothing acquires from the limiter yet, so this is inert at runtime. It builds, vets, and passes -race on ./config/ and ./internal/concurrency/.


First of a set of PRs splitting the larger wiring change (#782) into independently reviewable pieces. This one stands alone.


Note

Low Risk
Refactor of unused limiter/config surface with no production call sites; main operational note is breaking removal of INIT_PER_ENV_MAX_PERCENT for anyone who set it.

Overview
Simplifies the init-concurrency limiter ahead of poll/stream wiring by dropping per-environment fairness in favor of one global admission budget.

Removed: INIT_PER_ENV_MAX_PERCENT / PerEnvMaxPercent, limiter PerEnvMax, per-env sync.Map gates, and the envKey argument on Acquire. Admission is now only MaxConcurrent held slots plus a bounded FIFO queue (MaxQueued).

INIT_SEND_TIMEOUT docs now describe an absolute cap on how long a gated delivery may hold a slot (default 2m), with a 64 KB/s throughput floor expected to reclaim slow/stalling clients sooner.

Tests drop per-env isolation coverage and call Acquire(ctx) without an environment key. No runtime behavior change yet—nothing acquires from this limiter in production paths.

Reviewed by Cursor Bugbot for commit 8a9ec78. Bugbot is set up for automated code reviews on this repo. Configure here.

…dget

The limiter primitive added in #780 carried an optional per-environment gate
and a matching INIT_PER_ENV_MAX_PERCENT config option. Drop per-environment
fairness: the limiter now bounds concurrent admissions with one shared budget
(a fixed number of held slots plus a bounded FIFO queue of waiters), and
Acquire no longer takes an environment key.

Also revise the INIT_SEND_TIMEOUT documentation and default to describe an
absolute cap on how long one gated delivery may hold a slot.

No runtime effect on its own: nothing acquires from the limiter yet.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant