docs+config(worker): rename per-type WORKER_*_MAX_SLOTS to *_RESERVED_SLOTS (#2963) - #3016
Open
nicoloboschi wants to merge 1 commit into
Open
docs+config(worker): rename per-type WORKER_*_MAX_SLOTS to *_RESERVED_SLOTS (#2963)#3016nicoloboschi wants to merge 1 commit into
nicoloboschi wants to merge 1 commit into
Conversation
…_SLOTS (#2963) The per-operation `HINDSIGHT_API_WORKER_<TYPE>_MAX_SLOTS` env vars set a reservation *floor* (a guaranteed minimum), not a ceiling — despite the name a type overflows the shared pool up to WORKER_MAX_SLOTS. The reporter hit exactly this: consolidation ran 6-concurrent with "MAX_SLOTS=1". Rename them to `<TYPE>_RESERVED_SLOTS`, which says what they do. The old `<TYPE>_MAX_SLOTS` stays as a deprecated alias that logs a warning (setting both is an error), so no existing deployment changes behavior. Defaults unchanged (consolidation reserved=2). Docs (current + version-0.8) updated to state plainly that a reservation is a floor, not a cap — a type's real ceiling is WORKER_MAX_SLOTS. Tests cover the new env var, the deprecated-alias mapping + warning, and the both-set error. This is the issue's "part 1" — the cheap, high-value half. A genuine per-type concurrency ceiling is a separate follow-up if operators need one.
nicoloboschi
force-pushed
the
feat-worker-per-type-slot-limits
branch
from
July 28, 2026 15:28
b4fe912 to
c257003
Compare
nicoloboschi
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem (#2963)
HINDSIGHT_API_WORKER_<TYPE>_MAX_SLOTSreads like a per-type ceiling but is actually a reservation floor (a guaranteed minimum) — a type overflows the shared pool up toWORKER_MAX_SLOTS, so the name says the opposite of the behavior. The reporter hit exactly this: consolidation ran 6-concurrent withMAX_SLOTS=1, and it took a production latency investigation to discover the log line was working as designed.(History for context: the var was a genuine max at the slim split, became a strict-partition reservation in #1006, then #1199 added shared-pool overflow — which is where the name became a misnomer.)
Change (issue's "part 1" — the cheap, high-value half)
Rename the per-type env vars to say what they do:
<TYPE>_RESERVED_SLOTS— the reservation floor (guaranteed minimum).<TYPE>_MAX_SLOTS— kept as a deprecated alias that logs a warning (setting both is an error), so no existing deployment changes behavior.TYPE ∈ {consolidation, retain, file_convert_retain, refresh_mental_model, graph_maintenance, import_documents}. Defaults unchanged (consolidation reserved=2).Docs (current + version-0.8) now state plainly that a reservation is a floor, not a cap — a type's real ceiling is
WORKER_MAX_SLOTS.Scope
Deliberately rename-only. The issue's part 2 (an actual per-type concurrency ceiling) is a larger, behavior-changing follow-up that requires claim-time enforcement across both PG and Oracle; left out on purpose. The issue itself notes part 1 alone "would have saved us the investigation." Happy to do the ceiling as a separate PR if operators need it.
Tests
<TYPE>_RESERVED_SLOTSparses into the reservation floor.<TYPE>_MAX_SLOTSmaps to the floor and logs a warning.Full
test_worker.py+test_config_validation.pygreen; lint clean.Closes #2963. Supersedes #2987 (docs-only).