Skip to content

feat(perf-client): env flags to suppress cache-busting headers - #2613

Open
michaelfeil wants to merge 1 commit into
mainfrom
perf-client/suppress-cache-busting-headers
Open

feat(perf-client): env flags to suppress cache-busting headers#2613
michaelfeil wants to merge 1 commit into
mainfrom
perf-client/suppress-cache-busting-headers

Conversation

@michaelfeil

Copy link
Copy Markdown
Contributor

Summary

The performance client emits two headers that vary per request and break header-based downstream caching (e.g. when a cache server SHA-hashes request headers as the cache key):

  • Request-Deadline-Ms — time-based (now + timeout), recomputed every request (core/src/http_client.rs)
  • x-baseten-customer-request-id — unique per request (UUID + batch/retry/hedge counters)

When the user has no control over the cache server, the only lever is to stop emitting these headers at the client. This adds two opt-in env flags (default behavior unchanged):

  • PERFORMANCE_CLIENT_DISABLE_TIMEOUT_HEADERS — suppresses Request-Timeout-Ms / Request-Deadline-Ms
  • PERFORMANCE_CLIENT_DISABLE_REQUEST_ID_HEADER — suppresses x-baseten-customer-request-id (internal retry/hedge/tracing behavior is unaffected; only the emitted header is skipped)

Truthy values: 1, true, yes, on (case-insensitive). Values are read once and cached via OnceLock to avoid a syscall per request.

Changes

  • core/src/constants.rs: new env var name constants
  • core/src/http_client.rs: env_flag_enabled helper + cached flags; add_timeout_headers and both CUSTOMER_HEADER_NAME injection sites are now gated
  • README.md: document the two new env vars

Test plan

  • cargo test -p baseten_performance_client_core (49 passed, incl. new http_client::tests*`)
  • cargo clippy -p baseten_performance_client_core --all-targets -- -D warnings clean
  • cargo fmt --check clean
  • cargo check for python + node bindings (no public API change)

Notes

  • No API/builder changes, so Python/Node bindings require no updates.
  • Suppression is env-var-only per the requested design.

Add PERFORMANCE_CLIENT_DISABLE_TIMEOUT_HEADERS and
PERFORMANCE_CLIENT_DISABLE_REQUEST_ID_HEADER env vars (truthy:
1/true/yes/on). When set, the client omits the time-based
Request-Deadline-Ms / Request-Timeout-Ms headers and/or the
per-request x-baseten-customer-request-id header, which otherwise
break header-based downstream cache keys. Default behavior is
unchanged (headers emitted). Values are cached via OnceLock.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant