Skip to content

api/prometheus/v1: merge envelope and payload JSON decode, fixes #977 - #2078

Open
felpau05 wants to merge 1 commit into
prometheus:mainfrom
felpau05:merge-json-decode-977
Open

api/prometheus/v1: merge envelope and payload JSON decode, fixes #977#2078
felpau05 wants to merge 1 commit into
prometheus:mainfrom
felpau05:merge-json-decode-977

Conversation

@felpau05

@felpau05 felpau05 commented Jul 27, 2026

Copy link
Copy Markdown

Fixes #977.

Removes the redundant second JSON parse across all 20 methods affected by this issue, by decoding the response envelope and the typed payload in a single json.Unmarshal pass instead of decoding into an intermediate json.RawMessage that each method then re-parsed separately.

How

  • apiResponse.Data changes from json.RawMessage to interface{}.
  • apiClient.Do / DoGetFallback take a new data interface{} destination parameter. When a caller passes a typed pointer (e.g. &queryResult{}), json.Unmarshal decodes directly into it via Go's existing behavior of following a non-nil pointer already stored in an interface field — no second parse. When a caller passes nil, Do falls back to decoding into a local json.RawMessage and returns it as []byte, preserving the exact current behavior for any caller that only needs the raw bytes.
  • CleanTombstones and DeleteSeries are unaffected — they never read the response body, so they already pass nil.
  • FormatQuery is intentionally not converted. It doesn't call json.Unmarshal on the body at all — it does a raw string(body) cast — so it was never affected by the double-parse this issue describes. While verifying this, I found what looks like a separate, pre-existing bug (the raw cast leaves JSON quoting/escaping in the returned string instead of decoding it) — filing that separately rather than mixing an unrelated behavior change into this PR.

Why this is scoped differently from #1090

That attempt removed the buffering in httpClient.Do (api/client.go) itself to stream from resp.Body, which broke request cancellation handling and ended up with a benchmark that stopped measuring anything real. This PR doesn't touch api/client.go or cancellation at all — Do still receives a fully-buffered body exactly as before. The only change is not parsing that body twice.

Benchmark

New, committed BenchmarkQueryRange driving QueryRange through the public API over a real HTTP round trip, against a 100-series × 350-sample matrix response (~35k samples), -count=100, benchstat:

              │ /tmp/bench_old.txt │       /tmp/bench_new.txt        │
              │       sec/op       │   sec/op     vs base            │
QueryRange-20          48.01m ± 2%   37.80m ± 1%  -21.26% (n=100)

              │ /tmp/bench_old.txt │       /tmp/bench_new.txt        │
              │        B/op        │     B/op      vs base           │
QueryRange-20         7.522Mi ± 0%  6.732Mi ± 0%  -10.50% (n=100)

              │ /tmp/bench_old.txt │      /tmp/bench_new.txt         │
              │     allocs/op      │ allocs/op    vs base            │
QueryRange-20          75.18k ± 0%   74.77k ± 0%  -0.54% (n=100)
  • Memory drops ~10.5% — the old path fully materializes data into a RawMessage copy and then reparses it; removing that copy is the whole effect.
  • Time drops ~21.3%. Running a high-count (n=100) benchmark completely flattened the localhost HTTP round-trip noise, revealing a massive and highly consistent decode speedup.
  • Allocation count barely moves, since the same typed structures get built either way — the win is in not re-scanning and re-copying the payload, not in fewer objects.

Tests

  • Added a matrix-shaped case to the existing queryTests table in TestAPIs, alongside the existing scalar case, asserting reflect.DeepEqual on the fully decoded model.Matrix — the path this change (and the benchmark) actually exercises. Confirmed the assertion catches regressions by manually corrupting the expected value and checking the test fails, then reverting.
  • go build, go vet, gofmt -l ., and go test -count=1 ./api/... are all clean.

@kakkoyun — you triaged this issue originally; would appreciate a look when you have time.

Removes the redundant second JSON parse across all 20 methods affected by prometheus#977. Decodes the envelope and typed payload in a single json.Unmarshal pass instead of an intermediate json.RawMessage that each method re-parsed separately.

Changes apiResponse.Data from json.RawMessage to interface{}. Updates Do and DoGetFallback to take a new data interface{} destination. A typed pointer decodes directly via Go's existing behavior of following a non-nil pointer stored in an interface field. Passing nil falls back to the current json.RawMessage behavior unchanged (used by CleanTombstones and DeleteSeries).

FormatQuery remains unconverted as it uses a raw string(body) cast rather than json.Unmarshal on the body.

Fixes prometheus#977.

Signed-off-by: felpau05 <felpau05@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Client API: don't read entire response into a buffer before parsing it

1 participant