Skip to content

Measure and possibly bound the heap retained by the shared zstd decoder #87

Open

Description

@oss-amikos

Issue #83 and the EncoderProfileBoundedMemoryUncached profile (#86) cover the zstd encoder only. The decoder is also shared and never closed (sharedZstdDecoder in serialize.go). We have not measured what it keeps live after Decode.

What to measure:

  • Live heap after one Decode of a small index and of a large index, with a forced GC before and after.
  • The same at GOMAXPROCS 1, 4 and 16, since the decoder may scale its workers with it.
  • Whether the 64 MiB window and memory limits change retained heap or only cap it.

Proposed acceptance:

  • A benchmark in benchmark_test.go that prints retained MB per GOMAXPROCS.
  • A table in docs/ with the real numbers.
  • A decision: leave as is, or add an opt-in bounded profile for the decoder, as was done for the encoder.

No code change is proposed until the numbers show a problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions