Skip to content

Zarr v3 backend configuration#1749

Draft
alejoe91 wants to merge 2 commits into
mainfrom
zarrv3
Draft

Zarr v3 backend configuration#1749
alejoe91 wants to merge 2 commits into
mainfrom
zarrv3

Conversation

@alejoe91

@alejoe91 alejoe91 commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

@h-mayorquin

Copy link
Copy Markdown
Collaborator

I took a first look, very high level:

  1. The validator should check shard_shape against buffer_shape, not just chunk_shape. As far as I understand a shard is assembled whole in memory on every flush, so shard_shape sets a per-flush memory footprint a user can push until they run out of memory, and a shard bigger than the buffer also forces a read-modify-write since we write buffer by buffer. So I'd enforce shard_shape <= buffer_shape, ideally that it evenly tiles the buffer.
  2. There's no user-facing documentation, we should do this and make this easy to review.
  3. What do you think about defaults shard values? Should we write defaults equal bo the buffer size? non-shared by defaults?

@alejoe91 let me know if this is the type of feedback that you wanted or if I should look into other aspects of it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants