Skip to content

Preserve source extension in symlink writer (OME-Zarr support) - #19

Open
Arshya-Guru wants to merge 2 commits into
mainfrom
preserve-source-extension
Open

Preserve source extension in symlink writer (OME-Zarr support)#19
Arshya-Guru wants to merge 2 commits into
mainfrom
preserve-source-extension

Conversation

@Arshya-Guru

Copy link
Copy Markdown

Summary

The symlink backend hardcoded .ims as the link extension, so packaging an OME-Zarr source (documented as supported in the README) produced a link named _SPIM.ims pointing at a zarr directory — silently misidentifying the format for pybids, ZarrNii, and downstream tools like spimquant. This never surfaced before because all existing datasets (e.g. the ki3 aggregate) only used .ims/.ome.ims sources.

Hit in practice while building mouse_app_vaccine_aggregated, where batch1's only remaining sources are the SPIMprep OME-Zarrs.

Changes

  • SymlinkWriter now derives the link extension from the source asset via a new _bids_extension() helper:
    • .ome.zarr / .zarr stores keep their full extension
    • compound file extensions collapse to the plain format extension (x.ome.ims.ims), matching the naming in existing SPIMpack-generated datasets
  • Sidecar path is built with build_path using a .json extension, so zarr assets get _SPIM.json rather than _SPIM.ome.json (previously derived via with_suffix, which only replaces the last suffix)
  • Added a writer test covering both an .ome.zarr directory source and an .ome.ims file source

Testing

All writer/io/cli/validation tests pass (17 passed), and the change was exercised for real to build the 31-subject mouse_app_vaccine_aggregated dataset (mixed .ims, .ome.ims, and .ome.zarr sources; all links verified).

🤖 Generated with Claude Code

The symlink backend hardcoded .ims as the link extension, so OME-Zarr
sources (which the README already documents as supported) produced links
named _SPIM.ims pointing at zarr directories, misidentifying the format
for pybids/ZarrNii and downstream tools.

Derive the extension from the source asset instead: .ome.zarr/.zarr are
kept as-is, while compound file extensions collapse to the plain format
extension (x.ome.ims -> .ims), matching existing datasets. The sidecar
path is now built via build_path with a .json extension so zarr assets
get _SPIM.json rather than _SPIM.ome.json.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings August 14, 2026 01:32

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the symlink backend to correctly name BIDS symlinks and sidecars based on the true source asset format, enabling correct OME-Zarr packaging and avoiding misleading .ims outputs.

Changes:

  • Derive the symlink extension from the source path (preserving .ome.zarr / .zarr, collapsing compound file extensions like .ome.ims.ims).
  • Build sidecar paths via build_path(..., extension=".json") to avoid incorrect .ome.json naming for compound extensions.
  • Add tests covering .ome.zarr directory sources and .ome.ims file sources.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
src/spimpack/backends/symlink.py Derives BIDS link extension from the source and fixes sidecar path construction for compound extensions.
tests/test_writer.py Adds coverage for .ome.zarr and .ome.ims extension handling and sidecar placement.
Suppressed comments (2)

src/spimpack/backends/symlink.py:58

  • When an existing output path is a real directory (not a symlink), unlink() will raise IsADirectoryError. With .ome.zarr support, it’s more likely users may already have a directory at the intended link path (e.g., from a previous copy-based run), which would cause the writer to crash instead of overwriting cleanly. Handle directories explicitly before unlinking.
                if link_path.exists() or link_path.is_symlink():
                    link_path.unlink()

src/spimpack/backends/symlink.py:86

  • Path.suffix preserves the original casing, so sources like raw.OME.IMS would produce a link ending in .IMS (previously this was always .ims). Since BIDS filenames and the existing writer behavior use lowercase extensions, normalize the returned suffix to lowercase to keep output stable.
    return source.suffix or ".ims"

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +47 to +53
bids_entities["extension"] = _bids_extension(asset.spim_path)

rel_path = build_path(bids_entities, [BIDS_MICR_PATTERN])
link_path = output_dir / rel_path
json_path = link_path.with_suffix(".json")
json_path = output_dir / build_path(
{**bids_entities, "extension": ".json"}, [BIDS_MICR_PATTERN]
)

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — updated the README in 12b8fcc: the symlink bullet now describes extension derivation (.ome.zarr/.zarr kept, compound file extensions collapsed to the plain format extension), and the path pattern ends in _SPIM{extension}.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Arshya-Guru

Copy link
Copy Markdown
Author

Whoops, this was actually problematic for downstream SPIMquant analysis as path_templates expects consistent extensions, so mixed .ome.zarr/.ims extensions fail at config

@Arshya-Guru

Copy link
Copy Markdown
Author

Re-opening because downstream SPIMquant analysis failed anyways when a symlink named .ims was pointing to a directory.. SPIMquant can now handle extensions of both file types

@Arshya-Guru Arshya-Guru reopened this Aug 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants