XNAT dcm2niix conversion pinned to stale 2021 Docker Hub build silently drops slices from valid series

[atriaybagur]

Closes #980.

Problem

The XNAT Container Service converts DICOM→NIfTI with Docker Hub's mutable xnat/dcm2niix:latest, which resolves to dcm2niix v1.0.20211006 — a 2021 build upstream no longer updates (no newer tag exists on that repo). Its slice-ordering heuristic mis-fires on some of the seeded spleen series and silently drops one slice:

Warning: Instance number (0020,0013) not found: ...-no-value-for-SeriesNumber-0-....dcm
Slices skipped: image position not sequential, admonish your vendor (Siemens OOG?)
Convert 59 DICOM as ... (512x512x59x1)   # 60 DICOM in

The series are provably clean (unique 0-based InstanceNumbers, uniform 5 mm ImagePositionPatient steps, constant orientation; complete at every hop — Orthanc 60/60, XNAT DICOM resource 60/60). The converted input_*.nii.gz then has N−1 slices while the data-enrichment label has N, so the segmentation tutorials' shape QC (correctly) rejects the pair — 6 of GSTT's 21 spleen studies are silently lost from every spleen cohort, and always have been; the QC skips at INFO level so nobody noticed.

Fix

Build and publish a version-pinned in-house image, and reference it by immutable version tag everywhere (never latest):

• trust/xnat/dcm2niix/Dockerfile — debian-slim + the upstream release binary, pinned by version and sha256, with a build-time version smoke check. The Dockerfile's DCM2NIIX_VERSION ARG is the single source of truth for the pin.
• .github/workflows/docker_build_xnat_dcm2niix.yml — publishes ghcr.io/londonaicentre/xnat-dcm2niix with the usual sha/branch/stag/prod tags plus the tool-version tag read from the Dockerfile. (GHCR images in this org are anonymously pullable, so the Container Service's pull path needs no auth plumbing.) Also resolves the PR [Feature]: Add MONAILabel to OHIF viewer plugin XNAT #945 review note about the converter still coming from Docker Hub.
• The Dockerfile also carries org.opencontainers.image.source, which links the published package to this repo so it is listed under FLIP/packages alongside the other FLIP images. The siblings got that link implicitly (their packages were first created by a GITHUB_TOKEN push from this repo's workflows); this package already exists from the pathway test's manual push, so it needs the explicit label.
• References updated in the three places that must stay in sync (each cross-references the others):
  • trust/xnat/xnat/config/dcm2niix_command.json (Compose deploy; command version bumped 1.6 → 1.7)
  • deploy/providers/kubernetes/templates/xnat-init-job.yaml (the documented inline copy of the command)
  • trust/imaging-api — new Settings.DCM2NIIX_IMAGE used by the per-project event-subscription lookup (which finds the XNAT command by image string)
• trust/xnat/xnat/config/configure-dcm2niix.sh — additionally registers the pinned image's registry (ghcr.io, bare hostname, derived from the command JSON so it can't drift) as a credential-less Container Service image host, idempotently, with post-hoc validation. Load-bearing on the swarm backend — see the pathway-test finding below. The K8s init job deliberately doesn't (kubelet pulls there), and now documents why.
• Docs: trust/xnat/README.md, deploy/README.md, component-xnat.rst.

Verification

• Built image converts the previously slice-dropping series 60/60 (FAK76374103 / MSD spleen_63) and a control series 92/92 (FAK99198511), using the Container Service's exact command line (dcm2niix -b n -z y -o /output /input) against the live GSTT archive data. No warnings.
• imaging-api: ruff + mypy clean, 274 tests pass. trust/xnat: 174 tests pass. Workflow YAML + command JSON parse; helm template renders the chart with the new image ref.
• Full pathway test on the live dev stack (dummy GHCR package): pushed the built image to GHCR as a test-e2e tag (package made public — the CS pulls with no credentials), registered a parallel 1.7-test command on the dev XNAT (leaving the live command and its subscriptions untouched), ran the PR-branch imaging-api with DCM2NIIX_IMAGE pointed at the test tag, and drove a fresh platform project through cohort → approval → image pull:
  • imaging-api's new-project subscription bound the test command by image string (CommandActionProvider:2), old projects untouched.
  • Finding (fixed in this PR): every swarm-mode launch initially died with NullPointerException: authConfig was not specified — container-service 3.8.1 resolves the launch's registry auth by matching an image-host url against the bare registry hostname parsed from the image, and the swarm path passes a null resolution to docker-java unguarded. A registry with no image-host entry is unlaunchable even for a public image. Hence the configure-dcm2niix.sh image-host registration above; without the pathway test this would have shipped broken for every swarm deployment.
  • With the host registered: the CS pulled test-e2e from GHCR (no local copy existed — genuine registry pull), the conversion completed (container [Task]: Upgrade Python 3.12 → 3.13 across all services (and record why 3.14 is blocked) #944), and the previously slice-dropping accession FAK76374103 produced the full (512, 512, 60) NIfTI through the real event/launch machinery.
  • The dummy test-e2e tag and the package it created have since been deleted. That manual push created the package outside Actions, which left it unlinked from this repo (repository: null in the org packages API) — deleting it means the first post-merge publish recreates it from the workflow, labelled and linked, exactly as every other FLIP package came to be. See the post-merge checklist below.

Deployment notes

• workflow_dispatch of the new workflow is impossible until it exists on the default branch, so the first merge to develop publishes the image (the push event runs the workflow from the merged tree). Existing deployments pick the new image up on their next XNAT (re)configure — running stacks keep the old registered command until then.
• Found while verifying Migrate open NVFLARE tutorial PRs from archived flip-fl-base into FLIP #665's spleen e2e; sibling finding (0-based InstanceNumbers + empty SeriesNumber in the DICOM synthesis, which also gives XNAT 43-char UID-derived scan IDs) is noted in XNAT dcm2niix conversion pinned to stale 2021 Docker Hub build silently drops slices from valid series #980 for separate tracking.

Post-merge checklist (the package no longer exists, so the first publish creates it from scratch):

1. Confirm the publish ran, and that the package is linked: gh api /orgs/londonaicentre/packages/container/xnat-dcm2niix --jq .repository.full_name → londonaicentre/FLIP (it should now be listed under the repo's Packages).
2. Flip the new package's visibility to public in its package settings. New org packages default to private, and the Container Service pulls anonymously — leaving it private breaks DICOM→NIfTI conversion on every deployment that reconfigures after the merge.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[garciadias]

Re: deploy/README.md (line 269) — flagging at PR level because this line is unchanged context just outside the diff hunks.

This section states "All Docker containers in FLIP are hardened" and the Non-Root Users table enumerates every service, but the new xnat-dcm2niix image introduced in this PR deliberately keeps the root default user (see trust/xnat/dcm2niix/Dockerfile), and it is missing from the table. The blanket claim is now inaccurate and the hardening doc is incomplete for exactly the image being added.

Suggested change: add a xnat-dcm2niix | root | Container Service one-shot; keeps the root default user to match the replaced image's output-file ownership on the build mount row, and adjust the intro sentence to acknowledge this intentional exception.

[garciadias]

Correct pin of the dcm2niix image, but one operational gap blocks merge: the GHCR package visibility flip is a manual post-merge step recorded only in the PR body, so the first publish will produce a private image that deployments cannot pull — the exact silent failure this fixes. Please record it in committed docs (and ideally add an unauthenticated manifest probe).

[atriaybagur]

On the PR-level point about the "Non-Root Users" table in deploy/README.md omitting xnat-dcm2niix: added in 7e9db55.

The row records both halves of why it is an exception rather than an oversight:

• Deliberately root. The image keeps the base image's root default to match the output-file ownership the previous xnat/dcm2niix image produced on the Container Service's build mount — XNAT reads the converted NIfTIs back off that mount, so changing the writing UID here changes what XNAT finds. The Dockerfile already carried that rationale in a comment; the table now states it where the other services' user choices are listed.
• Outside the cap_drop regime. It is not a compose service. The Container Service launches it per archived scan and reaps it, so it never passes through the compose files that impose cap_drop: [ALL] on everything else — which is why it does not appear in the capability section either.

Also worth flagging for the merge itself, since it came up in the visibility thread: ghcr.io/londonaicentre/xnat-dcm2niix is currently not anonymously pullable — a credential-less probe of the token endpoint returns 403 today. The new final step in the publish workflow will therefore fail on the first post-merge run until the package visibility is flipped to public. That red run is the intended signal rather than a broken build, and it clears on a re-run once the flip is done; the procedure is in trust/xnat/README.md under "Operator action: the GHCR package must be public".

[garciadias]

Verified the fix is complete: grepping the full PR-head tree confirms no leftover xnat/dcm2niix:latest references anywhere in code, config, tests, or docs, and all four pin sites (Dockerfile ARG, dcm2niix_command.json, K8s init-job, imaging_api/config.py) agree on v1.0.20260724, enforced in sync by the new check_image_pin_sync.sh. Manual verification (re-converting the previously slice-dropping accession 60/60, a control series 92/92, plus a full live platform pathway test) is concrete and convincing.

Left two comments on the new publish workflow: it has no pull_request trigger, so a broken Dockerfile/checksum on a future version bump would only surface after merging to develop -- the same "nobody notices" pattern this PR fixes for the pin itself -- plus a small dead-code leftover in the tag-computation step.