Skip to content

fix(alias_table): walk parent groups iteratively and stop at the file root - #55

Merged
ggmarshall merged 2 commits into
mainfrom
fix/alias-table-recursion
Sep 6, 2026
Merged

ggmarshall merged 2 commits into
mainfrom
fix/alias-table-recursion

Conversation

@ggmarshall

Copy link
Copy Markdown
Contributor

The bug

convert_parents_to_structs recursed on h5group.parent — but in h5py the root group is its own parent, so the walk could only terminate via the branch requiring / to have zero attributes. Real LH5 files always carry a datatype attribute on the root, so every alias write into a new top-level group (dsp/<detector>, hit/<detector>, …) recursed until RecursionError after ~1000 frames. At the root, name.split('/')[-1] is '', which the field filter strips, so the membership early-return never fired and each pass appended an empty field — and because the file is open in append mode those partial writes are flushed, leaving a crashed file's root datatype corrupted as struct{...,dsp,,,…}.

Every build-tier-dsp/build-tier-hit job dies at the "Creating alias table" step, after the full processing time is already spent. In the v3.5.0dev1 production (which installs the released 0.4.0a1) this currently accounts for 1354 failed dsp jobs (p19 cal + fft).

The fix

Rewrite the walk iteratively with an explicit stop at the root. Deliberately preserved: the root's datatype is still extended with new top-level members (LGDO readers need it); a root with no datatype is left untouched; the early return when the child is already a member; the non-struct-parent ValueError and substring-safe field parsing. Incidental hardening: keying on 'datatype' in attrs avoids a KeyError on parents carrying only other attributes.

Regression test included (test_alias_table_annotated_root) reproducing the production file shape — a root group that already carries datatype — which is exactly the case the existing test fixture missed. Verified on real p18 data: the produced dsp file's root gains dsp exactly once with no empty fields, and all 135 detector aliases resolve as hard links readable through both paths.

Cherry-picked from integration-fable (973d031); 18/18 tests pass on current main.

🤖 Generated with Claude Code

… root

convert_parents_to_structs recursed on h5group.parent, but in h5py the
root group is its own parent, so a file whose root already carries a
datatype attribute sent the walk into infinite recursion. Rewrite the
walk as a loop that terminates at '/', never invents a datatype on the
root, and stops early when the child is already a registered member.
Add a test aliasing into a new top-level group of a file with an
annotated root.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit 973d031)
Copilot AI lite review requested due to automatic review settings September 6, 2026 19:23

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The iterative root-terminating walk addresses the documented recursion failure mode and is covered by a targeted regression test reproducing the production file shape.

Pull request overview

Fixes an HDF5 alias-table crash caused by convert_parents_to_structs recursing forever at the HDF5 root (in h5py the root group is its own parent), by switching to an iterative parent-walk with an explicit stop at / while preserving the prior “only update root if it already has datatype” behavior.

Changes:

  • Rewrite convert_parents_to_structs to walk parent groups iteratively and terminate reliably at the file root.
  • Harden parent handling by checking for the presence of the datatype attribute (avoids KeyError when parents have other attrs).
  • Add regression test covering the real-world LH5 shape where the root already carries a datatype struct.
File summaries
File Description
tests/test_utils_helpers.py Adds regression test asserting aliasing into a new top-level group correctly updates / exactly once and avoids empty members.
src/legenddataflowscripts/utils/alias_table.py Replaces recursive parent traversal with an iterative loop that stops at /, preventing infinite recursion and root corruption.
Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@ggmarshall
ggmarshall merged commit 4e18269 into main Sep 6, 2026
10 checks passed
@ggmarshall
ggmarshall deleted the fix/alias-table-recursion branch September 6, 2026 19:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants