Skip to content

Harden data-pipeline reliability and CLI failure modes - #705

Open
Ayush7614 wants to merge 1 commit into
google-deepmind:mainfrom
Ayush7614:fix/pipeline-reliability-and-cli-guardrails
Open

Harden data-pipeline reliability and CLI failure modes#705
Ayush7614 wants to merge 1 commit into
google-deepmind:mainfrom
Ayush7614:fix/pipeline-reliability-and-cli-guardrails

Conversation

@Ayush7614

@Ayush7614 Ayush7614 commented Jul 28, 2026

Copy link
Copy Markdown

Summary

  • Fail early with clear errors when HMMER binaries are missing from PATH (None previously caused a cryptic TypeError in os.path.exists), when --jax_backend=gpu finds no GPUs / an out-of-range --gpu_device, and when --input_dir contains no JSON inputs.
  • Always clean up sharded Jackhmmer/Nhmmer temp directories on failure (try/finally), instead of leaking large temp trees after shard errors.
  • Share a single cached StructureStore per PDB path and add close() / context-manager support so tar-backed stores do not accumulate open file handles across sequences.
  • Make Hmmsearch CPU count configurable (--hmmsearch_n_cpu, default min(cpu_count, 8)) instead of hardcoding --cpu 8; raise Jackhmmer/Nhmmer CPU lower bounds from 0 to 1.
  • Add unit tests for binary-path checks, StructureStore lifecycle, and Hmmsearch CPU flag wiring (pipeline_reliability_test.py, 8 tests, all passing).

Why

These are independent reliability gaps that hurt local installs and batch/sharded runs. None of them are covered by existing open/closed issues; they improve real failure modes and resource hygiene without changing model behavior.

Test plan

  • python src/alphafold3/data/pipeline_reliability_test.py — 8/8 passed
  • Run with HMMER unset on PATH and --run_data_pipeline=true — expect clear binary-missing error
  • Run --jax_backend=gpu with no GPU — expect clear no-device error
  • Interrupt/fail a sharded Jackhmmer query — confirm temp dir is removed
  • Confirm --hmmsearch_n_cpu=N is forwarded to the hmmsearch command line

Fail early on missing HMMER binaries, missing GPUs, and empty input
dirs; clean up sharded-search temp dirs on failure; share/close PDB
structure stores; and make Hmmsearch CPU count configurable.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant