Skip to content
Open
Show file tree
Hide file tree
Changes from 51 commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
b19e7be
feat(fusion-doctor): Add Python report generator and HTML template
alberto-miranda Feb 24, 2026
05b3ff7
test(fusion-doctor): Add unit tests for report generation
alberto-miranda Feb 24, 2026
ab418fb
feat(fusion-doctor): Add `FUSION_DOCTOR_GENERATE_REPORT` process
alberto-miranda Feb 24, 2026
b150c34
chore: Update test snapshots
alberto-miranda Feb 25, 2026
02a8dad
ci: Integrate Python tests in CI pipelines
alberto-miranda Feb 25, 2026
fc6768e
fix: Make `fusion_report_template.html` conformant with `prettier`
alberto-miranda Feb 25, 2026
33ea648
fix: Add `uv` container for `FUSION_DOCTOR_GENERATE_REPORT`
alberto-miranda Feb 25, 2026
8d0d96d
fix: Replace `uv:0.10.6` container with `uv:python3.12-bookworm-slim`
alberto-miranda Feb 25, 2026
e814e44
ci: Add `setup-uv` to `nf-test` job
alberto-miranda Feb 25, 2026
53afe7e
fix: Replace `uv:python3.12-bookworm-slim` with `jinja2_python_uv:711…
alberto-miranda Feb 25, 2026
047347b
chore: Rename HTML report to `fusion-report.html`
alberto-miranda Feb 27, 2026
09220c1
feat(fusion-doctor): Split profiles into tiers
alberto-miranda Mar 4, 2026
676f39c
fix: Stage report template file as an input
alberto-miranda Mar 4, 2026
6b77f7f
fix: Ensure `uv` does not check for project files
alberto-miranda Mar 4, 2026
5e8a31e
fix: Pass template file as parameter instead of copying to fixed loca…
alberto-miranda Mar 4, 2026
f3a7b72
fix: Handle real fusion doctor JSON schema in HTML report
alberto-miranda Mar 4, 2026
d581249
fix: Fix report generation
alberto-miranda Mar 4, 2026
9ee58b6
feat: Enhance Fusion diagnostic report with system, storage, and reso…
alberto-miranda Mar 4, 2026
f045e99
feat: Redesign fusion report with Platform design system and polish
alberto-miranda Mar 5, 2026
0e37e90
fix: Add humanized labels for new fusion doctor checks
alberto-miranda Mar 5, 2026
a173a18
feat: Add nvme, cpu_cores, and open_files checks to fusion profiles
alberto-miranda Mar 5, 2026
0398577
feat: validate single Fusion profile usage for TEST_FUSION_DOCTOR
alberto-miranda Mar 5, 2026
6b74eb6
fix: Update report names in tests
alberto-miranda Mar 5, 2026
c9af456
feat: Dark header matching Seqera Platform navigation bar
alberto-miranda Mar 5, 2026
aae734a
feat: Color-coded overview cards and status badge in overview grid
alberto-miranda Mar 5, 2026
a359e5c
feat: Convert validation checks from cards to table with collapsible …
alberto-miranda Mar 5, 2026
e99b57a
fix: Design critique polish for v4 report template
alberto-miranda Mar 5, 2026
c785947
feat: Allow TEST_FUSION_DOCTOR to continue with exit codes 1 and 3
alberto-miranda Mar 5, 2026
377a1e8
feat: Redesign fusion report with Platform design system
alberto-miranda Mar 6, 2026
6ab31b1
feat: Improve accessibility and template structure
alberto-miranda Mar 6, 2026
b3d77bb
feat: Add mobile responsive layout
alberto-miranda Mar 6, 2026
2d1423e
feat: Split bucket table into access + URI columns
alberto-miranda Mar 6, 2026
01f4a27
feat: Add footer and improve visual polish
alberto-miranda Mar 6, 2026
0fa6260
refactor: Simplify, deduplicate, and remove dead code
alberto-miranda Mar 7, 2026
2db3600
test: Improve Python tests
alberto-miranda Mar 8, 2026
d00005d
fix: Exclude Jinja2 templates from Prettier checks
alberto-miranda Mar 8, 2026
93145be
fix: Run Python tests via pytest in CI to fix module import
alberto-miranda Mar 8, 2026
b41373c
feat(fusion): Allow TEST_FUSION_DOCTOR to proceed on validation failures
alberto-miranda Mar 8, 2026
9f08932
fix(fusion): Improve report layout and resource limits display
alberto-miranda Mar 9, 2026
26a60ea
fix(fusion): Lower open_files_min to 65535 in low-tier profiles
alberto-miranda Mar 9, 2026
acfd727
feat(fusion): Add example doctor report fixtures and generator script
alberto-miranda Mar 9, 2026
bffe20e
refactor(fusion): Remove duplicate details from resource limits table
alberto-miranda Mar 9, 2026
90b8b8c
feat(fusion): Add `fusion_redact` param to mask PII in diagnostics
alberto-miranda Mar 9, 2026
0766a8e
chore: Leave profile handling to Nextflow
alberto-miranda Mar 10, 2026
18f2a2d
chore: Replace horizontal dividers with subheadings
alberto-miranda Mar 10, 2026
f887634
fix: Clarify NVMe auto-selection for AWS CEs
alberto-miranda Mar 10, 2026
01ccfe5
fix: Make path mandatory in `load_json_report`
alberto-miranda Mar 10, 2026
0e7735d
refactor(tests): Convert fusion doctor report test to process-level
alberto-miranda Mar 10, 2026
66e90f8
chore: Remove example report fixtures
alberto-miranda Mar 10, 2026
8c38d40
refactor: Remove try/catch from `load_json_report`, catch at `main()`…
alberto-miranda Mar 10, 2026
061ccc3
feat(fusion): add parameter sweep for fusion doctor validation
adamrtalbot Mar 12, 2026
1e4a587
Apply suggestions from code review
adamrtalbot Apr 28, 2026
5e234d4
Merge origin/main: integrate dedicated fusion-doctor container and HT…
Copilot Apr 28, 2026
3116670
chore: initial plan for merge conflict resolution
Copilot Jul 22, 2026
996f576
fix: resolve merge conflicts with origin/main
Copilot Jul 22, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,13 +33,28 @@ jobs:
- name: Run Prettier --check
run: prettier --check .

python-tests:
runs-on: ubuntu-latest
steps:
- name: Check out repository
uses: actions/checkout@v3

- name: Install uv
uses: astral-sh/setup-uv@v3

- name: Run generate_report tests
run: uv run --no-project --with "pytest==9.0.2" --with "jinja2==3.1.6" --with "humanize==4.15.0" --with "markupsafe" python -m pytest tests/ -v

nf-test:
runs-on: ubuntu-latest
env:
NXF_ANSI_LOG: false
steps:
- uses: actions/checkout@v3

- name: Install uv
uses: astral-sh/setup-uv@v3

- uses: actions/setup-java@v3
with:
distribution: "temurin"
Expand Down
1 change: 1 addition & 0 deletions .prettierignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,5 @@ results/
*.test
*.test.snap
*.txt
assets/templates/

60 changes: 50 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,25 +143,65 @@ This process tests the ability to use a GPU. It uses the `pytorch` conda environ

_Note: Enabled only if the parameter `--fusion` is specified (or set to `true` by a profile)._

This process runs `fusion doctor` to validate the Fusion filesystem configuration. It checks system requirements (kernel version, memory, and disk space), FUSE device availability, and cloud bucket accessibility. The process produces a JSON diagnostic report published to `${outdir}/fusion/`.

This test requires the `fusion` binary to be available in the task environment and will fail if it is not found.
This process runs the `fusion doctor` diagnostics tool to validate the Fusion filesystem configuration. It checks system requirements (e.g. kernel version, memory, disk space, etc.), FUSE device availability, and cloud bucket accessibility among others. The process produces a JSON diagnostic report published to `${outdir}/fusion/`.

#### Fusion Profiles

Use a built-in profile to enable Fusion validation with recommended thresholds:
Use a built-in profile to enable Fusion validation with predefined thresholds:

```bash
nextflow run seqeralabs/nf-canary -profile fusion_aws_recommended
```

Available profiles:
> [!TIP]
>
> **If in doubt, start with the `recommended` tier:**
>
> 1. Choose `low` for small workloads or quick smoke tests, and `high` for large-scale production with big datasets.
> 2. Choose `recommended` to match Seqera's documented minimum requirements for production workloads with Fusion.
> 3. Choose `high` (or a custom threshold of 400 GB+ storage) if your pipeline processes files larger than 100 GB.

##### AWS

> [!NOTE]
>
> Seqera Platform will auto-select NVMe-based instance families when Fusion is enabled (e.g. `m6id`, `c6id`, `r6id`) and the "Fast instance storage" toggle is active in the CE settings. Fusion can also work without NVMe instances, but in this case the EBS disk shall be bumped to 100 GB (`gp3`, 325 MB/s); this is what the `low` profile validates.

| Profile | Disk | Memory | Kernel | Based on |
| ------------------------ | ------ | ------ | ------ | ------------------- |
| `fusion_aws_low` | 100 GB | 4 GB | 5.10+ | EBS gp3 (no NVMe) |
| `fusion_aws_recommended` | 200 GB | 8 GB | 5.10+ | Seqera docs minimum |
| `fusion_aws_high` | 474 GB | 16 GB | 5.10+ | m6id.2xlarge NVMe |

**Google Cloud**

> [!NOTE]
>
> Seqera Platform auto-selects families supporting local SSDs (e.g. `n2`, `c2`, `n2d`) and provisions a 375 GB NVMe SSD per job.

| Profile | Disk | Memory | Kernel | Based on |
| --------------------------- | ------ | ------ | ------ | ------------------- |
| `fusion_google_low` | 50 GB | 4 GB | 5.15+ | GCP persistent disk |
| `fusion_google_recommended` | 375 GB | 8 GB | 5.15+ | 1x local NVMe SSD |
| `fusion_google_high` | 750 GB | 16 GB | 5.15+ | 2x local NVMe SSDs |

##### Azure

> [!note]
>
> Unlike AWS and Google Cloud, there is no auto-selection: the user must pick the VM size. Seqera recommends E-series with a `d` suffix (e.g. `Standard_E8d_v5`, `Standard_E16d_v5`).

| Profile | Disk | Memory | Kernel | Based on |
| -------------------------- | ------ | ------ | ------ | ---------------------------- |
| `fusion_azure_low` | 75 GB | 4 GB | 5.15+ | `Standard_E2d_v5` temp disk |
| `fusion_azure_recommended` | 300 GB | 8 GB | 5.15+ | `Standard_E8d_v5` temp disk |
| `fusion_azure_high` | 600 GB | 16 GB | 5.15+ | `Standard_E16d_v5` temp disk |

#### Caveats

In AWS Batch, the Seqera Platform UI allows selecting instance families (e.g. `m6id`) but not specific sizes. Small instances in an otherwise valid family may not meet the `recommended` disk threshold. For example, on AWS the `m6id` family NVMe ranges from 118 GB (`.large`) to 1,900 GB (`.8xlarge`), with only `.xlarge` and above meeting the 200 GB threshold.

| Profile | Description |
| --------------------------- | ----------------------------------------- |
| `fusion_aws_recommended` | AWS Batch recommended thresholds |
| `fusion_google_recommended` | Google Cloud Batch recommended thresholds |
| `fusion_azure_recommended` | Azure Batch recommended thresholds |
If `fusion doctor` reports a disk requirement failure, request more CPUs/memory to get a larger instance, or use the `low` profile for small tasks.
Comment thread
adamrtalbot marked this conversation as resolved.
Outdated

#### Custom Requirements

Expand Down
Loading