From 8bdb3ece087228827852929dad0017b560735f97 Mon Sep 17 00:00:00 2001 From: apooladi Date: Fri, 7 Aug 2026 15:30:09 -0400 Subject: [PATCH 1/4] Add LANTERN 5-fold deep-learning plaque segmentation Adds an opt-in alternative to the GMM/N4 plaque segmentation: the LANTERN ResEncL-UNet ensemble (apooladi/lantern-ki3-abeta), selected with --seg_method lantern. The model reads raw intensities -- it was never trained on N4-corrected data -- so this path bypasses the bias-field chain entirely and reads the SPIM input directly, as run_vesselfm does. It runs at plaque_level (1), the grid it was built for; the segmentation_level mask is that prediction upsampled by exact voxel replication rather than native inference at level 0. Two outputs: a level-1 float32 probability map holding the fraction of folds voting foreground ({0, .2, .4, .6, .8, 1} for five folds, the same form as LANTERN's own whole-brain probmask), and the level-0 binary mask valued 0/100 to match the segmentation.smk convention, so fieldfrac / regionprops / counts / segstats / heatmaps / QC all consume it with no changes to those rules. Writing the probability map unthresholded means the vote threshold can be changed without re-running the GPU; binarize compares against the midpoint between attainable values so float representation cannot flip a boundary voxel. ZarrNii.segment() could not be used: it dispatches through da.map_blocks with disjoint blocks, no block position and a hardcoded uint8 output, so it can express neither the stride-64 tile overlap the model requires nor brain-mask block skipping. da.map_overlap is driven directly instead, with a halo of tile - stride, brain restriction via the desc-brain mask (the n4_biasfield precedent), and chunks laid out so no chunk is shorter than the halo. Like vessels, the method is resolved to a single stain rather than fanned out: it is trained on amyloid-beta only, so it stays out of the generic seg_method expansions and the cross-stain aggregation the way vessel_seg_method does. Remaining stains fall back to the --seg_method default, read off the CLI definition so it cannot drift. Verified: against a run that never mentions lantern, the job plan differs only by the added lantern jobs. The architecture is vendored (lantern_model.py) rather than depended on, and loads the published checkpoints with zero missing/unexpected keys. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_018WbbraRDoezJrijVaAsL97 --- pixi.lock | 536 +++++++++++++++++- pyproject.toml | 1 + spimquant/config/snakebids.yml | 43 ++ spimquant/workflow/Snakefile | 139 ++++- spimquant/workflow/rules/plaques.smk | 128 +++++ .../scripts/binarize_lantern_plaques.py | 85 +++ spimquant/workflow/scripts/lantern_model.py | 73 +++ spimquant/workflow/scripts/lantern_plaques.py | 349 ++++++++++++ 8 files changed, 1316 insertions(+), 38 deletions(-) create mode 100644 spimquant/workflow/rules/plaques.smk create mode 100644 spimquant/workflow/scripts/binarize_lantern_plaques.py create mode 100644 spimquant/workflow/scripts/lantern_model.py create mode 100644 spimquant/workflow/scripts/lantern_plaques.py diff --git a/pixi.lock b/pixi.lock index 63a4d91..71705db 100644 --- a/pixi.lock +++ b/pixi.lock @@ -6,8 +6,6 @@ environments: - url: https://conda.anaconda.org/bioconda/ indexes: - https://pypi.org/simple - options: - pypi-prerelease-mode: if-necessary-or-explicit packages: linux-64: - conda: https://conda.anaconda.org/conda-forge/linux-64/_openmp_mutex-4.5-7_kmp_llvm.conda @@ -603,6 +601,8 @@ environments: - conda: https://conda.anaconda.org/conda-forge/linux-64/zstandard-0.25.0-py312h5253ce2_1.conda - conda: https://conda.anaconda.org/conda-forge/linux-64/zstd-1.5.7-hb78ec9c_6.conda - pypi: https://files.pythonhosted.org/packages/6f/4b/817da308fa1170da07ef01259585887a3bbb6ab80700b3e61ce4967301ec/dask_image-2025.11.0-py3-none-any.whl + - pypi: https://files.pythonhosted.org/packages/2f/e3/07a70c9b83e5446528dd5d222fdca88ab30efcb07094536a864546c0096f/dynamic_network_architectures-0.4.4-py3-none-any.whl + - pypi: https://files.pythonhosted.org/packages/2a/09/f8d8f8f31e4483c10a906437b4ce31bdf3d6d417b73fe33f1a8b59e34228/einops-0.8.2-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/9e/e9/1a19e42cd43cc1365e127db6aae85e1c671da1d9a5d746f4d34a50edb577/h5py-3.16.0-cp312-cp312-manylinux_2_28_x86_64.whl - pypi: https://files.pythonhosted.org/packages/ed/2f/1046d464ad1db29a4f6c70ba4e19b39baa8a6542c719eaa4e765108f07f1/hdf5plugin-6.0.0-py3-none-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl - pypi: https://files.pythonhosted.org/packages/c0/47/7dbdec41b18f4c9aa90240c8804482a8c4ada6db3d2844e287695cd42109/imaris_ims_zarr-0.2.0-py3-none-any.whl @@ -623,9 +623,12 @@ environments: - pypi: https://files.pythonhosted.org/packages/f6/17/7468d6cb45b0c537b235bb910812bb248c581a50ffc5eeaeba342f6fdf2d/pylibtiff-0.7.0.tar.gz - pypi: https://files.pythonhosted.org/packages/2f/43/d7e2b9ad768c07b5473bea3ac7db9ca4d995c09399cbea3d4df1c0bd4955/rangehttpserver-1.4.0-py2.py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/0b/35/1cceccc5fcb50fa2ed53e2aa278cd032f3902682a73e763fb1ac3be8e6fa/rich_argparse-1.8.0-py3-none-any.whl + - pypi: https://files.pythonhosted.org/packages/28/50/f203ff3a3ddfe19308efc83c5a3a29ed02bf786732ec35e68bf9162f3365/safetensors-0.8.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl - pypi: https://files.pythonhosted.org/packages/c3/5d/5e017f59bbc06af5b9c78331036478b2ddb371d7ff810c559c36e5389904/simpleitk-2.5.4-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl - pypi: https://files.pythonhosted.org/packages/e8/ae/fa6cd331b364ad2bbc31652d025f5747d89cbb75576733dfdf8efe3e4d62/slicerator-1.1.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/d1/14/7688e984cdd0be1438779825640b943574b89946ed868d76497b3cffb3d5/tensorstore-0.1.83-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl + - pypi: https://files.pythonhosted.org/packages/d6/14/fc04d491527b774ec7479897f5861959209de1480e4c4cd32ed098ff8bea/timm-1.0.22-py3-none-any.whl + - pypi: https://files.pythonhosted.org/packages/17/8b/155f99042f9319bd7759536779b2a5b67cbd4f89c380854670850f89a2f4/torchvision-0.22.1-cp312-cp312-manylinux_2_28_x86_64.whl - pypi: git+https://github.com/khanlab/vesselfm?rev=504baab#504baabb253c2b6bf7d5a4037a93c2f590c9da51 - pypi: https://files.pythonhosted.org/packages/c5/c1/9c64ed6c2d152b56482bb66297bfc03442d25449bfa91da7bf2e58a80f01/wasmtime-44.0.0-py3-none-manylinux1_x86_64.whl - pypi: https://files.pythonhosted.org/packages/bd/4d/5e4df18906ddeb6e64af6712e72056c73ee926d402c0a5fd16feb72f6fe9/zarrnii-0.21.2-py3-none-any.whl @@ -636,8 +639,6 @@ environments: - url: https://conda.anaconda.org/bioconda/ indexes: - https://pypi.org/simple - options: - pypi-prerelease-mode: if-necessary-or-explicit packages: linux-64: - conda: https://conda.anaconda.org/conda-forge/linux-64/_openmp_mutex-4.5-7_kmp_llvm.conda @@ -1410,6 +1411,8 @@ environments: - conda: https://conda.anaconda.org/conda-forge/linux-64/zstandard-0.25.0-py312h5253ce2_1.conda - conda: https://conda.anaconda.org/conda-forge/linux-64/zstd-1.5.7-hb78ec9c_6.conda - pypi: https://files.pythonhosted.org/packages/6f/4b/817da308fa1170da07ef01259585887a3bbb6ab80700b3e61ce4967301ec/dask_image-2025.11.0-py3-none-any.whl + - pypi: https://files.pythonhosted.org/packages/d8/ae/b8a29e6dd366327530390464bf7eee0e3b3bcb03deea98b1a6951edb68ba/dynamic_network_architectures-0.4.2.tar.gz + - pypi: https://files.pythonhosted.org/packages/2a/09/f8d8f8f31e4483c10a906437b4ce31bdf3d6d417b73fe33f1a8b59e34228/einops-0.8.2-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/ed/2f/1046d464ad1db29a4f6c70ba4e19b39baa8a6542c719eaa4e765108f07f1/hdf5plugin-6.0.0-py3-none-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl - pypi: https://files.pythonhosted.org/packages/c0/47/7dbdec41b18f4c9aa90240c8804482a8c4ada6db3d2844e287695cd42109/imaris_ims_zarr-0.2.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/aa/57/0a3499479cb19d7e4b7fc38b2ba15c0ea20e1e88cb2b634b75dd3e9ff8b8/itk_core-5.4.6-cp311-abi3-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl @@ -1440,8 +1443,6 @@ environments: - url: https://conda.anaconda.org/bioconda/ indexes: - https://pypi.org/simple - options: - pypi-prerelease-mode: if-necessary-or-explicit packages: linux-64: - conda: https://conda.anaconda.org/conda-forge/linux-64/_openmp_mutex-4.5-20_gnu.conda @@ -1677,18 +1678,27 @@ environments: - pypi: https://files.pythonhosted.org/packages/57/d8/8d44036d7eb7b6a8ec4c5494ea0c8c8b94fbc0ed3991c1a7adf230df03bf/aiohttp-3.13.5-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl - pypi: https://files.pythonhosted.org/packages/10/a1/510b0a7fadc6f43a6ce50152e69dbd86415240835868bb0bd9b5b88b1e06/aioitertools-0.13.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/fb/76/641ae371508676492379f16e2fa48f4e2c11741bd63c48be4b12a6b09cba/aiosignal-1.4.0-py3-none-any.whl + - pypi: https://files.pythonhosted.org/packages/3e/30/e900b21425a860e195f32e37657aa1f7c7f2b1bfb26f03ca209b90933c06/annotated_doc-0.0.5-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/bf/4b/afc1fef8a43bafb139f57f73bbd70df82807af5934321e8112ae50668827/botocore-1.43.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/88/39/799be3f2f0f38cc727ee3b4f1445fe6d5e4133064ec2e4115069418a5bb6/cloudpickle-3.1.2-py3-none-any.whl + - pypi: https://files.pythonhosted.org/packages/39/2a/6d2e9047d1fb243dbaa364b01e0297534b9ed7fd27dba1c9f361519cf69b/cuda_bindings-13.3.1-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl + - pypi: https://files.pythonhosted.org/packages/fc/b4/d088047afe39827556df21118cac9ffd20cc3f968c99a7681494d1eb333c/cuda_pathfinder-1.6.0-py3-none-any.whl + - pypi: https://files.pythonhosted.org/packages/d1/c7/a79086a62c98befcdb8349656c6f114e2db3b8b2422f6e25c97a7f2a9a3c/cuda_toolkit-13.0.3.0-py2.py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/4a/f3/00bb1e867fba351e2d784170955713bee200c43ea306c59f30bd7e748192/dask-2026.3.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/6f/4b/817da308fa1170da07ef01259585887a3bbb6ab80700b3e61ce4967301ec/dask_image-2025.11.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/84/d0/205d54408c08b13550c733c4b85429e7ead111c7f0014309637425520a9a/deprecated-1.3.1-py2.py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/b9/77/2ff7aefc09cf1306a81cd7a46af34f80ebefef81a2e8329b94b58ad813ae/distributed-2026.3.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/0c/d5/c5db1ea3394c6e1732fb3286b3bd878b59507a8f77d32a2cebda7d7b7cd4/donfig-0.8.1.post1-py3-none-any.whl + - pypi: https://files.pythonhosted.org/packages/2f/e3/07a70c9b83e5446528dd5d222fdca88ab30efcb07094536a864546c0096f/dynamic_network_architectures-0.4.4-py3-none-any.whl + - pypi: https://files.pythonhosted.org/packages/2a/09/f8d8f8f31e4483c10a906437b4ce31bdf3d6d417b73fe33f1a8b59e34228/einops-0.8.2-py3-none-any.whl + - pypi: https://files.pythonhosted.org/packages/c1/e8/72f8cef9fdfeffe06213fe8508039396ee48daa0e3259457ed766173bfd6/filelock-3.32.2-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/6a/bd/d91c5e39f490a49df14320f4e8c80161cfcce09f1e2cde1edd16a551abb3/frozenlist-1.8.0-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl - pypi: https://files.pythonhosted.org/packages/d5/0c/043d5e551459da400957a1395e0febbf771446ff34291afcbe3d8be2a279/fsspec-2026.4.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/ce/a9/a780cc66f86335a6019f557a8aaca8fbb970728f0efd2430d15ff1beae0e/google_crc32c-1.8.0-cp312-cp312-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl - pypi: https://files.pythonhosted.org/packages/9e/e9/1a19e42cd43cc1365e127db6aae85e1c671da1d9a5d746f4d34a50edb577/h5py-3.16.0-cp312-cp312-manylinux_2_28_x86_64.whl - pypi: https://files.pythonhosted.org/packages/ed/2f/1046d464ad1db29a4f6c70ba4e19b39baa8a6542c719eaa4e765108f07f1/hdf5plugin-6.0.0-py3-none-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl + - pypi: https://files.pythonhosted.org/packages/67/4e/a28359bf1c1ecf11eba22123168c138698f7cb576ac678f5a2e16cd5da08/hf_xet-1.6.0-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl + - pypi: https://files.pythonhosted.org/packages/49/79/621a7dbb80c70974f73a597275351ebe03ce5bc65cb5f8f4acb5859252bc/huggingface_hub-1.16.1-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/c0/47/7dbdec41b18f4c9aa90240c8804482a8c4ada6db3d2844e287695cd42109/imaris_ims_zarr-0.2.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/8a/db/55a262f3606bebcae07cc14095338471ad7c0bbcaa37707e6f0ee49725b7/importlib_resources-7.1.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/aa/57/0a3499479cb19d7e4b7fc38b2ba15c0ea20e1e88cb2b634b75dd3e9ff8b8/itk_core-5.4.6-cp311-abi3-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl @@ -1706,11 +1716,27 @@ environments: - pypi: https://files.pythonhosted.org/packages/b3/81/4da04ced5a082363ecfa159c010d200ecbd959ae410c10c0264a38cac0f5/markdown_it_py-4.2.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/b3/38/89ba8ad64ae25be8de66a6d463314cf1eb366222074cfda9ee839c56a4b4/mdurl-0.1.2-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/3a/cb/28ce52eb94390dda42599c98ea0204d74799e4d8047a0eb559b6fd648056/ml_dtypes-0.5.4-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl + - pypi: https://files.pythonhosted.org/packages/43/e3/7d92a15f894aa0c9c4b49b8ee9ac9850d6e63b03c9c32c0367a13ae62209/mpmath-1.3.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/65/92/a5100f7185a800a5d29f8d14041f61475b9de465ffcc0f3b9fba606e4505/msgpack-1.1.2-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl - pypi: https://files.pythonhosted.org/packages/f3/8d/5e5be3ced1d12966fefb5c4ea3b2a5b480afcea36406559442c6e31d4a48/multidict-6.7.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl - pypi: https://files.pythonhosted.org/packages/48/c4/7748c764c297c2469f26902fa70d72dbf026219e2a7a5ce2dc53fc519eff/ngff_zarr-0.34.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/3b/d7/601b6396b33536811668935faa790112266c70661be94555999be431f86f/nibabel-5.4.2-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/fb/53/78c98ef5c8b2b784453487f3e4d6c017b20747c58b470393e230c78d18e8/numcodecs-0.16.5-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl + - pypi: https://files.pythonhosted.org/packages/3b/cd/154ca20c38269e05eff77c1464e6c1da89f50a6390b565e9d82e06bc11e1/nvidia_cublas-13.1.1.3-py3-none-manylinux_2_27_x86_64.whl + - pypi: https://files.pythonhosted.org/packages/33/6d/737d164b4837a9bbd202f5ae3078975f0525a55730fe871d8ed4e3b952b0/nvidia_cuda_cupti-13.0.85-py3-none-manylinux_2_25_x86_64.whl + - pypi: https://files.pythonhosted.org/packages/c3/68/483a78f5e8f31b08fb1bb671559968c0ca3a065ac7acabfc7cee55214fd6/nvidia_cuda_nvrtc-13.0.88-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl + - pypi: https://files.pythonhosted.org/packages/2e/24/d1558f3b68b1d26e706813b1d10aa1d785e4698c425af8db8edc3dced472/nvidia_cuda_runtime-13.0.96-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl + - pypi: https://files.pythonhosted.org/packages/6e/5e/edb9c0ae051602c3ccaffe424256463636d639e27d7f302dde9975ef9e7a/nvidia_cudnn_cu13-9.20.0.48-py3-none-manylinux_2_27_x86_64.whl + - pypi: https://files.pythonhosted.org/packages/a8/2f/7b57e29836ea8714f81e9898409196f47d772d5ddedddf1592eadb8ab743/nvidia_cufft-12.0.0.61-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl + - pypi: https://files.pythonhosted.org/packages/3f/70/4f193de89a48b71714e74602ee14d04e4019ad36a5a9f20c425776e72cd6/nvidia_cufile-1.15.1.6-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl + - pypi: https://files.pythonhosted.org/packages/a5/9f/be0a41ca4a4917abf5cb9ae0daff1a6060cc5de950aec0396de9f3b52bc5/nvidia_curand-10.4.0.35-py3-none-manylinux_2_27_x86_64.whl + - pypi: https://files.pythonhosted.org/packages/5f/67/cba3777620cdacb99102da4042883709c41c709f4b6323c10781a9c3aa34/nvidia_cusolver-12.0.4.66-py3-none-manylinux_2_27_x86_64.whl + - pypi: https://files.pythonhosted.org/packages/fa/18/623c77619c31d62efd55302939756966f3ecc8d724a14dab2b75f1508850/nvidia_cusparse-12.6.3.3-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl + - pypi: https://files.pythonhosted.org/packages/34/7d/2661f2fb3ac4302f3a246f5fc030213ac60c1fe0bce84f9783dbd831dbb7/nvidia_cusparselt_cu13-0.8.1-py3-none-manylinux2014_x86_64.whl + - pypi: https://files.pythonhosted.org/packages/67/f4/58e4e91b6919367c7aafb8e36fce9aad1a3047e536bf7e2fd560927d3a4c/nvidia_nccl_cu13-2.29.7-py3-none-manylinux_2_18_x86_64.whl + - pypi: https://files.pythonhosted.org/packages/f0/ee/580ca6f29dcab0221db8706badca1bbbb084f1975c4d4e83329c3a7e31f0/nvidia_nvjitlink-13.3.33-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl + - pypi: https://files.pythonhosted.org/packages/3c/35/a9bf80a609e74e3b000fef598933235c908fcefcef9026042b8e6dfde2a9/nvidia_nvshmem_cu13-3.4.5-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl + - pypi: https://files.pythonhosted.org/packages/c2/f3/d86c845465a2723ad7e1e5c36dcd75ddb82898b3f53be47ebd429fb2fa5d/nvidia_nvtx-13.0.85-py3-none-manylinux1_x86_64.manylinux_2_5_x86_64.whl - pypi: https://files.pythonhosted.org/packages/99/81/4c41331f59e64b2a5eaec1c03819d5791e4dde321694fbda661f873184da/ome_zarr-0.16.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/71/e7/40fb618334dcdf7c5a316c0e7343c5cd82d3d866edc100d98e29bc945ecd/partd-1.4.2-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/b8/02/5bf3639f5b77e9b183011c08541c5039ba3d04f5316c70312b48a8e003a9/pims-0.7.tar.gz @@ -1721,11 +1747,19 @@ environments: - pypi: https://files.pythonhosted.org/packages/82/3b/64d4899d73f91ba49a8c18a8ff3f0ea8f1c1d75481760df8c68ef5235bf5/rich-15.0.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/0b/35/1cceccc5fcb50fa2ed53e2aa278cd032f3902682a73e763fb1ac3be8e6fa/rich_argparse-1.8.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/5d/a4/9d1ea10ebc9e028a289a72fec84da170689549a8102c8aacfcad26bc5035/s3fs-2026.4.0-py3-none-any.whl + - pypi: https://files.pythonhosted.org/packages/28/50/f203ff3a3ddfe19308efc83c5a3a29ed02bf786732ec35e68bf9162f3365/safetensors-0.8.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl + - pypi: https://files.pythonhosted.org/packages/e0/f9/0595336914c5619e5f28a1fb793285925a8cd4b432c9da0a987836c7f822/shellingham-1.5.4-py2.py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/c3/5d/5e017f59bbc06af5b9c78331036478b2ddb371d7ff810c559c36e5389904/simpleitk-2.5.4-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl - pypi: https://files.pythonhosted.org/packages/e8/ae/fa6cd331b364ad2bbc31652d025f5747d89cbb75576733dfdf8efe3e4d62/slicerator-1.1.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/32/46/9cb0e58b2deb7f82b84065f37f3bffeb12413f947f9388e4cac22c4621ce/sortedcontainers-2.4.0-py2.py3-none-any.whl + - pypi: https://files.pythonhosted.org/packages/a2/09/77d55d46fd61b4a135c444fc97158ef34a095e5681d0a6c10b75bf356191/sympy-1.14.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/02/be/5d2d47b1fb58943194fb59dcf222f7c4e35122ec0ffe8c36e18b5d728f0b/tblib-3.2.2-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/d1/14/7688e984cdd0be1438779825640b943574b89946ed868d76497b3cffb3d5/tensorstore-0.1.83-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl + - pypi: https://files.pythonhosted.org/packages/d6/14/fc04d491527b774ec7479897f5861959209de1480e4c4cd32ed098ff8bea/timm-1.0.22-py3-none-any.whl + - pypi: https://files.pythonhosted.org/packages/f3/82/fea946351658e6534db52d2cc12bc53087cbf87f9440c5f180f367c1950b/torch-2.13.0-cp312-cp312-manylinux_2_28_x86_64.whl + - pypi: https://files.pythonhosted.org/packages/93/e4/e9b2495d0d57b9f60d63c57d0a910410a81b4b073bf70917bef815291119/torchvision-0.28.0-cp312-cp312-manylinux_2_28_x86_64.whl + - pypi: https://files.pythonhosted.org/packages/c4/6f/fb96d15db6f36d6eae4cafb998c2e0353bf59d7c4ea1662d7497f269134a/triton-3.7.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl + - pypi: https://files.pythonhosted.org/packages/43/89/9518bc0c3929bee36b3a4a8e3daddd6e03f92f9961c66d4983b837160543/typer-0.27.1-py3-none-any.whl - pypi: git+https://github.com/khanlab/vesselfm?rev=504baab#504baabb253c2b6bf7d5a4037a93c2f590c9da51 - pypi: https://files.pythonhosted.org/packages/c5/c1/9c64ed6c2d152b56482bb66297bfc03442d25449bfa91da7bf2e58a80f01/wasmtime-44.0.0-py3-none-manylinux1_x86_64.whl - pypi: https://files.pythonhosted.org/packages/af/aa/ead46a88f9ec3a432a4832dfedb84092fc35af2d0ba40cd04aea3889f247/wrapt-2.1.2-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl @@ -1740,8 +1774,6 @@ environments: - url: https://conda.anaconda.org/bioconda/ indexes: - https://pypi.org/simple - options: - pypi-prerelease-mode: if-necessary-or-explicit packages: linux-64: - conda: https://conda.anaconda.org/conda-forge/linux-64/_openmp_mutex-4.5-7_kmp_llvm.conda @@ -2365,6 +2397,8 @@ environments: - conda: https://conda.anaconda.org/conda-forge/linux-64/zstandard-0.25.0-py312h5253ce2_1.conda - conda: https://conda.anaconda.org/conda-forge/linux-64/zstd-1.5.7-hb78ec9c_6.conda - pypi: https://files.pythonhosted.org/packages/6f/4b/817da308fa1170da07ef01259585887a3bbb6ab80700b3e61ce4967301ec/dask_image-2025.11.0-py3-none-any.whl + - pypi: https://files.pythonhosted.org/packages/2f/e3/07a70c9b83e5446528dd5d222fdca88ab30efcb07094536a864546c0096f/dynamic_network_architectures-0.4.4-py3-none-any.whl + - pypi: https://files.pythonhosted.org/packages/2a/09/f8d8f8f31e4483c10a906437b4ce31bdf3d6d417b73fe33f1a8b59e34228/einops-0.8.2-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/9e/e9/1a19e42cd43cc1365e127db6aae85e1c671da1d9a5d746f4d34a50edb577/h5py-3.16.0-cp312-cp312-manylinux_2_28_x86_64.whl - pypi: https://files.pythonhosted.org/packages/ed/2f/1046d464ad1db29a4f6c70ba4e19b39baa8a6542c719eaa4e765108f07f1/hdf5plugin-6.0.0-py3-none-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl - pypi: https://files.pythonhosted.org/packages/c0/47/7dbdec41b18f4c9aa90240c8804482a8c4ada6db3d2844e287695cd42109/imaris_ims_zarr-0.2.0-py3-none-any.whl @@ -2385,9 +2419,12 @@ environments: - pypi: https://files.pythonhosted.org/packages/f6/17/7468d6cb45b0c537b235bb910812bb248c581a50ffc5eeaeba342f6fdf2d/pylibtiff-0.7.0.tar.gz - pypi: https://files.pythonhosted.org/packages/2f/43/d7e2b9ad768c07b5473bea3ac7db9ca4d995c09399cbea3d4df1c0bd4955/rangehttpserver-1.4.0-py2.py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/0b/35/1cceccc5fcb50fa2ed53e2aa278cd032f3902682a73e763fb1ac3be8e6fa/rich_argparse-1.8.0-py3-none-any.whl + - pypi: https://files.pythonhosted.org/packages/28/50/f203ff3a3ddfe19308efc83c5a3a29ed02bf786732ec35e68bf9162f3365/safetensors-0.8.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl - pypi: https://files.pythonhosted.org/packages/c3/5d/5e017f59bbc06af5b9c78331036478b2ddb371d7ff810c559c36e5389904/simpleitk-2.5.4-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl - pypi: https://files.pythonhosted.org/packages/e8/ae/fa6cd331b364ad2bbc31652d025f5747d89cbb75576733dfdf8efe3e4d62/slicerator-1.1.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/d1/14/7688e984cdd0be1438779825640b943574b89946ed868d76497b3cffb3d5/tensorstore-0.1.83-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl + - pypi: https://files.pythonhosted.org/packages/d6/14/fc04d491527b774ec7479897f5861959209de1480e4c4cd32ed098ff8bea/timm-1.0.22-py3-none-any.whl + - pypi: https://files.pythonhosted.org/packages/17/8b/155f99042f9319bd7759536779b2a5b67cbd4f89c380854670850f89a2f4/torchvision-0.22.1-cp312-cp312-manylinux_2_28_x86_64.whl - pypi: git+https://github.com/khanlab/vesselfm?rev=504baab#504baabb253c2b6bf7d5a4037a93c2f590c9da51 - pypi: https://files.pythonhosted.org/packages/c5/c1/9c64ed6c2d152b56482bb66297bfc03442d25449bfa91da7bf2e58a80f01/wasmtime-44.0.0-py3-none-manylinux1_x86_64.whl - pypi: https://files.pythonhosted.org/packages/bd/4d/5e4df18906ddeb6e64af6712e72056c73ee926d402c0a5fd16feb72f6fe9/zarrnii-0.21.2-py3-none-any.whl @@ -2644,6 +2681,11 @@ packages: - pkg:pypi/amply?source=hash-mapping size: 21899 timestamp: 1734603085333 +- pypi: https://files.pythonhosted.org/packages/3e/30/e900b21425a860e195f32e37657aa1f7c7f2b1bfb26f03ca209b90933c06/annotated_doc-0.0.5-py3-none-any.whl + name: annotated-doc + version: 0.0.5 + sha256: 117bac03a25ede5df5440e855b32d556049ca169ead221505badf432fed4b101 + requires_python: '>=3.9' - conda: https://conda.anaconda.org/conda-forge/noarch/annotated-doc-0.0.4-pyhcf101f3_0.conda sha256: cc9fbc50d4ee7ee04e49ee119243e6f1765750f0fd0b4d270d5ef35461b643b1 md5: 52be5139047efadaeeb19c6a5103f92a @@ -4187,6 +4229,17 @@ packages: - pkg:pypi/cryptography?source=hash-mapping size: 2534262 timestamp: 1775637873338 +- pypi: https://files.pythonhosted.org/packages/39/2a/6d2e9047d1fb243dbaa364b01e0297534b9ed7fd27dba1c9f361519cf69b/cuda_bindings-13.3.1-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl + name: cuda-bindings + version: 13.3.1 + sha256: e32d08f71ebcdf00f0f41eab2eb37e8da94c8ed411cc9f7f7a019ce6b34abe3a + requires_dist: + - cuda-pathfinder>=1.4.2 + - cuda-toolkit[nvfatbin,nvjitlink,nvrtc,nvvm]==13.* ; extra == 'all' + - cuda-toolkit[cufile]==13.* ; sys_platform == 'linux' and extra == 'all' + - cuda-toolkit==13.* ; extra == 'all' + - nvidia-cudla==13.* ; platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all' + requires_python: '>=3.10' - conda: https://conda.anaconda.org/conda-forge/linux-64/cuda-crt-tools-12.6.85-ha770c72_0.conda sha256: 83b6f3332a17bc891f2ecdc9b1424658009e37e14e888d0bd0458b6aa4db59a2 md5: 4ab193b5fcdcf8d7b094221e3977a112 @@ -4307,6 +4360,75 @@ packages: purls: [] size: 10880815 timestamp: 1732132210850 +- pypi: https://files.pythonhosted.org/packages/fc/b4/d088047afe39827556df21118cac9ffd20cc3f968c99a7681494d1eb333c/cuda_pathfinder-1.6.0-py3-none-any.whl + name: cuda-pathfinder + version: 1.6.0 + sha256: 1503af579d8379c24bdd65528379bc57039b0455be9f5f9686cf8e473a1fce51 + requires_python: '>=3.10' +- pypi: https://files.pythonhosted.org/packages/d1/c7/a79086a62c98befcdb8349656c6f114e2db3b8b2422f6e25c97a7f2a9a3c/cuda_toolkit-13.0.3.0-py2.py3-none-any.whl + name: cuda-toolkit + version: 13.0.3.0 + sha256: d693caaa261214ddd7dbb60d68e71cbed884e68c2be7509778f3051da0b91c3f + requires_dist: + - nvidia-cublas==13.1.1.3.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') + - nvidia-cuda-cccl==13.0.85.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') + - nvidia-cuda-crt==13.0.88.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') + - nvidia-cuda-culibos==13.0.85.* ; (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') + - nvidia-cuda-cupti==13.0.85.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') + - nvidia-cuda-cuxxfilt==13.0.85.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') + - nvidia-cuda-nvcc==13.0.88.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') + - nvidia-cuda-nvrtc==13.0.88.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') + - nvidia-cuda-opencl==13.0.85.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') + - nvidia-cuda-profiler-api==13.0.85.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') + - nvidia-cuda-runtime==13.0.96.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') + - nvidia-cuda-sanitizer-api==13.0.85.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') + - nvidia-cufft==12.0.0.61.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') + - nvidia-cufile==1.15.1.6.* ; (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') + - nvidia-curand==10.4.0.35.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') + - nvidia-cusolver==12.0.4.66.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') + - nvidia-cusparse==12.6.3.3.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') + - nvidia-npp==13.0.1.2.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') + - nvidia-nvfatbin==13.0.85.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') + - nvidia-nvjitlink>=13.0.88,<14 ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') + - nvidia-nvjpeg==13.0.1.86.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') + - nvidia-nvml-dev==13.0.87.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') + - nvidia-nvptxcompiler==13.0.88.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') + - nvidia-nvtx==13.0.85.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') + - nvidia-nvvm==13.0.88.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') + - nvidia-cuda-cccl==13.0.85.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'cccl') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'cccl') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'cccl') + - nvidia-cuda-crt==13.0.88.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'crt') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'crt') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'crt') + - nvidia-cublas==13.1.1.3.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'cublas') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'cublas') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'cublas') + - nvidia-cuda-nvrtc==13.0.88.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'cublas') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'cublas') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'cublas') + - nvidia-cuda-runtime==13.0.96.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'cudart') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'cudart') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'cudart') + - nvidia-cufft==12.0.0.61.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'cufft') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'cufft') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'cufft') + - nvidia-nvjitlink>=13.0.88,<14 ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'cufft') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'cufft') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'cufft') + - nvidia-cufile==1.15.1.6.* ; (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'cufile') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'cufile') + - nvidia-cuda-culibos==13.0.85.* ; (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'culibos') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'culibos') + - nvidia-cuda-cupti==13.0.85.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'cupti') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'cupti') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'cupti') + - nvidia-curand==10.4.0.35.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'curand') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'curand') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'curand') + - nvidia-cublas==13.1.1.3.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'cusolver') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'cusolver') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'cusolver') + - nvidia-cusolver==12.0.4.66.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'cusolver') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'cusolver') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'cusolver') + - nvidia-cusparse==12.6.3.3.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'cusolver') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'cusolver') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'cusolver') + - nvidia-nvjitlink>=13.0.88,<14 ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'cusolver') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'cusolver') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'cusolver') + - nvidia-cusparse==12.6.3.3.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'cusparse') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'cusparse') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'cusparse') + - nvidia-nvjitlink>=13.0.88,<14 ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'cusparse') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'cusparse') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'cusparse') + - nvidia-cuda-cuxxfilt==13.0.85.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'cuxxfilt') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'cuxxfilt') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'cuxxfilt') + - nvidia-npp==13.0.1.2.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'npp') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'npp') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'npp') + - nvidia-cuda-crt==13.0.88.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'nvcc') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'nvcc') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'nvcc') + - nvidia-cuda-nvcc==13.0.88.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'nvcc') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'nvcc') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'nvcc') + - nvidia-cuda-runtime==13.0.96.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'nvcc') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'nvcc') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'nvcc') + - nvidia-nvvm==13.0.88.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'nvcc') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'nvcc') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'nvcc') + - nvidia-nvfatbin==13.0.85.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'nvfatbin') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'nvfatbin') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'nvfatbin') + - nvidia-nvjitlink>=13.0.88,<14 ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'nvjitlink') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'nvjitlink') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'nvjitlink') + - nvidia-nvjpeg==13.0.1.86.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'nvjpeg') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'nvjpeg') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'nvjpeg') + - nvidia-nvml-dev==13.0.87.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'nvml') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'nvml') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'nvml') + - nvidia-nvptxcompiler==13.0.88.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'nvptxcompiler') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'nvptxcompiler') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'nvptxcompiler') + - nvidia-cuda-nvrtc==13.0.88.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'nvrtc') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'nvrtc') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'nvrtc') + - nvidia-nvtx==13.0.85.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'nvtx') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'nvtx') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'nvtx') + - nvidia-nvvm==13.0.88.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'nvvm') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'nvvm') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'nvvm') + - nvidia-cuda-opencl==13.0.85.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'opencl') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'opencl') + - nvidia-cuda-profiler-api==13.0.85.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'profiler') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'profiler') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'profiler') + - nvidia-cuda-sanitizer-api==13.0.85.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'sanitizer') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'sanitizer') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'sanitizer') - conda: https://conda.anaconda.org/conda-forge/noarch/cuda-version-12.6-h7480c83_3.conda sha256: fd9104d73199040285b6a6ad56322b38af04828fabbac1f5a268a83509358425 md5: 1c8b99e65a4423b1e4ac2e4c76fb0978 @@ -4745,6 +4867,25 @@ packages: - pkg:pypi/dunamai?source=hash-mapping size: 30866 timestamp: 1775597880526 +- pypi: https://files.pythonhosted.org/packages/d8/ae/b8a29e6dd366327530390464bf7eee0e3b3bcb03deea98b1a6951edb68ba/dynamic_network_architectures-0.4.2.tar.gz + name: dynamic-network-architectures + version: 0.4.2 + sha256: f0f4cfa26e99e7c1cc44d3ac48ce802f109c403d109c4e4d813f682fa9d0ab84 + requires_dist: + - torch>=1.6.0a0 + - numpy + - timm + - einops +- pypi: https://files.pythonhosted.org/packages/2f/e3/07a70c9b83e5446528dd5d222fdca88ab30efcb07094536a864546c0096f/dynamic_network_architectures-0.4.4-py3-none-any.whl + name: dynamic-network-architectures + version: 0.4.4 + sha256: 7c51bc6518d5c3ee5449b81e00b4e140bb5fd26d035744c01538cdb47b55751d + requires_dist: + - torch>=1.6.0a0 + - numpy + - timm<1.0.23 + - einops + requires_python: '>=3.8' - conda: https://conda.anaconda.org/conda-forge/noarch/eido-0.2.5-pyhd8ed1ab_0.conda sha256: 6972edd5d9dffccd0afcbec077ef2f3a51abc6cc70060bcc818d02988b70983d md5: 679c2b63523ef75652e5690a39e0f0a2 @@ -4768,6 +4909,11 @@ packages: purls: [] size: 1311364 timestamp: 1773744756289 +- pypi: https://files.pythonhosted.org/packages/2a/09/f8d8f8f31e4483c10a906437b4ce31bdf3d6d417b73fe33f1a8b59e34228/einops-0.8.2-py3-none-any.whl + name: einops + version: 0.8.2 + sha256: 54058201ac7087911181bfec4af6091bb59380360f069276601256a76af08193 + requires_python: '>=3.9' - conda: https://conda.anaconda.org/conda-forge/noarch/email-validator-2.3.0-pyhd8ed1ab_0.conda sha256: c37320864c35ef996b0e02e289df6ee89582d6c8e233e18dc9983375803c46bb md5: 3bc0ac31178387e8ed34094d9481bfe8 @@ -5002,6 +5148,11 @@ packages: purls: [] size: 2195198 timestamp: 1776781721834 +- pypi: https://files.pythonhosted.org/packages/c1/e8/72f8cef9fdfeffe06213fe8508039396ee48daa0e3259457ed766173bfd6/filelock-3.32.2-py3-none-any.whl + name: filelock + version: 3.32.2 + sha256: 87dd94cf281e586d135fa51132b8e3d9a598b316e90377a288663c9321036c82 + requires_python: '>=3.10' - conda: https://conda.anaconda.org/conda-forge/noarch/filelock-3.29.0-pyhd8ed1ab_0.conda sha256: 6b471a18372bbd52bdf32fc965f71de3bc1b5219418b8e6b3875a67a7b08c483 md5: 8fa8358d022a3a9bd101384a808044c6 @@ -6084,6 +6235,13 @@ packages: - pkg:pypi/heapdict?source=hash-mapping size: 10169 timestamp: 1733762704231 +- pypi: https://files.pythonhosted.org/packages/67/4e/a28359bf1c1ecf11eba22123168c138698f7cb576ac678f5a2e16cd5da08/hf_xet-1.6.0-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl + name: hf-xet + version: 1.6.0 + sha256: d62671bb130879cef0ee4c9ebe47a14af6c66ec53e6d84dc15936e5ffdfac82f + requires_dist: + - pytest ; extra == 'tests' + requires_python: '>=3.8' - conda: https://conda.anaconda.org/conda-forge/linux-64/hf-xet-1.5.0-py310hb823017_0.conda noarch: python sha256: b0f16f161bae4bdef033d56678a9eda4d07f5aa300db19d58e5e73acb3028b3d @@ -6165,6 +6323,124 @@ packages: - pkg:pypi/httpx?source=hash-mapping size: 63082 timestamp: 1733663449209 +- pypi: https://files.pythonhosted.org/packages/49/79/621a7dbb80c70974f73a597275351ebe03ce5bc65cb5f8f4acb5859252bc/huggingface_hub-1.16.1-py3-none-any.whl + name: huggingface-hub + version: 1.16.1 + sha256: 64340de934b9ce37857ef85a82de72f5629e8a270f9119eabb12bf495eb53c22 + requires_dist: + - filelock>=3.10.0 + - fsspec>=2023.5.0 + - hf-xet>=1.4.3,<2.0.0 ; platform_machine == 'AMD64' or platform_machine == 'aarch64' or platform_machine == 'amd64' or platform_machine == 'arm64' or platform_machine == 'x86_64' + - httpx>=0.23.0,<1 + - packaging>=20.9 + - pyyaml>=5.1 + - tqdm>=4.42.1 + - typer>=0.20.0 + - typing-extensions>=4.1.0 + - authlib>=1.3.2 ; extra == 'oauth' + - fastapi ; extra == 'oauth' + - httpx ; extra == 'oauth' + - itsdangerous ; extra == 'oauth' + - torch ; extra == 'torch' + - safetensors[torch] ; extra == 'torch' + - toml ; extra == 'fastai' + - fastai>=2.4 ; extra == 'fastai' + - fastcore>=1.3.27 ; extra == 'fastai' + - hf-xet>=1.4.3,<2.0.0 ; extra == 'hf-xet' + - mcp>=1.8.0 ; extra == 'mcp' + - authlib>=1.3.2 ; extra == 'testing' + - fastapi ; extra == 'testing' + - httpx ; extra == 'testing' + - itsdangerous ; extra == 'testing' + - jedi ; extra == 'testing' + - jinja2 ; extra == 'testing' + - pytest>=8.4.2 ; extra == 'testing' + - pytest-cov ; extra == 'testing' + - pytest-env ; extra == 'testing' + - pytest-xdist ; extra == 'testing' + - pytest-vcr ; extra == 'testing' + - pytest-asyncio ; extra == 'testing' + - pytest-rerunfailures<16.0 ; extra == 'testing' + - pytest-mock ; extra == 'testing' + - urllib3<2.0 ; extra == 'testing' + - soundfile ; extra == 'testing' + - pillow ; extra == 'testing' + - numpy ; extra == 'testing' + - duckdb ; extra == 'testing' + - fastapi ; extra == 'testing' + - gradio>=5.0.0 ; extra == 'gradio' + - requests ; extra == 'gradio' + - typing-extensions>=4.8.0 ; extra == 'typing' + - types-pyyaml ; extra == 'typing' + - types-simplejson ; extra == 'typing' + - types-toml ; extra == 'typing' + - types-tqdm ; extra == 'typing' + - types-urllib3 ; extra == 'typing' + - ruff>=0.9.0 ; extra == 'quality' + - mypy==1.15.0 ; extra == 'quality' + - libcst>=1.4.0 ; extra == 'quality' + - ty ; extra == 'quality' + - authlib>=1.3.2 ; extra == 'all' + - fastapi ; extra == 'all' + - httpx ; extra == 'all' + - itsdangerous ; extra == 'all' + - jedi ; extra == 'all' + - jinja2 ; extra == 'all' + - pytest>=8.4.2 ; extra == 'all' + - pytest-cov ; extra == 'all' + - pytest-env ; extra == 'all' + - pytest-xdist ; extra == 'all' + - pytest-vcr ; extra == 'all' + - pytest-asyncio ; extra == 'all' + - pytest-rerunfailures<16.0 ; extra == 'all' + - pytest-mock ; extra == 'all' + - urllib3<2.0 ; extra == 'all' + - soundfile ; extra == 'all' + - pillow ; extra == 'all' + - numpy ; extra == 'all' + - duckdb ; extra == 'all' + - fastapi ; extra == 'all' + - ruff>=0.9.0 ; extra == 'all' + - mypy==1.15.0 ; extra == 'all' + - libcst>=1.4.0 ; extra == 'all' + - ty ; extra == 'all' + - typing-extensions>=4.8.0 ; extra == 'all' + - types-pyyaml ; extra == 'all' + - types-simplejson ; extra == 'all' + - types-toml ; extra == 'all' + - types-tqdm ; extra == 'all' + - types-urllib3 ; extra == 'all' + - authlib>=1.3.2 ; extra == 'dev' + - fastapi ; extra == 'dev' + - httpx ; extra == 'dev' + - itsdangerous ; extra == 'dev' + - jedi ; extra == 'dev' + - jinja2 ; extra == 'dev' + - pytest>=8.4.2 ; extra == 'dev' + - pytest-cov ; extra == 'dev' + - pytest-env ; extra == 'dev' + - pytest-xdist ; extra == 'dev' + - pytest-vcr ; extra == 'dev' + - pytest-asyncio ; extra == 'dev' + - pytest-rerunfailures<16.0 ; extra == 'dev' + - pytest-mock ; extra == 'dev' + - urllib3<2.0 ; extra == 'dev' + - soundfile ; extra == 'dev' + - pillow ; extra == 'dev' + - numpy ; extra == 'dev' + - duckdb ; extra == 'dev' + - fastapi ; extra == 'dev' + - ruff>=0.9.0 ; extra == 'dev' + - mypy==1.15.0 ; extra == 'dev' + - libcst>=1.4.0 ; extra == 'dev' + - ty ; extra == 'dev' + - typing-extensions>=4.8.0 ; extra == 'dev' + - types-pyyaml ; extra == 'dev' + - types-simplejson ; extra == 'dev' + - types-toml ; extra == 'dev' + - types-tqdm ; extra == 'dev' + - types-urllib3 ; extra == 'dev' + requires_python: '>=3.10.0' - conda: https://conda.anaconda.org/conda-forge/noarch/huggingface_hub-1.14.0-pyhd8ed1ab_0.conda sha256: 7a6810ae5ff35c04e7160f748fb536ca2be3ffb0997ef15ac0dbb6db3e4e874d md5: a293e6cbdb2daaa9419c9c3432f10b31 @@ -9944,6 +10220,19 @@ packages: purls: [] size: 491140 timestamp: 1730581373280 +- pypi: https://files.pythonhosted.org/packages/43/e3/7d92a15f894aa0c9c4b49b8ee9ac9850d6e63b03c9c32c0367a13ae62209/mpmath-1.3.0-py3-none-any.whl + name: mpmath + version: 1.3.0 + sha256: a0b2b9fe80bbcd81a6647ff13108738cfb482d481d826cc0e02f5b35e5c88d2c + requires_dist: + - pytest>=4.6 ; extra == 'develop' + - pycodestyle ; extra == 'develop' + - pytest-cov ; extra == 'develop' + - codecov ; extra == 'develop' + - wheel ; extra == 'develop' + - sphinx ; extra == 'docs' + - gmpy2>=2.1.0a4 ; platform_python_implementation != 'PyPy' and extra == 'gmpy' + - pytest>=4.6 ; extra == 'tests' - conda: https://conda.anaconda.org/conda-forge/noarch/mpmath-1.4.1-pyhd8ed1ab_0.conda sha256: 5bbf2f8179ec43d34d67ca8e4989d216c1bdb4b749fe6cb40e86ebf88c1b5300 md5: 2e81b32b805f406d23ba61938a184081 @@ -10692,6 +10981,72 @@ packages: - pkg:pypi/numpydoc?source=hash-mapping size: 65801 timestamp: 1764715638266 +- pypi: https://files.pythonhosted.org/packages/3b/cd/154ca20c38269e05eff77c1464e6c1da89f50a6390b565e9d82e06bc11e1/nvidia_cublas-13.1.1.3-py3-none-manylinux_2_27_x86_64.whl + name: nvidia-cublas + version: 13.1.1.3 + sha256: 37936a16db8fe4ac1f065c2139360608a543a09275cb1a1af612e08cfa065436 + requires_dist: + - nvidia-cuda-nvrtc + requires_python: '>=3' +- pypi: https://files.pythonhosted.org/packages/33/6d/737d164b4837a9bbd202f5ae3078975f0525a55730fe871d8ed4e3b952b0/nvidia_cuda_cupti-13.0.85-py3-none-manylinux_2_25_x86_64.whl + name: nvidia-cuda-cupti + version: 13.0.85 + sha256: 4eb01c08e859bf924d222250d2e8f8b8ff6d3db4721288cf35d14252a4d933c8 + requires_python: '>=3' +- pypi: https://files.pythonhosted.org/packages/c3/68/483a78f5e8f31b08fb1bb671559968c0ca3a065ac7acabfc7cee55214fd6/nvidia_cuda_nvrtc-13.0.88-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl + name: nvidia-cuda-nvrtc + version: 13.0.88 + sha256: ad9b6d2ead2435f11cbb6868809d2adeeee302e9bb94bcf0539c7a40d80e8575 + requires_python: '>=3' +- pypi: https://files.pythonhosted.org/packages/2e/24/d1558f3b68b1d26e706813b1d10aa1d785e4698c425af8db8edc3dced472/nvidia_cuda_runtime-13.0.96-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl + name: nvidia-cuda-runtime + version: 13.0.96 + sha256: 7f82250d7782aa23b6cfe765ecc7db554bd3c2870c43f3d1821f1d18aebf0548 + requires_python: '>=3' +- pypi: https://files.pythonhosted.org/packages/6e/5e/edb9c0ae051602c3ccaffe424256463636d639e27d7f302dde9975ef9e7a/nvidia_cudnn_cu13-9.20.0.48-py3-none-manylinux_2_27_x86_64.whl + name: nvidia-cudnn-cu13 + version: 9.20.0.48 + sha256: 0c45dd8eeb50b603f07995b1b300c62ffe6a1980482b82b3bcf94a4ca9d49304 + requires_dist: + - nvidia-cublas + requires_python: '>=3' +- pypi: https://files.pythonhosted.org/packages/a8/2f/7b57e29836ea8714f81e9898409196f47d772d5ddedddf1592eadb8ab743/nvidia_cufft-12.0.0.61-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl + name: nvidia-cufft + version: 12.0.0.61 + sha256: 6c44f692dce8fd5ffd3e3df134b6cdb9c2f72d99cf40b62c32dde45eea9ddad3 + requires_dist: + - nvidia-nvjitlink + requires_python: '>=3' +- pypi: https://files.pythonhosted.org/packages/3f/70/4f193de89a48b71714e74602ee14d04e4019ad36a5a9f20c425776e72cd6/nvidia_cufile-1.15.1.6-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl + name: nvidia-cufile + version: 1.15.1.6 + sha256: 08a3ecefae5a01c7f5117351c64f17c7c62efa5fffdbe24fc7d298da19cd0b44 + requires_python: '>=3' +- pypi: https://files.pythonhosted.org/packages/a5/9f/be0a41ca4a4917abf5cb9ae0daff1a6060cc5de950aec0396de9f3b52bc5/nvidia_curand-10.4.0.35-py3-none-manylinux_2_27_x86_64.whl + name: nvidia-curand + version: 10.4.0.35 + sha256: 1aee33a5da6e1db083fe2b90082def8915f30f3248d5896bcec36a579d941bfc + requires_python: '>=3' +- pypi: https://files.pythonhosted.org/packages/5f/67/cba3777620cdacb99102da4042883709c41c709f4b6323c10781a9c3aa34/nvidia_cusolver-12.0.4.66-py3-none-manylinux_2_27_x86_64.whl + name: nvidia-cusolver + version: 12.0.4.66 + sha256: 0a759da5dea5c0ea10fd307de75cdeb59e7ea4fcb8add0924859b944babf1112 + requires_dist: + - nvidia-cublas + - nvidia-nvjitlink + - nvidia-cusparse + requires_python: '>=3' +- pypi: https://files.pythonhosted.org/packages/fa/18/623c77619c31d62efd55302939756966f3ecc8d724a14dab2b75f1508850/nvidia_cusparse-12.6.3.3-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl + name: nvidia-cusparse + version: 12.6.3.3 + sha256: 2b3c89c88d01ee0e477cb7f82ef60a11a4bcd57b6b87c33f789350b59759360b + requires_dist: + - nvidia-nvjitlink + requires_python: '>=3' +- pypi: https://files.pythonhosted.org/packages/34/7d/2661f2fb3ac4302f3a246f5fc030213ac60c1fe0bce84f9783dbd831dbb7/nvidia_cusparselt_cu13-0.8.1-py3-none-manylinux2014_x86_64.whl + name: nvidia-cusparselt-cu13 + version: 0.8.1 + sha256: 786ce87568c303fadb5afcc7102d454cd3040d75f6f8626f5db460d1871f4dd0 - conda: https://conda.anaconda.org/conda-forge/noarch/nvidia-ml-py-13.595.45-pyhd8ed1ab_1.conda sha256: fb46556c423311638fbb26ed4b2a67cf598044919e42ab7571365a5b4ae3b663 md5: dc8587ae654e96031728802016e8258c @@ -10706,6 +11061,26 @@ packages: - pkg:pypi/nvidia-ml-py?source=hash-mapping size: 48878 timestamp: 1775592349734 +- pypi: https://files.pythonhosted.org/packages/67/f4/58e4e91b6919367c7aafb8e36fce9aad1a3047e536bf7e2fd560927d3a4c/nvidia_nccl_cu13-2.29.7-py3-none-manylinux_2_18_x86_64.whl + name: nvidia-nccl-cu13 + version: 2.29.7 + sha256: edd81538446786ec3b73972543e53bb43bcaf0bfc8ef76cb679fcc390ffe136d + requires_python: '>=3' +- pypi: https://files.pythonhosted.org/packages/f0/ee/580ca6f29dcab0221db8706badca1bbbb084f1975c4d4e83329c3a7e31f0/nvidia_nvjitlink-13.3.33-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl + name: nvidia-nvjitlink + version: 13.3.33 + sha256: 26a6de7fb4c8fdaa7703d3dad720d6d427ddfea5c48a528fd97c11733ad830e5 + requires_python: '>=3' +- pypi: https://files.pythonhosted.org/packages/3c/35/a9bf80a609e74e3b000fef598933235c908fcefcef9026042b8e6dfde2a9/nvidia_nvshmem_cu13-3.4.5-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl + name: nvidia-nvshmem-cu13 + version: 3.4.5 + sha256: 290f0a2ee94c9f3687a02502f3b9299a9f9fe826e6d0287ee18482e78d495b80 + requires_python: '>=3' +- pypi: https://files.pythonhosted.org/packages/c2/f3/d86c845465a2723ad7e1e5c36dcd75ddb82898b3f53be47ebd429fb2fa5d/nvidia_nvtx-13.0.85-py3-none-manylinux1_x86_64.manylinux_2_5_x86_64.whl + name: nvidia-nvtx + version: 13.0.85 + sha256: 4936d1d6780fbe68db454f5e72a42ff64d1fd6397df9f363ae786930fd5c1cd4 + requires_python: '>=3' - conda: https://conda.anaconda.org/conda-forge/noarch/oauthlib-3.3.1-pyhd8ed1ab_0.conda sha256: dfa8222df90736fa13f8896f5a573a50273af8347542d412c3bd1230058e56a5 md5: d4f3f31ee39db3efecb96c0728d4bdbf @@ -13062,6 +13437,48 @@ packages: - pkg:pypi/s3transfer?source=hash-mapping size: 65987 timestamp: 1757487748738 +- pypi: https://files.pythonhosted.org/packages/28/50/f203ff3a3ddfe19308efc83c5a3a29ed02bf786732ec35e68bf9162f3365/safetensors-0.8.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl + name: safetensors + version: 0.8.0 + sha256: fd6f3f93c9a0a7cc2788ee63fb763353d4bd2e89b0751bc78fcf7dda00bea774 + requires_dist: + - safetensors[torch] ; extra == 'all' + - safetensors[numpy] ; extra == 'all' + - safetensors[jax] ; extra == 'all' + - safetensors[paddlepaddle] ; extra == 'all' + - safetensors[convert] ; extra == 'all' + - safetensors[quality] ; extra == 'all' + - safetensors[testing] ; extra == 'all' + - safetensors[torch] ; extra == 'convert' + - huggingface-hub>=1.4 ; extra == 'convert' + - safetensors[all] ; extra == 'dev' + - safetensors[pinned-tf] ; extra == 'dev' + - safetensors[numpy] ; extra == 'jax' + - flax>=0.6.3 ; extra == 'jax' + - jax>=0.3.25 ; extra == 'jax' + - jaxlib>=0.3.25 ; extra == 'jax' + - mlx>=0.0.9 ; extra == 'mlx' + - numpy>=1.24.6 ; extra == 'numpy' + - safetensors[numpy] ; extra == 'paddlepaddle' + - paddlepaddle>=2.4.1 ; extra == 'paddlepaddle' + - safetensors[numpy] ; extra == 'pinned-tf' + - tensorflow==2.18.0 ; extra == 'pinned-tf' + - ruff ; extra == 'quality' + - safetensors[numpy] ; extra == 'tensorflow' + - tensorflow>=2.11.0 ; extra == 'tensorflow' + - safetensors[numpy] ; extra == 'testing' + - h5py>=3.7.0 ; extra == 'testing' + - setuptools-rust>=1.12.0 ; extra == 'testing' + - pytest>=9.0 ; extra == 'testing' + - pytest-benchmark>=5.2 ; extra == 'testing' + - hypothesis>=6.70.2 ; extra == 'testing' + - fsspec>=2024.6.0 ; extra == 'testing' + - s3fs>=2024.6.0 ; extra == 'testing' + - safetensors[numpy] ; extra == 'tf-nightly' + - tf-nightly ; extra == 'tf-nightly' + - safetensors[numpy] ; extra == 'torch' + - torch>=2.4 ; extra == 'torch' + requires_python: '>=3.10' - conda: https://conda.anaconda.org/conda-forge/linux-64/safetensors-0.7.0-py312h0ccc70a_0.conda sha256: 59886c90dbfe388eee79c78256125ef2346a6d18058cf9769758c35f2e836a8c md5: c11270acf79465903f1508314c3e86fd @@ -13327,6 +13744,11 @@ packages: purls: [] size: 113547 timestamp: 1756649518540 +- pypi: https://files.pythonhosted.org/packages/e0/f9/0595336914c5619e5f28a1fb793285925a8cd4b432c9da0a987836c7f822/shellingham-1.5.4-py2.py3-none-any.whl + name: shellingham + version: 1.5.4 + sha256: 7ecfff8f2fd72616f7481040475a65b2bf8af90a56c89140852d1120324e8686 + requires_python: '>=3.7' - conda: https://conda.anaconda.org/conda-forge/noarch/shellingham-1.5.4-pyhd8ed1ab_2.conda sha256: 1d6534df8e7924d9087bd388fbac5bd868c5bf8971c36885f9f016da0657d22b md5: 83ea3a2ddb7a75c1b09cea582aa4f106 @@ -13918,8 +14340,9 @@ packages: - pypi: ./ name: spimquant version: 0.1.0 - sha256: 822584aa22bfc1737bd444f82c39ef79c6df0f880a359ab332a1a98155899064 + sha256: 78e687dadf8c0111fa639960e2f1d8a830e2a15017815ec96243cd2f42a17604 requires_python: '>=3.11' + editable: true - conda: https://conda.anaconda.org/conda-forge/linux-64/spirv-tools-2025.5-hb700be7_0.conda sha256: 7547142ab1352132adf98d555ed955badd96c9f277cbd054ae52f7edd6cf6cb8 md5: 058d5f16eaa3018be91aa3508df00d7c @@ -14051,6 +14474,15 @@ packages: purls: [] size: 2619743 timestamp: 1769664536467 +- pypi: https://files.pythonhosted.org/packages/a2/09/77d55d46fd61b4a135c444fc97158ef34a095e5681d0a6c10b75bf356191/sympy-1.14.0-py3-none-any.whl + name: sympy + version: 1.14.0 + sha256: e091cc3e99d2141a0ba2847328f5479b05d94a6635cb96148ccb3f34671bd8f5 + requires_dist: + - mpmath>=1.1.0,<1.4 + - pytest>=7.1.0 ; extra == 'dev' + - hypothesis>=6.70.0 ; extra == 'dev' + requires_python: '>=3.9' - conda: https://conda.anaconda.org/conda-forge/noarch/sympy-1.14.0-pyh2585a3b_106.conda sha256: 1c8057e6875eba958aa8b3c1a072dc9a75d013f209c26fd8125a5ebd3abbec0c md5: 32d866e43b25275f61566b9391ccb7b5 @@ -14268,6 +14700,17 @@ packages: - pkg:pypi/tifffile?source=hash-mapping size: 209916 timestamp: 1777805970265 +- pypi: https://files.pythonhosted.org/packages/d6/14/fc04d491527b774ec7479897f5861959209de1480e4c4cd32ed098ff8bea/timm-1.0.22-py3-none-any.whl + name: timm + version: 1.0.22 + sha256: 888981753e65cbaacfc07494370138b1700a27b1f0af587f4f9b47bc024161d0 + requires_dist: + - torch + - torchvision + - pyyaml + - huggingface-hub + - safetensors + requires_python: '>=3.8' - conda: https://conda.anaconda.org/conda-forge/noarch/timm-1.0.27-pyhcf101f3_0.conda sha256: 77c69e2ea6d040200fd21e9b82889e9eb1fd4cade6b4eaa877b442af636e0654 md5: 59a0c3968e59e6ce52855ef9bc9ff421 @@ -14357,6 +14800,29 @@ packages: - pkg:pypi/toolz?source=hash-mapping size: 53978 timestamp: 1760707830681 +- pypi: https://files.pythonhosted.org/packages/f3/82/fea946351658e6534db52d2cc12bc53087cbf87f9440c5f180f367c1950b/torch-2.13.0-cp312-cp312-manylinux_2_28_x86_64.whl + name: torch + version: 2.13.0 + sha256: 796633c4cdf0fe2cdced72d8f88f22e73dbcfce83132763162f6d4bff13b820b + requires_dist: + - filelock + - typing-extensions>=4.10.0 + - setuptools>=77.0.3 + - sympy>=1.13.3 + - networkx>=2.5.1 + - jinja2 + - fsspec>=0.8.5 + - cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.3 ; sys_platform == 'linux' + - cuda-bindings>=13.0.3,<14 ; python_full_version < '3.15' and sys_platform == 'linux' + - nvidia-cudnn-cu13==9.20.0.48 ; sys_platform == 'linux' + - nvidia-cusparselt-cu13==0.8.1 ; sys_platform == 'linux' + - nvidia-nccl-cu13==2.29.7 ; sys_platform == 'linux' + - nvidia-nvshmem-cu13==3.4.5 ; sys_platform == 'linux' + - triton==3.7.1 ; python_full_version < '3.15' and sys_platform == 'linux' + - optree>=0.13.0 ; extra == 'optree' + - opt-einsum>=3.3 ; extra == 'opt-einsum' + - pyyaml ; extra == 'pyyaml' + requires_python: '>=3.10' - conda: https://conda.anaconda.org/conda-forge/noarch/torch_em-0.8.3-pyhd8ed1ab_0.conda sha256: 12b8b552f16f0115b62055f752b6bbde44363d5c36346a75097effd62cf814f4 md5: 43f7a6f953343a444a35ccc99614170b @@ -14397,6 +14863,28 @@ packages: - pkg:pypi/torchmetrics?source=hash-mapping size: 398600 timestamp: 1773827733172 +- pypi: https://files.pythonhosted.org/packages/17/8b/155f99042f9319bd7759536779b2a5b67cbd4f89c380854670850f89a2f4/torchvision-0.22.1-cp312-cp312-manylinux_2_28_x86_64.whl + name: torchvision + version: 0.22.1 + sha256: 699c2d70d33951187f6ed910ea05720b9b4aaac1dcc1135f53162ce7d42481d3 + requires_dist: + - numpy + - torch==2.7.1 + - pillow>=5.3.0,!=8.3.* + - gdown>=4.7.3 ; extra == 'gdown' + - scipy ; extra == 'scipy' + requires_python: '>=3.9' +- pypi: https://files.pythonhosted.org/packages/93/e4/e9b2495d0d57b9f60d63c57d0a910410a81b4b073bf70917bef815291119/torchvision-0.28.0-cp312-cp312-manylinux_2_28_x86_64.whl + name: torchvision + version: 0.28.0 + sha256: 028a3d481b37d785605620d7cdad897064c5a55bae2aa1f2658766333e291940 + requires_dist: + - numpy + - torch==2.13.0 + - pillow>=5.3.0,!=8.3.* + - gdown>=4.7.3 ; extra == 'gdown' + - scipy ; extra == 'scipy' + requires_python: '>=3.10,!=3.14.1' - conda: https://conda.anaconda.org/conda-forge/linux-64/torchvision-0.24.0-cpu_py312_h56cfa8b_0.conda sha256: a22214010fd1f3398ad7ea8311e0d76b0f7750590b502b0a73f57c14ca806595 md5: feb0c7bde000f25617408266492d2d2a @@ -14482,6 +14970,26 @@ packages: - pkg:pypi/traitlets?source=hash-mapping size: 115165 timestamp: 1778074251714 +- pypi: https://files.pythonhosted.org/packages/c4/6f/fb96d15db6f36d6eae4cafb998c2e0353bf59d7c4ea1662d7497f269134a/triton-3.7.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl + name: triton + version: 3.7.1 + sha256: 7e40869937a68206ec70d7f25bb7ec6433cb083f9135e1f36dbd318dc449a728 + requires_dist: + - importlib-metadata ; python_full_version < '3.10' + - cmake>=3.20,<4.0 ; extra == 'build' + - lit ; extra == 'build' + - autopep8 ; extra == 'tests' + - isort ; extra == 'tests' + - numpy ; extra == 'tests' + - pytest ; extra == 'tests' + - pytest-forked ; extra == 'tests' + - pytest-xdist ; extra == 'tests' + - scipy>=1.7.1 ; extra == 'tests' + - llnl-hatchet ; extra == 'tests' + - matplotlib ; extra == 'tutorials' + - pandas ; extra == 'tutorials' + - tabulate ; extra == 'tutorials' + requires_python: '>=3.10,<3.15' - conda: https://conda.anaconda.org/conda-forge/linux-64/triton-3.3.0-cuda126py312hebffaa9_1.conda sha256: 7089c27a38fc3ec199af4d51fcbba33720281f3098e984c49a9f010805d2de84 md5: a05b9a73fe6a9be82a2fc4af2b01e95f @@ -14518,6 +15026,16 @@ packages: - pkg:pypi/truststore?source=hash-mapping size: 24279 timestamp: 1766494826559 +- pypi: https://files.pythonhosted.org/packages/43/89/9518bc0c3929bee36b3a4a8e3daddd6e03f92f9961c66d4983b837160543/typer-0.27.1-py3-none-any.whl + name: typer + version: 0.27.1 + sha256: 53150287edd11baeb4e4722c8e394fcdf8181c0ae89485cba8d25c778d5edd56 + requires_dist: + - shellingham>=1.3.0 + - rich>=13.8.0 + - annotated-doc>=0.0.2 + - colorama ; sys_platform == 'win32' + requires_python: '>=3.10' - conda: https://conda.anaconda.org/conda-forge/noarch/typer-0.25.1-pyhcf101f3_0.conda sha256: 18fc3a27bc995318d09142fe16d01ea454e76f377bf8f68db03b8b18f11085ed md5: ef114c2eb2ff19f6bf616c81f4710841 diff --git a/pyproject.toml b/pyproject.toml index f60ae60..acd3c98 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -54,6 +54,7 @@ spimquant = { path = ".", editable = true } zarrnii = ">=0.21.2,<0.22.0" #zarrnii = { path = "/nfs/khan/trainees/akhan488/apps/zarrnii", editable = true } vesselfm = { git = "https://github.com/khanlab/vesselfm", rev = "504baab" } +dynamic-network-architectures = ">=0.4" h5py = ">=3.14.0" simpleitk = ">=2.4.0" pylibtiff = ">=0.7.0" diff --git a/spimquant/config/snakebids.yml b/spimquant/config/snakebids.yml index f518ff2..b4025b6 100644 --- a/spimquant/config/snakebids.yml +++ b/spimquant/config/snakebids.yml @@ -194,6 +194,18 @@ parse_args: ``threshold = expm1(mean_n + k * sigma_n)``. The float multiplier k may be encoded with 'p' as a decimal separator to keep BIDS-compatible filenames, e.g. ``gmm+n2k2p5`` → n=2, k=2.5. + + ``lantern`` – 5-fold LANTERN deep-learning ensemble for amyloid-beta + plaques. Unlike the histogram-based methods above it reads raw + intensities (not the bias-field-corrected image) and runs at + ``plaque_level``; the mask at ``segmentation_level`` is that prediction + upsampled. Requires a GPU. + + Also unlike them, it is stain-specific: it runs only on the first stain + from ``stains_for_plaques``. The remaining stains in ``stains_for_seg`` + use the other methods given here, or -- if ``lantern`` was the only one -- + this option's own default, so they behave exactly as if it had not been + passed. default: - gmm+n3k1 nargs: '+' @@ -447,6 +459,12 @@ templates: models: vesselfm: https://huggingface.co/bwittmann/vesselFM/resolve/main/vesselFM_base.pt + lantern_abeta: + fold0: https://huggingface.co/apooladi/lantern-ki3-abeta/resolve/main/fold0/seg_model.pt + fold1: https://huggingface.co/apooladi/lantern-ki3-abeta/resolve/main/fold1/seg_model.pt + fold2: https://huggingface.co/apooladi/lantern-ki3-abeta/resolve/main/fold2/seg_model.pt + fold3: https://huggingface.co/apooladi/lantern-ki3-abeta/resolve/main/fold3/seg_model.pt + fold4: https://huggingface.co/apooladi/lantern-ki3-abeta/resolve/main/fold4/seg_model.pt @@ -524,6 +542,31 @@ vessel_seg_method: vesselfm vessel_metrics: - fieldfrac +# LANTERN deep-learning plaque segmentation (opt in with --seg_method lantern). +# The ensemble is scale-sensitive and was built for plaque_level; the mask at +# segmentation_level is that prediction upsampled, not native inference. +plaque_seg_method: lantern +# The LANTERN ensemble is trained on amyloid-beta only, so unlike the +# intensity-based methods it is not applied to every stain in stains_for_seg. +# The first of these found is the one it runs on; the remaining stains are +# segmented exactly as they would be if --seg_method had never been passed. +stains_for_plaques: + - abeta + - Abeta + - BetaAmyloid +plaque_level: 1 +plaque_n_folds: 5 +plaque_vote_threshold: 3 +plaque_tile: 128 +plaque_stride: 64 +plaque_batch_size: 8 +plaque_chunk: 512 +# GPUs per inference job. The ensemble is replicated onto each card and dask +# blocks are handed to whichever card is free, so this scales one subject's +# wall-clock rather than throughput across subjects. Clamped at runtime to the +# number of visible devices. +plaque_n_gpus: 2 + coloc_seg_metrics: - overlapratio diff --git a/spimquant/workflow/Snakefile b/spimquant/workflow/Snakefile index 08230ec..bc4d994 100644 --- a/spimquant/workflow/Snakefile +++ b/spimquant/workflow/Snakefile @@ -175,6 +175,40 @@ else: do_vessels = False +# The LANTERN plaque ensemble is opt-in via --seg_method, but it is trained on +# amyloid-beta only, so it is resolved to a single stain the same way vessels are +# rather than being fanned out over every stain in stains_for_seg. +do_plaques = do_seg and config["plaque_seg_method"] in config["seg_method"] + +stain_for_plaques = None + +if do_plaques: + for stain in config["stains_for_plaques"]: + if stain in stains_for_seg: + stain_for_plaques = stain + break + + if stain_for_plaques is None: + raise ValueError( + f"--seg_method {config['plaque_seg_method']} was requested but none of " + f"{config['stains_for_plaques']} is in stains_for_seg ({stains_for_seg}). " + f"The LANTERN ensemble only segments amyloid-beta." + ) + +# Methods applied to every stain in stains_for_seg. The plaque method is excluded +# because it is stain-specific -- keeping it out of this list is what stops it from +# entering the desc x stain expansions and the cross-stain aggregation, exactly as +# vessel_seg_method stays out of them. +# +# If the plaque method was the ONLY one requested, the remaining stains must behave +# exactly as if --seg_method had never been passed. That default is read back off +# the CLI definition rather than restated here, so it cannot drift out of sync with +# the actual default if the latter ever changes. +seg_methods = [ + m for m in config["seg_method"] if m != config["plaque_seg_method"] +] or list(config["parse_args"]["--seg_method"]["default"]) + + # atlas segmentations to use if config["atlas_segs"] is None: @@ -540,6 +574,51 @@ rule all_vessels: ), +rule all_plaques: + input: + inputs["spim"].expand( + bids_oz_in( + root=root, + datatype="seg", + stain="{stain}", + level="{level}", + desc=config["plaque_seg_method"], + suffix="probseg.{ext}", + **inputs["spim"].wildcards, + ), + level=config["plaque_level"], + stain=stain_for_plaques, + ), + inputs["spim"].expand( + bids_oz_in( + root=root, + datatype="seg", + stain="{stain}", + level="{level}", + desc=config["plaque_seg_method"], + suffix="mask.{ext}", + **inputs["spim"].wildcards, + ), + level=config["segmentation_level"], + stain=stain_for_plaques, + ), + inputs["spim"].expand( + bids( + root=root, + datatype="seg", + stain="{stain}", + level="{level}", + desc=config["plaque_seg_method"], + space="{template}", + suffix="fieldfrac.nii.gz", + **inputs["spim"].wildcards, + ), + level=config["registration_level"], + template=config["template"], + stain=stain_for_plaques, + ), + + rule all_segment: input: inputs["spim"].expand( @@ -553,7 +632,7 @@ rule all_segment: **inputs["spim"].wildcards, ), seg=atlas_segs, - desc=config["seg_method"], + desc=seg_methods, template=config["template"], suffix=config["seg_metrics"], stain=stains_for_seg, @@ -585,7 +664,7 @@ rule all_segment: ), stain=stains_for_seg, level=config["registration_level"], - desc=config["seg_method"], + desc=seg_methods, template=config["template"], ), inputs["spim"].expand( @@ -600,7 +679,7 @@ rule all_segment: ), stain=stains_for_seg, level=config["registration_level"], - desc=config["seg_method"], + desc=seg_methods, ), inputs["spim"].expand( bids( @@ -614,7 +693,7 @@ rule all_segment: ), stain=stains_for_seg, template=config["template"], - desc=config["seg_method"], + desc=seg_methods, ), expand( bids( @@ -642,7 +721,7 @@ rule all_segment_coloc: **inputs["spim"].wildcards, ), seg=atlas_segs, - desc=config["seg_method"], + desc=seg_methods, template=config["template"], suffix=config["coloc_seg_metrics"], ), @@ -655,7 +734,7 @@ rule all_segment_coloc: suffix="coloccounts.nii.gz", **inputs["spim"].wildcards, ), - desc=config["seg_method"], + desc=seg_methods, template=config["template"], ), @@ -721,7 +800,7 @@ rule all_spim_patches: stain=stains_for_seg, seg=patch_atlas_segs, template=config["template"], - desc=config["seg_method"], + desc=seg_methods, level=config["segmentation_level"], ), @@ -759,7 +838,7 @@ rule all_group_stats: suffix="groupstats.{ext}", ), seg=atlas_segs, - desc=config["seg_method"], + desc=seg_methods, template=config["template"], pairwise_contrast=pairwise_contrast_labels, ext=["png", "tsv"], @@ -776,7 +855,7 @@ rule all_group_stats: suffix="{stat}.nii.gz", ), seg=atlas_segs, - desc=config["seg_method"], + desc=seg_methods, template=config["template"], stain=stains_for_seg, metric=config["seg_metrics"], @@ -792,7 +871,7 @@ rule all_group_stats: desc="{desc}", suffix="{stain}+count.nii.gz", ), - desc=config["seg_method"], + desc=seg_methods, template=config["template"], level=range(4), stain=stains_for_seg, @@ -813,7 +892,7 @@ rule all_group_stats_coloc: suffix="{stat}.nii.gz", ), seg=atlas_segs, - desc=config["seg_method"], + desc=seg_methods, template=config["template"], metric=config["coloc_seg_metrics"], stat=config["stats_maps"], @@ -828,7 +907,7 @@ rule all_group_stats_coloc: desc="{desc}", suffix="coloccount.nii.gz", ), - desc=config["seg_method"], + desc=seg_methods, template=config["template"], level=range(4), ), @@ -859,7 +938,7 @@ rule all_qc: **inputs["spim"].wildcards, ), stain=stains_for_seg, - desc=config["seg_method"], + desc=seg_methods, ) if do_seg else [], @@ -878,7 +957,7 @@ rule all_qc: seg=atlas_segs, template=config["template"], stain=stains_for_seg, - desc=config["seg_method"], + desc=seg_methods, ) if do_seg else [], @@ -897,7 +976,7 @@ rule all_qc: seg=atlas_segs, template=config["template"], stain=stains_for_seg, - desc=config["seg_method"], + desc=seg_methods, ) if do_seg else [], @@ -965,7 +1044,7 @@ rule all_qc: **inputs["spim"].wildcards, ), stain=stains_for_seg, - desc=config["seg_method"], + desc=seg_methods, ) if do_seg else [], @@ -980,7 +1059,7 @@ rule all_qc: **inputs["spim"].wildcards, ), stain=stains_for_seg, - desc=config["seg_method"], + desc=seg_methods, ) if do_seg else [], @@ -997,7 +1076,7 @@ rule all_qc: ), seg=atlas_segs, template=config["template"], - desc=config["seg_method"], + desc=seg_methods, ) if do_seg else [], @@ -1028,7 +1107,7 @@ rule all_qc: seg=atlas_segs, template=config["template"], stain=stains_for_seg, - desc=config["seg_method"], + desc=seg_methods, ) if do_seg and config["qc_instance_seg"] else [], @@ -1056,7 +1135,7 @@ rule all_qc: ], seg=atlas_segs, template=config["template"], - desc=config["seg_method"], + desc=seg_methods, ) if do_coloc and config["qc_instance_seg"] else [], @@ -1077,7 +1156,7 @@ rule all_tabular_sidecars: **inputs["spim"].wildcards, ), seg=atlas_segs, - desc=config["seg_method"], + desc=seg_methods, template=config["template"], ) if do_seg @@ -1095,7 +1174,7 @@ rule all_tabular_sidecars: **inputs["spim"].wildcards, ), seg=atlas_segs, - desc=config["seg_method"], + desc=seg_methods, template=config["template"], stain=stains_for_seg, ) @@ -1111,7 +1190,7 @@ rule all_tabular_sidecars: suffix="regionprops.json", **inputs["spim"].wildcards, ), - desc=config["seg_method"], + desc=seg_methods, template=config["template"], ) if do_seg @@ -1125,7 +1204,7 @@ rule all_tabular_sidecars: suffix="coloc.json", **inputs["spim"].wildcards, ), - desc=config["seg_method"], + desc=seg_methods, template=config["template"], ) if do_coloc @@ -1146,7 +1225,7 @@ rule all_group_stats_tabular_sidecars: suffix="groupstats.json", ), seg=atlas_segs, - desc=config["seg_method"], + desc=seg_methods, template=config["template"], pairwise_contrast=pairwise_contrast_labels, ) @@ -1162,7 +1241,7 @@ rule all_group_stats_tabular_sidecars: suffix="allsubjects.json", ), seg=atlas_segs, - desc=config["seg_method"], + desc=seg_methods, template=config["template"], ) if do_seg @@ -1175,7 +1254,7 @@ rule all_group_stats_tabular_sidecars: desc="{desc}", suffix="regionprops.json", ), - desc=config["seg_method"], + desc=seg_methods, template=config["template"], ) if do_seg @@ -1187,7 +1266,7 @@ rule all_group_stats_tabular_sidecars: desc="{desc}", suffix="coloc.json", ), - desc=config["seg_method"], + desc=seg_methods, template=config["template"], ) if do_coloc @@ -1207,7 +1286,7 @@ rule all_group_merged_tsvs: suffix="allsubjects.tsv", ), seg=atlas_segs, - desc=config["seg_method"], + desc=seg_methods, template=config["template"], ) if do_seg @@ -1219,6 +1298,7 @@ rule all_participant: input: rules.all_register.input, rules.all_vessels.input if do_vessels else [], + rules.all_plaques.input if do_plaques else [], rules.all_segment.input if do_seg else [], rules.all_mri_reg.input if config["register_to_mri"] else [], rules.all_segment_coloc.input if do_coloc else [], @@ -1240,6 +1320,7 @@ include: "rules/import.smk" include: "rules/masking.smk" include: "rules/templatereg.smk" include: "rules/vessels.smk" +include: "rules/plaques.smk" include: "rules/segmentation.smk" include: "rules/counts.smk" include: "rules/fieldfrac.smk" diff --git a/spimquant/workflow/rules/plaques.smk b/spimquant/workflow/rules/plaques.smk new file mode 100644 index 0000000..5d491e3 --- /dev/null +++ b/spimquant/workflow/rules/plaques.smk @@ -0,0 +1,128 @@ +rule import_lantern_abeta_fold: + """Download one fold of the LANTERN Abeta plaque ensemble.""" + input: + model=lambda wildcards: storage( + config["models"]["lantern_abeta"][f"fold{wildcards.fold}"] + ), + output: + "resources/models/lantern-ki3-abeta/fold{fold}/seg_model.pt", + wildcard_constraints: + fold="[0-9]+", + localrule: True + shell: + "cp {input} {output}" + + +rule run_lantern_plaques: + """Segment Abeta plaques with the 5-fold LANTERN ensemble. + + Runs at config['plaque_level'] on RAW intensities -- the model was not + trained on N4-corrected data, so this bypasses the bias-field chain that + the GMM and Otsu methods depend on. + + Tiles at 128^3 with stride 64 and combines overlapping tiles by max over + votes. The low-res brain mask restricts inference to blocks that touch + tissue, which is what makes whole-brain 5-fold inference tractable. + + Writes votes/n_folds rather than a binary mask, so the vote threshold can be + changed downstream without re-running inference. + """ + input: + spim=inputs["spim"].path, + models=expand( + "resources/models/lantern-ki3-abeta/fold{fold}/seg_model.pt", + fold=range(config["plaque_n_folds"]), + ), + mask=bids( + root=root, + datatype="micr", + stain=stain_for_reg, + level=config["correction_level"], + desc="brain", + suffix="mask.nii.gz", + **inputs["spim"].wildcards, + ), + params: + zarrnii_kwargs=zarrnii_in_kwargs, + tile=config["plaque_tile"], + stride=config["plaque_stride"], + batch_size=config["plaque_batch_size"], + chunk=config["plaque_chunk"], + n_gpus=config["plaque_n_gpus"], + output: + probseg=bids_oz_out( + root=root, + datatype="seg", + stain="{stain}", + level="{level}", + desc=config["plaque_seg_method"], + suffix="probseg.{ext}", + **inputs["spim"].wildcards, + ), + wildcard_constraints: + # Trained on amyloid-beta only. Constraining the stain per-rule (never + # globally -- a module-level constraint in an included smk applies to the + # whole workflow) means a target asking for a lantern mask on any other + # stain fails at DAG build instead of quietly burning a GPU on it. + stain=stain_for_plaques or "^$", + threads: 8 * config["plaque_n_gpus"] + resources: + gpu=config["plaque_n_gpus"], + cpus_per_gpu=8, + mem_mb=256000, + disk_mb=2097152, + runtime=lambda wildcards: max( + 1, + int( + 2880.0 + / (3.0 ** float(wildcards.level)) + / float(config["plaque_n_gpus"]) + ), + ), + # stride-64 tiling x 5 folds, split across GPUs + script: + "../scripts/lantern_plaques.py" + + +rule binarize_lantern_plaques: + """Threshold the LANTERN vote map and upsample it to the segmentation level. + + Emits 0/100 at config['segmentation_level'], matching the segmentation.smk + mask convention, so the standard fieldfrac / regionprops / counts / segstats + chain consumes it unchanged. + """ + input: + probseg=bids_oz_in( + root=root, + datatype="seg", + stain="{stain}", + level=config["plaque_level"], + desc=config["plaque_seg_method"], + suffix="probseg.{ext}", + **inputs["spim"].wildcards, + ), + spim=inputs["spim"].path, + params: + zarrnii_kwargs=zarrnii_in_kwargs, + n_folds=config["plaque_n_folds"], + vote_threshold=config["plaque_vote_threshold"], + target_level=config["segmentation_level"], + output: + mask=bids_oz_out( + root=root, + datatype="seg", + stain="{stain}", + level=config["segmentation_level"], + desc=config["plaque_seg_method"], + suffix="mask.{ext}", + **inputs["spim"].wildcards, + ), + wildcard_constraints: + stain=stain_for_plaques or "^$", + threads: 32 + resources: + mem_mb=64000, + disk_mb=2097152, + runtime=180, + script: + "../scripts/binarize_lantern_plaques.py" diff --git a/spimquant/workflow/scripts/binarize_lantern_plaques.py b/spimquant/workflow/scripts/binarize_lantern_plaques.py new file mode 100644 index 0000000..1684f68 --- /dev/null +++ b/spimquant/workflow/scripts/binarize_lantern_plaques.py @@ -0,0 +1,85 @@ +"""Threshold the LANTERN vote map and upsample it to the segmentation level. + +The ensemble runs at ``plaque_level`` because it is scale-sensitive and was built +for that grid; the segmentation level output is therefore the upsampled +prediction, NOT native inference at that level. + +Upsampling is exact replication (``da.repeat``) rather than interpolation, and +the per-axis factors are derived from the shape ratio so this stays correct when +the pyramid downsamples only x and y. + +Output is valued 0/100 rather than 0/1, matching ``gmmthresh`` / ``multiotsu`` / +``threshold`` in segmentation.smk, so that mean-pool downsampling in ``fieldfrac`` +yields a percentage directly. +""" + +import dask.array as da +import numpy as np +from dask.diagnostics import ProgressBar +from zarrnii import ZarrNii + + +def upsample_nearest(arr, target_shape): + """Replicate voxels to `target_shape`, then trim/pad to land on it exactly.""" + if len(target_shape) != arr.ndim: + raise ValueError(f"rank mismatch: {arr.shape} vs target {target_shape}") + + out = arr + for axis, (have, want) in enumerate(zip(arr.shape, target_shape)): + factor = max(1, int(round(want / have))) + if factor > 1: + out = da.repeat(out, factor, axis=axis) + + out = out[tuple(slice(0, min(h, w)) for h, w in zip(out.shape, target_shape))] + pad = [(0, w - h) for h, w in zip(out.shape, target_shape)] + if any(after for _, after in pad): + out = da.pad(out, pad, mode="edge") + return out + + +def main(): + n_folds = int(snakemake.params.n_folds) + vote_threshold = int(snakemake.params.vote_threshold) + + if not 1 <= vote_threshold <= n_folds: + raise ValueError( + f"plaque_vote_threshold={vote_threshold} is out of range for an " + f"{n_folds}-fold ensemble; the vote map holds counts in 0..{n_folds}" + ) + + probseg = ZarrNii.from_file(snakemake.input.probseg, level=0) + ref = ZarrNii.from_file( + snakemake.input.spim, + level=int(snakemake.params.target_level), + channel_labels=[snakemake.wildcards.stain], + **snakemake.params.zarrnii_kwargs, + ) + + # probseg holds votes/n_folds. Compare against the MIDPOINT between adjacent + # attainable values rather than the exact fraction: 2.5/5 = 0.5 sits safely + # between 0.4 and 0.6, so no float representation error can flip a voxel at the + # boundary the way `>= 3/5` could. + binary = probseg.data >= (vote_threshold - 0.5) / n_folds + + upsampled = upsample_nearest(binary, ref.data.shape) + + znimg_mask = ref.copy() + znimg_mask.data = (upsampled * 100).astype(np.uint8) + + print( + f"votes>={vote_threshold} of {n_folds} | " + f"{probseg.data.shape} -> {ref.data.shape}", + flush=True, + ) + + with ProgressBar(): + znimg_mask.to_ome_zarr( + snakemake.output.mask, + max_layer=5, + match_scale_factors_from=snakemake.input.spim, + **snakemake.config["zarrnii_out_kwargs"], + ) + + +if __name__ == "__main__": + main() diff --git a/spimquant/workflow/scripts/lantern_model.py b/spimquant/workflow/scripts/lantern_model.py new file mode 100644 index 0000000..8ab61bc --- /dev/null +++ b/spimquant/workflow/scripts/lantern_model.py @@ -0,0 +1,73 @@ +"""ResEncL U-Net architecture for the LANTERN Abeta plaque segmentation ensemble. + +Vendored from the LANTERN repository (``lantern/encoders/resenc_l.py`` and +``lantern/models/unet_seg.py``) so that SPIMquant can load the published +checkpoints without depending on that repository. The definitions must stay +byte-compatible with the checkpoints on HuggingFace +(``apooladi/lantern-ki3-abeta``) -- changing any architectural constant here +will make ``load_state_dict`` fail. + +The encoder is identical to nnssl's ``architecture_registry.get_res_enc_l()``; +the decoder is nnU-Net's ``UNetDecoder`` over that encoder's skips. +""" + +from __future__ import annotations + +import torch +import torch.nn as nn +from dynamic_network_architectures.building_blocks.residual_encoders import ( + ResidualEncoder, +) +from dynamic_network_architectures.building_blocks.unet_decoder import UNetDecoder + + +def build_resenc_l(num_input_channels: int = 1) -> ResidualEncoder: + """Build the ResEncL encoder. + + Architecture: 6 stages, features [32, 64, 128, 256, 320, 320], + strides [1,1,1] then [2,2,2] x 5, blocks per stage [1, 3, 4, 6, 6, 6], + InstanceNorm3d (eps=1e-5, affine=True), LeakyReLU(inplace=True). + """ + n_stages = 6 + encoder = ResidualEncoder( + input_channels=num_input_channels, + n_stages=n_stages, + features_per_stage=[32, 64, 128, 256, 320, 320], + conv_op=nn.Conv3d, + kernel_sizes=[[3, 3, 3]] * n_stages, + strides=[[1, 1, 1]] + [[2, 2, 2]] * 5, + n_blocks_per_stage=[1, 3, 4, 6, 6, 6], + conv_bias=True, + norm_op=nn.InstanceNorm3d, + norm_op_kwargs={"eps": 1e-5, "affine": True}, + nonlin=nn.LeakyReLU, + nonlin_kwargs={"inplace": True}, + return_skips=True, + disable_default_stem=False, + stem_channels=None, + ) + return encoder + + +class ResEncLUNet(nn.Module): + """ResEncL encoder + UNetDecoder, 2-class (background / plaque) logits.""" + + def __init__( + self, + num_classes: int = 2, + num_input_channels: int = 1, + n_conv_per_stage_decoder: int = 1, + deep_supervision: bool = False, + ) -> None: + super().__init__() + self.encoder = build_resenc_l(num_input_channels) + n_dec_stages = len(self.encoder.output_channels) - 1 + self.decoder = UNetDecoder( + self.encoder, + num_classes, + [n_conv_per_stage_decoder] * n_dec_stages, + deep_supervision=deep_supervision, + ) + + def forward(self, x: torch.Tensor) -> torch.Tensor: + return self.decoder(self.encoder(x)) diff --git a/spimquant/workflow/scripts/lantern_plaques.py b/spimquant/workflow/scripts/lantern_plaques.py new file mode 100644 index 0000000..97720a2 --- /dev/null +++ b/spimquant/workflow/scripts/lantern_plaques.py @@ -0,0 +1,349 @@ +"""5-fold LANTERN ensemble Abeta plaque segmentation. + +Runs the published LANTERN ensemble (``apooladi/lantern-ki3-abeta``) over the +Abeta channel and writes the per-fold agreement map -- the fraction of folds +voting foreground, so ``{0, .2, .4, .6, .8, 1}`` for five folds -- as an +OME-Zarr. Binarization is deliberately left to a downstream rule so the vote +threshold can be changed without re-running inference. + +Deviations from ``vesselfm.py``, and why: + +- ``ZarrNii.segment()`` cannot be used. It dispatches through ``da.map_blocks`` + with disjoint blocks, no block position and a hardcoded ``uint8`` output, so it + can express neither the stride-64 tile overlap the model requires nor + brain-mask-based block skipping. ``da.map_overlap`` is driven directly here. + +- The model reads RAW intensities. It was never trained on N4-corrected or + otherwise rescaled data, so this path takes ``input.spim`` directly rather than + the ``desc-corrected*`` store the GMM/Otsu rules consume. + +Inference contract (from the model card, must match training exactly): + tiles of 128^3 at stride 64, per-tile 0.5/99.5 percentile clip then z-score, + softmax over 2 classes, foreground where class-1 probability >= 0.5 per fold, + overlapping tiles combined by MAX over votes. +""" + +import contextlib +import queue + +import dask.array as da +import numpy as np +import torch +from dask.diagnostics import ProgressBar +from zarrnii import ZarrNii + +from dask_setup import get_dask_client +from lantern_model import ResEncLUNet + + +def tile_origins(extent, tile, stride): + """Tile origins covering [0, extent), with the last pulled back to hit the edge.""" + if extent <= tile: + return [0] + origins = list(range(0, extent - tile + 1, stride)) + if origins[-1] + tile < extent: + origins.append(extent - tile) + return origins + + +def normalize_tiles(x): + """Per-tile percentile clip then z-score, matching LANTERN's training transform.""" + out = torch.empty_like(x) + quantiles = torch.tensor([0.005, 0.995], device=x.device, dtype=x.dtype) + for i in range(x.shape[0]): + lo, hi = torch.quantile(x[i].reshape(-1), quantiles) + clipped = torch.clamp(x[i], lo, hi) + std = clipped.std() + out[i] = (clipped - clipped.mean()) / (std if std > 1e-8 else 1.0) + return out + + +def predict_volume(vol, nets, device, tile, stride, batch_size): + """Tiled 5-fold vote map for one 3D block, as uint8 in [0, len(nets)]. + + The caller owns `device` exclusively for the duration of this call (see + DevicePool), so no additional locking is needed here. + """ + orig_shape = vol.shape + pad = [max(0, tile - n) for n in orig_shape] + if any(pad): + # Blocks at the volume border can be thinner than one tile; pad, then crop back. + vol = np.pad(vol, [(0, p) for p in pad], mode="edge") + + votes = np.zeros(vol.shape, dtype=np.uint8) + coords = [ + (z, y, x) + for z in tile_origins(vol.shape[0], tile, stride) + for y in tile_origins(vol.shape[1], tile, stride) + for x in tile_origins(vol.shape[2], tile, stride) + ] + + for start in range(0, len(coords), batch_size): + batch = coords[start : start + batch_size] + arr = np.stack( + [vol[z : z + tile, y : y + tile, x : x + tile] for z, y, x in batch] + ) + tensor = torch.from_numpy(arr).to(device) + tensor = normalize_tiles(tensor)[:, None] + batch_votes = torch.zeros( + (len(batch), tile, tile, tile), dtype=torch.uint8, device=device + ) + with ( + torch.no_grad(), + torch.autocast( + device.type, dtype=torch.bfloat16, enabled=device.type == "cuda" + ), + ): + for net in nets: + prob = torch.softmax(net(tensor).float(), dim=1)[:, 1] + batch_votes += (prob >= 0.5).to(torch.uint8) + batch_votes = batch_votes.cpu().numpy() + + for j, (z, y, x) in enumerate(batch): + region = votes[z : z + tile, y : y + tile, x : x + tile] + # MAX, not sum: a plaque clipped at one tile edge is recovered by the + # tile that contains it whole. + np.maximum(region, batch_votes[j], out=region) + + return votes[: orig_shape[0], : orig_shape[1], : orig_shape[2]] + + +def load_folds(model_paths, device): + """Instantiate every fold on `device`, once, in the main process. + + ~2 GB of fp32 weights for a 5-fold ensemble, so replicating it per GPU is + cheap. DevicePool then guarantees only one dask block uses a given device at + a time, which bounds peak VRAM without needing a lock inside the forward pass. + + ``weights_only=True`` because these checkpoints are downloaded over the + network; they carry only tensors and plain dicts, so nothing is lost. + """ + nets = [] + for path in model_paths: + ckpt = torch.load(path, map_location="cpu", weights_only=True) + if ckpt.get("model_type") not in (None, "ResEncLUNet"): + raise ValueError( + f"{path} declares model_type={ckpt['model_type']!r}, " + "but this script only builds ResEncLUNet" + ) + net = ResEncLUNet(num_classes=2, num_input_channels=1) + net.load_state_dict(ckpt["state_dict"]) + nets.append(net.eval().to(device)) + return nets + + +def resolve_devices(requested): + """The devices to run on, clamped to what is actually visible.""" + if not torch.cuda.is_available(): + print("WARNING: no CUDA device visible; falling back to CPU", flush=True) + return [torch.device("cpu")] + available = torch.cuda.device_count() + if requested > available: + print( + f"WARNING: {requested} GPUs requested but only {available} visible; " + f"using {available}", + flush=True, + ) + return [torch.device(f"cuda:{i}") for i in range(max(1, min(requested, available)))] + + +class DevicePool: + """Lends out one GPU at a time, with the full ensemble resident on each. + + The ensemble is replicated per device (~2 GB of fp32 weights for 5 folds), so + each GPU works independently on a whole dask block. Handing out devices from + a queue rather than striping blocks by index balances the load for free: + blocks vary enormously in cost because most of a frame is background, so a + static assignment would leave one card idle -- the same imbalance LANTERN's + own two-GPU split had to correct for by splitting on cumulative tile count. + """ + + def __init__(self, devices, model_paths): + self._free = queue.Queue() + self.nets = {} + for device in devices: + self.nets[device] = load_folds(model_paths, device) + self._free.put(device) + + @contextlib.contextmanager + def acquire(self): + device = self._free.get() + try: + yield device, self.nets[device] + finally: + self._free.put(device) + + +def build_brain_block_mask(mask_path, image_shape, chunks): + """Per-block "does this block touch brain" flags, aligned with the image chunking. + + The brain mask lives at a much coarser pyramid level, so rather than + interpolating it up to the inference grid it is held in memory and indexed by + each block's array location -- the same approach LANTERN's whole-brain + inference uses to skip background tiles. + """ + mask_znimg = ZarrNii.from_nifti(mask_path, axes_order="ZYX") + mask_arr = np.asarray(mask_znimg.data).squeeze() > 0 + if mask_arr.ndim != 3: + raise ValueError(f"expected a 3D brain mask, got shape {mask_arr.shape}") + + scales = [m / i for m, i in zip(mask_arr.shape, image_shape[-3:])] + + def block_has_brain(block_info=None): + shape = block_info[None]["chunk-shape"] + location = block_info[None]["array-location"] + index = [] + for axis, (lo, hi) in enumerate(location[-3:]): + i0 = int(np.floor(lo * scales[axis])) + i1 = int(np.ceil(hi * scales[axis])) + i0 = min(max(i0, 0), mask_arr.shape[axis] - 1) + i1 = min(max(i1, i0 + 1), mask_arr.shape[axis]) + index.append(slice(i0, i1)) + present = bool(mask_arr[index[0], index[1], index[2]].any()) + return np.broadcast_to(np.bool_(present), shape) + + return da.map_blocks( + block_has_brain, + chunks=chunks, + dtype=bool, + meta=np.array([], dtype=bool), + ) + + +def plan_chunks(shape, block, halo): + """Chunk sizes for `shape` at `block`, with no chunk shorter than `halo`. + + A plain rechunk leaves a remainder chunk that can be far shorter than the + halo (z=1030 at block 512 leaves a 6-voxel tail), and dask refuses to overlap + a chunk smaller than the depth. A short tail is absorbed into its + predecessor instead. Every chunk start stays a multiple of `block`, so the + per-block tile grid stays aligned with the global one. + """ + chunks = [(shape[0],)] + for extent in shape[1:]: + if extent <= block: + chunks.append((extent,)) + continue + full, remainder = divmod(extent, block) + if remainder == 0: + chunks.append((block,) * full) + elif remainder < halo: + chunks.append((block,) * (full - 1) + (block + remainder,)) + else: + chunks.append((block,) * full + (remainder,)) + return tuple(chunks) + + +def vote_map(data, brain, predict, tile, stride): + """Lazily map `predict` over `data`, skipping blocks that carry no brain. + + The halo is `tile - stride`, so every voxel in a block's core is covered by + at least one tile lying wholly inside the block that produced it -- which is + what makes the stride-64 overlap correct across block boundaries and not just + within them. `brain` is halo-expanded alongside `data`, so the skip test is + conservative: a block is processed if its core *or* its halo touches brain. + + An axis that fits in a single chunk gets no halo: there is no block boundary + to bridge, and dask rejects a depth larger than the chunk. + """ + halo = tile - stride + + depth = {} + for axis, axis_chunks in enumerate(data.chunks): + if axis == 0 or len(axis_chunks) == 1: + depth[axis] = 0 + continue + if min(axis_chunks) < halo: + raise ValueError( + f"axis {axis} has a chunk of {min(axis_chunks)} voxels, shorter " + f"than the {halo}-voxel halo; use plan_chunks() to lay out the " + f"chunks before calling vote_map()" + ) + depth[axis] = halo + + def segment_block(image_block, brain_block): + if not brain_block.any(): + return np.zeros(image_block.shape, dtype=np.uint8) + out = np.empty(image_block.shape, dtype=np.uint8) + for c in range(image_block.shape[0]): + out[c] = predict(image_block[c].astype(np.float32)) + return out + + return da.map_overlap( + segment_block, + data, + brain, + depth=depth, + boundary="none", + trim=True, + allow_rechunk=False, + dtype=np.uint8, + meta=np.array([], dtype=np.uint8), + ) + + +def main(): + tile = int(snakemake.params.tile) + stride = int(snakemake.params.stride) + batch_size = int(snakemake.params.batch_size) + chunk = int(snakemake.params.chunk) + n_gpus = int(snakemake.params.n_gpus) + + devices = resolve_devices(n_gpus) + + with get_dask_client("threads", snakemake.threads): + znimg = ZarrNii.from_file( + snakemake.input.spim, + level=int(snakemake.wildcards.level), + channel_labels=[snakemake.wildcards.stain], + **snakemake.params.zarrnii_kwargs, + ) + + if znimg.data.ndim != 4 or znimg.data.shape[0] != 1: + raise ValueError( + f"expected a single-channel (c,z,y,x) image, got shape {znimg.data.shape}" + ) + data = znimg.data.rechunk(plan_chunks(znimg.data.shape, chunk, tile - stride)) + + brain = build_brain_block_mask(snakemake.input.mask, data.shape, data.chunks) + + pool = DevicePool(devices, snakemake.input.models) + n_folds = len(pool.nets[devices[0]]) + print( + f"loaded {n_folds} folds on each of {[str(d) for d in devices]}; " + f"grid {data.shape} chunk {chunk} halo {tile - stride} " + f"tile {tile} stride {stride} batch {batch_size}", + flush=True, + ) + + def predict(vol): + with pool.acquire() as (device, nets): + return predict_volume(vol, nets, device, tile, stride, batch_size) + + votes = vote_map(data, brain, predict, tile, stride) + + # The fraction of folds voting foreground: float32 in {0, .2, .4, .6, .8, 1} + # for a 5-fold ensemble. Same form as LANTERN's own whole-brain probmask. + # + # Deliberately a probability rather than the raw count or a 0-100 rescaling: + # it is directly readable as model confidence in a viewer, and independent of + # how many folds the ensemble happens to have. The 0/100 mask convention does + # not apply -- that exists so fieldfrac can mean-pool a *mask* into a + # percentage, and nothing but binarize_lantern_plaques reads this store. + # + # float32 costs ~4 bytes/voxel, so a level-1 whole brain is ~100 GB raw; it + # compresses hard, being overwhelmingly zero. + znimg_prob = znimg.copy() + znimg_prob.data = votes.astype(np.float32) / float(n_folds) + + with ProgressBar(): + znimg_prob.to_ome_zarr( + snakemake.output.probseg, + max_layer=5, + match_scale_factors_from=snakemake.input.spim, + **snakemake.config["zarrnii_out_kwargs"], + ) + + +if __name__ == "__main__": + main() From 9bede9ccd08a8456eaa769fcbe5479a1ccf4bbc4 Mon Sep 17 00:00:00 2001 From: apooladi Date: Tue, 11 Aug 2026 14:09:34 -0400 Subject: [PATCH 2/4] Treat segmentation as (method, stain) pairs, not a cross product Addresses the Copilot review on #192. The plaque method is stain-specific, so excluding it from `seg_methods` and letting every target rule expand `desc x stain` had two consequences: - With `--seg_method lantern` as the only method, the fallback to the CLI default expanded across all of stains_for_seg, so the amyloid channel was segmented twice -- once by LANTERN and once by the default GMM. A dry run confirms this scheduled a redundant gmmthresh, n4_biasfield and n4_pre_quant for that stain. - LANTERN itself appeared in none of the desc expansions, so it produced no regionprops, counts, segstats, QC or sidecars, and could not be compared against the method it is meant to replace. Replace the cross product with `stains_for_desc(desc)` and the `seg_expand` / `seg_expand_plain` helpers, which expand each method against its own stains. Cross-stain targets now use `coloc_seg_methods`, which drops single-stain methods, and `do_coloc` is derived from it. The cross-stain aggregations in segstats.smk and regionprops.smk become input functions of `wildcards.desc` for the same reason; this also removes the unguarded `stains_for_seg[0]`. LANTERN now produces the same downstream set as gmm+n3k1 apart from the coloc-* products, which are undefined for a single-stain method. Also from the same review: - Scope dynamic-network-architectures to the `runtime` feature. At workspace level it pulled a PyPI torch and CUDA stack into the lint-only `dev-only` environment, which has no other torch dependency. - Correct the plaque_vote_threshold error message, which described the vote map as raw counts after it changed to normalized fractions. Not changed: folds are still summed within a tile before tiles are combined with max. Those steps do not commute, but this is the order LANTERN's own infer_wholebrain.py uses and plaque_vote_threshold was calibrated under it. Documented in place so it is not "fixed" later by mistake. Verified by dry run against a synthetic BIDS tree: per-rule job counts for a run that never mentions lantern are byte-identical to the previous commit. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018WbbraRDoezJrijVaAsL97 --- pixi.lock | 427 +----------------- pyproject.toml | 8 +- spimquant/workflow/Snakefile | 167 ++++--- spimquant/workflow/rules/regionprops.smk | 39 +- spimquant/workflow/rules/segstats.smk | 115 +++-- .../scripts/binarize_lantern_plaques.py | 2 +- spimquant/workflow/scripts/lantern_plaques.py | 7 + .../merge_indiv_and_coloc_segstats_tsv.py | 4 +- 8 files changed, 215 insertions(+), 554 deletions(-) diff --git a/pixi.lock b/pixi.lock index 71705db..fd342c2 100644 --- a/pixi.lock +++ b/pixi.lock @@ -1678,27 +1678,18 @@ environments: - pypi: https://files.pythonhosted.org/packages/57/d8/8d44036d7eb7b6a8ec4c5494ea0c8c8b94fbc0ed3991c1a7adf230df03bf/aiohttp-3.13.5-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl - pypi: https://files.pythonhosted.org/packages/10/a1/510b0a7fadc6f43a6ce50152e69dbd86415240835868bb0bd9b5b88b1e06/aioitertools-0.13.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/fb/76/641ae371508676492379f16e2fa48f4e2c11741bd63c48be4b12a6b09cba/aiosignal-1.4.0-py3-none-any.whl - - pypi: https://files.pythonhosted.org/packages/3e/30/e900b21425a860e195f32e37657aa1f7c7f2b1bfb26f03ca209b90933c06/annotated_doc-0.0.5-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/bf/4b/afc1fef8a43bafb139f57f73bbd70df82807af5934321e8112ae50668827/botocore-1.43.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/88/39/799be3f2f0f38cc727ee3b4f1445fe6d5e4133064ec2e4115069418a5bb6/cloudpickle-3.1.2-py3-none-any.whl - - pypi: https://files.pythonhosted.org/packages/39/2a/6d2e9047d1fb243dbaa364b01e0297534b9ed7fd27dba1c9f361519cf69b/cuda_bindings-13.3.1-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl - - pypi: https://files.pythonhosted.org/packages/fc/b4/d088047afe39827556df21118cac9ffd20cc3f968c99a7681494d1eb333c/cuda_pathfinder-1.6.0-py3-none-any.whl - - pypi: https://files.pythonhosted.org/packages/d1/c7/a79086a62c98befcdb8349656c6f114e2db3b8b2422f6e25c97a7f2a9a3c/cuda_toolkit-13.0.3.0-py2.py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/4a/f3/00bb1e867fba351e2d784170955713bee200c43ea306c59f30bd7e748192/dask-2026.3.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/6f/4b/817da308fa1170da07ef01259585887a3bbb6ab80700b3e61ce4967301ec/dask_image-2025.11.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/84/d0/205d54408c08b13550c733c4b85429e7ead111c7f0014309637425520a9a/deprecated-1.3.1-py2.py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/b9/77/2ff7aefc09cf1306a81cd7a46af34f80ebefef81a2e8329b94b58ad813ae/distributed-2026.3.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/0c/d5/c5db1ea3394c6e1732fb3286b3bd878b59507a8f77d32a2cebda7d7b7cd4/donfig-0.8.1.post1-py3-none-any.whl - - pypi: https://files.pythonhosted.org/packages/2f/e3/07a70c9b83e5446528dd5d222fdca88ab30efcb07094536a864546c0096f/dynamic_network_architectures-0.4.4-py3-none-any.whl - - pypi: https://files.pythonhosted.org/packages/2a/09/f8d8f8f31e4483c10a906437b4ce31bdf3d6d417b73fe33f1a8b59e34228/einops-0.8.2-py3-none-any.whl - - pypi: https://files.pythonhosted.org/packages/c1/e8/72f8cef9fdfeffe06213fe8508039396ee48daa0e3259457ed766173bfd6/filelock-3.32.2-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/6a/bd/d91c5e39f490a49df14320f4e8c80161cfcce09f1e2cde1edd16a551abb3/frozenlist-1.8.0-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl - pypi: https://files.pythonhosted.org/packages/d5/0c/043d5e551459da400957a1395e0febbf771446ff34291afcbe3d8be2a279/fsspec-2026.4.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/ce/a9/a780cc66f86335a6019f557a8aaca8fbb970728f0efd2430d15ff1beae0e/google_crc32c-1.8.0-cp312-cp312-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl - pypi: https://files.pythonhosted.org/packages/9e/e9/1a19e42cd43cc1365e127db6aae85e1c671da1d9a5d746f4d34a50edb577/h5py-3.16.0-cp312-cp312-manylinux_2_28_x86_64.whl - pypi: https://files.pythonhosted.org/packages/ed/2f/1046d464ad1db29a4f6c70ba4e19b39baa8a6542c719eaa4e765108f07f1/hdf5plugin-6.0.0-py3-none-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl - - pypi: https://files.pythonhosted.org/packages/67/4e/a28359bf1c1ecf11eba22123168c138698f7cb576ac678f5a2e16cd5da08/hf_xet-1.6.0-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl - - pypi: https://files.pythonhosted.org/packages/49/79/621a7dbb80c70974f73a597275351ebe03ce5bc65cb5f8f4acb5859252bc/huggingface_hub-1.16.1-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/c0/47/7dbdec41b18f4c9aa90240c8804482a8c4ada6db3d2844e287695cd42109/imaris_ims_zarr-0.2.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/8a/db/55a262f3606bebcae07cc14095338471ad7c0bbcaa37707e6f0ee49725b7/importlib_resources-7.1.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/aa/57/0a3499479cb19d7e4b7fc38b2ba15c0ea20e1e88cb2b634b75dd3e9ff8b8/itk_core-5.4.6-cp311-abi3-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl @@ -1716,27 +1707,11 @@ environments: - pypi: https://files.pythonhosted.org/packages/b3/81/4da04ced5a082363ecfa159c010d200ecbd959ae410c10c0264a38cac0f5/markdown_it_py-4.2.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/b3/38/89ba8ad64ae25be8de66a6d463314cf1eb366222074cfda9ee839c56a4b4/mdurl-0.1.2-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/3a/cb/28ce52eb94390dda42599c98ea0204d74799e4d8047a0eb559b6fd648056/ml_dtypes-0.5.4-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl - - pypi: https://files.pythonhosted.org/packages/43/e3/7d92a15f894aa0c9c4b49b8ee9ac9850d6e63b03c9c32c0367a13ae62209/mpmath-1.3.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/65/92/a5100f7185a800a5d29f8d14041f61475b9de465ffcc0f3b9fba606e4505/msgpack-1.1.2-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl - pypi: https://files.pythonhosted.org/packages/f3/8d/5e5be3ced1d12966fefb5c4ea3b2a5b480afcea36406559442c6e31d4a48/multidict-6.7.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl - pypi: https://files.pythonhosted.org/packages/48/c4/7748c764c297c2469f26902fa70d72dbf026219e2a7a5ce2dc53fc519eff/ngff_zarr-0.34.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/3b/d7/601b6396b33536811668935faa790112266c70661be94555999be431f86f/nibabel-5.4.2-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/fb/53/78c98ef5c8b2b784453487f3e4d6c017b20747c58b470393e230c78d18e8/numcodecs-0.16.5-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl - - pypi: https://files.pythonhosted.org/packages/3b/cd/154ca20c38269e05eff77c1464e6c1da89f50a6390b565e9d82e06bc11e1/nvidia_cublas-13.1.1.3-py3-none-manylinux_2_27_x86_64.whl - - pypi: https://files.pythonhosted.org/packages/33/6d/737d164b4837a9bbd202f5ae3078975f0525a55730fe871d8ed4e3b952b0/nvidia_cuda_cupti-13.0.85-py3-none-manylinux_2_25_x86_64.whl - - pypi: https://files.pythonhosted.org/packages/c3/68/483a78f5e8f31b08fb1bb671559968c0ca3a065ac7acabfc7cee55214fd6/nvidia_cuda_nvrtc-13.0.88-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl - - pypi: https://files.pythonhosted.org/packages/2e/24/d1558f3b68b1d26e706813b1d10aa1d785e4698c425af8db8edc3dced472/nvidia_cuda_runtime-13.0.96-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl - - pypi: https://files.pythonhosted.org/packages/6e/5e/edb9c0ae051602c3ccaffe424256463636d639e27d7f302dde9975ef9e7a/nvidia_cudnn_cu13-9.20.0.48-py3-none-manylinux_2_27_x86_64.whl - - pypi: https://files.pythonhosted.org/packages/a8/2f/7b57e29836ea8714f81e9898409196f47d772d5ddedddf1592eadb8ab743/nvidia_cufft-12.0.0.61-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl - - pypi: https://files.pythonhosted.org/packages/3f/70/4f193de89a48b71714e74602ee14d04e4019ad36a5a9f20c425776e72cd6/nvidia_cufile-1.15.1.6-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl - - pypi: https://files.pythonhosted.org/packages/a5/9f/be0a41ca4a4917abf5cb9ae0daff1a6060cc5de950aec0396de9f3b52bc5/nvidia_curand-10.4.0.35-py3-none-manylinux_2_27_x86_64.whl - - pypi: https://files.pythonhosted.org/packages/5f/67/cba3777620cdacb99102da4042883709c41c709f4b6323c10781a9c3aa34/nvidia_cusolver-12.0.4.66-py3-none-manylinux_2_27_x86_64.whl - - pypi: https://files.pythonhosted.org/packages/fa/18/623c77619c31d62efd55302939756966f3ecc8d724a14dab2b75f1508850/nvidia_cusparse-12.6.3.3-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl - - pypi: https://files.pythonhosted.org/packages/34/7d/2661f2fb3ac4302f3a246f5fc030213ac60c1fe0bce84f9783dbd831dbb7/nvidia_cusparselt_cu13-0.8.1-py3-none-manylinux2014_x86_64.whl - - pypi: https://files.pythonhosted.org/packages/67/f4/58e4e91b6919367c7aafb8e36fce9aad1a3047e536bf7e2fd560927d3a4c/nvidia_nccl_cu13-2.29.7-py3-none-manylinux_2_18_x86_64.whl - - pypi: https://files.pythonhosted.org/packages/f0/ee/580ca6f29dcab0221db8706badca1bbbb084f1975c4d4e83329c3a7e31f0/nvidia_nvjitlink-13.3.33-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl - - pypi: https://files.pythonhosted.org/packages/3c/35/a9bf80a609e74e3b000fef598933235c908fcefcef9026042b8e6dfde2a9/nvidia_nvshmem_cu13-3.4.5-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl - - pypi: https://files.pythonhosted.org/packages/c2/f3/d86c845465a2723ad7e1e5c36dcd75ddb82898b3f53be47ebd429fb2fa5d/nvidia_nvtx-13.0.85-py3-none-manylinux1_x86_64.manylinux_2_5_x86_64.whl - pypi: https://files.pythonhosted.org/packages/99/81/4c41331f59e64b2a5eaec1c03819d5791e4dde321694fbda661f873184da/ome_zarr-0.16.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/71/e7/40fb618334dcdf7c5a316c0e7343c5cd82d3d866edc100d98e29bc945ecd/partd-1.4.2-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/b8/02/5bf3639f5b77e9b183011c08541c5039ba3d04f5316c70312b48a8e003a9/pims-0.7.tar.gz @@ -1747,19 +1722,11 @@ environments: - pypi: https://files.pythonhosted.org/packages/82/3b/64d4899d73f91ba49a8c18a8ff3f0ea8f1c1d75481760df8c68ef5235bf5/rich-15.0.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/0b/35/1cceccc5fcb50fa2ed53e2aa278cd032f3902682a73e763fb1ac3be8e6fa/rich_argparse-1.8.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/5d/a4/9d1ea10ebc9e028a289a72fec84da170689549a8102c8aacfcad26bc5035/s3fs-2026.4.0-py3-none-any.whl - - pypi: https://files.pythonhosted.org/packages/28/50/f203ff3a3ddfe19308efc83c5a3a29ed02bf786732ec35e68bf9162f3365/safetensors-0.8.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl - - pypi: https://files.pythonhosted.org/packages/e0/f9/0595336914c5619e5f28a1fb793285925a8cd4b432c9da0a987836c7f822/shellingham-1.5.4-py2.py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/c3/5d/5e017f59bbc06af5b9c78331036478b2ddb371d7ff810c559c36e5389904/simpleitk-2.5.4-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl - pypi: https://files.pythonhosted.org/packages/e8/ae/fa6cd331b364ad2bbc31652d025f5747d89cbb75576733dfdf8efe3e4d62/slicerator-1.1.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/32/46/9cb0e58b2deb7f82b84065f37f3bffeb12413f947f9388e4cac22c4621ce/sortedcontainers-2.4.0-py2.py3-none-any.whl - - pypi: https://files.pythonhosted.org/packages/a2/09/77d55d46fd61b4a135c444fc97158ef34a095e5681d0a6c10b75bf356191/sympy-1.14.0-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/02/be/5d2d47b1fb58943194fb59dcf222f7c4e35122ec0ffe8c36e18b5d728f0b/tblib-3.2.2-py3-none-any.whl - pypi: https://files.pythonhosted.org/packages/d1/14/7688e984cdd0be1438779825640b943574b89946ed868d76497b3cffb3d5/tensorstore-0.1.83-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl - - pypi: https://files.pythonhosted.org/packages/d6/14/fc04d491527b774ec7479897f5861959209de1480e4c4cd32ed098ff8bea/timm-1.0.22-py3-none-any.whl - - pypi: https://files.pythonhosted.org/packages/f3/82/fea946351658e6534db52d2cc12bc53087cbf87f9440c5f180f367c1950b/torch-2.13.0-cp312-cp312-manylinux_2_28_x86_64.whl - - pypi: https://files.pythonhosted.org/packages/93/e4/e9b2495d0d57b9f60d63c57d0a910410a81b4b073bf70917bef815291119/torchvision-0.28.0-cp312-cp312-manylinux_2_28_x86_64.whl - - pypi: https://files.pythonhosted.org/packages/c4/6f/fb96d15db6f36d6eae4cafb998c2e0353bf59d7c4ea1662d7497f269134a/triton-3.7.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl - - pypi: https://files.pythonhosted.org/packages/43/89/9518bc0c3929bee36b3a4a8e3daddd6e03f92f9961c66d4983b837160543/typer-0.27.1-py3-none-any.whl - pypi: git+https://github.com/khanlab/vesselfm?rev=504baab#504baabb253c2b6bf7d5a4037a93c2f590c9da51 - pypi: https://files.pythonhosted.org/packages/c5/c1/9c64ed6c2d152b56482bb66297bfc03442d25449bfa91da7bf2e58a80f01/wasmtime-44.0.0-py3-none-manylinux1_x86_64.whl - pypi: https://files.pythonhosted.org/packages/af/aa/ead46a88f9ec3a432a4832dfedb84092fc35af2d0ba40cd04aea3889f247/wrapt-2.1.2-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl @@ -2681,11 +2648,6 @@ packages: - pkg:pypi/amply?source=hash-mapping size: 21899 timestamp: 1734603085333 -- pypi: https://files.pythonhosted.org/packages/3e/30/e900b21425a860e195f32e37657aa1f7c7f2b1bfb26f03ca209b90933c06/annotated_doc-0.0.5-py3-none-any.whl - name: annotated-doc - version: 0.0.5 - sha256: 117bac03a25ede5df5440e855b32d556049ca169ead221505badf432fed4b101 - requires_python: '>=3.9' - conda: https://conda.anaconda.org/conda-forge/noarch/annotated-doc-0.0.4-pyhcf101f3_0.conda sha256: cc9fbc50d4ee7ee04e49ee119243e6f1765750f0fd0b4d270d5ef35461b643b1 md5: 52be5139047efadaeeb19c6a5103f92a @@ -4229,17 +4191,6 @@ packages: - pkg:pypi/cryptography?source=hash-mapping size: 2534262 timestamp: 1775637873338 -- pypi: https://files.pythonhosted.org/packages/39/2a/6d2e9047d1fb243dbaa364b01e0297534b9ed7fd27dba1c9f361519cf69b/cuda_bindings-13.3.1-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl - name: cuda-bindings - version: 13.3.1 - sha256: e32d08f71ebcdf00f0f41eab2eb37e8da94c8ed411cc9f7f7a019ce6b34abe3a - requires_dist: - - cuda-pathfinder>=1.4.2 - - cuda-toolkit[nvfatbin,nvjitlink,nvrtc,nvvm]==13.* ; extra == 'all' - - cuda-toolkit[cufile]==13.* ; sys_platform == 'linux' and extra == 'all' - - cuda-toolkit==13.* ; extra == 'all' - - nvidia-cudla==13.* ; platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all' - requires_python: '>=3.10' - conda: https://conda.anaconda.org/conda-forge/linux-64/cuda-crt-tools-12.6.85-ha770c72_0.conda sha256: 83b6f3332a17bc891f2ecdc9b1424658009e37e14e888d0bd0458b6aa4db59a2 md5: 4ab193b5fcdcf8d7b094221e3977a112 @@ -4360,75 +4311,6 @@ packages: purls: [] size: 10880815 timestamp: 1732132210850 -- pypi: https://files.pythonhosted.org/packages/fc/b4/d088047afe39827556df21118cac9ffd20cc3f968c99a7681494d1eb333c/cuda_pathfinder-1.6.0-py3-none-any.whl - name: cuda-pathfinder - version: 1.6.0 - sha256: 1503af579d8379c24bdd65528379bc57039b0455be9f5f9686cf8e473a1fce51 - requires_python: '>=3.10' -- pypi: https://files.pythonhosted.org/packages/d1/c7/a79086a62c98befcdb8349656c6f114e2db3b8b2422f6e25c97a7f2a9a3c/cuda_toolkit-13.0.3.0-py2.py3-none-any.whl - name: cuda-toolkit - version: 13.0.3.0 - sha256: d693caaa261214ddd7dbb60d68e71cbed884e68c2be7509778f3051da0b91c3f - requires_dist: - - nvidia-cublas==13.1.1.3.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') - - nvidia-cuda-cccl==13.0.85.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') - - nvidia-cuda-crt==13.0.88.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') - - nvidia-cuda-culibos==13.0.85.* ; (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') - - nvidia-cuda-cupti==13.0.85.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') - - nvidia-cuda-cuxxfilt==13.0.85.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') - - nvidia-cuda-nvcc==13.0.88.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') - - nvidia-cuda-nvrtc==13.0.88.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') - - nvidia-cuda-opencl==13.0.85.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') - - nvidia-cuda-profiler-api==13.0.85.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') - - nvidia-cuda-runtime==13.0.96.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') - - nvidia-cuda-sanitizer-api==13.0.85.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') - - nvidia-cufft==12.0.0.61.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') - - nvidia-cufile==1.15.1.6.* ; (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') - - nvidia-curand==10.4.0.35.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') - - nvidia-cusolver==12.0.4.66.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') - - nvidia-cusparse==12.6.3.3.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') - - nvidia-npp==13.0.1.2.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') - - nvidia-nvfatbin==13.0.85.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') - - nvidia-nvjitlink>=13.0.88,<14 ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') - - nvidia-nvjpeg==13.0.1.86.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') - - nvidia-nvml-dev==13.0.87.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') - - nvidia-nvptxcompiler==13.0.88.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') - - nvidia-nvtx==13.0.85.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') - - nvidia-nvvm==13.0.88.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'all') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'all') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'all') - - nvidia-cuda-cccl==13.0.85.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'cccl') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'cccl') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'cccl') - - nvidia-cuda-crt==13.0.88.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'crt') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'crt') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'crt') - - nvidia-cublas==13.1.1.3.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'cublas') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'cublas') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'cublas') - - nvidia-cuda-nvrtc==13.0.88.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'cublas') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'cublas') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'cublas') - - nvidia-cuda-runtime==13.0.96.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'cudart') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'cudart') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'cudart') - - nvidia-cufft==12.0.0.61.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'cufft') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'cufft') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'cufft') - - nvidia-nvjitlink>=13.0.88,<14 ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'cufft') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'cufft') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'cufft') - - nvidia-cufile==1.15.1.6.* ; (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'cufile') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'cufile') - - nvidia-cuda-culibos==13.0.85.* ; (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'culibos') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'culibos') - - nvidia-cuda-cupti==13.0.85.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'cupti') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'cupti') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'cupti') - - nvidia-curand==10.4.0.35.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'curand') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'curand') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'curand') - - nvidia-cublas==13.1.1.3.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'cusolver') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'cusolver') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'cusolver') - - nvidia-cusolver==12.0.4.66.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'cusolver') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'cusolver') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'cusolver') - - nvidia-cusparse==12.6.3.3.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'cusolver') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'cusolver') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'cusolver') - - nvidia-nvjitlink>=13.0.88,<14 ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'cusolver') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'cusolver') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'cusolver') - - nvidia-cusparse==12.6.3.3.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'cusparse') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'cusparse') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'cusparse') - - nvidia-nvjitlink>=13.0.88,<14 ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'cusparse') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'cusparse') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'cusparse') - - nvidia-cuda-cuxxfilt==13.0.85.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'cuxxfilt') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'cuxxfilt') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'cuxxfilt') - - nvidia-npp==13.0.1.2.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'npp') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'npp') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'npp') - - nvidia-cuda-crt==13.0.88.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'nvcc') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'nvcc') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'nvcc') - - nvidia-cuda-nvcc==13.0.88.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'nvcc') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'nvcc') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'nvcc') - - nvidia-cuda-runtime==13.0.96.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'nvcc') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'nvcc') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'nvcc') - - nvidia-nvvm==13.0.88.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'nvcc') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'nvcc') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'nvcc') - - nvidia-nvfatbin==13.0.85.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'nvfatbin') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'nvfatbin') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'nvfatbin') - - nvidia-nvjitlink>=13.0.88,<14 ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'nvjitlink') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'nvjitlink') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'nvjitlink') - - nvidia-nvjpeg==13.0.1.86.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'nvjpeg') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'nvjpeg') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'nvjpeg') - - nvidia-nvml-dev==13.0.87.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'nvml') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'nvml') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'nvml') - - nvidia-nvptxcompiler==13.0.88.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'nvptxcompiler') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'nvptxcompiler') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'nvptxcompiler') - - nvidia-cuda-nvrtc==13.0.88.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'nvrtc') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'nvrtc') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'nvrtc') - - nvidia-nvtx==13.0.85.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'nvtx') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'nvtx') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'nvtx') - - nvidia-nvvm==13.0.88.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'nvvm') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'nvvm') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'nvvm') - - nvidia-cuda-opencl==13.0.85.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'opencl') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'opencl') - - nvidia-cuda-profiler-api==13.0.85.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'profiler') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'profiler') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'profiler') - - nvidia-cuda-sanitizer-api==13.0.85.* ; (platform_machine == 'AMD64' and sys_platform == 'win32' and extra == 'sanitizer') or (platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'sanitizer') or (platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'sanitizer') - conda: https://conda.anaconda.org/conda-forge/noarch/cuda-version-12.6-h7480c83_3.conda sha256: fd9104d73199040285b6a6ad56322b38af04828fabbac1f5a268a83509358425 md5: 1c8b99e65a4423b1e4ac2e4c76fb0978 @@ -5148,11 +5030,6 @@ packages: purls: [] size: 2195198 timestamp: 1776781721834 -- pypi: https://files.pythonhosted.org/packages/c1/e8/72f8cef9fdfeffe06213fe8508039396ee48daa0e3259457ed766173bfd6/filelock-3.32.2-py3-none-any.whl - name: filelock - version: 3.32.2 - sha256: 87dd94cf281e586d135fa51132b8e3d9a598b316e90377a288663c9321036c82 - requires_python: '>=3.10' - conda: https://conda.anaconda.org/conda-forge/noarch/filelock-3.29.0-pyhd8ed1ab_0.conda sha256: 6b471a18372bbd52bdf32fc965f71de3bc1b5219418b8e6b3875a67a7b08c483 md5: 8fa8358d022a3a9bd101384a808044c6 @@ -6235,13 +6112,6 @@ packages: - pkg:pypi/heapdict?source=hash-mapping size: 10169 timestamp: 1733762704231 -- pypi: https://files.pythonhosted.org/packages/67/4e/a28359bf1c1ecf11eba22123168c138698f7cb576ac678f5a2e16cd5da08/hf_xet-1.6.0-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl - name: hf-xet - version: 1.6.0 - sha256: d62671bb130879cef0ee4c9ebe47a14af6c66ec53e6d84dc15936e5ffdfac82f - requires_dist: - - pytest ; extra == 'tests' - requires_python: '>=3.8' - conda: https://conda.anaconda.org/conda-forge/linux-64/hf-xet-1.5.0-py310hb823017_0.conda noarch: python sha256: b0f16f161bae4bdef033d56678a9eda4d07f5aa300db19d58e5e73acb3028b3d @@ -6323,124 +6193,6 @@ packages: - pkg:pypi/httpx?source=hash-mapping size: 63082 timestamp: 1733663449209 -- pypi: https://files.pythonhosted.org/packages/49/79/621a7dbb80c70974f73a597275351ebe03ce5bc65cb5f8f4acb5859252bc/huggingface_hub-1.16.1-py3-none-any.whl - name: huggingface-hub - version: 1.16.1 - sha256: 64340de934b9ce37857ef85a82de72f5629e8a270f9119eabb12bf495eb53c22 - requires_dist: - - filelock>=3.10.0 - - fsspec>=2023.5.0 - - hf-xet>=1.4.3,<2.0.0 ; platform_machine == 'AMD64' or platform_machine == 'aarch64' or platform_machine == 'amd64' or platform_machine == 'arm64' or platform_machine == 'x86_64' - - httpx>=0.23.0,<1 - - packaging>=20.9 - - pyyaml>=5.1 - - tqdm>=4.42.1 - - typer>=0.20.0 - - typing-extensions>=4.1.0 - - authlib>=1.3.2 ; extra == 'oauth' - - fastapi ; extra == 'oauth' - - httpx ; extra == 'oauth' - - itsdangerous ; extra == 'oauth' - - torch ; extra == 'torch' - - safetensors[torch] ; extra == 'torch' - - toml ; extra == 'fastai' - - fastai>=2.4 ; extra == 'fastai' - - fastcore>=1.3.27 ; extra == 'fastai' - - hf-xet>=1.4.3,<2.0.0 ; extra == 'hf-xet' - - mcp>=1.8.0 ; extra == 'mcp' - - authlib>=1.3.2 ; extra == 'testing' - - fastapi ; extra == 'testing' - - httpx ; extra == 'testing' - - itsdangerous ; extra == 'testing' - - jedi ; extra == 'testing' - - jinja2 ; extra == 'testing' - - pytest>=8.4.2 ; extra == 'testing' - - pytest-cov ; extra == 'testing' - - pytest-env ; extra == 'testing' - - pytest-xdist ; extra == 'testing' - - pytest-vcr ; extra == 'testing' - - pytest-asyncio ; extra == 'testing' - - pytest-rerunfailures<16.0 ; extra == 'testing' - - pytest-mock ; extra == 'testing' - - urllib3<2.0 ; extra == 'testing' - - soundfile ; extra == 'testing' - - pillow ; extra == 'testing' - - numpy ; extra == 'testing' - - duckdb ; extra == 'testing' - - fastapi ; extra == 'testing' - - gradio>=5.0.0 ; extra == 'gradio' - - requests ; extra == 'gradio' - - typing-extensions>=4.8.0 ; extra == 'typing' - - types-pyyaml ; extra == 'typing' - - types-simplejson ; extra == 'typing' - - types-toml ; extra == 'typing' - - types-tqdm ; extra == 'typing' - - types-urllib3 ; extra == 'typing' - - ruff>=0.9.0 ; extra == 'quality' - - mypy==1.15.0 ; extra == 'quality' - - libcst>=1.4.0 ; extra == 'quality' - - ty ; extra == 'quality' - - authlib>=1.3.2 ; extra == 'all' - - fastapi ; extra == 'all' - - httpx ; extra == 'all' - - itsdangerous ; extra == 'all' - - jedi ; extra == 'all' - - jinja2 ; extra == 'all' - - pytest>=8.4.2 ; extra == 'all' - - pytest-cov ; extra == 'all' - - pytest-env ; extra == 'all' - - pytest-xdist ; extra == 'all' - - pytest-vcr ; extra == 'all' - - pytest-asyncio ; extra == 'all' - - pytest-rerunfailures<16.0 ; extra == 'all' - - pytest-mock ; extra == 'all' - - urllib3<2.0 ; extra == 'all' - - soundfile ; extra == 'all' - - pillow ; extra == 'all' - - numpy ; extra == 'all' - - duckdb ; extra == 'all' - - fastapi ; extra == 'all' - - ruff>=0.9.0 ; extra == 'all' - - mypy==1.15.0 ; extra == 'all' - - libcst>=1.4.0 ; extra == 'all' - - ty ; extra == 'all' - - typing-extensions>=4.8.0 ; extra == 'all' - - types-pyyaml ; extra == 'all' - - types-simplejson ; extra == 'all' - - types-toml ; extra == 'all' - - types-tqdm ; extra == 'all' - - types-urllib3 ; extra == 'all' - - authlib>=1.3.2 ; extra == 'dev' - - fastapi ; extra == 'dev' - - httpx ; extra == 'dev' - - itsdangerous ; extra == 'dev' - - jedi ; extra == 'dev' - - jinja2 ; extra == 'dev' - - pytest>=8.4.2 ; extra == 'dev' - - pytest-cov ; extra == 'dev' - - pytest-env ; extra == 'dev' - - pytest-xdist ; extra == 'dev' - - pytest-vcr ; extra == 'dev' - - pytest-asyncio ; extra == 'dev' - - pytest-rerunfailures<16.0 ; extra == 'dev' - - pytest-mock ; extra == 'dev' - - urllib3<2.0 ; extra == 'dev' - - soundfile ; extra == 'dev' - - pillow ; extra == 'dev' - - numpy ; extra == 'dev' - - duckdb ; extra == 'dev' - - fastapi ; extra == 'dev' - - ruff>=0.9.0 ; extra == 'dev' - - mypy==1.15.0 ; extra == 'dev' - - libcst>=1.4.0 ; extra == 'dev' - - ty ; extra == 'dev' - - typing-extensions>=4.8.0 ; extra == 'dev' - - types-pyyaml ; extra == 'dev' - - types-simplejson ; extra == 'dev' - - types-toml ; extra == 'dev' - - types-tqdm ; extra == 'dev' - - types-urllib3 ; extra == 'dev' - requires_python: '>=3.10.0' - conda: https://conda.anaconda.org/conda-forge/noarch/huggingface_hub-1.14.0-pyhd8ed1ab_0.conda sha256: 7a6810ae5ff35c04e7160f748fb536ca2be3ffb0997ef15ac0dbb6db3e4e874d md5: a293e6cbdb2daaa9419c9c3432f10b31 @@ -10220,19 +9972,6 @@ packages: purls: [] size: 491140 timestamp: 1730581373280 -- pypi: https://files.pythonhosted.org/packages/43/e3/7d92a15f894aa0c9c4b49b8ee9ac9850d6e63b03c9c32c0367a13ae62209/mpmath-1.3.0-py3-none-any.whl - name: mpmath - version: 1.3.0 - sha256: a0b2b9fe80bbcd81a6647ff13108738cfb482d481d826cc0e02f5b35e5c88d2c - requires_dist: - - pytest>=4.6 ; extra == 'develop' - - pycodestyle ; extra == 'develop' - - pytest-cov ; extra == 'develop' - - codecov ; extra == 'develop' - - wheel ; extra == 'develop' - - sphinx ; extra == 'docs' - - gmpy2>=2.1.0a4 ; platform_python_implementation != 'PyPy' and extra == 'gmpy' - - pytest>=4.6 ; extra == 'tests' - conda: https://conda.anaconda.org/conda-forge/noarch/mpmath-1.4.1-pyhd8ed1ab_0.conda sha256: 5bbf2f8179ec43d34d67ca8e4989d216c1bdb4b749fe6cb40e86ebf88c1b5300 md5: 2e81b32b805f406d23ba61938a184081 @@ -10981,72 +10720,6 @@ packages: - pkg:pypi/numpydoc?source=hash-mapping size: 65801 timestamp: 1764715638266 -- pypi: https://files.pythonhosted.org/packages/3b/cd/154ca20c38269e05eff77c1464e6c1da89f50a6390b565e9d82e06bc11e1/nvidia_cublas-13.1.1.3-py3-none-manylinux_2_27_x86_64.whl - name: nvidia-cublas - version: 13.1.1.3 - sha256: 37936a16db8fe4ac1f065c2139360608a543a09275cb1a1af612e08cfa065436 - requires_dist: - - nvidia-cuda-nvrtc - requires_python: '>=3' -- pypi: https://files.pythonhosted.org/packages/33/6d/737d164b4837a9bbd202f5ae3078975f0525a55730fe871d8ed4e3b952b0/nvidia_cuda_cupti-13.0.85-py3-none-manylinux_2_25_x86_64.whl - name: nvidia-cuda-cupti - version: 13.0.85 - sha256: 4eb01c08e859bf924d222250d2e8f8b8ff6d3db4721288cf35d14252a4d933c8 - requires_python: '>=3' -- pypi: https://files.pythonhosted.org/packages/c3/68/483a78f5e8f31b08fb1bb671559968c0ca3a065ac7acabfc7cee55214fd6/nvidia_cuda_nvrtc-13.0.88-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl - name: nvidia-cuda-nvrtc - version: 13.0.88 - sha256: ad9b6d2ead2435f11cbb6868809d2adeeee302e9bb94bcf0539c7a40d80e8575 - requires_python: '>=3' -- pypi: https://files.pythonhosted.org/packages/2e/24/d1558f3b68b1d26e706813b1d10aa1d785e4698c425af8db8edc3dced472/nvidia_cuda_runtime-13.0.96-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl - name: nvidia-cuda-runtime - version: 13.0.96 - sha256: 7f82250d7782aa23b6cfe765ecc7db554bd3c2870c43f3d1821f1d18aebf0548 - requires_python: '>=3' -- pypi: https://files.pythonhosted.org/packages/6e/5e/edb9c0ae051602c3ccaffe424256463636d639e27d7f302dde9975ef9e7a/nvidia_cudnn_cu13-9.20.0.48-py3-none-manylinux_2_27_x86_64.whl - name: nvidia-cudnn-cu13 - version: 9.20.0.48 - sha256: 0c45dd8eeb50b603f07995b1b300c62ffe6a1980482b82b3bcf94a4ca9d49304 - requires_dist: - - nvidia-cublas - requires_python: '>=3' -- pypi: https://files.pythonhosted.org/packages/a8/2f/7b57e29836ea8714f81e9898409196f47d772d5ddedddf1592eadb8ab743/nvidia_cufft-12.0.0.61-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl - name: nvidia-cufft - version: 12.0.0.61 - sha256: 6c44f692dce8fd5ffd3e3df134b6cdb9c2f72d99cf40b62c32dde45eea9ddad3 - requires_dist: - - nvidia-nvjitlink - requires_python: '>=3' -- pypi: https://files.pythonhosted.org/packages/3f/70/4f193de89a48b71714e74602ee14d04e4019ad36a5a9f20c425776e72cd6/nvidia_cufile-1.15.1.6-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl - name: nvidia-cufile - version: 1.15.1.6 - sha256: 08a3ecefae5a01c7f5117351c64f17c7c62efa5fffdbe24fc7d298da19cd0b44 - requires_python: '>=3' -- pypi: https://files.pythonhosted.org/packages/a5/9f/be0a41ca4a4917abf5cb9ae0daff1a6060cc5de950aec0396de9f3b52bc5/nvidia_curand-10.4.0.35-py3-none-manylinux_2_27_x86_64.whl - name: nvidia-curand - version: 10.4.0.35 - sha256: 1aee33a5da6e1db083fe2b90082def8915f30f3248d5896bcec36a579d941bfc - requires_python: '>=3' -- pypi: https://files.pythonhosted.org/packages/5f/67/cba3777620cdacb99102da4042883709c41c709f4b6323c10781a9c3aa34/nvidia_cusolver-12.0.4.66-py3-none-manylinux_2_27_x86_64.whl - name: nvidia-cusolver - version: 12.0.4.66 - sha256: 0a759da5dea5c0ea10fd307de75cdeb59e7ea4fcb8add0924859b944babf1112 - requires_dist: - - nvidia-cublas - - nvidia-nvjitlink - - nvidia-cusparse - requires_python: '>=3' -- pypi: https://files.pythonhosted.org/packages/fa/18/623c77619c31d62efd55302939756966f3ecc8d724a14dab2b75f1508850/nvidia_cusparse-12.6.3.3-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl - name: nvidia-cusparse - version: 12.6.3.3 - sha256: 2b3c89c88d01ee0e477cb7f82ef60a11a4bcd57b6b87c33f789350b59759360b - requires_dist: - - nvidia-nvjitlink - requires_python: '>=3' -- pypi: https://files.pythonhosted.org/packages/34/7d/2661f2fb3ac4302f3a246f5fc030213ac60c1fe0bce84f9783dbd831dbb7/nvidia_cusparselt_cu13-0.8.1-py3-none-manylinux2014_x86_64.whl - name: nvidia-cusparselt-cu13 - version: 0.8.1 - sha256: 786ce87568c303fadb5afcc7102d454cd3040d75f6f8626f5db460d1871f4dd0 - conda: https://conda.anaconda.org/conda-forge/noarch/nvidia-ml-py-13.595.45-pyhd8ed1ab_1.conda sha256: fb46556c423311638fbb26ed4b2a67cf598044919e42ab7571365a5b4ae3b663 md5: dc8587ae654e96031728802016e8258c @@ -11061,26 +10734,6 @@ packages: - pkg:pypi/nvidia-ml-py?source=hash-mapping size: 48878 timestamp: 1775592349734 -- pypi: https://files.pythonhosted.org/packages/67/f4/58e4e91b6919367c7aafb8e36fce9aad1a3047e536bf7e2fd560927d3a4c/nvidia_nccl_cu13-2.29.7-py3-none-manylinux_2_18_x86_64.whl - name: nvidia-nccl-cu13 - version: 2.29.7 - sha256: edd81538446786ec3b73972543e53bb43bcaf0bfc8ef76cb679fcc390ffe136d - requires_python: '>=3' -- pypi: https://files.pythonhosted.org/packages/f0/ee/580ca6f29dcab0221db8706badca1bbbb084f1975c4d4e83329c3a7e31f0/nvidia_nvjitlink-13.3.33-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl - name: nvidia-nvjitlink - version: 13.3.33 - sha256: 26a6de7fb4c8fdaa7703d3dad720d6d427ddfea5c48a528fd97c11733ad830e5 - requires_python: '>=3' -- pypi: https://files.pythonhosted.org/packages/3c/35/a9bf80a609e74e3b000fef598933235c908fcefcef9026042b8e6dfde2a9/nvidia_nvshmem_cu13-3.4.5-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl - name: nvidia-nvshmem-cu13 - version: 3.4.5 - sha256: 290f0a2ee94c9f3687a02502f3b9299a9f9fe826e6d0287ee18482e78d495b80 - requires_python: '>=3' -- pypi: https://files.pythonhosted.org/packages/c2/f3/d86c845465a2723ad7e1e5c36dcd75ddb82898b3f53be47ebd429fb2fa5d/nvidia_nvtx-13.0.85-py3-none-manylinux1_x86_64.manylinux_2_5_x86_64.whl - name: nvidia-nvtx - version: 13.0.85 - sha256: 4936d1d6780fbe68db454f5e72a42ff64d1fd6397df9f363ae786930fd5c1cd4 - requires_python: '>=3' - conda: https://conda.anaconda.org/conda-forge/noarch/oauthlib-3.3.1-pyhd8ed1ab_0.conda sha256: dfa8222df90736fa13f8896f5a573a50273af8347542d412c3bd1230058e56a5 md5: d4f3f31ee39db3efecb96c0728d4bdbf @@ -13744,11 +13397,6 @@ packages: purls: [] size: 113547 timestamp: 1756649518540 -- pypi: https://files.pythonhosted.org/packages/e0/f9/0595336914c5619e5f28a1fb793285925a8cd4b432c9da0a987836c7f822/shellingham-1.5.4-py2.py3-none-any.whl - name: shellingham - version: 1.5.4 - sha256: 7ecfff8f2fd72616f7481040475a65b2bf8af90a56c89140852d1120324e8686 - requires_python: '>=3.7' - conda: https://conda.anaconda.org/conda-forge/noarch/shellingham-1.5.4-pyhd8ed1ab_2.conda sha256: 1d6534df8e7924d9087bd388fbac5bd868c5bf8971c36885f9f016da0657d22b md5: 83ea3a2ddb7a75c1b09cea582aa4f106 @@ -14340,7 +13988,7 @@ packages: - pypi: ./ name: spimquant version: 0.1.0 - sha256: 78e687dadf8c0111fa639960e2f1d8a830e2a15017815ec96243cd2f42a17604 + sha256: f51bae3a4dc97ad20fa034c702e598b9becd0971f6d09d50f79df5f8b72c8d1d requires_python: '>=3.11' editable: true - conda: https://conda.anaconda.org/conda-forge/linux-64/spirv-tools-2025.5-hb700be7_0.conda @@ -14474,15 +14122,6 @@ packages: purls: [] size: 2619743 timestamp: 1769664536467 -- pypi: https://files.pythonhosted.org/packages/a2/09/77d55d46fd61b4a135c444fc97158ef34a095e5681d0a6c10b75bf356191/sympy-1.14.0-py3-none-any.whl - name: sympy - version: 1.14.0 - sha256: e091cc3e99d2141a0ba2847328f5479b05d94a6635cb96148ccb3f34671bd8f5 - requires_dist: - - mpmath>=1.1.0,<1.4 - - pytest>=7.1.0 ; extra == 'dev' - - hypothesis>=6.70.0 ; extra == 'dev' - requires_python: '>=3.9' - conda: https://conda.anaconda.org/conda-forge/noarch/sympy-1.14.0-pyh2585a3b_106.conda sha256: 1c8057e6875eba958aa8b3c1a072dc9a75d013f209c26fd8125a5ebd3abbec0c md5: 32d866e43b25275f61566b9391ccb7b5 @@ -14800,29 +14439,6 @@ packages: - pkg:pypi/toolz?source=hash-mapping size: 53978 timestamp: 1760707830681 -- pypi: https://files.pythonhosted.org/packages/f3/82/fea946351658e6534db52d2cc12bc53087cbf87f9440c5f180f367c1950b/torch-2.13.0-cp312-cp312-manylinux_2_28_x86_64.whl - name: torch - version: 2.13.0 - sha256: 796633c4cdf0fe2cdced72d8f88f22e73dbcfce83132763162f6d4bff13b820b - requires_dist: - - filelock - - typing-extensions>=4.10.0 - - setuptools>=77.0.3 - - sympy>=1.13.3 - - networkx>=2.5.1 - - jinja2 - - fsspec>=0.8.5 - - cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.3 ; sys_platform == 'linux' - - cuda-bindings>=13.0.3,<14 ; python_full_version < '3.15' and sys_platform == 'linux' - - nvidia-cudnn-cu13==9.20.0.48 ; sys_platform == 'linux' - - nvidia-cusparselt-cu13==0.8.1 ; sys_platform == 'linux' - - nvidia-nccl-cu13==2.29.7 ; sys_platform == 'linux' - - nvidia-nvshmem-cu13==3.4.5 ; sys_platform == 'linux' - - triton==3.7.1 ; python_full_version < '3.15' and sys_platform == 'linux' - - optree>=0.13.0 ; extra == 'optree' - - opt-einsum>=3.3 ; extra == 'opt-einsum' - - pyyaml ; extra == 'pyyaml' - requires_python: '>=3.10' - conda: https://conda.anaconda.org/conda-forge/noarch/torch_em-0.8.3-pyhd8ed1ab_0.conda sha256: 12b8b552f16f0115b62055f752b6bbde44363d5c36346a75097effd62cf814f4 md5: 43f7a6f953343a444a35ccc99614170b @@ -14874,17 +14490,6 @@ packages: - gdown>=4.7.3 ; extra == 'gdown' - scipy ; extra == 'scipy' requires_python: '>=3.9' -- pypi: https://files.pythonhosted.org/packages/93/e4/e9b2495d0d57b9f60d63c57d0a910410a81b4b073bf70917bef815291119/torchvision-0.28.0-cp312-cp312-manylinux_2_28_x86_64.whl - name: torchvision - version: 0.28.0 - sha256: 028a3d481b37d785605620d7cdad897064c5a55bae2aa1f2658766333e291940 - requires_dist: - - numpy - - torch==2.13.0 - - pillow>=5.3.0,!=8.3.* - - gdown>=4.7.3 ; extra == 'gdown' - - scipy ; extra == 'scipy' - requires_python: '>=3.10,!=3.14.1' - conda: https://conda.anaconda.org/conda-forge/linux-64/torchvision-0.24.0-cpu_py312_h56cfa8b_0.conda sha256: a22214010fd1f3398ad7ea8311e0d76b0f7750590b502b0a73f57c14ca806595 md5: feb0c7bde000f25617408266492d2d2a @@ -14970,26 +14575,6 @@ packages: - pkg:pypi/traitlets?source=hash-mapping size: 115165 timestamp: 1778074251714 -- pypi: https://files.pythonhosted.org/packages/c4/6f/fb96d15db6f36d6eae4cafb998c2e0353bf59d7c4ea1662d7497f269134a/triton-3.7.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl - name: triton - version: 3.7.1 - sha256: 7e40869937a68206ec70d7f25bb7ec6433cb083f9135e1f36dbd318dc449a728 - requires_dist: - - importlib-metadata ; python_full_version < '3.10' - - cmake>=3.20,<4.0 ; extra == 'build' - - lit ; extra == 'build' - - autopep8 ; extra == 'tests' - - isort ; extra == 'tests' - - numpy ; extra == 'tests' - - pytest ; extra == 'tests' - - pytest-forked ; extra == 'tests' - - pytest-xdist ; extra == 'tests' - - scipy>=1.7.1 ; extra == 'tests' - - llnl-hatchet ; extra == 'tests' - - matplotlib ; extra == 'tutorials' - - pandas ; extra == 'tutorials' - - tabulate ; extra == 'tutorials' - requires_python: '>=3.10,<3.15' - conda: https://conda.anaconda.org/conda-forge/linux-64/triton-3.3.0-cuda126py312hebffaa9_1.conda sha256: 7089c27a38fc3ec199af4d51fcbba33720281f3098e984c49a9f010805d2de84 md5: a05b9a73fe6a9be82a2fc4af2b01e95f @@ -15026,16 +14611,6 @@ packages: - pkg:pypi/truststore?source=hash-mapping size: 24279 timestamp: 1766494826559 -- pypi: https://files.pythonhosted.org/packages/43/89/9518bc0c3929bee36b3a4a8e3daddd6e03f92f9961c66d4983b837160543/typer-0.27.1-py3-none-any.whl - name: typer - version: 0.27.1 - sha256: 53150287edd11baeb4e4722c8e394fcdf8181c0ae89485cba8d25c778d5edd56 - requires_dist: - - shellingham>=1.3.0 - - rich>=13.8.0 - - annotated-doc>=0.0.2 - - colorama ; sys_platform == 'win32' - requires_python: '>=3.10' - conda: https://conda.anaconda.org/conda-forge/noarch/typer-0.25.1-pyhcf101f3_0.conda sha256: 18fc3a27bc995318d09142fe16d01ea454e76f377bf8f68db03b8b18f11085ed md5: ef114c2eb2ff19f6bf616c81f4710841 diff --git a/pyproject.toml b/pyproject.toml index acd3c98..31b28ff 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -54,7 +54,6 @@ spimquant = { path = ".", editable = true } zarrnii = ">=0.21.2,<0.22.0" #zarrnii = { path = "/nfs/khan/trainees/akhan488/apps/zarrnii", editable = true } vesselfm = { git = "https://github.com/khanlab/vesselfm", rev = "504baab" } -dynamic-network-architectures = ">=0.4" h5py = ">=3.14.0" simpleitk = ">=2.4.0" pylibtiff = ">=0.7.0" @@ -80,6 +79,13 @@ features = ["runtime"] [tool.pixi.environments.gpu] features = ["runtime", "gpu"] +# Scoped to `runtime` rather than the workspace-global pypi-dependencies: it is a +# torch package, and putting it at workspace level pulls a PyPI torch + CUDA stack +# into the lint-only `dev-only` environment, which has no other torch dependency. +[tool.pixi.feature.runtime.pypi-dependencies] +dynamic-network-architectures = ">=0.4" + + [tool.pixi.feature.gpu.system-requirements] cuda = "12.0" diff --git a/spimquant/workflow/Snakefile b/spimquant/workflow/Snakefile index bc4d994..4af152e 100644 --- a/spimquant/workflow/Snakefile +++ b/spimquant/workflow/Snakefile @@ -195,18 +195,79 @@ if do_plaques: f"The LANTERN ensemble only segments amyloid-beta." ) -# Methods applied to every stain in stains_for_seg. The plaque method is excluded -# because it is stain-specific -- keeping it out of this list is what stops it from -# entering the desc x stain expansions and the cross-stain aggregation, exactly as -# vessel_seg_method stays out of them. +# Segmentation is a set of (method, stain) pairs, not a method x stain cross +# product: the generic intensity-based methods run on every stain, while the plaque +# method runs only on stain_for_plaques. Everything below exists to keep those two +# groups from contaminating each other. # -# If the plaque method was the ONLY one requested, the remaining stains must behave -# exactly as if --seg_method had never been passed. That default is read back off -# the CLI definition rather than restated here, so it cannot drift out of sync with -# the actual default if the latter ever changes. -seg_methods = [ +# Generic methods, i.e. everything the user asked for except the plaque method. +generic_seg_methods = [ m for m in config["seg_method"] if m != config["plaque_seg_method"] -] or list(config["parse_args"]["--seg_method"]["default"]) +] + +if generic_seg_methods: + # The user asked for generic methods explicitly, so they apply everywhere -- + # including the plaque stain, which is how `--seg_method gmm+n3k1 lantern` + # produces a like-for-like comparison on the same channel. + seg_methods = generic_seg_methods + stains_for_seg_methods = stains_for_seg +else: + # The plaque method was the ONLY one requested. The remaining stains must behave + # exactly as if --seg_method had never been passed -- but the plaque stain must + # NOT, or it would be segmented twice, once by LANTERN and once by the default + # GMM. The default is read back off the CLI definition rather than restated here + # so it cannot drift out of sync if the actual default ever changes. + seg_methods = list(config["parse_args"]["--seg_method"]["default"]) + stains_for_seg_methods = [s for s in stains_for_seg if s != stain_for_plaques] + +# Every method that produces a mask, plaque method included. +all_seg_methods = seg_methods + ([config["plaque_seg_method"]] if do_plaques else []) + + +def stains_for_desc(desc): + """Stains that segmentation method `desc` produces a mask for.""" + if do_plaques and desc == config["plaque_seg_method"]: + return [stain_for_plaques] + return stains_for_seg_methods + + +# Methods that segment two or more stains, and so can be colocalized across them. +# The plaque method is single-stain by construction, so it drops out here. +coloc_seg_methods = [m for m in all_seg_methods if len(stains_for_desc(m)) > 1] + +# Recomputed now that the plaque stain is known: colocalization is only meaningful +# if some method segments two or more stains. Requesting the plaque method alone on +# a single-stain dataset leaves nothing to colocalize. +do_coloc = do_seg and len(coloc_seg_methods) > 0 + + +def seg_expand(paths, **wildcards): + """Expand `paths` over every (desc, stain) segmentation pair. + + Replaces `inputs["spim"].expand(..., desc=seg_methods, stain=stains_for_seg)`. + Expanding each method against its own stains and concatenating gives the plaque + method the same per-subject downstream products as every other method + (featuremaps, fieldfrac, counts, regionprops, segstats, QC) without the generic + methods leaking onto the plaque stain. + + With no plaque method requested this returns exactly what the cross product did. + """ + out = [] + for desc in all_seg_methods: + stains = stains_for_desc(desc) + if stains: + out += inputs["spim"].expand(paths, desc=[desc], stain=stains, **wildcards) + return out + + +def seg_expand_plain(paths, **wildcards): + """`seg_expand` for group-level paths, which carry no subject wildcards.""" + out = [] + for desc in all_seg_methods: + stains = stains_for_desc(desc) + if stains: + out += expand(paths, desc=[desc], stain=stains, **wildcards) + return out # atlas segmentations to use @@ -621,7 +682,7 @@ rule all_plaques: rule all_segment: input: - inputs["spim"].expand( + seg_expand( bids( root=root, datatype="featuremap", @@ -632,10 +693,8 @@ rule all_segment: **inputs["spim"].wildcards, ), seg=atlas_segs, - desc=seg_methods, template=config["template"], suffix=config["seg_metrics"], - stain=stains_for_seg, ), inputs["spim"].expand( bids( @@ -651,7 +710,7 @@ rule all_segment: level=config["registration_level"], template=config["template"], ), - inputs["spim"].expand( + seg_expand( bids( root=root, datatype="seg", @@ -662,12 +721,10 @@ rule all_segment: suffix="fieldfrac.nii.gz", **inputs["spim"].wildcards, ), - stain=stains_for_seg, level=config["registration_level"], - desc=seg_methods, template=config["template"], ), - inputs["spim"].expand( + seg_expand( bids( root=root, datatype="seg", @@ -677,11 +734,9 @@ rule all_segment: suffix="counts.nii.gz", **inputs["spim"].wildcards, ), - stain=stains_for_seg, level=config["registration_level"], - desc=seg_methods, ), - inputs["spim"].expand( + seg_expand( bids( root=root, datatype="seg", @@ -691,9 +746,7 @@ rule all_segment: suffix="counts.nii.gz", **inputs["spim"].wildcards, ), - stain=stains_for_seg, template=config["template"], - desc=seg_methods, ), expand( bids( @@ -721,7 +774,7 @@ rule all_segment_coloc: **inputs["spim"].wildcards, ), seg=atlas_segs, - desc=seg_methods, + desc=coloc_seg_methods, template=config["template"], suffix=config["coloc_seg_metrics"], ), @@ -734,7 +787,7 @@ rule all_segment_coloc: suffix="coloccounts.nii.gz", **inputs["spim"].wildcards, ), - desc=seg_methods, + desc=coloc_seg_methods, template=config["template"], ), @@ -785,7 +838,7 @@ rule all_spim_patches: level=config["segmentation_level"], desc=["raw", "corrected" + config["correction_method"]], ), - inputs["spim"].expand( + seg_expand( bids( root=root, datatype="seg", @@ -797,10 +850,8 @@ rule all_spim_patches: suffix="mask.patches", **inputs["spim"].wildcards, ), - stain=stains_for_seg, seg=patch_atlas_segs, template=config["template"], - desc=seg_methods, level=config["segmentation_level"], ), @@ -838,13 +889,13 @@ rule all_group_stats: suffix="groupstats.{ext}", ), seg=atlas_segs, - desc=seg_methods, + desc=all_seg_methods, template=config["template"], pairwise_contrast=pairwise_contrast_labels, ext=["png", "tsv"], ), # Per-contrast NIfTI stat maps for each seg metric - expand( + seg_expand_plain( bids( root=group_stats_root, seg="{seg}", @@ -855,15 +906,13 @@ rule all_group_stats: suffix="{stat}.nii.gz", ), seg=atlas_segs, - desc=seg_methods, template=config["template"], - stain=stains_for_seg, metric=config["seg_metrics"], stat=config["stats_maps"], pairwise_contrast=pairwise_contrast_labels, ), # All-subjects count maps (not contrast-specific) - expand( + seg_expand_plain( bids( root=group_stats_root, level="{level}", @@ -871,10 +920,8 @@ rule all_group_stats: desc="{desc}", suffix="{stain}+count.nii.gz", ), - desc=seg_methods, template=config["template"], level=range(4), - stain=stains_for_seg, ), @@ -892,7 +939,7 @@ rule all_group_stats_coloc: suffix="{stat}.nii.gz", ), seg=atlas_segs, - desc=seg_methods, + desc=coloc_seg_methods, template=config["template"], metric=config["coloc_seg_metrics"], stat=config["stats_maps"], @@ -907,7 +954,7 @@ rule all_group_stats_coloc: desc="{desc}", suffix="coloccount.nii.gz", ), - desc=seg_methods, + desc=coloc_seg_methods, template=config["template"], level=range(4), ), @@ -928,7 +975,7 @@ rule all_qc: stain=stains, ), # Segmentation overview figures (per stain, per seg method) - inputs["spim"].expand( + seg_expand( bids( root=root, datatype="qc", @@ -937,13 +984,11 @@ rule all_qc: suffix="segslices.png", **inputs["spim"].wildcards, ), - stain=stains_for_seg, - desc=seg_methods, ) if do_seg else [], # Segmentation ROI zoom montage (per stain, per atlas, per seg method) - inputs["spim"].expand( + seg_expand( bids( root=root, datatype="qc", @@ -956,13 +1001,11 @@ rule all_qc: ), seg=atlas_segs, template=config["template"], - stain=stains_for_seg, - desc=seg_methods, ) if do_seg else [], # Segmentation ROI zoom montage - no mask overlay - inputs["spim"].expand( + seg_expand( bids( root=root, datatype="qc", @@ -975,8 +1018,6 @@ rule all_qc: ), seg=atlas_segs, template=config["template"], - stain=stains_for_seg, - desc=seg_methods, ) if do_seg else [], @@ -1034,7 +1075,7 @@ rule all_qc: if do_vessels else [], # Z-profile QC (per stain, per seg method) - inputs["spim"].expand( + seg_expand( bids( root=root, datatype="qc", @@ -1043,13 +1084,11 @@ rule all_qc: suffix="zprofile.png", **inputs["spim"].wildcards, ), - stain=stains_for_seg, - desc=seg_methods, ) if do_seg else [], # Object-level statistics (per stain, per seg method) - inputs["spim"].expand( + seg_expand( bids( root=root, datatype="qc", @@ -1058,8 +1097,6 @@ rule all_qc: suffix="objectstats.png", **inputs["spim"].wildcards, ), - stain=stains_for_seg, - desc=seg_methods, ) if do_seg else [], @@ -1076,12 +1113,12 @@ rule all_qc: ), seg=atlas_segs, template=config["template"], - desc=seg_methods, + desc=all_seg_methods, ) if do_seg else [], # Instance-centred animated GIFs (per stain, per atlas seg, per seg method) - inputs["spim"].expand( + seg_expand( [ bids( root=root, @@ -1106,8 +1143,6 @@ rule all_qc: ], seg=atlas_segs, template=config["template"], - stain=stains_for_seg, - desc=seg_methods, ) if do_seg and config["qc_instance_seg"] else [], @@ -1135,7 +1170,7 @@ rule all_qc: ], seg=atlas_segs, template=config["template"], - desc=seg_methods, + desc=coloc_seg_methods, ) if do_coloc and config["qc_instance_seg"] else [], @@ -1156,12 +1191,12 @@ rule all_tabular_sidecars: **inputs["spim"].wildcards, ), seg=atlas_segs, - desc=seg_methods, + desc=all_seg_methods, template=config["template"], ) if do_seg else [], - inputs["spim"].expand( + seg_expand( bids( root=root, datatype="tabular", @@ -1174,9 +1209,7 @@ rule all_tabular_sidecars: **inputs["spim"].wildcards, ), seg=atlas_segs, - desc=seg_methods, template=config["template"], - stain=stains_for_seg, ) if do_seg else [], @@ -1190,7 +1223,7 @@ rule all_tabular_sidecars: suffix="regionprops.json", **inputs["spim"].wildcards, ), - desc=seg_methods, + desc=all_seg_methods, template=config["template"], ) if do_seg @@ -1204,7 +1237,7 @@ rule all_tabular_sidecars: suffix="coloc.json", **inputs["spim"].wildcards, ), - desc=seg_methods, + desc=coloc_seg_methods, template=config["template"], ) if do_coloc @@ -1225,7 +1258,7 @@ rule all_group_stats_tabular_sidecars: suffix="groupstats.json", ), seg=atlas_segs, - desc=seg_methods, + desc=all_seg_methods, template=config["template"], pairwise_contrast=pairwise_contrast_labels, ) @@ -1241,7 +1274,7 @@ rule all_group_stats_tabular_sidecars: suffix="allsubjects.json", ), seg=atlas_segs, - desc=seg_methods, + desc=all_seg_methods, template=config["template"], ) if do_seg @@ -1254,7 +1287,7 @@ rule all_group_stats_tabular_sidecars: desc="{desc}", suffix="regionprops.json", ), - desc=seg_methods, + desc=all_seg_methods, template=config["template"], ) if do_seg @@ -1266,7 +1299,7 @@ rule all_group_stats_tabular_sidecars: desc="{desc}", suffix="coloc.json", ), - desc=seg_methods, + desc=coloc_seg_methods, template=config["template"], ) if do_coloc @@ -1286,7 +1319,7 @@ rule all_group_merged_tsvs: suffix="allsubjects.tsv", ), seg=atlas_segs, - desc=seg_methods, + desc=all_seg_methods, template=config["template"], ) if do_seg diff --git a/spimquant/workflow/rules/regionprops.smk b/spimquant/workflow/rules/regionprops.smk index d3594d6..78293fd 100644 --- a/spimquant/workflow/rules/regionprops.smk +++ b/spimquant/workflow/rules/regionprops.smk @@ -68,24 +68,35 @@ rule transform_regionprops_to_template: "../scripts/transform_regionprops_to_template.py" +def get_regionprops_parquets_to_aggregate(wildcards): + """Per-stain regionprops for the stains this method actually segmented. + + Cannot be resolved at parse time: a single-stain method such as the plaque + ensemble aggregates over only its own stain, not all of stains_for_seg. + """ + paths = expand( + bids( + root=root, + datatype="tabular", + stain="{stain}", + desc="{desc}", + space="{template}", + suffix="regionprops.parquet", + **inputs["spim"].wildcards, + ), + stain=stains_for_desc(wildcards.desc), + allow_missing=True, + ) + return [path.format(**wildcards) for path in paths] + + rule aggregate_regionprops_across_stains: """Aggregate transformed regionprops across stains.""" input: - regionprops_parquets=expand( - bids( - root=root, - datatype="tabular", - stain="{stain}", - desc="{desc}", - space="{template}", - suffix="regionprops.parquet", - **inputs["spim"].wildcards, - ), - stain=stains_for_seg, - allow_missing=True, - ), + regionprops_parquets=get_regionprops_parquets_to_aggregate, params: - stains=stains_for_seg, + # must stay aligned with regionprops_parquets, which is per-method + stains=lambda wildcards: stains_for_desc(wildcards.desc), output: regionprops_aggregated_parquet=bids( root=root, diff --git a/spimquant/workflow/rules/segstats.smk b/spimquant/workflow/rules/segstats.smk index 39e016b..f9e7afc 100644 --- a/spimquant/workflow/rules/segstats.smk +++ b/spimquant/workflow/rules/segstats.smk @@ -149,6 +149,33 @@ rule merge_into_segstats_tsv: "../scripts/merge_into_segstats_tsv.py" +def get_coloc_reference_fieldfrac(wildcards): + """Fieldfrac table used only to recover ROI volume, so count becomes density. + + Colocalization is cross-stain, so every stain this method segmented carries the + same ROI volume column and the first one is an arbitrary but stable choice. + Which stains a method segments depends on the method -- the plaque method covers + only its own stain -- so this cannot be resolved at parse time. + """ + stains = stains_for_desc(wildcards.desc) + if not stains: + raise ValueError( + f"--seg_method {wildcards.desc} segments no stains, so colocalization " + f"statistics cannot be computed for it" + ) + return bids( + root=root, + datatype="tabular", + seg="{seg}", + from_="{template}", + stain=stains[0], + level=config["registration_level"], + desc="{desc}", + suffix="fieldfracstats.tsv", + **inputs["spim"].wildcards, + ).format(**wildcards) + + rule merge_into_colocsegstats_tsv: """ also includes fieldfracstats.tsv to obtain the volume to turn count into density""" input: @@ -170,17 +197,7 @@ rule merge_into_colocsegstats_tsv: suffix="coloccountstats.tsv", **inputs["spim"].wildcards, ), - fieldfrac_tsv=bids( - root=root, - datatype="tabular", - seg="{seg}", - from_="{template}", - stain=stains_for_seg[0], - level=config["registration_level"], - desc="{desc}", - suffix="fieldfracstats.tsv", - **inputs["spim"].wildcards, - ), + fieldfrac_tsv=get_coloc_reference_fieldfrac, params: columns_to_drop=["fieldfrac"], output: @@ -203,44 +220,54 @@ rule merge_into_colocsegstats_tsv: "../scripts/merge_into_segstats_tsv.py" -def get_coloc_tsv_input_kwargs(): - """return coloc_tsv only if we have multiple stains to segment""" - if len(stains_for_seg) == 1: - return {} - else: - return { - "coloc_tsv": bids( - root=root, - datatype="tabular", - seg="{seg}", - from_="{template}", - desc="{desc}", - suffix="colocsegstats.tsv", - **inputs["spim"].wildcards, - ) - } +def get_coloc_tsv_input(wildcards): + """Colocalization table, only for methods that segment two or more stains. + + Single-stain methods (the plaque ensemble) have nothing to colocalize, so this + resolves to an empty list and the merge falls back to the per-stain tables. + """ + if wildcards.desc not in coloc_seg_methods: + return [] + return bids( + root=root, + datatype="tabular", + seg="{seg}", + from_="{template}", + desc="{desc}", + suffix="colocsegstats.tsv", + **inputs["spim"].wildcards, + ).format(**wildcards) + + +def get_indiv_segstats_tsvs(wildcards): + """Per-stain segstats tables for the stains this method actually segmented.""" + paths = expand( + bids( + root=root, + datatype="tabular", + seg="{seg}", + from_="{template}", + stain="{stain}", + level=config["registration_level"], + desc="{desc}", + suffix="segstats.tsv", + **inputs["spim"].wildcards, + ), + stain=stains_for_desc(wildcards.desc), + allow_missing=True, + ) + # Input functions are handed to snakemake verbatim, so the remaining wildcards + # have to be substituted here rather than left for the usual resolution pass. + return [path.format(**wildcards) for path in paths] rule merge_indiv_and_coloc_segstats_tsv: input: - **get_coloc_tsv_input_kwargs(), - indiv_tsvs=expand( - bids( - root=root, - datatype="tabular", - seg="{seg}", - from_="{template}", - stain="{stain}", - level=config["registration_level"], - desc="{desc}", - suffix="segstats.tsv", - **inputs["spim"].wildcards, - ), - stain=stains_for_seg, - allow_missing=True, - ), + coloc_tsv=get_coloc_tsv_input, + indiv_tsvs=get_indiv_segstats_tsvs, params: - stains=stains_for_seg, + # must stay aligned with indiv_tsvs, which is per-method + stains=lambda wildcards: stains_for_desc(wildcards.desc), output: merged_tsv=bids( root=root, diff --git a/spimquant/workflow/scripts/binarize_lantern_plaques.py b/spimquant/workflow/scripts/binarize_lantern_plaques.py index 1684f68..83b9115 100644 --- a/spimquant/workflow/scripts/binarize_lantern_plaques.py +++ b/spimquant/workflow/scripts/binarize_lantern_plaques.py @@ -44,7 +44,7 @@ def main(): if not 1 <= vote_threshold <= n_folds: raise ValueError( f"plaque_vote_threshold={vote_threshold} is out of range for an " - f"{n_folds}-fold ensemble; the vote map holds counts in 0..{n_folds}" + f"{n_folds}-fold ensemble; it must be between 1 and {n_folds}" ) probseg = ZarrNii.from_file(snakemake.input.probseg, level=0) diff --git a/spimquant/workflow/scripts/lantern_plaques.py b/spimquant/workflow/scripts/lantern_plaques.py index 97720a2..0eae48e 100644 --- a/spimquant/workflow/scripts/lantern_plaques.py +++ b/spimquant/workflow/scripts/lantern_plaques.py @@ -103,6 +103,13 @@ def predict_volume(vol, nets, device, tile, stride, batch_size): region = votes[z : z + tile, y : y + tile, x : x + tile] # MAX, not sum: a plaque clipped at one tile edge is recovered by the # tile that contains it whole. + # + # Folds are summed WITHIN a tile (above) and tiles are combined with max + # (here). Those two steps do not commute -- max-then-sum would score a + # voxel higher when different folds find it from different tiles -- but + # this order is the one LANTERN's own inference uses + # (scripts/infer_wholebrain.py), and plaque_vote_threshold was calibrated + # under it. Swapping them would silently change what "3 of 5" means. np.maximum(region, batch_votes[j], out=region) return votes[: orig_shape[0], : orig_shape[1], : orig_shape[2]] diff --git a/spimquant/workflow/scripts/merge_indiv_and_coloc_segstats_tsv.py b/spimquant/workflow/scripts/merge_indiv_and_coloc_segstats_tsv.py index 2aee731..12924fd 100644 --- a/spimquant/workflow/scripts/merge_indiv_and_coloc_segstats_tsv.py +++ b/spimquant/workflow/scripts/merge_indiv_and_coloc_segstats_tsv.py @@ -10,7 +10,9 @@ import pandas as pd indiv_files = snakemake.input.indiv_tsvs -coloc_file = getattr(snakemake.input, "coloc_tsv", None) +# Single-stain methods resolve coloc_tsv to an empty list rather than dropping the +# key, so an empty value has to be treated the same as a missing one. +coloc_file = getattr(snakemake.input, "coloc_tsv", None) or None output_file = snakemake.output.merged_tsv stains = snakemake.params.stains # list aligned to indiv_files From e4d1dadecfc530726f97be949a71ba48297a2f02 Mon Sep 17 00:00:00 2001 From: apooladi Date: Tue, 11 Aug 2026 14:40:06 -0400 Subject: [PATCH 3/4] Drop segmentation methods that are left covering no stains Requesting only the plaque method on a single-stain dataset (the actual production invocation: --seg_method lantern --stains_for_seg Abeta) removes that stain from stains_for_seg_methods and leaves the fallback generic method covering nothing. seg_expand already skipped it, but all_seg_methods still listed it, so the stain-less cross-stain targets kept requesting its products. Those rules then received an empty input list and failed at runtime -- merge_indiv_and_coloc_segstats_tsv died on `merged.to_csv` with merged=None. Filter all_seg_methods to methods that actually segment something. The previous commit's verification used two stains, which left one for the generic method and hid the degenerate case. Verified by dry run on a synthetic BIDS tree: - single stain + lantern: zero desc-gmm+n3k1 targets scheduled (was 12 merge_indiv_and_coloc_segstats_tsv + 3 aggregate_regionprops_across_stains, all of which failed on the cluster) - two stains + lantern: unchanged, gmm covers Iba1 and lantern covers Abeta - no lantern: per-rule job counts still byte-identical to 8bdb3ec Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018WbbraRDoezJrijVaAsL97 --- spimquant/workflow/Snakefile | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/spimquant/workflow/Snakefile b/spimquant/workflow/Snakefile index 4af152e..eee1ebb 100644 --- a/spimquant/workflow/Snakefile +++ b/spimquant/workflow/Snakefile @@ -220,9 +220,6 @@ else: seg_methods = list(config["parse_args"]["--seg_method"]["default"]) stains_for_seg_methods = [s for s in stains_for_seg if s != stain_for_plaques] -# Every method that produces a mask, plaque method included. -all_seg_methods = seg_methods + ([config["plaque_seg_method"]] if do_plaques else []) - def stains_for_desc(desc): """Stains that segmentation method `desc` produces a mask for.""" @@ -231,6 +228,21 @@ def stains_for_desc(desc): return stains_for_seg_methods +# Every method that produces a mask, plaque method included. +# +# Methods left with no stains are dropped. This is not hypothetical: requesting +# only the plaque method on a single-stain dataset removes that stain from +# stains_for_seg_methods and leaves the fallback generic method covering nothing. +# Keeping it here would schedule cross-stain aggregations (mergedsegstats, +# aggregated regionprops) with an empty input list, which fail at runtime rather +# than at parse time. +all_seg_methods = [ + m + for m in seg_methods + ([config["plaque_seg_method"]] if do_plaques else []) + if stains_for_desc(m) +] + + # Methods that segment two or more stains, and so can be colocalized across them. # The plaque method is single-stain by construction, so it drops out here. coloc_seg_methods = [m for m in all_seg_methods if len(stains_for_desc(m)) > 1] From 4a28212e5e40b30c992b1bf8b637561b37df2fda Mon Sep 17 00:00:00 2001 From: apooladi Date: Tue, 11 Aug 2026 15:09:35 -0400 Subject: [PATCH 4/4] Stop SLURM launching the plaque job once per GPU With `gpu: 2` and no explicit task setting, the SLURM executor appends `--ntasks-per-gpu=1` (submit_string.py:71-83). Combined with `--gpus=2` that requests two tasks, so srun runs the entire job twice, each task pinned to one device. Observed on the cluster: the log contains "loaded 5 folds" twice, "localrule run_lantern_plaques" twice, and WARNING: 2 GPUs requested but only 1 visible; using 1 twice. Both copies then infer over the whole volume and race to write the same probseg store. The executor already provides an escape hatch for this -- its own comment cites python process management interfering with SLURM for pytorch -- and skips the flag when the value is < 1. Set tasks_per_gpu=0, which yields --gpus=2 --cpus-per-gpu=8 i.e. one task that sees both devices, which is what DevicePool was written for. run_vesselfm is unaffected: gpu=1 with --ntasks-per-gpu=1 is a single task. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018WbbraRDoezJrijVaAsL97 --- spimquant/workflow/rules/plaques.smk | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/spimquant/workflow/rules/plaques.smk b/spimquant/workflow/rules/plaques.smk index 5d491e3..f4ac85d 100644 --- a/spimquant/workflow/rules/plaques.smk +++ b/spimquant/workflow/rules/plaques.smk @@ -68,6 +68,13 @@ rule run_lantern_plaques: threads: 8 * config["plaque_n_gpus"] resources: gpu=config["plaque_n_gpus"], + # Suppress the executor's default --ntasks-per-gpu=1. With --gpus=N that + # asks SLURM for N tasks, so srun launches the whole job N times, each task + # pinned to one GPU: the ensemble runs N times over the same volume, every + # copy sees a single device, and they race to write the same output store. + # Setting this to 0 skips the flag entirely (submit_string.py:82), leaving + # one task that sees all N GPUs -- which is what DevicePool expects. + tasks_per_gpu=0, cpus_per_gpu=8, mem_mb=256000, disk_mb=2097152,