v0.16.2

Changelog

Breaking Changes

• Split the ilamb dataset registry: it now holds only the 3 ILAMB reference products without an obs4REF equivalent (mrsol/WangMao, tas/CRU4.02, evspsbl/GLEAMv3.3a), and the two region masks moved to a new ilamb-regions registry.
  The iomb registry has been removed entirely as all diagnostics have been migrated to obs4REF.
  The ILAMB and IOMB diagnostic configuration files (configure/ilamb.yaml, configure/iomb.yaml) now declare realm: land / realm: ocean and, where relevant, region_masks: ilamb-regions in place of the old registry: key.
  The gpp-WECANN diagnostic's pr relationship now reads GPCP from obs4REF (NOAA-NCEI/GPCP-2-3/mon) instead of the retired ilamb/pr/GPCPv2.3/pr.nc (sha1 e1b942863ec76a75aa972b6d75e2e08646741259).
  This is a different processing of the same underlying product, so if it moves an ILAMB regression baseline the baseline will be re-minted separately. (#815)

Features

• Added dataset retraction.
  ref datasets retract <slug> marks a dataset ineligible for future solve-time selection
  without deleting its row or its execution provenance links,
  and ref datasets unretract <slug> reverses it.
  Retracting an already-retracted dataset (or unretracting an already-active one) is a no-op.
  ref datasets list now surfaces retraction state via a retracted_at column.
  Retracted datasets are excluded by default, matching what the solver selects,
  and can be shown with the new --include-retracted flag,
  which also lists every version of a dataset instead of only the latest.
  "Retract" follows the term ESGF uses for pulling a published dataset from recommended use. (#802)

• Added two new source dataset types, obs4REF and ESMValToolReference, together with their database tables.

  obs4REF is REF-curated observational data that follows the obs4MIPs metadata conventions that has not yet been published to the obs4MIPs ESGF archive.
  ESMValToolReference is ESMValTool's own reference data, which is not CMOR compliant and so carries a smaller set of metadata.
  The shared obs4MIPs-style column block used by obs4MIPs, PMPClimatology and now obs4REF was extracted into a ReferenceDatasetMixin. (#805)

• Adds dataset adapters for the obs4REF and ESMValTool reference dataset types, so both can now be ingested with ref datasets ingest --source-type obs4ref and ref datasets ingest --source-type esmvaltool-reference. obs4REF datasets also participate in solving, while ESMValTool reference datasets are ingest-only for now. (#814)

• Reworked the ILAMB provider to draw its reference observations from the ingested obs4MIPs and obs4REF datasets rather than its own bundled registry.
  The few observations not yet published there (currently WangMao, GLEAMv3 and CRU4.02) are still fetched from the provider registry at execute time. ILAMB region masks moved to their own support registry.

  Each dataset registry now declares which source type its files map to, if any, and whether it holds reference or support data. This is what lets the solver provision reference observations from the database. (#815)

• Added a --exclude-diagnostic option to ref solve.
  Diagnostics can be excluded by slug or by provider/diagnostic pair, matched exactly and case-insensitively.
  This uses the same token format as the REF_TEST_CASES_SKIP environment variable. (#820)

Improvements

• Reworked the cloud radiative effects diagnostic so it runs on CMIP7.
  Both source types now use the ref/recipe_ref_cre_cmip7.yml recipe,
  with the time range rewritten per source type: 2002-2014 for CMIP6 and 2002-2021 for CMIP7. (#683)
• Adds CMIP7 support to the ozone diagnostics.
  All five now build from the new ref/recipe_ref_ozone_cmip7.yml recipe.
  The lat-time, SH October, NH March and annual-cycle diagnostics run on both CMIP6 and CMIP7, and the stratospheric zonal-mean profile stays on CMIP6 with its reference switched to the ESACCI-OZONE L3-SAGE-OMPS product. (#704)
• Switched the PMP extratropical modes-of-variability diagnostics that use sea-level pressure
  (NAO, NAM, PNA, NPO, SAM) to the obs4MIPs 20CR-V2 reference, fetched from ESGF,
  in place of the obs4REF 20CR copy.
  Their regression baselines were re-minted against the new reference.
  The shared ExtratropicalModesOfVariability version bumps from 2 to 3.
  The surface-temperature modes (PDO, NPGO) share that class, so their baselines were
  re-minted under the new version too. Their reference is unchanged and only the
  last few significant figures move. (#801)
• ESMValTool reference data (the OBS, OBS6, native6 and obs4MIPs files in the provider's data.txt registry) is now ingested as the esmvaltool-reference source type, so it is recorded in the database. Selecting it at solve time is a separate follow-up. This also adds the CMIP5-era OImon sea-ice table to the MIP-table to frequency mapping, which the OSI-450 sea-ice references use. (#815)
• Reduced the climate-ref unit test runtime by reusing migrated database templates, parallelising pull request checks, and moving full archive solver regressions to the main slow test suite. (#825)

Bug Fixes

• Made the transient climate response to cumulative CO2 emissions (TCRE) diagnostic compatible with CMIP7 data. (#702)

• Moved the test-case catalog.paths.yaml sidecar out of the provider packages and into the dataset cache,
  at $REF_DATASET_CACHE_DIR/test-case-paths/{provider}/{diagnostic}/{test_case}/. (#806)

• Fixed ref test-cases fetch aborting the whole fetch loop when the solver found no
  matching datasets for a single test case.
  The failure is now logged and the remaining test cases are still fetched,
  so one broken case no longer leaves later diagnostics without catalogs and paths sidecars.

  Fixed the global-sst-bias example diagnostic to accumulate its global means in float64.
  The float32 reduction differed by ~1e-6 between platforms,
  which tripped the regression gate when baselines were minted on a different machine.
  Its regression baselines were re-minted and the diagnostic version bumps from 1 to 2.

  Loosened the regression comparison's default relative tolerance from 1e-6 to 1e-5,
  so platform-level floating-point noise no longer trips the gate.
  Committed bundles keep their 7 significant figures,
  which is now two digits finer than the tolerance. (#810)

• Handles an ESGF search that returns no results by raising DatasetResolutionError.
  This lets ref test-cases fetch log and skip a test case whose dataset has been de-indexed,
  instead of crashing the whole prefetch. (#811)

• Enables the CEDA Solr index as an additional ESGF search index when fetching data.
  The obs4MIPs ozone reference C3S-GTO-ECV-9-0 is currently missing from the default
  ESGF2-US-1.5-Catalog index but is still served by CEDA,
  so this lets ref test-cases fetch and scripts/fetch-esgf.py resolve it again. (#812)

• Fixed the PMP regression baselines drifting on CI.
  The annual-cycle climatology version stamp is now derived from the diagnostic version instead of the run date,
  the conda platform in CMEC provenance is redacted to a placeholder,
  and the affected baselines were re-minted on linux-64 to match the CI platform. (#816)

• Fixed the ozone-zonal diagnostic by shipping the ESACCI-OZONE L3-SAGE-OMPS ozone profile file
  that the updated ESMValTool recipe references in the dataset registry.
  Also re-minted the ESMValTool regression baselines that drifted after the ESMValTool version update
  and improved the error message for catalogs built from unfinalised CMIP7 datasets. (#817)

• Fixes the ILAMB lai-avh15c1 test case running the CI runner out of memory and getting killed with no logs.

  • Coarsens the very fine AVHRR LAI reference one time block at a time instead of loading the whole ~0.05 degree cube into memory at once
  • Caches the coarsened reference to disk, keyed on the source data and target resolution, because every model comparison in a solve reuses the same coarsened field. The first execution computes it, later ones read the cached file. Set REF_ILAMB_COARSEN_NO_CACHE to disable.
  • Logs wall clock, CPU time, and peak memory after each test case in ref test-cases run and the per-provider drift tests.
  • Re-mints the lai-avh15c1 baselines against the chunked coarsening, and the gpp-fluxnet2015 baselines to adopt the CF long_name a newer ILAMB now sets on the region traces.

  (#818)

• Fixed the GitHub release listing a stray default.gitignore alongside the wheels and sdists.
  uv build writes a .gitignore into the build output directory,
  and the release workflow's dist/* glob was uploading it as a release asset.
  The upload is now restricted to the built wheels and source distributions. ([#8...

v0.16.1

Announcements

Changelog

Bug Fixes

• default_ignore_datasets.yaml is now shipped inside the climate_ref package and resolved with importlib.resources.
  The config and solve_config pytest fixtures previously located it relative to the monorepo source tree, so they raised ValueError for any project that installs climate-ref from a wheel. (#800)

Changes

• 325ceaa Bump version: 0.16.0 → 0.16.1
• d89903c docs(changelog): add fragment for the packaging fix
• fb03dff fix(conftest_plugin): resolve default_ignore_datasets.yaml from the package

v0.16.0

Changelog

Features

• Metric values now carry a first-class kind of either model or reference,
  so model and reference (observation) values can be distinguished and filtered directly
  rather than inferred from provider-specific conventions.
  Reference series additionally record a stable reference_id content hash,
  so an identical observation produced by different executions is recognised as the same value.
  Series can also carry typed presentation metadata (value_units, value_long_name, index_units, and calendar). (#772)

• Every PMP scalar metric value now keys its reference identity on reference_source_id
  and declares its role with the first-class kind of model.
  The ENSO diagnostics previously keyed reference identity on reference_datasets,
  so a consumer now reads reference identity the same way for every PMP diagnostic.
  ENSO references that the metrics package scores against two observation datasets jointly
  (for example HadISST_GPCP-Monthly-3-2) are kept as a single combined reference_source_id value,
  because the score is one value that depends on both references and cannot be split per reference. (#775)

• ESMValTool series now carry an explicit kind of either model or reference,
  so an observation curve can be distinguished from a model curve directly
  rather than inferred from the presence of a reference source. (#776)

• ILAMB metric values now carry the shared kind field, distinguishing reference (observation) series from model series without the previous "Reference" sentinel.
  Series units are presented in a clean CF form so a model series and its reference agree, the typed presentation fields (value_units, value_long_name, index_units, calendar) are populated, and comparison scalars record the reference they were scored against so they can be grouped by it. (#777)

• The obs4REF reference-data registry gained 33 new files covering 12 CEDA
  obs_for_ref_v2 observational datasets,
  including cloud and radiation observations (ESACCI-CLOUD),
  burnt-area (GFED-5-0), ozone (SAGE-CCI-OMPS), FLUXNET site fluxes,
  soil carbon (HWSD-2-0), CALIPSO ice cloud, WOA-23 ocean biogeochemistry,
  RAPID AMOC transport, snow cover (CCI-CryoClim), and ocean carbon fluxes (Hoffman).
  Older WECANN and WOA2023 entries were replaced by their newer versions. (#778)

• Ten ILAMB standard diagnostics now evaluate models against the obs4REF reference datasets rather than the bundled ILAMB files.

  This updates the version of:

  • gross primary productivity (WECANN, FLUXNET2015)
  • runoff (LORA)
  • soil carbon (HWSD-2-0)
  • net biome productivity (Hoffman)
  • snow cover (CCI-CryoClim),
  • burnt area (GFED-5-0)
  • sea-surface temperature and salinity (WOA-23),
  • RAPID Atlantic overturning transport.

  (#779)

• Added climate_ref.results, a typed read layer for querying results from notebooks and other tools.
  A single Reader exposes per-domain readers for metric values, executions, datasets, diagnostics, and output artifacts, each returning frozen objects that remain usable after the database session closes and can be turned into a pandas DataFrame.

  Added a ref diagnostics list command that lists the registered diagnostics with their promoted version and per-diagnostic successful, in-flight, and total execution-group counts.
  Metric values now expose a first-class kind (model or reference) field, and ref datasets list returns only the latest version of each dataset. (#780)

• Added REF_TEST_CASES_SKIP,
  a comma-separated list of diagnostic slugs or provider/diagnostic pairs
  that are excluded from both ref test-cases fetch and the per-provider no-drift test.
  This keeps diagnostics whose full-resolution ESGF input cannot be fetched within CI limits
  (for example ilamb/thetao-woa2023-surface, a 3D ocean field)
  from being downloaded or executed.
  ESGF fetching is now confined to the populate-cache CI job. (#797)

• The ESGF fetch script can now retrieve the obs4MIPs reference datasets used by the PMP diagnostics,
  namely GPCP-Monthly-3-2, TropFlux-1-0, CERES-EBAF-4-2, HadISST-1-1 and 20CR-V2.
  Previously these were only available through the obs4REF registry.

  Note that the psl modes of variability diagnostics request 20CR,
  which ESGF publishes as 20CR-V2,
  so those diagnostics continue to source their reference data from the obs4REF registry. (#799)

Improvements

• Added a register argument to ProviderRegistry.build_from_config.
  Registration is the only step that writes to the database, so read-only consumers such as the API can now build the registry with register=False and serve a database mounted read-only. (#791)

• Removed the ERA-20C and ERA-Interim reference datasets from the obs4REF and sample data registries,
  reducing the volume of data fetched by ref datasets fetch-data --registry obs4ref.
  No diagnostic used either dataset,
  and neither is available from obs4MIPs.
  The unrelated ERA-5 dataset is unaffected.

  If you have already fetched the sample data,
  re-fetch it with fetch_sample_data(force_cleanup=True) so that the removed files are cleared from your local copy. (#799)

Bug Fixes

• Made ECS diagnostic work with CMIP7 data. (#671)
• Made TCR diagnostic work with CMIP7 data. (#686)
• Widened the contiguous-timerange tolerance so monthly datasets split across multiple files are no longer dropped as false-positive gaps. (#774)
• Committed regression baselines no longer embed the machine-specific source-checkout path or the
  host operating system in provider command-line provenance.
  Both are now redacted to portable <SOURCE_DIR> and <OS> placeholders,
  so a baseline minted on one platform (for example macOS) reproduces one re-derived on another
  (for example Linux in CI) instead of drifting. (#775)
• Fixed execution listings duplicating an execution group when two of its executions shared the same timestamp; each group now resolves to exactly one latest execution. (#786)
• Decoupled fetching of the ignore datasets file from configuration loading;
  the file is now fetched lazily at solve time rather than while the configuration is loaded,
  so read-only commands no longer perform network I/O.
  Added an ignore_datasets_url setting (env REF_IGNORE_DATASETS_URL)
  whose empty-string value disables fetching entirely for offline and air-gapped deployments.
  A failed download no longer creates an empty placeholder file;
  an existing cached copy is reused unchanged,
  and a solve with no cached copy fails loudly rather than running without ignore-dataset protections. (#792)
• Fixed the LocalExecutor per-task timeout counting time spent waiting in the process-pool queue as execution time. The budget is now measured from when a worker actually starts a task, so a large backlog of executions queued behind other work is no longer culled en masse once it ages past the timeout. (#795)
• Fixed ref datasets ingest --n-jobs greater than 1 crashing (SIGSEGV/SIGBUS) with the defaultcomplete CMIP6 parser.
  Parsing previously fanned out across threads, but the netCDF4/HDF5 wheels published on PyPI are not thread-safe.
  Parsing now runs using a process pool. (#796)
• Fixed three integration-test failures caused by the self-hosted runner's shared cache
  and stable output directory leaking into tests that assumed a clean, default environment. (#797)
• The ESGF fetch script now exits with a non-zero status if any request failed,
  rather than reporting 0 datasets and exiting successfully.
  Transient ESGF access errors are retried with an exponential backoff,
  configurable via --max-attempts and --retry-delay.
  A request that matches no datasets is reported but is not treated as a failure. (#799)

Trivial/Internal Changes

• #670, #783, #787, #789, #794

Changes

• 59e5311 Bump version: 0.15.0 → 0.16.0
• c166238 test(core): update obs4ref registry expectations after ERA removal
• a460a86 fix(scripts): fail the ESGF fetch when a request could not be retrieved
• ce7d63c docs: add changelog entries for #799
• 963589d chore(datasets): drop unused ERA-20C and ERA-Interim reference data
• fa1ceef feat(scripts): fetch PMP obs4MIPs reference data from ESGF
• 72994b8 chore: docs
• 03e2ee7 docs: add changelog fragments for #797
• 3a09276 feat(test-cases): confine ESGF fetching to populate-cache and allo...

v0.15.0

Changelog

Breaking Changes

• The hash that identifies a diagnostic's input datasets is now computed deterministically across pandas versions and platforms.
  It previously relied on pandas.util.hash_pandas_object, whose output changes between pandas releases and platforms.

  Because the underlying hash values change, the committed regression catalog_hash values have been regenerated,
  and existing databases will re-run each execution once on first use after upgrading. (#741)

• The minimum supported Python version is now 3.12; Python 3.11 is no longer supported.
  Update your environment to Python 3.12 or newer before upgrading. (#745)

• Removed the legacy central regression-testing framework in favour of the native baselines managed by ref test-cases.
  The DiagnosticValidator and ExecutionRegression helpers, the diagnostic_validation, execution_regression, and regression_data_dir pytest fixtures (from climate_ref.conftest_plugin), the central tests/test-data/regression/ baseline tree, and the per-provider test_diagnostics / test_build_results tests have all been removed.

  Provider regression baselines are now recorded and verified entirely through the ref test-cases CLI and the per-package committed bundles (see Regression baselines). (#757)

Features

• Added ref test-cases mint, replay, and sync commands for managing native regression baselines.
  mint records a test case's native outputs into the baseline store,
  replay regenerates a bundle from the stored outputs and checks it against the committed copy,
  and sync downloads the native outputs referenced by committed baselines into the local cache. (#724)

• Added the ref test-cases ci-gate command, which decides how CI should verify each
  test case's regression baseline against the base branch: skip it, replay the cached
  native baseline, re-run the diagnostic in full, or fail an unauthorised change.
  Baselines are now coupled to their input catalogs, so changing a test case's inputs
  without regenerating its baseline is reported as a failure rather than passing silently. (#727)

• Added a Cloudflare R2 object-store backend for diagnostic regression baselines, so ref test-cases mint can publish a test case's native outputs to the shared baseline store rather than only a local directory.
  The store endpoint and bucket are configured through the REF_NATIVE_STORE_* settings (defaulting to the project's public baseline bucket), and write credentials are supplied via the environment or a named AWS/R2 profile, never the persisted configuration.

  Minting is also easier to get right.
  ref test-cases mint now checks the store credentials and bucket up front and stops with a clear message when they are misconfigured, the new ref test-cases check-store command verifies store connectivity without minting, ref test-cases mint --dry-run previews what would be minted, and ref test-cases fetch restores a missing local paths file on its own so a fresh checkout no longer needs --force. (#732)

• Regression baselines are now verified automatically in CI.
  Every pull request runs the coupling gate and replays each test case's cached native baseline, a nightly job re-checks every baseline for silent drift, and a manually gated workflow mints new baselines to the shared object store behind required reviewers — so write credentials never run on untrusted pull-request code.

  The diagnostic testing guide also gained a "pull request workflow" section explaining what the gate decides for each change and how to publish a new baseline. (#733)

• ILAMB regression baselines are migrated to the per-package Framework B layout, starting with the mrsos-wangmao, gpp-fluxnet2015 and lai-avh15c1 cmip6 test cases. Each carries a committed CMEC bundle and a manifest; native blobs are published separately through the gated mint workflow. ILAMB executions now also declare their scalar CSV and netCDF outputs in the CMEC output bundle so they are persisted with the results and can be replayed. (#738)

• Extended the ILAMB cmip6 regression-baseline migration to the per-package Framework B layout with nine more test cases:
  gpp-wecann, mrro-lora, csoil-hwsd2, nbp-hoffman, snc-esacci, burntfractionall-gfed, emp-gleamgpcp2.3, thetao-woa2023-surface and so-woa2023-surface. (#743)

• ref test-cases runs now materialise their outputs into an inspectable, gitignored output slot
  (<case>/output/<label>/) holding the curated native files, the rebuilt committed bundle, and a source stamp,
  so you can see and diff exactly what a run produced.
  Added a build verb that rebuilds the committed bundle from an existing slot without re-executing the diagnostic,
  a mint --from-replay option that re-authors a baseline from the stored native instead of re-running it,
  and a --label option that keeps several runs side by side for comparison. (#748)

• Added a --format json option to ref datasets list, ref providers list, ref executions list-groups, and ref config list for machine-readable output with full, untruncated identifiers and paths. (#765)

• Added ref config init, get, set, unset, edit, and validate commands to make configuration onboarding and management easier from the CLI. (#768)

• Added a five-minute Quick Start guide
  and a new model-vs-observation diagnostic to the example provider,
  global-sst-bias,
  which compares modelled global-mean sea surface temperature (tos, area-weighted by areacello)
  against the HadISST-1-1 observations and emits scalar metrics, the model/reference/bias time series, and figures.
  A curated quickstart data registry provides the single reference dataset it needs,
  avoiding the multi-gigabyte obs4REF download. (#769)

• The CI coupling gate now couples each regression baseline to its diagnostic's Diagnostic.version.
  A test case fails the gate when the diagnostic's in-code version no longer matches the version recorded
  when its baseline was minted, so an execution-affecting change can no longer slip through as a green replay
  against a stale committed bundle.
  The new author-declared diagnostic_version field lives in the test case manifest.json (schema version 2),
  and the new ref test-cases migrate-manifests command backfills it across existing baselines.
  A legitimate revert is still permitted: lower the diagnostic version and re-mint the baseline. (#770)

Improvements

• ILAMB reference (observational) series are now tagged with their source via the existing
  reference_source_id dimension (for example reference_source_id="WangMao"),
  so a reference series is self-identifying without adding a new schema field. (#743)
• Committed regression baseline bundles (series.json and diagnostic.json) now round floating-point values to seven significant figures when written, giving stable, reviewable bytes that no longer churn between local and CI runs while staying well within the regression comparison tolerance. (#744)
• Reduced database size by deduplicating the index of series metric values into a shared index_axis table, referenced by each series rather than stored inline on every row.
  A database migration backfills existing data losslessly; on SQLite, run VACUUM afterwards to release the freed space on disk. (#747)
• Sped up ref executions reingest by eager-loading datasets and batching the database writes, so bulk reingests now scale with the number of executions rather than the number of data files. (#750)

Bug Fixes

• Regression series.json baselines are now written with a stable, dimension-sorted order. Previously a diagnostic could emit its series in a platform-dependent order, causing a committed bundle minted on one platform to falsely fail the regression gate on another even when every value matched within tolerance. (#738)
• Selecting the latest dataset version now compares version numbers numerically, so versions like v10 are correctly preferred over v2 (previously the older version could be chosen). (#739)
• Fixed a bug where a diagnostic whose results failed to ingest was incorrectly recorded as successful with no metric values; such executions are now marked failed and retried on the next solve. (#740)
• ILAMB regression output.json bundles are now written with a stable, sorted block order
  (the plot and data sections are sorted by filename),
  so a baseline regenerated on one machine reproduces byte-for-byte on another
  instead of reordering and producing a spurious diff. (#743)
• Fixed ref executions reingest reporting a reingest as...

v0.14.7

Changelog

Bug Fixes

• Fixed ref solve --timeout 0 discarding completed work instead of waiting for it.
  A timeout of 0 (or any non-positive value) now means "wait with no time limit",
  so executions are collected, copied to the results directory, and ingested as expected;
  previously they ran but their outputs were left in the scratch directory.
  Use --no-wait to queue executions and exit immediately,
  and recover any orphaned scratch outputs from an earlier run with ref executions reingest --include-failed. (#735)
• Fixed ref executions reingest failing for ESMValTool diagnostics.
  Reingest copies an execution's outputs to a new directory, but the absolute paths recorded in ESMValTool's provenance files still pointed at the original location,
  which caused an error that silently rolled back the reingest.
  These embedded paths are now re-pointed at the new output directory so the results are reingested as expected. (#736)

Changes

• 5b239c3 Bump version: 0.14.6 → 0.14.7
• 12be1fa refactor(reingest): reuse core rewrite_tree for path rewriting
• 3807459 docs: add changelog for #736
• 6f35062 fix(reingest): rewrite embedded absolute paths when copying execution outputs
• 6eba410 chore: clean up docs
• 08a55e0 docs: add changelog for #735
• e4ada47 fix(solve): wait and collect results when --timeout 0
• 66acade chore(deps): bump bleach from 6.3.0 to 6.4.0
• d04fb98 chore(deps): bump cryptography from 46.0.7 to 48.0.1
• 0f8f00d chore(deps): bump pyjwt from 2.12.0 to 2.13.0

v0.14.6

Changelog

Bug Fixes

• Fixed loading the default configuration in environments where the user cache directory cannot be written (for example, read-only or restricted HPC systems).
  Previously, running a ref command without an existing configuration file could fail with Error loading configuration because the default ignore-datasets file could not be cached.
  The REF now degrades gracefully and continues without the cached ignore-datasets file. (#734)

Changes

• 306f5ea Bump version: 0.14.5 → 0.14.6
• fd95be2 docs: add changelog for #734
• 0142b9f fix(config): don't fail config load when ignore-datasets cache is unwritable

v0.14.5

Changelog

Bug Fixes

• Made ESMValTool regression baselines deterministic by giving each captured run a stable execution directory instead of a timestamped one, so re-running a test case no longer churns the committed output. (#714)
• Fixed fetching datasets from ESGF failing with Must have equal len keys and value when setting with an iterable under pandas 3.0,
  by running the intake-esgf catalogue build with the legacy object-string dtype. (#726)

Trivial/Internal Changes

• #719, #720, #723, #725

Changes

• da4461c Bump version: 0.14.4 → 0.14.5
• 27cdeb4 docs: cleanup
• 31eca3d ci: stop per-job coverage xml from enforcing the fail_under gate
• 8633217 docs: update changelog for #725
• 999ff88 ci: split integration tests into a parallel job and cap PR concurrency
• 250d55d fix(esgf): build intake-esgf catalogue with legacy object-string dtype
• a9f3df2 docs: add changelog entry for #725
• da3f60c ci: parallelise tests-providers into a per-provider matrix
• f2be958 chore: cleanup
• cfea48b docs: add changelog entry for #723
• 48bd0b5 test: silence deprecation warnings and speed up the climate-ref suite
• a75c835 fix(core): address PR review comments on regression primitives
• 468f823 chore: remove unneeded annotations
• d2e336c chore: remove unneeded annotations
• 1bd11e3 chore: pull out the extra layer of tolerance redirection
• 0733172 chore(deps): bump tornado from 6.5.5 to 6.5.6
• 6a66af6 fix(core): apply review fixes to regression-baseline primitives
• 9455f7b test(core): expand output-file coverage and apply review fixes
• c83e316 chore(deps): bump codecov/codecov-action in the github-actions group
• c2a7ccc docs(changelog): add fragment for native regression-baseline primitives
• 9cabe73 feat(core): add native regression-baseline primitives (RFC 0005)
• 3a0ebd4 docs(changelog): add fragment for output-file operations refactor
• 6936a27 refactor(core): extract shared output-file operations into climate_ref_core.output_files
• 4765fcf chore: clean ups
• 835a371 docs(changelog): add fragment for #714
• cacd200 fix(esmvaltool): stabilise execution dir for regression capture
• 6ddfa89 chore(deps): bump gitpython from 3.1.47 to 3.1.50
• 8065f73 chore(deps): bump astral-sh/setup-uv in the github-actions group

v0.14.4

Changelog

Improvements

• Added GPCP dataset version to 3.3 for the PMP precipitation climatology reference dataset. (#672)
• Updated GPCP dataset version to 3.3 for the PMP precipitation climatology reference dataset. (#684)
• Removed the pandas upper bound and fixed pandas 3 compatibility in catalog and ILAMB result handling. (#708)

Bug Fixes

• Updates the PMP provider data directory (#685)
• Fixed a KeyError: 'branded_variable' when solving diagnostics against CMIP7 datasets.
  The branded_variable facet is now reconstructed when a data catalog is loaded from the database,
  so it is available to data requirement filters. (#712)

Improved Documentation

• Added a Zenodo DOI badge to the README and documentation landing page so users can easily find how to cite the REF. (#634)
• Added a getting-started guide for modelling centres that explains how to evaluate local or pre-publication CMOR-compliant model output with Climate-REF, including deployment options, accessing results, and a suggested first run. (#709)

Trivial/Internal Changes

• #706

Changes

• d26bdce Bump version: 0.14.3 → 0.14.4
• dfee0e3 refactor: declare derived catalog columns on the adapter
• b191910 docs: add changelog fragment for PR 712
• 9d43fee fix: derive branded_variable when loading CMIP7 catalog from DB
• baa358b docs: add Zenodo DOI badge to README and docs landing page
• 673b6fa docs: add changelog fragment for modelling centre guide
• 5206586 docs: add modelling centre onboarding guide
• cf96a14 test: Add tests for the helper
• 42cea87 chore: rename changelog fragment to PR 708
• 182f088 chore: remove pandas upper bound and fix pandas 3 compatibility
• 3b9973c docs(changelog): add fragment for series regression check
• 4995b17 test(core): add series.json regression comparison
• ef7ab30 chore: update the regression.
• aebe346 chore: increment version and fix test
• 22dfaeb feat: add example package to deployment
• b2e1a76 fix: tests
• 7b76f57 fix: test
• 2f41764 docs: Changelog
• 1d17468 fix: correct the data path for the new pmp cache
• 9ab3365 Update GPCP dataset version in changelog
• a69c4c0 Update reference dataset from GPCP-Monthly-3-2 to GPCP-3-3
• 3b6d2e1 chore(deps): bump pymdown-extensions from 10.21 to 10.21.3
• 368334f chore(deps): bump idna from 3.11 to 3.15
• 5734953 Update 672.improvement.md
• 5a5812a add changelog
• 1f72461 Add new climatology dataset entry
• cbbd022 chore(deps-dev): bump the python-dependencies group with 4 updates

v0.14.3

Changelog

Bug Fixes

• Fixed a race condition when trying to create the same ExecutionGroup concurrently.
  Database.get_or_create and Database.update_or_create now wrap the INSERT in a SAVEPOINT
  and re-fetch the winning row on conflict instead of aborting the transaction. (#679)

Changes

• c07e141 Bump version: 0.14.2 → 0.14.3
• 10c9dd0 fix(database): address PR review comments
• 3aae695 docs: changelog
• 9f6d74c docs: changelog for #679
• c60b8ed fix(database): make get_or_create / update_or_create race-safe

v0.14.2

Changelog

Bug Fixes

• Changed the behaviour for register_dataset treatment of files absent from the current ingest slice as kept-in-place with a warning, instead of raising NotImplementedError. (#677)

Changes

• c1d9ce2 Bump version: 0.14.1 → 0.14.2
• 787e62b docs: changelog
• 82fbd00 fix(ingest): keep files absent from current ingest, warn instead of abort