Skip to content

CONTRIBUTING.md

Latest commit

 

History

History
214 lines (141 loc) · 11.9 KB

CONTRIBUTING.md

File metadata and controls

214 lines (141 loc) · 11.9 KB
Raw

Contributing to Mosaic

Mosaic grows as the community adds solver backends, improves configurations, and extends to new domains. This guide covers the contribution paths and the local tooling you need. Any contributions you make are greatly appreciated.

Code of Conduct

Ensure your contributions adhere to the Code of Conduct.

Code

Mosaic is developed under the Apache 2.0 license. By contributing to the Mosaic project you agree that your code contributions are governed by this license. We require you to sign our Contributor License Agreement to state so.

Adding a solver backend

A solver backend is a Tesseract wrapper that implements the forward map and (optionally) its VJP for an existing benchmark domain. The full walkthrough — directory layout, schemas, apply / abstract_eval / vector_jacobian_product, the metadata.mosaic: config block, and domain-specific overrides (exclusions, input_overrides, explained_anomalies) — is in Part A of the Add a Backend tutorial (source: docs/tutorial.qmd). It uses a short pseudo-spectral solver on the ns-grid domain as a running example.

Tuning an existing solver

Results reflect out-of-the-box configurations at the time of each tagged release. If you believe a solver can perform better with different settings, submit a PR with the improved configuration.

What to change

Solver behavior can be adjusted at three levels:

  • Solver-internal parameters in tesseract_api.py — sub-stepping counts, relaxation factors, convergence tolerances, time integration order, etc.
  • Input overrides in the problem config (mosaic/benchmarks/problems/<domain>/config.py, via _SOLVERS["<solver>"].input_overrides = {...}) — solver-specific values for shared schema fields (e.g. forcing a particular dt or density).
  • Schema defaults in tesseract_config.yaml or the solver's InputSchema subclass — changing the default value of an exposed parameter.

How to test

  1. Snapshot the current state:

    mosaic status --format json > before.json

  2. Make your changes and run the affected suites:

    # Quick iteration (small problem size)
mosaic run -p <problem> --suites gradient,optimization -s <solver> --debug

# Full run (production sizes)
mosaic run -p <problem> --suites gradient,optimization -s <solver>

    After an edit, mosaic run --only failed,stale -s <solver> re-runs only the cells that aren't already fresh-ok, leaving the rest of the campaign untouched.

  3. Compare against the baseline:

    mosaic status --format json > after.json
mosaic status --format md --diff-against before.json

Review expectations

  • The change must not regress any other solver's results.
  • CI requires a maintainer to add a benchmark label to any PR that touches mosaic/ code. The options are:
    • benchmark:none — skip benchmarks (maintainer trusts no answer-changing code).
    • benchmark:solver — run benchmarks only on the solvers whose Tesseract code changed, scoped to the problems containing them (any number of solvers; harness/core changes don't widen the run — use benchmark:all for that).
    • benchmark:domain — run every solver in the domain whose problem definition changed (derived from the changed files under mosaic/benchmarks/problems/). Use when a change to a problem's physics/config alters the reference the whole domain is benchmarked against. Single-domain by design: a PR changing more than one domain's problem must use benchmark:all. Solver code changes are benchmark:solver's job and don't trigger this label.
    • benchmark:all — run the full benchmark suite from scratch.
  • Include the diff output in your PR description so reviewers can see the impact at a glance.

Adding a benchmark domain

A new domain bundles canonical schemas (mosaic/mosaic_shared/problems/<domain>/schemas.py), a Problem package (mosaic/benchmarks/problems/<domain>/, split into config.py, ics.py, physics.py, plus any per-problem optimization.py / plots.py), and at least one reference solver. The fastest way to start is mosaic new-domain <name> --from-template <template>, which scaffolds the file tree from a built-in template. The end-to-end walkthrough — schemas, suite defaults, IC generators, error functions, and a working reference solver — is in Part B of the Add a Backend tutorial (source: docs/tutorial.qmd).

Building the docs

The Quarto site at https://docs.pasteurlabs.ai/projects/mosaic/latest is generated from docs/*.qmd plus two auto-generated files: docs/solvers.qmd (gitignored) and docs/results_*.qmd.

Install the Quarto CLI — this is a standalone binary, not a pip package. Grab it from https://quarto.org/docs/get-started/ (the page autodetects your OS) or:

# Linux x86_64 example — check quarto.org for the latest version
wget https://github.com/quarto-dev/quarto-cli/releases/download/v1.5.57/quarto-1.5.57-linux-amd64.deb
sudo dpkg -i quarto-1.5.57-linux-amd64.deb
quarto --version   # should print 1.5.x

Then, mirroring .readthedocs.yaml:

uv sync --extra dev

# Generate the solver reference page from tesseract_config.yaml metadata
uv run python docs/generate.py

# (Optional) Regenerate the per-domain results pages from mosaic-results/.
# Skip this if you have no local benchmark results; the tracked copies will be used.
uv run python docs/generate_results.py

# Render the full site to _site/
quarto render

# Or run a live-reloading preview
quarto preview

Quarto will error with a missing-file message if you run quarto render before docs/generate.py — docs/solvers.qmd is referenced in _quarto.yml but produced on the fly.

Code style

  • Python 3.10+, formatted with ruff (ruff check --fix && ruff format)
  • Pre-commit hooks enforce formatting on every commit
  • Run tests with pytest

Local development setup

Clone the repository and install the dependencies with pre-commit hooks. With uv this is a single uv sync --extra dev; with pip, install the shared workspace member first:

$ git clone git@github.com:pasteurlabs/mosaic.git
$ cd mosaic
$ python -m venv venv
$ . venv/bin/activate
$ pip install -e mosaic/mosaic_shared
$ pip install -e ".[dev]"
$ pre-commit install

Pull request workflow

  1. Fork the repository and create a feature branch.
  2. Make your changes, ensuring ruff check and pytest pass.
  3. Open a pull request with a clear description of what you're adding.
  4. CI runs lint, tests, and tesseract config validation on every PR. If your PR touches mosaic/ code, a maintainer will add one of four benchmark labels (benchmark:none, benchmark:solver, benchmark:domain, or benchmark:all) to control whether and how benchmarks run.

CI/CD

The project uses several GitHub Actions workflows that work together.

Lint and test (ci.yml)

Runs on every PR: linting (ruff), tests (pytest), and Tesseract config validation. Must pass before merge.

Benchmarks (benchmark.yml)

Runs on PRs that carry a benchmark label. The workflow has four stages:

  1. Plan — detects which solvers changed, reads the benchmark label, and builds a (suite, problem, hardware) matrix.
  2. Build — builds Docker images for any changed solvers and pushes them to GHCR.
  3. Run — executes the benchmark matrix across CPU and GPU runners.
  4. Report — merges CPU/GPU results, generates a status snapshot, renders a docs preview, posts a PR comment with the diff, and publishes results.

For dev PRs the report step overlays the PR's results on top of the current rolling baseline so the preview shows all solvers, not just the ones that ran.

For release PRs the report step uploads results the same way; publish-results.yml later creates both the updated baseline and an immutable release-<version> artifact.

Read the Docs (.readthedocs.yaml)

Builds the documentation site (production and PR previews). RTD fetches the baseline artifact from GitHub Actions so docs always reflect the latest merged benchmark results. PR preview builds also overlay PR-specific results when available.

RTD builds are triggered explicitly by GitHub Actions workflows (not by the default webhook) to avoid races — results must be published before the docs build starts:

  • Push to main — publish-results.yml uploads the baseline artifact, then triggers an RTD build for latest.
  • Release — publish-results.yml uploads a release-<version> artifact and triggers RTD builds for stable and latest.
  • PR — benchmark.yml uploads benchmark results, then triggers an RTD PR preview rebuild.

Release (release.yml)

Handles the full release lifecycle (see Release process below).

Benchmark results artifacts

Benchmark results are stored as GitHub Actions artifacts (not a git branch):

  • baseline — rolling accumulation of results merged from dev PRs, overwritten atomically on each merge to main.
  • release-<version> (e.g. release-v0.4.0) — immutable release results with long retention.
  • benchmark-results-<sha> — per-PR results uploaded during the benchmark workflow, used for PR preview builds.

Commit and pull request message guidelines

We follow the Conventional Commits specification for all commits that reach the main branch. Each commit is crafted from a pull request that is squash-merged. The commit title and message comes from the pull request title and message, respectively. As such, they should be structured following the specification.

The title consists of a type, an optional scope, and a short description: type[(scope)]: description. The types we use are:

  • chore: for changes that affect the build system, external dependencies, or general housekeeping.
  • ci: for changes in the CI.
  • doc: for documentation only changes.
  • feat: for a new feature.
  • fix: for fixing a bug.
  • perf: for a code change that improves performance.
  • refactor: for a code change that neither adds a feature nor fixes a bug.
  • security: for a change that fixes a security issue.
  • test: for adding new tests or fixing existing ones.

The scopes we use are:

  • harness: for changes that affect the benchmark harness or CLI.
  • solver: for changes to solver backends.
  • domain: for changes to benchmark domains.
  • deps: for changes in the dependencies.

In case there are breaking changes in your code, this should be indicated in the message either by appending an exclamation mark (!) after the type/scope or by adding a BREAKING CHANGE: trailer to the message.

Versioning

Mosaic follows semantic versioning.

Release process

(code owners only)

Releases are done via GitHub Actions, which automatically build the release artifacts and publish them to PyPI. To create a new release:

  1. Make sure the code is in a good state, all tests pass, and the documentation is up to date.
  2. Trigger the release workflow through the GitHub UI. This opens a new pull request with the release notes and the version number. Full benchmarks run automatically on the release PR.
  3. Add any additional release notes to the pull request message.
  4. Once the pull request is ready, merge it into main.
  5. GitHub Actions will then automatically release the new version and publish the benchmark results as both the updated baseline and an immutable release-<version> artifact. Verify that the release artifacts are correctly built and published.