Skip to content
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 70 additions & 0 deletions docs/docs/python/goals/test.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -578,6 +578,76 @@ branch = true

When generating HTML, XML, and JSON reports, you can automatically open the reports through the option `--test-open-coverage`.

### Coverage with test sharding

When using the `--shard` flag to split tests across CI runners, each shard only exercises a fraction of your test targets. The per-shard coverage report will be artificially low. To get accurate coverage you need to combine the binary `.coverage` files from all shards.

#### Configuration

Add `"raw"` to your coverage reports so Pants writes the `.coverage` binary, and set `relative_files = true` so that `coverage combine` can match paths across different sandbox directories:

```toml title="pants.ci.toml"
[coverage-py]
report = ["raw", "xml", "console"]
output_dir = "coverage-report"
```

```ini title=".coveragerc"
[run]
relative_files = true
branch = true
```

:::caution Don't set `fail_under` in `[coverage-py]` when sharding
Each shard only runs a fraction of your targets, so per-shard coverage is intentionally incomplete. Setting `fail_under` in `pants.toml` or `pants.ci.toml` will cause every shard to fail. Enforce the threshold after combining all shards instead.
:::

#### Merging coverage reports

After all shards complete, collect their `.coverage` binaries, combine them with `coverage combine`, and generate the final report. The following example uses GitHub Actions, but the same approach applies to any CI system:

```yaml title=".github/workflows/ci.yml"
# In each shard job: upload the coverage output
- name: Upload coverage
uses: actions/upload-artifact@v4
with:
name: coverage-shard-${{ matrix.shard }}
include-hidden-files: true # .coverage is a dotfile
path: coverage-report/

# Post-shard job: combine and enforce threshold
coverage-report:
needs: [test]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Download all shard coverage
uses: actions/download-artifact@v4
with:
pattern: coverage-shard-*
path: all-coverage/
merge-multiple: false

- name: Combine and report
run: |
pip install coverage

idx=0
for f in $(find all-coverage/ -name ".coverage" -not -name ".coverage.*"); do
cp "$f" ".coverage.shard${idx}"
idx=$((idx + 1))
done

coverage combine --rcfile=.coveragerc .coverage.shard*
coverage xml --rcfile=.coveragerc -o coverage-report/coverage.xml
coverage report --rcfile=.coveragerc --fail-under=80
```

:::note `global_report` and sharding
With `[coverage-py] global_report = true`, per-shard reports show 0% for untouched files. Consider applying this setting only in the post-merge step rather than in `pants.ci.toml`.
:::

## JUnit XML results

Pytest can generate [JUnit XML result files](https://docs.pytest.org/en/6.2.x/usage.html#creating-junitxml-format-files). This allows you to hook up your results, for example, to dashboards.
Expand Down
4 changes: 4 additions & 0 deletions docs/docs/using-pants/advanced-target-selection.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -206,6 +206,10 @@ For other goals, you can leverage shell piping to partition the input targets in
pants list :: | awk 'NR % 5 == 0' | xargs pants package
```

:::tip Coverage and sharding
When using `--shard` with test coverage enabled, each shard only exercises a fraction of your targets, producing artificially low coverage numbers. You need to combine the binary `.coverage` files from all shards in a post-shard CI step to get accurate results. See [Coverage with test sharding](../python/goals/test.mdx#coverage-with-test-sharding) for the full configuration and CI workflow.
:::

## Using CLI aliases

If setting tags on individual targets is not feasible, there are a few other options available to refer to multiple targets.
Expand Down
Loading