[miroslavpojer]

Overview

coverage-matrix now computes what the canon's coverage rules compute: coverage per declared aspect, the keyword, and a warning for each scenario link it does not count as written. Nothing else in the command changes: the inputs, the entities and their order, planned_summary, --view, --fail-under, the envelope and the documentation entry's selected_stats.

Rows with aspects. A counted criterion that declares aspects, with Aspect: or a keyword, gets one aspects[] entry per declared value, in declared order. A scenario goes under the aspect its link names. A bare @AC:<id> link puts it under every aspect. Each scenario is listed once per aspect, in input order. An aspect is covered with a scenario and not_covered without one. The row is covered when every aspect is, not_covered when none is, and partially_covered otherwise. Its own scenario_ids stays empty, as in the contract's sample. A criterion with aspects and no scenario is not_covered. The summaries need no new code: CoverageSummary.from_rows already weighs a partially_covered row as its covered aspects over its declared ones.

The keyword. A keyword criterion is counted per value exactly as an Aspect: one. Its row carries keyword, the criterion's one placeholder_values key as stored (rule), which labels the breakdown. An Aspect: row and a row without aspects carry none. ui-tests.json does not store a keyword tag's name, only its value, so the name is not checked.

Rows without aspects are unchanged. Every link to the criterion covers it, including a link that names an aspect.

The warnings. The builder adds them after every forwarded warning, in scenario order, then link order:

• STALE_AC_REF: the link names a criterion that no User Story or Functionality in the documentation input has, or an aspect the criterion does not declare.
• PLANNED_AC_HAS_TESTS / IN_REVIEW_AC_HAS_TESTS: the criterion is planned / in_review, whatever the link's aspect. The criterion has no row, and the scenario keeps its link in ui-test-catalog.
• There is one warning per scenario and criterion; for an undeclared aspect, one per scenario, criterion and aspect.
• ac_id is the link's id, which AcLink guarantees is valid, and entity_id is that id without its last -<n>. line_no, path and source_url stay unset, because a collector fills them. The message names the scenario, the criterion and any undeclared aspect. The context holds them as scenario_id='<id>', plus aspect='<value>' for an undeclared aspect.
• The run still succeeds. No command logs a warning, and no generator shows a transform's warning, so the three warnings reach the JSON only.

Why an undeclared aspect still covers a criterion without aspects, and never one with aspects. A criterion documented without aspects can meet scenarios tagged with aspects, for example when its documentation is mined from an issue and its tests come from .feature files that declare them. living-doc's examples README expects exactly this STALE_AC_REF on its gh-issues chain. The scenario names the criterion, and that criterion has only one thing to cover, so the link covers it, with the warning. On a criterion with aspects, a value it does not declare counts nowhere. Treating it as a bare link would let a typo cover every aspect.

The coverage percentage moves. It changes wherever criteria declare aspects, because a partially_covered row now weighs less than 1. The fixture pair goes from 66.7 to 55.6 (6 counted, 3 covered, 1 partially covered, 2 not covered). A --fail-under threshold set against the old figure may need a look. DEVELOPER.md's local example ran the fixture pair with --fail-under 60, so it is now 50.

Lineage. Under both user_stories[] and functionalities[], acceptance_criteria[].aspect[] now maps to entities[].acceptance_criteria[].aspects[].aspect, and acceptance_criteria[].placeholder_values to entities[].acceptance_criteria[].keyword. Both are added to MAPPED_ENTITY_PATHS. The link's aspect is Dropped, like its id, with its own reason: it only decides which aspect of the criterion's row lists the scenario. The entry that dropped these paths until per-aspect coverage existed is gone. Every other entry stays, and so does the run-time check: once, before the write, against the documentation entry's selected_stats over the counted criteria. A criterion's aspects lost from one list while the other list fills the path is still a partial drop, which R11 does not detect, and the package README says so.

Version impact. No bump; every package stays 0.1.0. No input, option, exit code or error line changes, and no artifact contract changes. The output now fills aspects[] and keyword and adds three transform warnings, all of which coverage-matrix-v1.0.0 already defines.

Release Notes

• coverage-matrix counts coverage per aspect: a criterion that declares aspects or a keyword is covered, partially_covered or not_covered from its aspects, with a breakdown per aspect; a bare @AC:<id> tag covers every aspect.
• A keyword criterion's row names its keyword, which labels the breakdown.
• coverage-matrix warns about a scenario link it does not count as written: STALE_AC_REF for an unknown criterion or an undeclared aspect, PLANNED_AC_HAS_TESTS and IN_REVIEW_AC_HAS_TESTS for a planned or in_review criterion. The run still succeeds, and a tag with an aspect still covers a criterion that declares none.
• The matrix's coverage percentage changes wherever criteria declare aspects: a partially_covered criterion weighs its covered aspects over its declared ones.

Framework

• Stays inside the frame. Per-aspect coverage and the test-link warnings are the toolkit's cross-referencing (F3); they read only typed fields (aspect, placeholder_values, AcLink.aspect), and the change mirrors no contract model and commits no schema (F7).

Related

Closes #95

Contract Rules

• Per-aspect status, a bare link under every aspect, the keyword as the breakdown's label, and a row with aspects whose own scenario_ids stays empty: rendering, § Coverage.
• The weights and the single rounding: rendering, § Coverage summary, through CoverageSummary.from_rows.
• The three warnings and their fields, with each field set only when the emitter knows it: errors, § Codes and § How codes are reported. That either warning names the scenario and the criterion: rendering, § Coverage.
• A stale tag that still covers a criterion without aspects: rendering § Coverage's status table ("without aspects: the criterion has one"), and errors § Codes, where the effect of STALE_AC_REF is "reported".
• A keyword's values fill aspect, and placeholder_values keeps its one name: AC grammar, § Variants.
• Lineage and the full-sample test: component checks, § R11 and § Check 3.
• R12 checks:
  • check 1: no schema is vendored, and make no-vendored-schemas is clean;
  • check 2: both inputs are read through read_artifact and the matrix is written with write_artifact;
  • check 3: the written full samples run through the command in both views, input validation included.

No disagreement with the canon was found. One reading the pages leave open: an aspect value is matched exactly as written, so Minimum-Length is not minimum-length. The canon folds keyword names, but it states no folding for values.

Verification

• make qa passed: core 67 tests, normalize-docs 70, normalize-tests 20, coverage-matrix 60 (100 % coverage), CLI 47, root 59. Pylint 10.00 in every package; Black, ruff, mypy and deptry clean; no-vendored-schemas clean.
• The fixture pair's matrix (FUNC-001's rows and the root):
  • FUNC-001-01: covered over minimum-length, character-classes and no-username, one scenario each.
  • FUNC-001-02: not_covered.
  • FUNC-001-03: keyword rule, partially_covered 1/3, with minimum-length covered by one scenario.
  • FUNC-001: 3 counted, 1 covered, 1 partially covered, 1 not covered, 44.4. US-001 is unchanged at 66.7, and FUNC-002 still has no row.
  • Root: 6 counted, 3 covered, 1 partially covered, 2 not covered, 55.6. planned_summary: total 2, backlog 1, {"1.1.0": 1}.
  • The two forwarded warnings stay, and the command adds none of its own. The output equals the regenerated golden with generated_at masked; the release view differs only in document.view.
• The written full samples, in both views: no FIELD_LOSS; every entity, row, aspect, keyword and summary, and planned_summary, equal full_sample("coverage-matrix-v1.0.0")'s by entity id; root 7 counted, 2 covered, 1 partially covered, 4 not covered, 33.3; no warning.
• The new integration.yml assertions ran locally on the repository's venv over a release-view matrix: action-run's (6, 3, 1, 2), 55.6 and FUNC-001-03's partially_covered and rule, and the isolation job's (6, 55.6).

Acceptance criteria

A bare builder.py, lineage.py, service.py or tests/test_*.py path is under packages/services/coverage_matrix/ (src/living_doc_service_coverage_matrix/ for the modules).

• Aspect rows in declared order, a bare link under every aspect, each scenario once in input order, status from the aspects, the row's own scenario_ids empty; no scenario → not_covered; a row without aspects unchanged: builder.py:58-102; tests/test_builder.py:123, :185, :199, :216, :225, :235, :279, :311.
• A keyword criterion counted as an Aspect: one, with keyword; none on an Aspect: row or a plain row: builder.py:99-100; tests/test_builder.py:145, :252.
• Regenerated golden: FUNC-001-01 covered over three aspects, FUNC-001-03 rule 1/3, FUNC-001 44.4, root 6 / 3 / 1 / 2 at 55.6: tests/expected/coverage-matrix.json, tests/expected/README.md; tests/test_builder.py:171, tests/test_service.py:61, :79.
• Warnings on built inputs, with their fields, no row for planned / in_review, and the count rule: builder.py:125-191, :306; tests/test_builder.py:328, :358, :394, :417, :441, :462; the run succeeds: tests/test_service.py:245, apps/cli/tests/test_coverage_matrix.py:145.
• Full samples, both views, equal to full_sample("coverage-matrix-v1.0.0") by entity id: tests/test_service.py:92.
• Lineage maps the aspects and the keyword, assert_complete passes for both contracts, and a builder that drops the aspects or the keyword fails with FIELD_LOSS and writes no file: lineage.py:83-84, :110-113; tests/test_lineage.py:10, :14, :20, :48; tests/test_service.py:316 (fixture pair, Functionalities), :338 (full samples, User Stories). selected_stats unchanged: builder.py:220-258.
• action-run and the isolation job assert the new numbers: .github/workflows/integration.yml:155-162, :253. No page, test or comment says the matrix is per criterion or that aspects come later.
• make qa green; integration.yml runs on this PR.
• In sync with the canon: see Contract Rules.
• No planning id in the diff or this body.
• ## Framework above.

[coderabbitai]

No actionable comments were generated in the recent review.

Recent review info

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: eb4ae5de-a501-4cff-9c4d-a1e35389477d

Commits

Reviewing files that changed from the base of the PR and between 1506e5f and adda5af.

Files selected for processing: 15

• .github/workflows/integration.yml
• CONTEXT.md
• DEVELOPER.md
• apps/cli/README.md
• apps/cli/tests/test_coverage_matrix.py
• packages/services/CONTEXT.md
• packages/services/coverage_matrix/README.md
• packages/services/coverage_matrix/src/living_doc_service_coverage_matrix/builder.py
• packages/services/coverage_matrix/src/living_doc_service_coverage_matrix/lineage.py
• packages/services/coverage_matrix/tests/expected/README.md
• packages/services/coverage_matrix/tests/expected/coverage-matrix.json
• packages/services/coverage_matrix/tests/test_builder.py
• packages/services/coverage_matrix/tests/test_lineage.py
• packages/services/coverage_matrix/tests/test_service.py
• tests/test_main.py

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

Walkthrough

The coverage-matrix builder now evaluates declared aspects individually, retains keyword values, and reports warnings for scenario links it does not count. Tests, expected outputs, documentation, and integration assertions reflect the revised coverage results.

Changes

Aspect-aware coverage matrix

Layer / File(s)	Summary
Per-aspect coverage rows packages/services/coverage_matrix/src/living_doc_service_coverage_matrix/builder.py, .../lineage.py, packages/services/coverage_matrix/tests/test_builder.py, .../test_lineage.py, .../expected/*, packages/services/coverage_matrix/README.md	Criteria with declared aspects now have ordered aspect coverage entries, and row status reflects the aspect statuses. Bare links cover each declared aspect; aspect-specific links cover matching aspects. Keyword values and lineage mappings are retained.
Warnings for uncounted links packages/services/coverage_matrix/src/living_doc_service_coverage_matrix/builder.py, packages/services/coverage_matrix/tests/test_builder.py, .../test_service.py, apps/cli/tests/test_coverage_matrix.py, packages/services/coverage_matrix/README.md, apps/cli/README.md	The builder reports warnings for links to unknown criteria, undeclared aspects, and planned or in-review criteria. The tests check warning contents, ordering, and criteria omitted from rows.
Fixtures, command checks, and documentation .github/workflows/integration.yml, apps/cli/tests/test_coverage_matrix.py, tests/test_main.py, packages/services/coverage_matrix/tests/test_service.py, packages/services/coverage_matrix/tests/expected/*, CONTEXT.md, packages/services/CONTEXT.md, DEVELOPER.md, apps/cli/README.md, packages/services/coverage_matrix/README.md	Fixtures and command tests use totals of 6 counted criteria, 3 covered, 1 partially covered, and 2 not covered, with 55.6% coverage. Threshold checks and documentation reflect the updated output and checks.

Priority: Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Feature · Severity of issue fixed: Medium

Sequence Diagram(s)

sequenceDiagram
  participant DocSourceResult
  participant UITestsResult
  participant build_coverage_matrix
  participant CLI
  participant CoverageMatrixArtifact
  DocSourceResult->>build_coverage_matrix: Criteria and declared aspects
  UITestsResult->>build_coverage_matrix: Scenario links
  build_coverage_matrix->>CLI: Matrix rows and warnings
  CLI->>CoverageMatrixArtifact: Write coverage matrix

Loading

Possibly related PRs

• AbsaOSS/living-doc-toolkit#90: Adds the UI-test acceptance-criterion links and aspects that this change uses to assign scenarios to coverage aspects and generate link warnings.
• AbsaOSS/living-doc-toolkit#94: Adds the initial coverage-matrix service that this change extends with per-aspect coverage and link warnings.

Suggested labels: enhancement

Suggested reviewers: tmikula-dev

Merge Risk: Minimal · up to adda5

The updated coverage behavior and checks present no identified merge-blocking risk. Merge after normal checks.

Pre-merge checks: 3 · 2

Failed checks: 2 warnings

Check name	Status	Explanation	Resolution
Out of Scope Changes check		The new root CONTEXT.md documents the whole monorepo, including all commands, general conventions, CI structure, and project invariants. This content does not implement or test issue #95. The report…	Remove the unrelated CONTEXT.md additions from this pull request, or move them to a separate documentation pull request.
Docstring Coverage		Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documenta…	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 3 passed

Check name	Status	Explanation
Title check		The title clearly identifies the main change: adding aspect-based coverage to the coverage matrix. It is concise and related to the pull request objective.
Linked Issues check		Issue #95 is open and directly linked. The builder now creates ordered per-aspect coverage, applies bare and aspect-specific links, preserves plain-criterion behavior, sets keyword metadata, and emits…
Description check		The pull request description includes the required Overview, Release Notes, and Related sections. It provides detailed scope, behavior, verification results, acceptance criteria, and closes issue #95.

Full details: Linked Issues check

Explanation

Issue #95 is open and directly linked. The builder now creates ordered per-aspect coverage, applies bare and aspect-specific links, preserves plain-criterion behavior, sets keyword metadata, and emits the three required warning codes. The lineage table maps aspects and keyword values and routes link aspects. The reported tests, fixture updates, full-sample checks, integration assertions, and make qa results cover the remaining coding requirements.

Full details: Out of Scope Changes check

Explanation

The new root CONTEXT.md documents the whole monorepo, including all commands, general conventions, CI structure, and project invariants. This content does not implement or test issue #95. The reported packages/services/CONTEXT.md change also documents the services generally rather than the requested coverage-matrix behavior. These additions are not needed for the per-aspect coverage, warning, lineage, fixture, or integration objectives.

Resolution

Remove the unrelated CONTEXT.md additions from this pull request, or move them to a separate documentation pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 30 functions across 7 files. (8 skipped: 8 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches: 💡 1

📝 Generate docstrings: 💡

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{A rabbit checks each aspect row,
And watches coverage statuses grow.
A bare link visits aspects all,
A named link finds its matching call.
Keywords stay; warnings mark the trail.
The matrix writes the measured tale.}

_{Comment @coderabbitai help to get the list of available commands.}