Skip to content

test(backend): add the assertions that mutation testing shows are missing #409

Description

@simonvanlierde

Follow-up to #169 / #399. The first full mutation run left 4351 survivors across 583 functions, now in backend/tests/mutation-baseline.txt. Survivors cluster in a few areas where tests pass without pinning the behaviour. Work through them by group: add the missing assertion, rerun the group with its glob, and lower its baseline lines.

Groups, by risk

  1. Auth flows. OAuth login and associate callbacks (60 and 52 survivors), complete_mfa_challenge (41); app.api.auth.services survives 29% overall.
    just mutation 'app.api.auth.services.oauth*' 'app.api.auth.services.mfa*'
  2. Upload safety. ClamAVScanner.scan (34) and recompute_user_upload_quota (32).
    just mutation 'app.api.file_storage.upload_*'
  3. RPi camera plugin, 38% survival, the highest of any large package: cross_worker_relay._execute_and_respond (91), YouTube livestream and recording setup (73, 62), capture_and_store_image (46).
    just mutation 'app.api.plugins.rpi_cam*'
  4. Error details and keys. About 1150 survivors change a string the code relies on: detail= and reason= arguments, dict keys, return values, exception messages. Tests check the status but not the message (see "Assertions" in backend/tests/README.md).

Router factories (build_categorized_admin_router, the OAuth build_*_router) survive mostly through OpenAPI summary and description strings; leave those in the baseline.

Also

  • mutmut skips decorated functions, so route handler bodies are never mutated. The handlers with real logic are a handful of rpi_cam ones (receive_camera_upload, receive_preview_thumbnail_upload, claim_pairing_code, poll_pairing_status, proxy_hls). Either check their guards with the if False: method or move the logic into plain functions.
  • Rebaseline per function by editing its line; just mutation-baseline replaces the whole file and needs a full run.

Activity

  1. simonvanlierde commented on Oct 8, 2026

    @simonvanlierde
    ContributorAuthor

    Group 2 (upload safety) is in #415: 144 survivors down to 81. The ones left are equivalent mutants (subquery labels, signed vs unsigned length prefix, default codec, the unreachable else_ in _quota_by_role) or not visible through the fake socket (receive buffer size).

    Still open:

    • Group 1 (auth flows). Starting counts match the baseline: handle_oauth_login_callback 60, handle_oauth_associate_callback 52, complete_mfa_challenge 41.
    • app.api.auth.services.mfa_flow.x_require_account_update_step_up has 11 survivors but no line in mutation-baseline.txt. It was added after the baseline was made, so the next scheduled run will likely flag it. Add its line or pin it as part of group 1.
    • When the scanner is unavailable, the reply text from clamd goes into the 503's details, which is neither logged nor returned. Log it, so a scan failure can be diagnosed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions