Skip to content

selfdrive/car/tests: add test_panda_safety_tx_fuzzy#38053

Closed
GitMonsters wants to merge 1 commit into
commaai:masterfrom
GitMonsters:feat/tx-fuzz-test
Closed

selfdrive/car/tests: add test_panda_safety_tx_fuzzy#38053
GitMonsters wants to merge 1 commit into
commaai:masterfrom
GitMonsters:feat/tx-fuzz-test

Conversation

@GitMonsters

@GitMonsters GitMonsters commented May 16, 2026

Copy link
Copy Markdown

Closes #32425

What

Adds test_panda_safety_tx_fuzzy to selfdrive/car/tests/test_models.py — a hypothesis-based fuzz test that verifies openpilot and panda agree on which TX messages are safe to send.

How it works

  1. Warm-up (300 CAN frames): Feeds real route CAN data through both the CarInterface and panda's safety_rx_hook, establishing valid vehicle speed, steering angle, torque sample state, and cruise state in panda before any TX checks. This is key to catching warm-state bugs like Toyota LTA: fix the way the torque sample_t's are used panda#1948 where panda's sample_t built up incorrect state.

  2. Fuzzing: Draws random actuation values with deliberately wider-than-nominal ranges using hypothesis:

    • Torque-controlled: torque ∈ [-2.0, 2.0], accel ∈ [-8, 6] m/s²
    • Angle-controlled: steeringAngleDeg ∈ [-200°, 200°], accel ∈ [-8, 6] m/s²
    • Wide ranges exercise openpilot's clamping logic on every example
  3. Assertion: Runs 10 s of CarController frames with controls_allowed=True and cruise_engaged_prev=True. Asserts panda's safety_tx_hook rejects zero messages.

Design goals from the issue

  • Detects mismatches: Any panda rejection with controls_allowed=True is a real bug (missing clamp in CarController, or limit mismatch between openpilot and panda)
  • Extensible: Follows the same hypothesis + @pytest.mark.nocapture pattern as test_panda_safety_carstate_fuzzy. Additional state (e.g. set_torque_meas, set_angle_meas) can be added straightforwardly
  • Performant: Uses MAX_EXAMPLES env var (default 300) and Phase.reuse for fast hypothesis replay, same as the existing carstate fuzzer

Skips

  • dashcamOnly cars (no controls, nothing to test)
  • notCar platforms

Adds a hypothesis-based fuzz test that verifies every CAN message sent
by the CarController with controls_allowed=True is accepted by panda's
safety_tx_hook. The test:

- Warms up panda and the CarInterface with 300 frames of real CAN data
  so vehicle speed, steering angle, and other internal state are valid
- Draws random actuation values with wider-than-nominal ranges to
  exercise openpilot's clamping logic (torque ±2.0, accel -8..6 m/s²,
  steer angle ±200° for angle-control cars)
- Runs 10 s of CarController frames and asserts panda rejects nothing

Mismatches would reveal bugs in openpilot's value clamping or panda's
safety limit configuration.

Closes commaai#32425

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions github-actions Bot added CI / testing car vehicle-specific labels May 16, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Process replay diff report

Replays driving segments through this PR and compares the behavior to master.
Please review any changes carefully to ensure they are expected.

✅ 0 changed, 66 passed, 0 errors

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

car vehicle-specific CI / testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test_models: add a test that fuzzes the tx messages

2 participants