selfdrive/car/tests: add test_panda_safety_tx_fuzzy#38053
Closed
GitMonsters wants to merge 1 commit into
Closed
Conversation
Adds a hypothesis-based fuzz test that verifies every CAN message sent by the CarController with controls_allowed=True is accepted by panda's safety_tx_hook. The test: - Warms up panda and the CarInterface with 300 frames of real CAN data so vehicle speed, steering angle, and other internal state are valid - Draws random actuation values with wider-than-nominal ranges to exercise openpilot's clamping logic (torque ±2.0, accel -8..6 m/s², steer angle ±200° for angle-control cars) - Runs 10 s of CarController frames and asserts panda rejects nothing Mismatches would reveal bugs in openpilot's value clamping or panda's safety limit configuration. Closes commaai#32425 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
Process replay diff reportReplays driving segments through this PR and compares the behavior to master. ✅ 0 changed, 66 passed, 0 errors |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #32425
What
Adds
test_panda_safety_tx_fuzzytoselfdrive/car/tests/test_models.py— a hypothesis-based fuzz test that verifies openpilot and panda agree on which TX messages are safe to send.How it works
Warm-up (300 CAN frames): Feeds real route CAN data through both the CarInterface and panda's
safety_rx_hook, establishing valid vehicle speed, steering angle, torque sample state, and cruise state in panda before any TX checks. This is key to catching warm-state bugs like Toyota LTA: fix the way the torquesample_t's are used panda#1948 where panda'ssample_tbuilt up incorrect state.Fuzzing: Draws random actuation values with deliberately wider-than-nominal ranges using hypothesis:
torque ∈ [-2.0, 2.0],accel ∈ [-8, 6] m/s²steeringAngleDeg ∈ [-200°, 200°],accel ∈ [-8, 6] m/s²Assertion: Runs 10 s of CarController frames with
controls_allowed=Trueandcruise_engaged_prev=True. Asserts panda'ssafety_tx_hookrejects zero messages.Design goals from the issue
@pytest.mark.nocapturepattern astest_panda_safety_carstate_fuzzy. Additional state (e.g.set_torque_meas,set_angle_meas) can be added straightforwardlyMAX_EXAMPLESenv var (default 300) andPhase.reusefor fast hypothesis replay, same as the existing carstate fuzzerSkips
dashcamOnlycars (no controls, nothing to test)notCarplatforms