fix(capa): reject empty submissions with a clear message#38321
fix(capa): reject empty submissions with a clear message#38321kingoftech-v01 wants to merge 1 commit intoopenedx:masterfrom
Conversation
submit_problem previously forwarded empty answer dicts straight to grade_answers, which raised a StudentInputError whose text was mangled by the codejail-traceback parser and surfaced verbatim to learners. Guard the empty case before grading so learners see a translatable "You must provide an answer before submitting." message, leave the attempt counter untouched, and publish a problem_check_fail event with failure="missing_answer" for observability parity with the other pre-grading guards. Closes openedx#36829
|
Thanks for the pull request, @kingoftech-v01! This repository is currently maintained by Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review. 🔘 Get product approvalIf you haven't already, check this list to see if your contribution needs to go through the product review process.
🔘 Provide contextTo help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:
🔘 Submit a signed contributor agreement (CLA)
If you've signed an agreement in the past, you may need to re-sign. Once you've signed the CLA, please allow 1 business day for it to be processed. 🔘 Get a green buildIf one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green. DetailsWhere can I find more information?If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources: When can I expect my changes to be merged?Our goal is to get community contributions seen and reviewed as efficiently as possible. However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:
💡 As a result it may take up to several weeks or months to complete a review and merge your PR. |
Description
Submitting a CAPA problem with no answer selected returned a cryptic, fragile error message to the learner instead of a clear "you must provide an answer" hint. The underlying flow had three defects:
grade_answers({})raised a crypticStudentInputError.\n-separated format and crashes / falls back to the raw message on single-line errors.User impact (Learner): Learners who click Submit without selecting an answer now see "You must provide an answer before submitting." and are not charged an attempt.
Root Cause
xmodule/capa_block.py:1753callsmake_dict_of_responses(data)on whatever POST body comes in. No validation.xmodule/capa_block.py:1814sends the empty dict toself.lcp.grade_answers(answers), which raisesStudentInputError.xmodule/capa_block.py:1845-1850runsfull_error.split("\\n")[-2].split(": ", 1)[1]. For a single-line error the split yields[full_error],[-2]raisesIndexError, and the handler falls back tomsg = full_error— exposing the raw, technical English message viagentle_alertin the frontend.The fragile parser was added in commit
b9e768410f(2017-02-01, Peter Pinch @ MIT, "return only error value to students when codejail raises an exception"). It was designed to handle codejail tracebacks containing a\nseparator — not ordinaryStudentInputErrormessages. Empty-answer handling was never considered.Fix
Single, surgical early-return in
capa_block.pyjust beforeself.lcp.grade_answers(...). Ifanswersis empty or every value is blank ("",[],{},None), return a clear translatable message without incrementing the attempt counter. Publish aproblem_check_failtracking event withfailure="missing_answer"for observability parity with the existing pre-grading guards (closed,unreset,wait-time).The predicate deliberately does NOT match
0orFalse(valid numeric/boolean answers) and does NOT match file uploads (which are file objects, not containers).The fragile codejail-traceback parser at lines 1845-1850 is intentionally left alone — it is still correct for genuine codejail tracebacks and changing it risks regressing
test_submit_problem_error_with_codejail_exception.Supporting Information
b9e768410f(Peter Pinch, 2017-02-01)9ee7ef1acf(reduce escaping, 2018) and file moves (b0d41f6ebb,4c005e86e8)Testing Instructions
Four new tests in
xmodule/tests/test_capa_block.py:test_unit_submit_problem_empty_answer— Unit: empty dict rejectiontest_integration_submit_problem_all_blank_values— Integration: all blank values pathtest_submit_problem_zero_is_not_empty— Regression guard: "0" is a valid answertest_bug_36829_regression_empty_answer_publishes_fail_event— Regression: tracking event emissionDeadline
None
Other Information
submit_problemreturn shape is already used by the other pre-grading guards"You must provide an answer before submitting.") follows the same style as existing sibling strings ("Problem is closed.","You must wait at least...") and will be picked up automatically bymanage.py makemessagesdisplay.js:932-1001has a hardcoded English"Select an option"check that is orthogonal to this bug and should be addressed in a separate UI-focused PRCloses #36829