Skip to content

fix(capa): reject empty submissions with a clear message#38321

Open
kingoftech-v01 wants to merge 1 commit intoopenedx:masterfrom
kingoftech-v01:fix/bug-36829
Open

fix(capa): reject empty submissions with a clear message#38321
kingoftech-v01 wants to merge 1 commit intoopenedx:masterfrom
kingoftech-v01:fix/bug-36829

Conversation

@kingoftech-v01
Copy link
Copy Markdown

Description

Submitting a CAPA problem with no answer selected returned a cryptic, fragile error message to the learner instead of a clear "you must provide an answer" hint. The underlying flow had three defects:

  1. No server-side empty-answer validation.
  2. grade_answers({}) raised a cryptic StudentInputError.
  3. The error message was passed through a codejail-traceback parser that expects a specific \n-separated format and crashes / falls back to the raw message on single-line errors.

User impact (Learner): Learners who click Submit without selecting an answer now see "You must provide an answer before submitting." and are not charged an attempt.

Root Cause

  • xmodule/capa_block.py:1753 calls make_dict_of_responses(data) on whatever POST body comes in. No validation.
  • xmodule/capa_block.py:1814 sends the empty dict to self.lcp.grade_answers(answers), which raises StudentInputError.
  • xmodule/capa_block.py:1845-1850 runs full_error.split("\\n")[-2].split(": ", 1)[1]. For a single-line error the split yields [full_error], [-2] raises IndexError, and the handler falls back to msg = full_error — exposing the raw, technical English message via gentle_alert in the frontend.

The fragile parser was added in commit b9e768410f (2017-02-01, Peter Pinch @ MIT, "return only error value to students when codejail raises an exception"). It was designed to handle codejail tracebacks containing a \n separator — not ordinary StudentInputError messages. Empty-answer handling was never considered.

Fix

Single, surgical early-return in capa_block.py just before self.lcp.grade_answers(...). If answers is empty or every value is blank ("", [], {}, None), return a clear translatable message without incrementing the attempt counter. Publish a problem_check_fail tracking event with failure="missing_answer" for observability parity with the existing pre-grading guards (closed, unreset, wait-time).

The predicate deliberately does NOT match 0 or False (valid numeric/boolean answers) and does NOT match file uploads (which are file objects, not containers).

The fragile codejail-traceback parser at lines 1845-1850 is intentionally left alone — it is still correct for genuine codejail tracebacks and changing it risks regressing test_submit_problem_error_with_codejail_exception.

Supporting Information

Testing Instructions

  1. Open any CAPA problem in the LMS (multiple choice, checkbox, text input).
  2. Click Submit without selecting any answer.
  3. Before the fix: a cryptic technical message is shown, or the problem is marked incorrect.
  4. After the fix: a clear "You must provide an answer before submitting." alert is shown, and the attempt counter is NOT incremented.
  5. Verify normal answered submissions still work correctly (regression check).

Four new tests in xmodule/tests/test_capa_block.py:

  • test_unit_submit_problem_empty_answer — Unit: empty dict rejection
  • test_integration_submit_problem_all_blank_values — Integration: all blank values path
  • test_submit_problem_zero_is_not_empty — Regression guard: "0" is a valid answer
  • test_bug_36829_regression_empty_answer_publishes_fail_event — Regression: tracking event emission

Deadline

None

Other Information

  • No database migrations
  • No new dependencies
  • No deprecations
  • No schema/API change — the submit_problem return shape is already used by the other pre-grading guards
  • New translatable string ("You must provide an answer before submitting.") follows the same style as existing sibling strings ("Problem is closed.", "You must wait at least...") and will be picked up automatically by manage.py makemessages
  • Client-side guard in display.js:932-1001 has a hardcoded English "Select an option" check that is orthogonal to this bug and should be addressed in a separate UI-focused PR
  • Safe to backport (teak label on the issue)

Closes #36829

submit_problem previously forwarded empty answer dicts straight to
grade_answers, which raised a StudentInputError whose text was mangled
by the codejail-traceback parser and surfaced verbatim to learners.
Guard the empty case before grading so learners see a translatable
"You must provide an answer before submitting." message, leave the
attempt counter untouched, and publish a problem_check_fail event with
failure="missing_answer" for observability parity with the other
pre-grading guards.

Closes openedx#36829
@openedx-webhooks openedx-webhooks added the open-source-contribution PR author is not from Axim or 2U label Apr 10, 2026
@openedx-webhooks
Copy link
Copy Markdown

Thanks for the pull request, @kingoftech-v01!

This repository is currently maintained by @openedx/wg-maintenance-openedx-platform.

Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review.

🔘 Get product approval

If you haven't already, check this list to see if your contribution needs to go through the product review process.

  • If it does, you'll need to submit a product proposal for your contribution, and have it reviewed by the Product Working Group.
    • This process (including the steps you'll need to take) is documented here.
  • If it doesn't, simply proceed with the next step.
🔘 Provide context

To help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:

  • Dependencies

    This PR must be merged before / after / at the same time as ...

  • Blockers

    This PR is waiting for OEP-1234 to be accepted.

  • Timeline information

    This PR must be merged by XX date because ...

  • Partner information

    This is for a course on edx.org.

  • Supporting documentation
  • Relevant Open edX discussion forum threads
🔘 Submit a signed contributor agreement (CLA)

⚠️ We ask all contributors to the Open edX project to submit a signed contributor agreement or indicate their institutional affiliation.
Please see the CONTRIBUTING file for more information.

If you've signed an agreement in the past, you may need to re-sign.
See The New Home of the Open edX Codebase for details.

Once you've signed the CLA, please allow 1 business day for it to be processed.
After this time, you can re-run the CLA check by adding a comment below that you have signed it.
If the CLA check continues to fail, you can tag the @openedx/cla-problems team in a comment for further assistance.

🔘 Get a green build

If one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green.

Details
Where can I find more information?

If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:

When can I expect my changes to be merged?

Our goal is to get community contributions seen and reviewed as efficiently as possible.

However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:

  • The size and impact of the changes that it introduces
  • The need for product review
  • Maintenance status of the parent repository

💡 As a result it may take up to several weeks or months to complete a review and merge your PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

open-source-contribution PR author is not from Axim or 2U

Projects

Status: Needs Triage

Development

Successfully merging this pull request may close these issues.

[Test failure] <TC_LEARNER_45>: <Incorrect error message displays incase of Clicking on Submit button on courses screen without adding answers>

2 participants