AI validation and auto-correction of submissions (Claude API) #11

Depends on: #5

Background / Context

We want submissions to auto-publish without a human reviewing every one, because the team can't review continuously. To make that safe, new and edited submissions should be run through the Claude API on the backend to confirm a submission looks like a real hackathon, normalize its fields, enrich missing data from the provided URL where possible, and flag spam/garbage.

This hooks into the backend submission endpoint POST /api/hackathons (and edits via PATCH /api/hackathons/:id and POST /api/hackathons/:id/change-request) defined in the Backend REST API issue. Today there is no backend persistence and no API calls at all — the frontend sources hackathons from frontend/src/lib/sample-hackathons.ts merged with localStorage (see frontend/src/pages/Home.tsx and frontend/src/components/SubmitHackathonModal.tsx), and the backend backend/main.py is a Flask stub with only /health and /ping. This issue adds the AI validation step that runs server-side once that real backend exists.

Seam with backend-api (single source of truth): the backend issue (backend-api) owns the server implementation of POST /api/hackathons, PATCH /api/hackathons/:id, and POST /api/hackathons/:id/change-request, and exposes a validation hook point at each. This issue only fills that hook with the AI logic — it does not define, route, or own those endpoints. No code-ownership conflict: backend-api implements the endpoints and the seam; ai-validation provides the validator the seam calls.

The validation result decides the status of a submission: a clean submission becomes published automatically; a suspicious or incomplete one becomes pending for periodic human review. The submission rule is permissive — only one of { name, url } is required — so the validator must be able to work from very little (often just a URL) and enrich the rest.

Goal

Add a backend AI-validation step that runs on submit/edit (via the hook points backend-api exposes), normalizes and (where possible) enriches fields, and sets status to published (clean) or pending (suspicious/incomplete) — failing safe to pending whenever anything goes wrong.

Tasks

• Fill the validation hook that backend-api exposes on POST /api/hackathons for new submissions, and on PATCH /api/hackathons/:id / POST /api/hackathons/:id/change-request for edits. This issue does not add those endpoints — backend-api owns them; here we only provide the validator the hook invokes.
• Implement a Claude API call (Anthropic Python SDK, pip install anthropic, add to backend/requirements.txt) that takes the raw submission and returns a structured decision.
• Use the cheap model claude-haiku-4-5 for routine checks (pricing: $1.00 / 1M input tokens, $5.00 / 1M output tokens; 200K context). Only escalate to a larger model (e.g. claude-opus-4-8) if a routine check is inconclusive — keep escalation optional/secondary for now.
• Constrain the model's output with a JSON schema so the backend always gets a predictable object. Use output_config={"format": {"type": "json_schema", "schema": {...}}} (or the Python SDK's client.messages.parse() with a Pydantic model, or strict tool use). The schema should cover at least: is_real_hackathon (bool), is_spam (bool), decision ("publish" | "flag"), normalized (the cleaned Hackathon fields), and notes (string explaining the decision).
• In the structured prompt, instruct the model to:
  • Confirm the input plausibly describes a real hackathon (vs. spam/garbage/ads).
  • Normalize fields: dates → ISO 8601 (startDate, endDate), tidy location (free-text city/venue), and map mode to one of "in-person" | "online" | "hybrid".
  • Enrich missing fields from the provided url where possible (e.g. infer name, description, organizer, dates) — this is important since a low-friction submit may provide only url.
• Map the model's decision to the canonical status lifecycle: clean → status: "published" (the default auto-publish path); suspicious/incomplete/spam → status: "pending" for periodic human review. (A change-request to a published item produces a "needs-changes" pending edit; the live item stays untouched until an admin/AI approves it.)
• Persist the AI decision/notes alongside the hackathon record (e.g. an internal aiNotes / aiDecision field, plus updatedAt) so admins reviewing pending / needs-changes items can see why something was flagged.
• Make the API key configurable via environment variable (ANTHROPIC_API_KEY) — never hardcode it; document it in the backend README/.env.example.
• Implement a graceful fallback: if the Claude API errors, times out, rate-limits, or returns an unparseable/refused result, default the submission to status: "pending" (fail safe — never auto-publish on failure) and record a note that validation didn't run.

Acceptance criteria

• Submitting a hackathon via POST /api/hackathons runs AI validation before the resource is finalized, and the returned resource includes a status decided by that validation.
• Fields are normalized: dates come back as ISO 8601 in startDate / endDate, and mode is one of "in-person" | "online" | "hybrid" when it can be inferred.
• A submission that provides only a url gets enriched with at least name where the URL makes that possible.
• An obvious spam/garbage submission is flagged and ends up as status: "pending", not published.
• A clean, complete submission auto-publishes (status: "published") with no human action.
• The AI decision/notes are stored on the record and visible to an admin reviewing pending items.
• The API key is read from configuration (env var), and the app starts/handles a missing key without crashing on unrelated routes.
• When the Claude API is forced to fail (e.g. bad/empty key, simulated timeout), the submission is accepted and stored as status: "pending" rather than erroring out or auto-publishing.

Dependencies

• backend-api — this issue fills the validation hook that backend-api exposes on POST /api/hackathons, PATCH /api/hackathons/:id, and POST /api/hackathons/:id/change-request. Those endpoints, their persistence, and the hook point itself are owned by backend-api; this issue owns only the AI validator that runs inside the hook. The AI step can be stubbed/feature-flagged until those endpoints and persistence land, so backend work can proceed in parallel.

Out of scope / notes

• No frontend changes are required here — the validator runs entirely server-side. The frontend just renders whatever status the backend returns.
• The admin review UI for pending / needs-changes items is a separate concern (a known follow-up not yet drafted); this issue only needs to store the decision/notes so that UI can later surface them.
• interestCount and the Discord-driven interest feature are explicitly deferred and not part of validation.
• Helpful references for implementation: the claude-api skill / Anthropic Python SDK. Model IDs and pricing were verified against that skill: claude-haiku-4-5 ($1.00 / 1M input, $5.00 / 1M output, 200K context) for routine checks, escalating to claude-opus-4-8 only when needed; structured output via output_config={"format": {"type": "json_schema", "schema": {...}}} or client.messages.parse() with a Pydantic model.
• Suggested new labels (not applied here, since only existing repo labels may be used): backend and ai. An admin may want to add and apply these.