Skip to content

Make SQL syntax errors readable and actionable (#128) - #139

Merged
laaielichter merged 3 commits into
laaielichter:mainfrom
amritkang165:fix/readable-syntax-errors
Nov 16, 2026
Merged

laaielichter merged 3 commits into
laaielichter:mainfrom
amritkang165:fix/readable-syntax-errors

Conversation

@amritkang165

@amritkang165 amritkang165 commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

What and why

SQL syntax errors currently expose SQLGlot’s internal token details and tell users to send a SELECT even when the problem is a typo. This formats the structured parser description, line, column and highlighted token into a readable message. Leading whitespace is preserved when parsing so positions refer to the supplied SQL.

Tokenizer failures have no structured position, so they use an explicitly labelled end-of-input fallback with a hint to check for an unterminated quote or comment. Lint, preflight, rehearse and run give syntax-specific retry advice while keeping status block and exit code 2. Existing advice for multiple statements, non-SELECT statements and security blocks is preserved. File lint uses the same wording and keeps its GV900 severity policy.

Fixes #128

How it was tested

  • Added 36 regression cases covering parser diagnostics in four dialects, leading whitespace, tokenizer failures, CLI exit codes/advice and file lint.
  • Before the fix, the initial new cases reported 31 failures and two passing non-syntax cases.
  • Focused SQL safety, CLI/MCP and lint-report suite: 124 passed.
  • Full suite after merging current main: 1037 passed, 10 skipped (no live PostgreSQL test database configured).
  • Security self-test: all 151 corpus cases and all sandbox checks passed.
  • ruff check . and git diff --check pass.
  • README, agent docs and skill guide updated.

Tested on macOS arm64 with Python 3.14.7. The three issue examples were also checked through the CLI; all return readable diagnostics and exit code 2. Hosted CI results are tracked separately on this PR.

Merged current main on 2026-11-15, preserving both branches’ CLI/MCP tests and both documentation sections. CHANGELOG.md matches main, following the updated contribution guide.

@laaielichter laaielichter left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @amritkang165. #128 is yours (your claim at 20:14 on 2026-11-10 was the only one; I have assigned it), and this PR does what the issue asks, carefully.

What I checked. I merged f26440d onto current main (c48adf8) in a scratch worktree (a fast-forward), on Python 3.13 / Linux: ruff check . passes and pytest -q gives 907 passed, 10 skipped. gulliver selftest security passes all 151 corpus cases, which matters because _classify is the read-only gate and this PR changes what it parses.

Against #128's "Done when":

  • The three examples now read at line 1, column 13, near "FROM": Expected table name., at line 1, column 12, near "c": Invalid expression / Unexpected token. and at line 1, column 12 (end of input): Could not tokenize SQL; check for an unterminated quote or comment. None has <Token or TokenType, status stays block, exit code stays 2.
  • next_step for a syntax error is the exact text the issue suggests. DROP TABLE t, SELECT 1; SELECT 2 and a data-changing CTE keep their old messages and hints. SELECT 1; and ;; with whitespace still pass.
  • Parsing the original sql instead of the stripped text is the right way to get real positions, and I could not find a case where the gate decides differently. Passing parse_error as a flag, rather than matching the message text, is a good choice.
  • Tests cover all four dialects, a mid-query error, leading whitespace, unterminated quote and comment, lint/preflight/rehearse/run exit codes and advice, and file lint (GV900). The docs, AGENTS.md, llms.txt and the skill are updated. Nothing in the diff is outside the scope.

Two small notes, neither blocks the merge:

  • The CHANGELOG line is at the end of ### Added. That is fine for an enhancement and keeps it away from the other open PRs, which add to ### Fixed.
  • Other open PRs also add tests at the end of tests/test_cli_and_mcp.py. Whichever lands second needs a git merge origin/main and to keep both blocks.

CI: ci and codeql for this commit are action_required (a maintainer has to approve a first-time contributor's run), so Linux, macOS and Windows have not run on GitHub yet; please do not worry, it is not a failure.

Next free one if you want it, after #129 and #128: #126 (gulliver.toml with a wrong-type section crashes with a traceback).


Generated by Claude Code

Copy link
Copy Markdown
Owner

@amritkang165 #105 just merged, so CHANGELOG.md and tests/test_cli_and_mcp.py now conflict with this PR (both add lines at the end). Could you merge or rebase main into your branch and keep both blocks? The code is approved. The CI run also still needs my approval, which I will give once the branch is updated.


Generated by Claude Code

@amritkang165

Copy link
Copy Markdown
Contributor Author

Merged main and kept both test blocks. All 919 tests pass, with 10 PostgreSQL tests skipped. Ruff and the security self-test pass too. Thanks!

@laaielichter laaielichter left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @amritkang165, the merge of main is clean and both test blocks are kept.

What I checked on 3b2421a. The code is line for line the same as f26440d, which I approved before; the only differences are the merge itself and the CHANGELOG line, which now sits under ## Unreleased / ### Added above ## 0.4.0, where it belongs. With current main merged in: ruff check . passes, pytest -q gives 922 passed, 10 skipped, and gulliver selftest security passes all 151 corpus cases.

One more merge of main, sorry. #147 landed today and opened ## Unreleased / ### Fixed in CHANGELOG.md at the same spot, so this PR conflicts in that file again (nothing else). Please keep both under the one ## Unreleased heading: your ### Added block and the ### Fixed list from main.

CI: ci and codeql for this head are action_required, so the 11 required checks have not run on GitHub yet; the maintainer has to approve the runs. Once main is merged in and they are green, this is ready to merge.


Generated by Claude Code

Copy link
Copy Markdown
Owner

@amritkang165 a small heads-up: #147 and its changelog credit landed on main right after my approval, so this PR now conflicts with main in CHANGELOG.md only. Could you merge main into your branch once more and keep both lines, yours at the end of ### Fixed under ## Unreleased? The code stays approved. I do not push to contributor branches, so this needs to come from you. After that, CI can run, and its runs wait for the maintainer's approval as usual.


Generated by Claude Code

Copy link
Copy Markdown
Owner

Good news for part of the conflict: the CHANGELOG part got easier. We changed the rule so contributors don't edit CHANGELOG.md any more; we add your entry with credit when we merge. When you merge main, just take main's version of CHANGELOG.md (or drop your change to it). The conflicts in docs/features/lint-ci.md and tests/test_cli_and_mcp.py still need a normal merge of main.


Generated by Claude Code

@laaielichter laaielichter left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked the head locally: full suite 1037 passed, 10 skipped; ruff check . clean. Reproduced #128 on main (TypeError/raw token repr) and on this head the message gives the line, column and the "Expected table name" detail with exit code 2. The new tests fail on main, so they cover the change.

Thanks @amritkang165 for the clear messages and the regression tests.


Generated by Claude Code

@laaielichter
laaielichter merged commit b57109e into laaielichter:main Nov 16, 2026
14 checks passed
laaielichter added a commit that referenced this pull request Nov 16, 2026
laaielichter added a commit that referenced this pull request Nov 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SQL syntax errors show sqlglot's internal Token repr, and the next_step talks about statement types instead of the syntax error

2 participants