Skip to content

fix(incremental): refuse a --time-column or --unique-key that does not exist (#157) - #165

Merged
laaielichter merged 2 commits into
laaielichter:mainfrom
Ipsita-Mondal-30:fix/incremental-validate-names
Nov 16, 2026
Merged

laaielichter merged 2 commits into
laaielichter:mainfrom
Ipsita-Mondal-30:fix/incremental-validate-names

Conversation

@Ipsita-Mondal-30

Copy link
Copy Markdown
Contributor

Closes #157

gulliver incremental copied --time-column and --unique-key into the generated model without checking them, and returned status: ok with exit 0.

Change (src/gulliver/features.py)
_check_incremental_names runs after the strategy is known and before the savings query:

  • --time-column must be a column of the source table (time_column_source). Otherwise: status: error, Time column 'nosuch' is not a column of events., and a next_step listing the timestamp-like columns (or all columns if none look like timestamps). Exit code 1.
  • every --unique-key name must be an output column of the model. Otherwise: Unique key column(s) not returned by the model: nosuch, with the model's columns in next_step.
  • names are compared case-insensitively.
  • column names are read with SELECT * ... LIMIT 0 through _gated_run, so the extra queries go through the same guarded path as the savings queries.
  • no connection (features.incremental(None, ...)): nothing is checked, output unchanged. If the source or the rendered model cannot be queried (for example a dbt model that does not run as plain SQL), the check is skipped and a notes entry says so.
  • valid names give the same output as before (a test compares the checked and offline results).

Tests (tests/test_features.py): unknown time column, unknown unique key, mixed case, valid names equal to the offline advice, no connection, and the skipped-check note. CHANGELOG line added.

pytest (892 passed, 10 skipped) and ruff check . pass.

Check the time column against the source table and the unique key against
the model's output columns (case-insensitive) before generating advice.
Column names are read with LIMIT 0 through the guarded path. Without a
connection, or when the columns cannot be read, nothing is checked and a
note says so.

Closes laaielichter#157
@Ipsita-Mondal-30

Copy link
Copy Markdown
Contributor Author

Ran locally before opening: pytest (full suite) and ruff check . pass. The CI run is waiting for a maintainer to approve the workflow for a first-time contributor, so there is no CI result yet; could you approve it when you have a moment? Happy to adjust the approach or the wording of the messages.

@laaielichter laaielichter left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved on eeebf15. Thank you, @Ipsita-Mondal-30, a second solid PR. Reading names with SELECT * ... LIMIT 0 through _gated_run, and turning every "cannot check" case into a note instead of a failure, is what the issue asked for.

What I checked, on this head merged with current main (no conflicts with main):

  • ruff check . clean; full pytest 892 passed, 10 skipped.
  • Before (main): --time-column nosuch gives status: ok, exit 0, with a failing savings line; --unique-key nosuch gives status: ok. After: exit 1, Time column 'nosuch' is not a column of events. with ts listed, and Unique key column(s) not returned by the model: nosuch with user_id, last_ts, n listed. A bad name in --unique-key user_id,nosuch is caught too.
  • Valid names, TS / USER_ID in other case, and events.ts give the same output as main.
  • A missing table and a dbt {{ ref() }} model still return their advice (the missing table with a note that the check was skipped).
  • Mutation check: with src/gulliver/features.py reverted to main, the three refusal and note tests fail.
  • The diff is limited to features.py, tests/test_features.py and CHANGELOG.md.

One small non-blocking suggestion, which can be a follow-up or an extra commit if you like: a quoted name is compared with its quotes, so a column that does exist is refused. --unique-key '"user_id"' now gives Unique key column(s) not returned by the model: "user_id" (main returned ok), and --time-column '"ts"' says '"ts"' is not a column of events. Stripping one pair of surrounding double quotes before the comparison (and a test for it) would fix both. I am not holding the PR for it.

Ready to merge from my side. It still needs the maintainer to approve the first-time CI runs (action_required) and the 14 checks have to pass. I will add the "Thanks to @Ipsita-Mondal-30" credit to the CHANGELOG line myself. #164 adds a line at the same spot in CHANGELOG.md, so whichever of the two lands second will need main merged and both lines kept.


Generated by Claude Code

Copy link
Copy Markdown
Owner

@Ipsita-Mondal-30 a heads-up: #164 landed just now and added its CHANGELOG line in ## Unreleased, right where this PR adds its own, so this PR now conflicts with main in that file only. Could you merge main into your branch and keep both lines (yours at the end of the list)? The code stays approved and CI was green on this head. After the merge, the CI runs wait for the maintainer's approval once more.


Generated by Claude Code

Copy link
Copy Markdown
Owner

Good news: you no longer need to resolve the CHANGELOG conflict. We changed the rule so contributors don't edit CHANGELOG.md any more; we add your entry with credit when we merge. Easiest fix: drop your CHANGELOG.md change (or merge main and take main's version of the file) and push; the rest of the PR is unchanged.


Generated by Claude Code

@Hayabusa-YL

Copy link
Copy Markdown

Thanks — #140 is closed in favour of #161, understood. I'd like to take #165 (refuse a --time-column/--unique-key that doesn't exist) unless someone's already on it; I'll open a PR with tests.

Copy link
Copy Markdown
Owner

Hi @Hayabusa-YL, welcome to Gulliver, and thanks for the careful write-up on #140.

#165 is not free to take over: #157 is assigned to @Ipsita-Mondal-30, and #165 is her open PR for it. Please don't open a second PR for #157 or #165.

A free issue to start with is #159, a good first issue with no code needed: translate the README quick start into a language of your choice. Comment on #159 with the language you take and it is yours. If you would rather write code, #125 (lint rule GV019, with a same-rows autofix) is free too.

Contributing guide: CONTRIBUTING.md. Setup is pip install -e ".[dev]", then pytest and ruff check .. Please don't edit CHANGELOG.md in your PR; the maintainer adds the entry with credit when it is merged.

If Gulliver is useful to you, a ⭐ helps other people find it.


Generated by Claude Code

@Ipsita-Mondal-30

Copy link
Copy Markdown
Contributor Author

Merged main into the branch and took main's CHANGELOG.md, so this PR no longer touches it and the conflict is gone. Full suite (1089 passed, 10 skipped) and ruff pass locally. Could you approve the workflow run again when you have a moment?

@laaielichter laaielichter left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-verified on d25017d, merged with current main, CHANGELOG.md taken from main:

  • ruff check . clean; full pytest 1089 passed, 10 skipped.
  • Repro of #157 on main and on this head: --time-column nosuch and --unique-key nosuch now give status: error with the columns listed. Valid names, and mixed case such as TS / USER_ID, give the same output as main.
  • Mutation check: with features.py reverted to main, the three new tests fail (unknown time column, unknown unique key, skipped-check note).
  • All 14 checks are green on this head.

Still open from my earlier review, non-blocking: a quoted name such as --time-column '"ts"' is refused although main accepted it. Stripping one pair of surrounding double quotes would fix that, as a follow-up or a small extra commit.

Thanks, @Ipsita-Mondal-30. Merging now; the CHANGELOG credit goes in with the credits PR.


Generated by Claude Code

@laaielichter
laaielichter merged commit 2ee274e into laaielichter:main Nov 16, 2026
14 checks passed
laaielichter added a commit that referenced this pull request Nov 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

incremental: a --time-column or --unique-key that does not exist is accepted, and the generated model uses it (status ok, exit code 0)

3 participants