Repository navigation
Conversation
Co-authored-by: ebhills <53243273+ebhills@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Fix not_columns to ignore nonexistent columns without error
Fix missing Oct 11, 2026
not_columns entries in write filters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Linked issue
What changes
writefilters were treating a missingnot_columnsname as a hard failure, even though the expected behavior is to ignore absent columns. This patch makes the recipe dataframe filter treat unknownnot_columnsentries as a no-op while preserving normal wildcard and regex exclusion behavior.wrangles.recipe._filter_dataframeto skip names that do not resolve to real columns before applying exclusions.not_columnsselection behavior unchanged for real column names, wildcard patterns, and regex matches.not_columnsentry alongside a valid exclusion.This now correctly returns the remaining columns instead of raising an error.
How it was verified
python3 -m pytest -c pytest-local.ini tests/recipes/test_read.py tests/recipes/test_write.py -qCompatibility and risk
Low risk. The change only relaxes the failure mode for missing
not_columnsentries; valid exclusions still apply, and the behavior is directly covered by focused regression tests. Safe rollback is a revert of the filter change if needed.Ready-for-review checklist
mainand has no merge conflictsSee the pull request workflow.