Skip to content

[CONTRIB] Promote ExpectColumnValuesToMatchStrftimeFormat to supported-core - #12009

Merged
joshua-stauffer merged 3 commits into
fivetran:developfrom
nanjeshramesh:promote-strftime-format-expectation
Aug 5, 2026
Merged

[CONTRIB] Promote ExpectColumnValuesToMatchStrftimeFormat to supported-core#12009
joshua-stauffer merged 3 commits into
fivetran:developfrom
nanjeshramesh:promote-strftime-format-expectation

Conversation

@nanjeshramesh

@nanjeshramesh nanjeshramesh commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Summary

Closes #12004.

ExpectColumnValuesToMatchStrftimeFormat already existed and was exported, but wasn't in the Expectation Gallery, had no generated schema, carried none of the support metadata, and had an undefined backend matrix. This brings it up to the supported-core bar:

  • Fixes a docstring bug: the old docstring contained the literal, never-interpolated token {FAILURE_SEVERITY_DESCRIPTION}. Converted to an __doc__ = f"""...""" f-string in the standard Gallery format (short description, Args, Supported Data Sources, Data Quality Issues, Example Data, passing/failing Code Examples), following the ExpectColumnValuesToMatchRegex exemplar.
  • Adds Gallery support metadata: EXPECTATION_SHORT_DESCRIPTION, DATA_QUALITY_ISSUES, SUPPORTED_DATA_SOURCES, a library_metadata ClassVar with _library_metadata alias, and Config.schema_extra.
  • Adds the class to supported_expectations in tasks.py and generates great_expectations/expectations/core/schemas/ExpectColumnValuesToMatchStrftimeFormat.json via invoke schemas --sync.

Backend matrix decision: shipping outcome (A) from the issue — Pandas + Spark supported, SQL explicitly declared out of scope. The backing metric (column_values_match_strftime_format.py) has no SqlAlchemy branch, and generic strftime-format validation doesn't map cleanly onto SQL dialects' date-format models (as noted in the issue, with precedent in ExpectColumnValuesToMatchRegex, which also omits some SQL dialects). Happy to follow up with a SQL implementation (outcome B) separately if maintainers want it split out.

Changes

  • great_expectations/expectations/core/expect_column_values_to_match_strftime_format.py — Gallery-format docstring, support metadata, schema config
  • great_expectations/expectations/core/schemas/ExpectColumnValuesToMatchStrftimeFormat.json — generated schema (new file)
  • tasks.py — added to supported_expectations
  • tests/integration/data_sources_and_expectations/expectations/test_expect_column_values_to_match_strftime_format.py — extended from Spark-only to also cover Pandas, plus added mostly-threshold success/failure cases

Test plan

  • ruff check / ruff format --check pass on all changed files
  • pytest tests/expectations/core/test_core_model_schemas.py passes (schema-sync check)
  • pytest tests/expectations/ -m unit — 1498 passed, 0 failed
  • Full pytest tests/ -m unit — 4330 passed, 0 failed
  • New integration tests pass locally against Pandas (-m "not spark"): timezone-aware format success (regression test for Validating datetime strings with timezone produces an error #9203), non-matching format failure, mostly-threshold met/not-met
  • Spark integration tests require pyspark, which isn't installed in my local dev environment — these are unchanged in logic from the existing Spark-only test and should run in maintainer CI

Brings the expectation up to the Gallery-supported bar: fixes the
broken docstring (the FAILURE_SEVERITY_DESCRIPTION token was never
interpolated), adds Gallery support metadata and a generated JSON
schema, and declares Pandas + Spark as the supported backend matrix
(SQL is out of scope for now, per the discussion in the issue -
strftime tokens don't map cleanly onto SQL dialects' date-format
models).

Extends the integration tests to cover Pandas alongside the existing
Spark cases, plus mostly-threshold coverage.

Closes fivetran#12004

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@netlify

netlify Bot commented Jul 28, 2026

Copy link
Copy Markdown

👷 Deploy request for niobium-lead-7998 pending review.

Visit the deploys page to approve it

Name Link
🔨 Latest commit f026d35

@github-actions github-actions Bot added the cla-not-signed https://github.com/fivetran/great_expectations/blob/develop/CLA.md label Jul 28, 2026
@github-actions

github-actions Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

All committers have signed the CLA. ✅

@nanjeshramesh

Copy link
Copy Markdown
Contributor Author

@cla-bot check

@github-actions github-actions Bot added cla-signed and removed cla-not-signed https://github.com/fivetran/great_expectations/blob/develop/CLA.md labels Jul 28, 2026
@joshua-stauffer joshua-stauffer changed the title Promote ExpectColumnValuesToMatchStrftimeFormat to supported-core [CONTRIB] Promote ExpectColumnValuesToMatchStrftimeFormat to supported-core Aug 3, 2026

@joshua-stauffer joshua-stauffer left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hey @nanjeshramesh, thanks for the PR! this looks great, I just have some notes for the docstring. The passing case which fails is the critical one, the other two are more nits, but would be nice to clean up while we're making changes.

Once those changes are in we'll get this merged. Thanks again!

}},
"meta": {{}},
"success": false
}}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is labeled as a passing case, but success is (correctly) false. We could use the pattern in great_expectations/expectations/core/expect_column_values_to_match_regex.py and have the example data use two columns instead, then use one for the passing case and the other for the failing case.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the docstrings are rendered in the Expectations gallery, so we want one actual passing case, and one failing case.


SQL data sources are not currently supported: strftime format tokens do not map cleanly \
onto the date-format models of SQL dialects.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the newline continuation here is likely to cause extra whitespace to be rendered. Can we remove it, and move this section up to the bottom of the Column Map Expectations are one of the most common types of Expectation. paragraph?

mostly (None or a float between 0 and 1): \
Successful if at least mostly fraction of values match the expectation. \
{MOSTLY_DESCRIPTION} \
For more detail, see [mostly](https://docs.greatexpectations.io/docs/reference/expectations/standard_arguments/#mostly).

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we append Default 1. to this line, like in great_expectations/expectations/core/expect_column_values_to_match_regex.py?

nanjeshramesh and others added 2 commits August 4, 2026 14:24
- Rework the Gallery Code Examples to use two columns (event_date /
  invalid_date), so the Passing Case genuinely passes (success: true)
  and the Failing Case genuinely fails, following the pattern in
  ExpectColumnValuesToMatchRegex. Both examples were run against the
  actual expectation to confirm the JSON output is accurate.
- Move the SQL-unsupported caveat out of the indented Supported Data
  Sources block (where its line continuation rendered with extra
  whitespace) and onto the end of the introductory Column Map
  Expectations paragraph instead.
- Append "Default 1." to the mostly parameter description, matching
  the regex exemplar.

Regenerated the JSON schema to match the updated docstring.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
My previous commit moved the SQL-unsupported sentence to the end of
the intro paragraph but kept the backslash line-continuation between
its two lines, reproducing the exact rendering issue that was flagged
instead of fixing it. The surrounding sentences in that paragraph use
plain lines with no continuation, so match that: each sentence is now
its own line with no trailing backslash.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@nanjeshramesh

Copy link
Copy Markdown
Contributor Author

Hey @joshua-stauffer, thanks for the review! Pushed fixes for all three:

  1. Reworked the Passing/Failing Code Examples to use two columns (event_date / invalid_date), following the ExpectColumnValuesToMatchRegex pattern — the Passing Case now genuinely passes (success: true) and the Failing Case genuinely fails. Ran both against the actual expectation to confirm the JSON output is accurate.
  2. Moved the SQL-unsupported caveat to the bottom of the intro paragraph and removed the line-continuation backslash so it renders as a plain sentence, consistent with the rest of that paragraph.
  3. Appended "Default 1." to the mostly description.

Ready for another look whenever you get a chance. Thanks again!

@joshua-stauffer joshua-stauffer left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks great, thanks @nanjeshramesh

@joshua-stauffer
joshua-stauffer added this pull request to the merge queue Aug 5, 2026
Merged via the queue into fivetran:develop with commit bdd9446 Aug 5, 2026
78 of 80 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make ExpectColumnValuesToMatchStrftimeFormat a fully-supported, Gallery-published Expectation

2 participants