Skip to content

source-{various}: harden conditional reduction annotations for hard deletes#4362

Open
Alex-Bair wants to merge 1 commit into
mainfrom
bair/source-various-harden-deletion-reduction
Open

source-{various}: harden conditional reduction annotations for hard deletes#4362
Alex-Bair wants to merge 1 commit into
mainfrom
bair/source-various-harden-deletion-reduction

Conversation

@Alex-Bair
Copy link
Copy Markdown
Member

@Alex-Bair Alex-Bair commented May 5, 2026

Description:

The current conditional reduction annotation of:

if:
  properties:
    _meta:
      properties:
        op: { const: "d" }
then:
  reduce: { delete: true, strategy: merge }
else:
  reduce: { strategy: merge }

also matches when either _meta or op are not present. Meaning, if either of those are absent, the if conditional evaluates to true and the delete: true, strategy: merge in the then branch is applied. That's not what we want; we only want that deletion reduction to be applied when both fields are present and op is "d".

This PR hardens the if check. By requiring _meta and op to be present, the if conditional only evaluates to true when both are present in the document and op is "d":

if:
  required: ["_meta"]
  properties:
    _meta:
      required: ["op"]
      properties:
        op: { const: "d" }
then:
  reduce: { delete: true, strategy: merge }
else:
  reduce: { strategy: merge }

The multitude of snapshot updates are expected due to the additional requireds added to the if/then/else block in discovered schemas.

Notes for reviewers:

For reference on the if/then/else behavior, check out the official JSON schema docs. This callout on that page is relevant:

Because the if schema also doesn't require the "country" property, it will pass and the "then" schema will apply.

I'm not aware of any reported bugs due to the laxer matching behavior, likely because all documents emitted by these connectors have a _meta and _meta/op field present. I'd still like to close the gaps with these conditional reduction annotations so they're better references for future connectors or derivations that need to signal hard deletes via the same conditional reduction annotation.

The current deletion reduction annotation of:
```yaml
if:
    properties:
      _meta:
        properties:
          op: { const: "d" }
  then:
    reduce: { delete: true, strategy: merge }
  else:
    reduce: { strategy: merge }
```
also matches when either `_meta` or `op` are not present. Meaning, if
either of those are absent, the `if` conditional evaluates to `true` and
the `delete: true, strategy: merge` in the `then` branch is applied.
That's not what we want; we only want that deletion reduction to be
applied when when both fields are present and `op` is `"d"`.

This commit hardens the `if` check. By requiring `_meta` and `op` to be
present, the `if` conditional only evaluates to `true` when both are
present in the document _and_ `op` is `"d"`.

The multitude of snapshot updates are expected due to the additional
`required`s added to the discovered schemas.
@Alex-Bair Alex-Bair marked this pull request as ready for review May 5, 2026 14:47
@Alex-Bair Alex-Bair requested a review from a team May 5, 2026 14:47
@Alex-Bair Alex-Bair changed the title source-{various}: harden conditional reduction annotations source-{various}: harden conditional reduction annotations for hard deletes May 6, 2026
@Alex-Bair Alex-Bair self-assigned this May 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant