fix: Make the JSON type schema match the JSON that the encoder writes

[stoovon]

Summary

The JSON type schema now moves embedded struct fields up into the parent, skips json:"-" fields, and writes json or utf8 for a type with its own marshaller unless its json tags describe the output, as the encoder does.

Fixes https://linear.app/env-zero/issue/ATH-913

[Copilot]

🟡 Changes recommended

Custom pointer-only marshaler detection is incorrect for value fields, and embedded-field traversal can grow exponentially.

2 open findings

• Deduplicate embedded types to prevent exponential schema generation · New
• Avoid treating non-pointer fields as pointer-only custom marshalers · New

What changed in this PR

Aligns generated JSON type schemas with encoder output.

Changes:

• Flattens embedded fields and resolves naming collisions.
• Detects custom JSON/text marshalers.
• Adds comprehensive schema/encoder tests.

File	Description
transformers/​struct.go	Adds custom-marshaler schema handling.
transformers/​json_schema_fields.go	Implements embedded-field discovery and dominance.
transformers/​json_schema_fields_test.go	Tests schema alignment and edge cases.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟡 Changes recommended

Merged embedded paths can incorrectly propagate addressability and produce a schema that differs from the encoder.

1 open finding

• Preserve first-path addressability for deduplicated embedded types · New

2 resolved since last review

• Avoid treating non-pointer fields as pointer-only custom marshalers
• Deduplicate embedded types to prevent exponential schema generation

🧠 Review effort: Balanced

---

Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

[Copilot]

🔵 Needs a closer look

Map-key schema generation incorrectly applies value-marshaler precedence instead of encoding/json’s map-key rules.

0 open findings

1 resolved since last review

• Preserve first-path addressability for deduplicated embedded types

Previously missed (1)

In code that hasn't changed since last review

Use map-key encoding rules when classifying schema keys

transformers/​struct.go:355

Map keys are not encoded through json.Marshaler; encoding/json uses native string/integer formatting or encoding.TextMarshaler. Routing a key through the value-oriented helper means a comparable key type that implements both interfaces is classified as "json", although the encoder invokes MarshalText and writes string keys. Handle map-key method selection separately so the schema matches the stored object.

🧠 Review effort: Balanced

---

Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

[Copilot]

🟢 Approval recommended

The implementation matches encoder behavior and includes strong regression coverage.

0 open findings

🧠 Review effort: Balanced

---

Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

[erezrokah]

Amazing work! The flattening of embedded structs and the json:"-" fix are correct, for example for k8s pod volumes and probes, stripe and aws.

I ran an audit of this PR on 108 source plugins. For each plugin, I built all tables before and after the change and compared each JSON type schema with the keys that json.Marshal actually writes (types filled with faker data).

Results (8,141 JSON columns in the 41 plugins that change):

	Before	This PR	This PR + #2628
Columns with wrong keys	1,117	173	217
Columns that lose correct detail	–	472	2
Columns with new wrong keys	–	0	0

Problem: 472 columns lose correct detail and become "json". Most generated API clients (Google APIs, OpenAPI Generator, oapi-codegen, auth0) have a MarshalJSON that writes the same keys as the json tags. Their old schema was correct, except for a "" key from json:"-" fields.

Performance: jsonSchemaFields runs again for each struct at each depth. On github, table init goes from 84ms to 165ms and allocations from 94MB to 287MB. On stripe, it goes from 194ms to 596ms.

Proposal (#2628, into this branch):

• A type with MarshalJSON keeps the schema of its tagged fields.
• It uses "json" only when it has no json tags, or when it hides a struct variant in an exported json:"-" field (for example stripe PaymentSource.Card).
• Untagged fields of such types are skipped, for example AdditionalProperties.
• The field lookup is cached per type. github goes to 72ms/88MB, stripe to 78ms/98MB.

With #2628, only 2 columns lose detail: jira Unknowns, 3 of 710 keys. The extra 44 columns with wrong keys are faker artifacts (empty omitempty fields), and these keys were in the schema before this PR too.

Can you review #2628 and merge it into this branch if it looks good?

[erezrokah]

A note on "Columns with wrong keys" (173 vs 217): this number is not a regression in #2628.

• The audit counts a key as wrong when the schema has it but the faker-filled value does not write it.
• A "json" schema has no keys, so it can never have wrong keys. This PR collapses 472 columns to "json", which hides their keys and their detail.
• fix: Keep the JSON type schema for types whose MarshalJSON follows their json tags #2628 restores the full schema in these columns. 44 of them then have keys that the faker value does not write, but all of these keys were already in the schema before this PR.
• The cause is faker, not the schema. For example, datadog image_digest is a datadog.NullableString, and MarshalJSON writes it only when the unexported isSet flag is true. Faker cannot set unexported fields, so the key is missing from the test value, but real API data includes it. The okta profile fields look like the same pattern, and the deep bitbucket, github and gitlab paths look like faker depth limits.

So #2628 adds no new wrong keys and keeps the detail.