{source-materialize}-postgres: support JSONB#4400
Conversation
mdibaiee
force-pushed
the
mahdi/postgres-jsonb
branch
from
cdb342e to
cad0b04
Compare
danielnelson
left a comment
danielnelson
left a comment
There was a problem hiding this comment.
Both json and jsonb are encoded in the collection in the same way, so it doesn't seem like the media type is describing anything about the actual format of the data. It also see that contentMediaType is described only for strings.
I feel like we potentially could need to invent media types for every possible source type? I'm also imagining things like a request for mirroring aspects such as string collation. We do already have the information in the description and while it's not super machine readable we could try parsing.
Or perhaps we would be better served with a new custom annotation for source type?
| "doc/bin": { | ||
| "description": "(source type: non-nullable jsonb)" | ||
| "description": "(source type: non-nullable jsonb)", | ||
| "contentMediaType": "application/vnd.postgresql.jsonb+json" |
There was a problem hiding this comment.
I noticed that PostgREST uses a similar media type format, but they define it on their own vendor: application/vnd.pgrst.object+json. Should we instead use something specific to us as a vendor such as application/vnd.estuary.jsonb+json?
I know this specifically meant to handle same db round-tripping, but I wonder if there really won't be any ideas to have things like mysql -> postgres customizations based on the source type, will we have a combinatorial issue?
| // source type annotation flips. | ||
| "json": {sql.NewMigrationSpec([]string{"jsonb"})}, | ||
| "jsonb": {sql.NewMigrationSpec([]string{"json"})}, | ||
| "*": {sql.NewMigrationSpec([]string{"json"}, sql.WithCastSQL(toJsonCast))}, |
There was a problem hiding this comment.
Should * to jsonb also be permitted?
3 participants
Description:
contentMediaType: "application/vnd.postgresql.jsonb+json"annotation in JSONSchema to mark an object as originating from a postgresqlJSONBcolumn typeJSONBfor creating such a columnJSONtoJSONBand vice versaI have identified customers that would be affected by this and I am coordinating with our support team to make sure we can transition them to this new change without breaking their pipelines down the line.
Workflow steps:
(How does one use this feature, and how has it changed)
Documentation links affected:
(list any documentation links that you created, or existing ones that you've identified as needing updates, along with a brief description)
Notes for reviewers:
(anything that might help someone review this PR)