SnowflakeIO yaml - #39742
Conversation
| import org.apache.beam.sdk.values.Row; | ||
|
|
||
| /** A {@link SchemaTransformProvider} for reading rows from Snowflake. */ | ||
| @SuppressWarnings({ |
There was a problem hiding this comment.
Prefer not introducing SuppressWarnings for new code
| }) | ||
| public class SnowflakeSchemaTransformUtils { | ||
|
|
||
| public static SnowflakeIO.DataSourceConfiguration createDataSourceConfiguration( |
There was a problem hiding this comment.
Why not just make SnowflakeReadSchemaTransformProvider.Configuration as createDataSourceConfiguration's parameter?
There was a problem hiding this comment.
It is shared by both the read and write providers, which have different configuration classes. Passing the individual fields keeps this utility independent of either provider-specific configuration.
|
|
||
| String debugMode = getDebugMode(); | ||
| if (debugMode != null) { | ||
| parseStreamingLogLevel(debugMode); |
There was a problem hiding this comment.
return value is unused. I understand it's for validation. Consider add a comment noting it.
|
Assigning reviewers: R: @claudevdm for label python. Note: If you would like to opt out of this review, comment Available commands:
The PR bot will only process comments in the main thread (not review comments). |
Adds Beam YAML support for Snowflake IO.
ReadFromSnowflakefor batch reads from a table or query.WriteToSnowflakefor batch writes and streaming writes via Snowpipe.Row/ Snowflake schema conversion.standard_io.yaml.https://github.com/apache/beam/actions/runs/31692701363
https://github.com/apache/beam/actions/runs/31691059139
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, commentfixes #<ISSUE NUMBER>instead.CHANGES.mdwith noteworthy changes.See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.