Add StarRocks connector#29189
Conversation
|
Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: ivan.torres.
|
0211bd2 to
a958bff
Compare
|
Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla |
|
@cla-bot check |
|
The cla-bot has been summoned, and re-checked this pull request! |
12fd482 to
74da5e8
Compare
Use StarRocks JDBC for metadata and Arrow Flight SQL for reads. The connector is read-only in v1 and avoids MySQL-specific metadata assumptions that break on StarRocks, including reliance on JDBC COLUMN_SIZE during type discovery. It adds docs, unit tests, connector smoke tests, and live integration smoke tests for schema, table, view, and basic SELECT query coverage.
74da5e8 to
549d864
Compare
Description
Add an initial native read-only StarRocks connector.
The connector uses the StarRocks JDBC driver for metadata discovery and Apache Arrow Flight SQL for reads. This avoids MySQL-specific metadata assumptions that can break on StarRocks, such as relying on JDBC
COLUMN_SIZEbeing present during type discovery.The initial version focuses on:
DESCRIBEandSHOW CREATE TABLEsupportSELECTqueries over Arrow Flight SQLARRAY,MAP, andSTRUCTvaluesOut of scope for v1 are writes, schema mutations, connector-managed DDL for complex types, and advanced optimizations beyond Flight SQL partition descriptors returned by StarRocks.
Additional context and related issues
This work follows the design direction discussed in #28735 and is informed by the Doris Flight SQL connector in #29120, while remaining a StarRocks-specific implementation rather than extending the generic MySQL connector path.
Prior StarRocks-specific work for context:
The StarRocks-specific choices in this PR are:
INFORMATION_SCHEMArather than MySQL connector metadata codestarrocks.catalog-name, with fallback for StarRocks versions whereINFORMATION_SCHEMA.COLUMNS.TABLE_CATALOGis absent or not populatedValidation includes:
BaseConnectorSmokeTestpatternerrorprone-compilervalidation matching the previously failing CI jobLocal validation commands:
./mvnw -pl plugin/trino-starrocks test./mvnw -pl plugin/trino-starrocks clean compile test-compile -DskipTests -Dair.check.skip-all=true -P errorprone-compiler./mvnw -pl plugin/trino-starrocks -Dtest=TestStarRocksIntegrationSmokeTest -Dstarrocks.test.integration.enabled=true -Dstarrocks.test.jdbc-url=jdbc:starrocks://127.0.0.1:9031 -Dstarrocks.test.flight-sql-host=127.0.0.1 -Dstarrocks.test.flight-sql-port=9408 testLocal validation notes:
4.0.9-f6475899031and Arrow Flight SQL on9408SHOW SCHEMAS,SHOW TABLES,DESCRIBE,SHOW CREATE TABLE,SELECT count(*), datetime reads, views, largeint fallback, and basic aggregation pushdownSHOW TABLES, 198.6 ms forDESCRIBE, 241.6 ms forSELECT count(*), 252.2 ms forSELECT sum(amount), 253.0 ms for a grouped aggregate, and 261.8 ms for a filtered projection over 1k rowsRelease notes
( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(x) Release notes are required, with the following suggested text: