feat(hive): add Hive built-in driver plugin for data export#3314
Closed
actiontech-bot wants to merge 8 commits into
Closed
feat(hive): add Hive built-in driver plugin for data export#3314actiontech-bot wants to merge 8 commits into
actiontech-bot wants to merge 8 commits into
Conversation
Add DriverTypeHive = "Hive" to the driver type constants block in sqle/driver/v2/util.go, matching DMS-EE's DBTypeHive value exactly. This enables the SQLE plugin system to recognize Hive as a valid database driver type. issue: #2859
Add Hive plugin directory with PluginProcessor and HiveDriverImpl implementing the full Plugin interface. Features: - init() registration to BuiltInPluginProcessors - GetDriverMetas with PluginName="Hive", port=10000 - additionalParams: auth (NOSASL/NONE/LDAP/KERBEROS), transport_mode (binary/http) - Parse with SQL splitting and keyword-prefix classification (DQL/DML/DDL) - Audit returning empty results (no rules in skeleton phase) - Stub implementations for unsupported methods issue: #2859
Map-case style tests covering: - Plugin registration in BuiltInPluginProcessors - GetDriverMetas: PluginName, DefaultPort, auth/transport_mode params, empty Rules and EnabledOptionalModule - classifySQL: DQL (SELECT/WITH/SHOW/DESCRIBE/DESC/EXPLAIN), DML (INSERT/UPDATE/DELETE/MERGE/LOAD/EXPORT), DDL (CREATE/ALTER/DROP/GRANT) - splitSQL: single/multiple/trailing semicolon/empty/whitespace - Audit returns correct-length empty results - Parse with single/multiple/empty SQL - Ping with nil DSN returns error - Open with nil DSN succeeds (offline mode) issue: #2859
Add github.com/beltran/gohive v1.6.0 for real HiveServer2 connectivity in the SQLE Hive plugin. Transitive deps: apache/thrift v0.19.0, beltran/gosasl, beltran/gssapi, go-zookeeper/zk v1.0.3. Version aligned with DMS-EE go.mod to avoid dependency conflicts.
Replace placeholder Ping/Close with real gohive connectivity: - Add newHiveConnection() that creates gohive.Connection from DSN parameters (host, port, user, password, database, auth, transport_mode) following DMS-EE NewHiveConn pattern - Update Open() to establish real connection when DSN is provided - Update Ping() to execute SELECT 1 via gohive cursor - Update Close() to close gohive connection - Add HiveDriverImpl.conn field for connection lifecycle - Add unit tests for nil conn boundary cases Offline audit mode (nil DSN) continues to work without connection.
The Hive built-in plugin's init() function was never called because sqle/server/sqled.go only imported the MySQL driver. Adding the blank import for github.com/actiontech/sqle/sqle/driver/hive ensures the Hive plugin registers itself into BuiltInPluginProcessors at startup, making the Hive data source type available in the DMS driver list. Fixes #2859
…ASES
The Schemas() method was returning an error "hive plugin does not support
Schemas", causing the /v1/projects/{project}/instances/{instance}/schemas
API to return 500. This blocked the data export workflow as users could
not select a database from the dropdown.
Now executes SHOW DATABASES via gohive cursor and returns the result list.
Fixes BUG-2.1-1
Add vendor entries for github.com/beltran/gohive v1.6.0 and its transitive deps (apache/thrift, beltran/gosasl, beltran/gssapi, go-zookeeper/zk). Vendor copies match the go.mod entries in the preceding commit and were validated by go build/vet/test against the new sqle/driver/hive package.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add the Hive built-in driver plugin in SQLE so the data export workflow can connect to HiveServer2, audit Hive SELECT statements, and list schemas via
SHOW DATABASES.DriverTypeHive(sqle/driver/v2/util.go).sqle/driver/hivepackage containing the PluginProcessor + Plugin skeleton, realgohiveOpen/Ping/Close/Schemas implementation, and unit tests.sqle/server/sqled.go.github.com/beltran/gohive v1.6.0+ apache/thrift, beltran/gosasl, beltran/gssapi, go-zookeeper/zk; vendor synced.Fixes actiontech/sqle-ee#2859
Test plan
go vet -mod=vendor ./sqle/...(golang:1.24.1, offline vendor)go test -mod=vendor -count=1 ./sqle/driver/hive/...(golang:1.24.1)go build -mod=vendor ./sqle/...