Skip to content

[Adhoc] Fix bigquery data characterization flow run - #3162

Merged
brandantck merged 5 commits into
developfrom
brandantck/ADHOC-fix-bigquery-datacharacterization
Sep 1, 2026
Merged

[Adhoc] Fix bigquery data characterization flow run#3162
brandantck merged 5 commits into
developfrom
brandantck/ADHOC-fix-bigquery-datacharacterization

Conversation

@brandantck

Copy link
Copy Markdown
Collaborator
  1. Adds D2E_BIGQUERY_JDBC_URL env which is required in entrypoint.sh for dataflow-gen-worker.
  2. Add EnableSession=1 to bigquery jdbc connection so that transaction control statements are valid.

Merge Checklist

Please cross check this list if additions / modifications needs to be done on top of your core changes and tick them off. Reviewer can as well glance through and help the developer if something is missed out.

  • Automated Tests (Jasmine integration tests, Unit tests, and/or Performance tests)
  • Updated Manual tests / Demo Config
  • Documentation (Application guide, Admin guide, Markdown, Readme and/or Wiki)
  • Verified that local development environment is working with latest changes (integrated with latest develop branch)
  • following best practices in code review doc

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds BigQuery JDBC session support and forwards the driver download URL to the dataflow worker.

Changes:

  • Enables BigQuery JDBC sessions for transaction statements.
  • Exposes D2E_BIGQUERY_JDBC_URL to the worker container.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
plugins/flows/_shared_flow_utils/dao/daobase.py Enables JDBC sessions in BigQuery connections.
docker-compose.yml Passes the BigQuery JDBC URL to the worker.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread plugins/flows/_shared_flow_utils/dao/daobase.py
@brandantck
brandantck added this pull request to the merge queue Sep 1, 2026
Merged via the queue into develop with commit af096be Sep 1, 2026
70 checks passed
@brandantck
brandantck deleted the brandantck/ADHOC-fix-bigquery-datacharacterization branch September 1, 2026 10:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants