You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit c2787be
Browse filesBrowse the repository at this point in the historyBrowse files
deltalake tutorial: drop the \${VAR} / envsubst dance — plain placeholders
The export-as-you-go scaffolding doesn't help in practice: most readers
run the GRANT block in the Databricks SQL editor, which doesn't expand
shell vars, so they end up substituting by hand anyway. Strip the
\`export\` lines from 1c/1d/1e and put plain placeholders
(\`<sp-client-id>\`, \`aidbox_staging_loc\`,
\`<target-external-location>\`) back into the 1f grants.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: docs/tutorials/subscriptions-tutorials/data-lakehouse-aidboxtopicdestination.md
+18-32Lines changed: 18 additions & 32 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -529,22 +529,14 @@ In both `managed-*` modes the module **automatically issues `ALTER TABLE ADD COL
529
529
530
530
#### 1c. SQL warehouse
531
531
532
-
Compute → SQL Warehouses → use an existing warehouse or create a new one. Copy the **Warehouse ID** — you'll use it as `databricksWarehouseId`. Stash it in your shell for the grants in 1f:
533
-
534
-
```sh
535
-
export WAREHOUSE_ID=<paste-the-warehouse-id>
536
-
```
532
+
Compute → SQL Warehouses → use an existing warehouse or create a new one. Copy the **Warehouse ID** — you'll use it as `databricksWarehouseId`.
537
533
538
534
#### 1d. Service principal
539
535
540
536
1. In your Databricks workspace, go to **Settings → Identity and access → Service principals → Add service principal**.
541
537
2. Give it a name (e.g. `aidbox-topic-destination`) and create.
542
538
3. Click the new SP, open the **Secrets** tab, click **Generate secret**.
543
-
4. Copy the **Client ID** and **Secret** — you'll use these as `databricksClientId` / `databricksClientSecret`. Stash the Client ID in your shell:
544
-
545
-
```sh
546
-
export SP_CLIENT_ID=<paste-the-client-id>
547
-
```
539
+
4. Copy the **Client ID** and **Secret** — you'll use these as `databricksClientId` / `databricksClientSecret`.
548
540
549
541
#### 1e. (Skip if you don't need an initial bulk copy of existing data) Staging location
`CREATE EXTERNAL LOCATION`itself runs a list + put + delete probe against the bucket as part of the command. If it returned without an error, the IAM trust + permission policies are correct. If it fails with `PERMISSION_DENIED` / 403, the External ID round-trip above almost certainly hasn't propagated yet — wait \~10 s and retry, or confirm the External ID in `trust-policy.json` matches the one shown on the credential's detail page.
Grant only the set that matches the `writeMode` you'll use. The blocks below reference `${SP_CLIENT_ID}`, `${WAREHOUSE_ID}`, `${STAGING_EXTERNAL_LOCATION}` and (for `external-direct`) `${TARGET_EXTERNAL_LOCATION}` — these are the shell variables you exported in steps 1c–1e. Pipe the SQL through `envsubst` to substitute them, or paste the literal values into the Databricks SQL editor. For `external-direct`, the target External Location is the one you registered around your target table's bucket prefix — give it a name like `aidbox_patients_loc` and export it now: `export TARGET_EXTERNAL_LOCATION=aidbox_patients_loc`.
681
+
Grant only the set that matches the `writeMode` you'll use. Substitute `<sp-client-id>` with the service principal Client ID from step 1d, and the External Location name with whatever you picked in step 1e (or registered around the target bucket for `external-direct`).
696
682
697
683
{% tabs %}
698
684
{% tab title="managed-zerobus" %}
@@ -706,13 +692,13 @@ Grant only the set that matches the `writeMode` you'll use. The blocks below ref
GRANT USE CATALOG ON CATALOG aidbox_export TO `${SP_CLIENT_ID}`;
710
-
GRANT USE SCHEMA ON SCHEMA aidbox_export.fhir TO `${SP_CLIENT_ID}`;
711
-
GRANT SELECT, MODIFY ON TABLE aidbox_export.fhir.patients TO `${SP_CLIENT_ID}`;
695
+
GRANT USE CATALOG ON CATALOG aidbox_export TO `<sp-client-id>`;
696
+
GRANT USE SCHEMA ON SCHEMA aidbox_export.fhir TO `<sp-client-id>`;
697
+
GRANT SELECT, MODIFY ON TABLE aidbox_export.fhir.patients TO `<sp-client-id>`;
712
698
-- initial-export only:
713
-
GRANT EXTERNAL USE SCHEMA ON SCHEMA aidbox_export.fhir TO `${SP_CLIENT_ID}`;
699
+
GRANT EXTERNAL USE SCHEMA ON SCHEMA aidbox_export.fhir TO `<sp-client-id>`;
714
700
GRANT READ FILES, WRITE FILES, CREATE EXTERNAL TABLE
715
-
ON EXTERNAL LOCATION `${STAGING_EXTERNAL_LOCATION}` TO `${SP_CLIENT_ID}`;
701
+
ON EXTERNAL LOCATION `aidbox_staging_loc` TO `<sp-client-id>`;
716
702
```
717
703
718
704
Warehouse permissions are **not grantable in SQL** — there's no `GRANT … ON WAREHOUSE` syntax. Do it through the UI: **SQL Warehouses → your warehouse → Permissions (top right) → Add principal → pick the service principal → Can use**.
@@ -729,13 +715,13 @@ Warehouse permissions are **not grantable in SQL** — there's no `GRANT … ON
GRANT USE CATALOG ON CATALOG aidbox_export TO `${SP_CLIENT_ID}`;
733
-
GRANT USE SCHEMA ON SCHEMA aidbox_export.fhir TO `${SP_CLIENT_ID}`;
734
-
GRANTSELECT, MODIFY ON TABLE aidbox_export.fhir.patients TO `${SP_CLIENT_ID}`;
718
+
GRANT USE CATALOG ON CATALOG aidbox_export TO `<sp-client-id>`;
719
+
GRANT USE SCHEMA ON SCHEMA aidbox_export.fhir TO `<sp-client-id>`;
720
+
GRANTSELECT, MODIFY ON TABLE aidbox_export.fhir.patients TO `<sp-client-id>`;
735
721
-- initial-export only:
736
-
GRANT EXTERNAL USE SCHEMA ON SCHEMA aidbox_export.fhir TO `${SP_CLIENT_ID}`;
722
+
GRANT EXTERNAL USE SCHEMA ON SCHEMA aidbox_export.fhir TO `<sp-client-id>`;
737
723
GRANT READ FILES, WRITE FILES, CREATE EXTERNAL TABLE
738
-
ON EXTERNAL LOCATION `${STAGING_EXTERNAL_LOCATION}` TO `${SP_CLIENT_ID}`;
724
+
ON EXTERNAL LOCATION `aidbox_staging_loc`TO `<sp-client-id>`;
739
725
```
740
726
741
727
Warehouse permissions are **not grantable in SQL** — there's no `GRANT … ON WAREHOUSE` syntax. Do it through the UI: **SQL Warehouses → your warehouse → Permissions (top right) → Add principal → pick the service principal → Can use**.
@@ -751,12 +737,12 @@ Warehouse permissions are **not grantable in SQL** — there's no `GRANT … ON
0 commit comments