Skip to content

Commit c2787be

Browse files
spicyfalafelclaude
andcommitted
deltalake tutorial: drop the \${VAR} / envsubst dance — plain placeholders
The export-as-you-go scaffolding doesn't help in practice: most readers run the GRANT block in the Databricks SQL editor, which doesn't expand shell vars, so they end up substituting by hand anyway. Strip the \`export\` lines from 1c/1d/1e and put plain placeholders (\`<sp-client-id>\`, \`aidbox_staging_loc\`, \`<target-external-location>\`) back into the 1f grants. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 5973738 commit c2787be

1 file changed

Lines changed: 18 additions & 32 deletions

File tree

‎docs/tutorials/subscriptions-tutorials/data-lakehouse-aidboxtopicdestination.md‎

Lines changed: 18 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -529,22 +529,14 @@ In both `managed-*` modes the module **automatically issues `ALTER TABLE ADD COL
529529

530530
#### 1c. SQL warehouse
531531

532-
Compute → SQL Warehouses → use an existing warehouse or create a new one. Copy the **Warehouse ID** — you'll use it as `databricksWarehouseId`. Stash it in your shell for the grants in 1f:
533-
534-
```sh
535-
export WAREHOUSE_ID=<paste-the-warehouse-id>
536-
```
532+
Compute → SQL Warehouses → use an existing warehouse or create a new one. Copy the **Warehouse ID** — you'll use it as `databricksWarehouseId`.
537533

538534
#### 1d. Service principal
539535

540536
1. In your Databricks workspace, go to **Settings → Identity and access → Service principals → Add service principal**.
541537
2. Give it a name (e.g. `aidbox-topic-destination`) and create.
542538
3. Click the new SP, open the **Secrets** tab, click **Generate secret**.
543-
4. Copy the **Client ID** and **Secret** — you'll use these as `databricksClientId` / `databricksClientSecret`. Stash the Client ID in your shell:
544-
545-
```sh
546-
export SP_CLIENT_ID=<paste-the-client-id>
547-
```
539+
4. Copy the **Client ID** and **Secret** — you'll use these as `databricksClientId` / `databricksClientSecret`.
548540

549541
#### 1e. (Skip if you don't need an initial bulk copy of existing data) Staging location
550542

@@ -682,17 +674,11 @@ CREATE EXTERNAL LOCATION aidbox_staging_loc
682674

683675
`CREATE EXTERNAL LOCATION` itself runs a list + put + delete probe against the bucket as part of the command. If it returned without an error, the IAM trust + permission policies are correct. If it fails with `PERMISSION_DENIED` / 403, the External ID round-trip above almost certainly hasn't propagated yet — wait \~10 s and retry, or confirm the External ID in `trust-policy.json` matches the one shown on the credential's detail page.
684676

685-
Stash the location name in your shell:
686-
687-
```sh
688-
export STAGING_EXTERNAL_LOCATION=aidbox_staging_loc
689-
```
690-
691677
</details>
692678

693679
#### 1f. Grant the service principal
694680

695-
Grant only the set that matches the `writeMode` you'll use. The blocks below reference `${SP_CLIENT_ID}`, `${WAREHOUSE_ID}`, `${STAGING_EXTERNAL_LOCATION}` and (for `external-direct`) `${TARGET_EXTERNAL_LOCATION}` — these are the shell variables you exported in steps 1c–1e. Pipe the SQL through `envsubst` to substitute them, or paste the literal values into the Databricks SQL editor. For `external-direct`, the target External Location is the one you registered around your target table's bucket prefix — give it a name like `aidbox_patients_loc` and export it now: `export TARGET_EXTERNAL_LOCATION=aidbox_patients_loc`.
681+
Grant only the set that matches the `writeMode` you'll use. Substitute `<sp-client-id>` with the service principal Client ID from step 1d, and the External Location name with whatever you picked in step 1e (or registered around the target bucket for `external-direct`).
696682

697683
{% tabs %}
698684
{% tab title="managed-zerobus" %}
@@ -706,13 +692,13 @@ Grant only the set that matches the `writeMode` you'll use. The blocks below ref
706692
| `READ FILES`, `WRITE FILES`, `CREATE EXTERNAL TABLE` | staging External Location | write the bulk Parquet via Unity-Catalog-vended STS (initial-export only) |
707693

708694
```sql
709-
GRANT USE CATALOG ON CATALOG aidbox_export TO `${SP_CLIENT_ID}`;
710-
GRANT USE SCHEMA ON SCHEMA aidbox_export.fhir TO `${SP_CLIENT_ID}`;
711-
GRANT SELECT, MODIFY ON TABLE aidbox_export.fhir.patients TO `${SP_CLIENT_ID}`;
695+
GRANT USE CATALOG ON CATALOG aidbox_export TO `<sp-client-id>`;
696+
GRANT USE SCHEMA ON SCHEMA aidbox_export.fhir TO `<sp-client-id>`;
697+
GRANT SELECT, MODIFY ON TABLE aidbox_export.fhir.patients TO `<sp-client-id>`;
712698
-- initial-export only:
713-
GRANT EXTERNAL USE SCHEMA ON SCHEMA aidbox_export.fhir TO `${SP_CLIENT_ID}`;
699+
GRANT EXTERNAL USE SCHEMA ON SCHEMA aidbox_export.fhir TO `<sp-client-id>`;
714700
GRANT READ FILES, WRITE FILES, CREATE EXTERNAL TABLE
715-
ON EXTERNAL LOCATION `${STAGING_EXTERNAL_LOCATION}` TO `${SP_CLIENT_ID}`;
701+
ON EXTERNAL LOCATION `aidbox_staging_loc` TO `<sp-client-id>`;
716702
```
717703

718704
Warehouse permissions are **not grantable in SQL** — there's no `GRANT … ON WAREHOUSE` syntax. Do it through the UI: **SQL Warehouses → your warehouse → Permissions (top right) → Add principal → pick the service principal → Can use**.
@@ -729,13 +715,13 @@ Warehouse permissions are **not grantable in SQL** — there's no `GRANT … ON
729715
| `READ FILES`, `WRITE FILES`, `CREATE EXTERNAL TABLE` | staging External Location | write the bulk Parquet via Unity-Catalog-vended STS (initial-export only) |
730716

731717
```sql
732-
GRANT USE CATALOG ON CATALOG aidbox_export TO `${SP_CLIENT_ID}`;
733-
GRANT USE SCHEMA ON SCHEMA aidbox_export.fhir TO `${SP_CLIENT_ID}`;
734-
GRANT SELECT, MODIFY ON TABLE aidbox_export.fhir.patients TO `${SP_CLIENT_ID}`;
718+
GRANT USE CATALOG ON CATALOG aidbox_export TO `<sp-client-id>`;
719+
GRANT USE SCHEMA ON SCHEMA aidbox_export.fhir TO `<sp-client-id>`;
720+
GRANT SELECT, MODIFY ON TABLE aidbox_export.fhir.patients TO `<sp-client-id>`;
735721
-- initial-export only:
736-
GRANT EXTERNAL USE SCHEMA ON SCHEMA aidbox_export.fhir TO `${SP_CLIENT_ID}`;
722+
GRANT EXTERNAL USE SCHEMA ON SCHEMA aidbox_export.fhir TO `<sp-client-id>`;
737723
GRANT READ FILES, WRITE FILES, CREATE EXTERNAL TABLE
738-
ON EXTERNAL LOCATION `${STAGING_EXTERNAL_LOCATION}` TO `${SP_CLIENT_ID}`;
724+
ON EXTERNAL LOCATION `aidbox_staging_loc` TO `<sp-client-id>`;
739725
```
740726

741727
Warehouse permissions are **not grantable in SQL** — there's no `GRANT … ON WAREHOUSE` syntax. Do it through the UI: **SQL Warehouses → your warehouse → Permissions (top right) → Add principal → pick the service principal → Can use**.
@@ -751,12 +737,12 @@ Warehouse permissions are **not grantable in SQL** — there's no `GRANT … ON
751737
| `READ FILES`, `WRITE FILES`, `CREATE EXTERNAL TABLE` | target's External Location | write Parquet + Delta commits directly to the bucket |
752738

753739
```sql
754-
GRANT USE CATALOG ON CATALOG aidbox_export TO `${SP_CLIENT_ID}`;
755-
GRANT USE SCHEMA ON SCHEMA aidbox_export.fhir TO `${SP_CLIENT_ID}`;
756-
GRANT SELECT, MODIFY ON TABLE aidbox_export.fhir.patients TO `${SP_CLIENT_ID}`;
757-
GRANT EXTERNAL USE SCHEMA ON SCHEMA aidbox_export.fhir TO `${SP_CLIENT_ID}`;
740+
GRANT USE CATALOG ON CATALOG aidbox_export TO `<sp-client-id>`;
741+
GRANT USE SCHEMA ON SCHEMA aidbox_export.fhir TO `<sp-client-id>`;
742+
GRANT SELECT, MODIFY ON TABLE aidbox_export.fhir.patients TO `<sp-client-id>`;
743+
GRANT EXTERNAL USE SCHEMA ON SCHEMA aidbox_export.fhir TO `<sp-client-id>`;
758744
GRANT READ FILES, WRITE FILES, CREATE EXTERNAL TABLE
759-
ON EXTERNAL LOCATION `${TARGET_EXTERNAL_LOCATION}` TO `${SP_CLIENT_ID}`;
745+
ON EXTERNAL LOCATION `<target-external-location>` TO `<sp-client-id>`;
760746
```
761747

762748
{% hint style="warning" %}

0 commit comments

Comments
 (0)