From 94b4a09270afb2ae60bc32cc0ced5b36da1b607b Mon Sep 17 00:00:00 2001 From: "tejas.sp" <241722411+tejassp-db@users.noreply.github.com> Date: Thu, 14 May 2026 12:45:37 +0530 Subject: [PATCH] docs(databricks): document use_describe_as_json_for_relation_metadata flag Add an entry on the Databricks adapter behavior changes page for the new flag introduced in dbt-databricks 1.12.0, including the false/true behaviors, Unity Catalog / DBR 17.3+ requirements, per-relation automatic fallback, and the dbt_project.yml enablement snippet. --- .../global-configs/databricks-changes.md | 27 +++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/website/docs/reference/global-configs/databricks-changes.md b/website/docs/reference/global-configs/databricks-changes.md index b6bb94b4058..fce0352522e 100644 --- a/website/docs/reference/global-configs/databricks-changes.md +++ b/website/docs/reference/global-configs/databricks-changes.md @@ -13,6 +13,7 @@ The following are the current [behavior change flags](/docs/reference/global-con | [`use_materialization_v2`](#use-restructured-materializations) | 1.10.0 | TBD | Active | | [`use_managed_iceberg`](#use-managed-iceberg) | 1.11.0 | 1.12.0 | Active | | [`use_replace_on_for_insert_overwrite`](#use-replace-on-for-insert_overwrite-strategy) | 1.11.0 | 1.12.0 | Active, defaults to `true` | +| [`use_describe_as_json_for_relation_metadata`](#use-describe-as-json-for-relation-metadata) | 1.12.0 | TBD | Active, defaults to `false` | ## Use information schema for columns @@ -221,3 +222,29 @@ The `use_replace_on_for_insert_overwrite` flag controls which SQL syntax dbt gen If you previously relied on this behavior to get full table replacement without dropping existing metadata, that behavior continues to exist with the flag set to `true`, provided you do not use any partitions or liquid clustering clusters. These data layout optimizations only tend to have a significant effect for tables that are approximately 1 TB large or greater, at which point regular replacement of all of the data is probably not the best approach. + +## Use `DESCRIBE AS JSON` for relation metadata + +The `use_describe_as_json_for_relation_metadata` flag controls how dbt fetches relation-level metadata — primary key, foreign key, and non-null constraints, column masks, row filters, and view descriptions — for Databricks tables and views. This flag defaults to `false`. + +- **`false` (default)**: dbt issues multiple `information_schema` queries per relation to fetch this metadata. +- **`true`**: dbt fetches all of this metadata in a single [`DESCRIBE TABLE EXTENDED ... AS JSON`](https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-aux-describe-table) call per relation. + +Enabling this flag is most useful for projects that have many models with constraints, column masks, or row filters, since it reduces the number of metadata queries dbt has to issue. + +### Requirements + +For the `DESCRIBE AS JSON` path to be used on a given relation, all of the following must be true: + +- The relation is in Unity Catalog (not the Hive metastore). +- The relation is not a foreign table. +- The compute is a SQL warehouse, or runs on DBR 17.3 or later. + +If any of these conditions aren't met for a specific relation, dbt falls back to `information_schema` queries for that relation. The fallback is per-relation, so projects that use a mix of compute types can safely enable the flag. + +To enable the flag, set it in your `dbt_project.yml`: + +```yaml +flags: + use_describe_as_json_for_relation_metadata: true +```