Connect Databricks

Databricks SQL Warehouse, via the Statement Execution API. Dashies exchanges a service principal's OAuth credentials for a bearer token and runs statements over HTTPS.

Before you start

In your Databricks workspace:

• Create a service principal with the Databricks SQL and Workspace entitlements.
• Grant it CAN USE on the SQL warehouse Dashies will run against.
• Grant it read on the catalog and the schemas you want readable.
• Mint an OAuth machine-to-machine client secret for it. You need the client id and the secret.

You also need the SQL warehouse's id, which appears in its connection details in the Databricks UI.

If your workspace restricts access with an IP access list, Dashies cannot connect to it yet: adding the data source and building a dashboard do not come from fixed addresses. IP addresses says which data sources do.

1. Add the data source

Open dashies.ai/app/connections, click to add a data source, and pick Databricks.

Field	Required	Example	Notes
Workspace host	yes	dbc-xxxxxxxx-xxxx.cloud.databricks.com	Host only, no scheme and no path.
Warehouse ID	yes	1234567890abcdef	Letters and digits, up to 64 characters.
Client ID	yes	a UUID	The application id of the service principal.
Catalog	no	main	Letters, digits, underscores; up to 128 characters.
Client secret	yes		20 to 256 characters.
Display name	no	Analytics warehouse	Up to 120 characters.

There is no schemas field. Databricks discovers the schemas during verification and stores up to 200 of them as the allowlist. To change what is readable, adjust the service principal's grants in Databricks and press Test.

BigQuery, Snowflake, and Redshift all derive their host from the fields above. Databricks is the one API-based engine that takes the host from you, and it is constrained by a vendor suffix allowlist rather than by resolving and range-checking the address. The host must end in .cloud.databricks.com, .azuredatabricks.net, or .gcp.databricks.com. An IP address in any form, an embedded port, a path, a scheme, or a user prefix are all refused with:

workspace_host is not a valid Databricks workspace host

2. It verifies in one step

Databricks is a single-step connect. Creating the data source exchanges the client credentials for a bearer token, lists the schemas the service principal can read, and comes back active with what it found, or error with a class. There is no Resync button on Databricks.

If it fails:

Message	Meaning
Databricks denied access. Check the client ID and secret, and that the service principal can use the SQL warehouse and read the catalog.	Authentication, or a missing grant.
We couldn't reach Databricks. Check the workspace host and try again.	Network-level failure reaching the host.
We couldn't read that workspace. Check the warehouse ID and catalog, then try again.	The token worked but the target was not usable.

Cost

Connecting is free. Verification uses the Unity Catalog REST API rather than a statement, so it starts no warehouse and burns no warehouse compute.

Introspection normally uses the same zero-warehouse path. It falls back to a information_schema.columns query, which does start the warehouse, only when the service principal is not authorized for the Unity Catalog REST API. If schema introspection is unexpectedly costing you compute, that fallback is why, and the fix is to grant the service principal Unity Catalog read access.

Where the data lives

A Databricks dashboard is served. It keeps its data with Dashies rather than inside the published file: each refresh runs your SQL, writes the result to Dashies' own storage, and the page asks for what it needs when a reader opens it. Every warehouse a refreshing dashboard can publish against is on that path.

The in-file materializations - cube, lattice, hybrid and rows, which ship the numbers inside the page - are not available on a warehouse you connect, so a publish that asks for one on this connection is refused. They are available on the data Dashies provides; Dataset modes is where that and its one plan-dependent exception are set out.

One consequence to design for: the refresh is asynchronous. A newly published dashboard reads "Updating" until the first refresh lands, rather than carrying numbers from the moment you published.

Caps

The first three bound one result read through the adapter, which is the path the authoring tools take. They are not what bounds a served dashboard's own data: that is streamed by the refresh rather than assembled by the adapter. The fourth is engine-independent.

Limit	Value	What happens past it
Rows per dataset query	100,000	Refused before results are fetched.
Inline result envelope	about 25 MiB	The statement fails server-side.
Statement wait	30 seconds, then polling	Query errors if it never completes.
Compiled dashboard body	5,242,880 bytes	Publish is refused.

The envelope cap is a server-side Databricks limit, so a result that exceeds it fails rather than being truncated. Larger results page through presigned external links instead.

How these ceilings relate to each other, and which one binds first, is in sizes and ceilings.

Dialect notes

Cube SQL is Databricks SQL, which is Spark SQL. It is not a PostgreSQL dialect.

• Table references are three-level and backtick-quoted where quoting is needed: from `main`.`analytics`.`orders`. The built-in samples catalog (samples.nyctaxi.trips, samples.tpch.*) is useful for a first test with no seed table of your own.

• Bucket a date with date_trunc('MONTH', ts) or date_format(ts, 'yyyy-MM'):

  select date_trunc('MONTH', ordered_at) as month,
         sum(amount) as revenue
  from main.analytics.orders
  where ordered_at >= current_timestamp() - interval 12 months
  group by 1
  order by 1

• Relative window: current_timestamp() - interval 12 months. Conditional count: count_if(c).

• Databricks preserves an unquoted output alias exactly as written, unlike Postgres and Redshift. sum(amount) as revenue comes back revenue.

• A TIMESTAMP value arrives in the data island as an ISO-8601 UTC string with a T separator and a trailing Z, such as 2024-01-15T10:30:00.123Z. Bucket and format it in SQL rather than parsing the raw text. Big integers keep full precision, as strings.

• The cube must be a single read-only SELECT. Dashies enforces that; Databricks itself would happily run DML, so that guard is the only thing stopping it.

• An ARRAY, MAP, or STRUCT column has to be addressed into before it is usable, and explode multiplies rows. Nothing rejects that, so the cube runs and the numbers are simply wrong. Collapse it back to one row per underlying record, never to the grain the dashboard reports at: a Databricks dashboard is served (Where the data lives), so a statement that has already aggregated gets aggregated a second time and a count comes back as the number of groups you made rather than the number of records. Group the exploded rows back by the key of the record they came from, and declare any column that collapse counts as a total rather than a count. See Verify your numbers.

Rotating the secret

Edit the data source and paste a new OAuth client secret under Client secret.

Check it worked

1. The data source reads active on dashies.ai/app/connections, and lists the schemas it discovered.

2. Ask your AI tool to introspect it and run one query:

   Introspect my Databricks data source, then validate this cube SQL against it: select 1 as ok

3. Then author a dashboard against it.