> For the complete documentation index, see [llms.txt](https://docs.hockeystack.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.hockeystack.com/integrations/datasyncs/connecting-your-warehouse/authenticate-databricks.md). # Authenticate Databricks #### Ensure that DataSyncs has correct permissions to access your Databricks workspace HockeyStack connects to Databricks using OAuth machine-to-machine (M2M) authentication with a service principal. For incremental syncs (specifically for tracking new rows and changes), ensure your source table is a Delta table with Change Data Feed enabled. #### What you need Before configuring the Data Sync, prepare: * A Databricks service principal assigned to the target workspace. * An OAuth secret generated for that service principal. * A Databricks SQL warehouse that the service principal can use. * One exact Unity Catalog table that the service principal can read. An account or workspace administrator normally performs this setup. *** ### 1. Create Service Principal and Generate OAuth Credentials A workspace administrator must set up the service principal. As a workspace admin, log in to the Databricks workspace and: * Create or select a service principal. * Assign it to the workspace containing the SQL warehouse and source table. In the service principal details page: * Click the **Secrets** tab * Under OAuth secrets, click Generate secret * Set the secret's lifetime in days (maximum 730 days) * **Important**: Copy both values immediately as Databricks displays them only once: * **Client ID**: (the service principal's application ID) * **Client secret**: (the generated OAuth secret) * Click **Done** when finished {% hint style="info" %} Databricks displays the secret only once. Store it securely and do not add it to source control, documentation, logs, or screenshots. {% endhint %} *** ### 2. Grant Required Permissions Grant the service principal: * `CAN USE` on the selected SQL warehouse. * `USE CATALOG` on the table's catalog. * `USE SCHEMA` on the table's schema. * `SELECT` on the source table. These privileges can be granted directly, inherited from a parent object, or provided through a group. Follow least privilege: the service principal only needs access to the warehouse and source data used by the import. *** ### 3. Find Your SQL Warehouse Connection Details In your Databricks workspace: * Go to **SQL > SQL Warehouses** * Select the warehouse you created for DataSyncs * Click **Connection Details** * Copy the **Server hostname** (e.g., `dbc-a1b2c3d4-e5f6.cloud.databricks.com`) * Copy the **HTTP path** (e.g., `/sql/1.0/warehouses/a1b2c3d4e5f67890`) Enter the hostname in HockeyStack as a full HTTPS workspace URL, for example: ``` https://dbc-a1b2c3d4-e5f6.cloud.databricks.com ``` The HTTP path must identify a SQL warehouse, for example: ``` /sql/1.0/warehouses/a1b2c3d4e5f67890 ``` HockeyStack supports commercial Azure, AWS, and GCP Databricks workspace URLs. Use the workspace URL, not an account-console URL, and do not append `/api` or another path. *** ### Fields to Enter in HockeyStack Collect the following information to enter in HockeyStack: | Field | Value | Example | | --------------------------- | ------------------------------------------- | ------------------------------------------------ | | **Workspace URL** | Full HTTPS workspace URL | `https://dbc-a1b2c3d4-e5f6.cloud.databricks.com` | | **SQL Warehouse HTTP path** | HTTP path from warehouse Connection Details | `/sql/1.0/warehouses/a1b2c3d4e5f67890` | | **Catalog** | Exact Unity Catalog name | `prod` | | **Schema** | Exact schema name in that catalog | `analytics` | | **Table** | Exact source table name | `customer_data` | | **OAuth Client ID** | Service principal application ID | (from step 2) | | **OAuth Client secret** | OAuth secret from step 2 | (from step 2) | > **Important**: Catalog, schema, and table are separate fields. Enter their exact names—do not enter a combined value like `catalog.schema.table` or use wildcards. *** ### Test Your Connection After completing all fields in HockeyStack's Data Sync configuration: 1. Click **Test Connection**. 2. HockeyStack verifies: * The OAuth client ID and secret are valid * Access to the selected SQL warehouse * Access to the exact catalog, schema, and table * The source table schema for use with mappings and filters > Changing a connection, credential, or table field after a successful test requires another connection test. *** ### IP Whitelisting (optional) If your Databricks workspace has IP restrictions, whitelist the following HockeyStack IPs: * 18.184.228.143 * 18.192.106.69 * 35.157.54.242 *** ### Common Failures * **OAuth credentials rejected:** verify the client ID, recopy the secret, check that it has not expired, and ensure the service principal is assigned to the workspace. Do not add spaces around the client ID or secret. * **Warehouse unavailable or forbidden:** confirm the workspace URL and HTTP path and grant `CAN USE` on the warehouse. * **Table cannot be accessed:** verify the exact catalog, schema, and table names and grant `USE CATALOG`, `USE SCHEMA`, and `SELECT`. * **Timeout or connection error:** verify that the workspace and warehouse are available and that network controls allow HTTPS access to Databricks. For incremental **new rows** imports, the source must also be a Delta table with [Change Data Feed](https://docs.databricks.com/aws/en/delta/delta-change-data-feed) enabled. HockeyStack does not enable it automatically. *** ### Security * HockeyStack treats the client secret as a password and never displays a saved secret. * Saved connection details do not expose the client ID, client secret, OAuth tokens, or internal import offsets. * OAuth M2M (machine-to-machine) is the only supported authentication method for this integration. *** ### Databricks References * [Authorize a service principal with OAuth M2M](https://docs.databricks.com/aws/en/dev-tools/auth/oauth-m2m) * [Manage service principals and workspace assignment](https://docs.databricks.com/aws/en/admin/users-groups/manage-service-principals) * [Find SQL warehouse connection details and grant warehouse access](https://docs.databricks.com/aws/en/integrations/odbc/compute) * [Unity Catalog privileges required to read a table](https://docs.databricks.com/aws/en/data-governance/unity-catalog/access-control/privileges-reference) --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.hockeystack.com/integrations/datasyncs/connecting-your-warehouse/authenticate-databricks.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.