Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
A Azure Databricks secret stores a sensitive value, such as a password, API key, or hashing salt, so that notebooks, jobs, and functions can use it without the value appearing in code. You can manage secrets in two places:
- Workspace secret scopes: Named collections of secrets in one workspace. Permissions apply to the whole scope. See Workspace secret scopes.
- Secrets in Unity Catalog: Securable objects (
catalog.schema.secret) that are available in every workspace attached to the metastore. You can grant access to each secret. See Secrets in Unity Catalog.
Unity Catalog secrets can be Azure Databricks-managed or backed by Azure Key Vault. See Azure Key Vault secrets in Unity Catalog.
Choose where to store a secret
The following table compares where each option stores secret values, which workspaces can use them, and how you control access.
| Workspace secret scope | Azure Databricks-managed secret in Unity Catalog | Azure Key Vault-backed secret in Unity Catalog (Beta) | |
|---|---|---|---|
| Where the value is stored | Azure Databricks, in the workspace, or your Azure Key Vault with a Key Vault-backed scope | Azure Databricks, in Unity Catalog | Azure Key Vault (Azure Databricks stores only the secret name in Unity Catalog) |
| Encryption with your own key | Yes, with a customer-managed key for managed services | Yes, with a customer-managed key on the catalog | |
| Available in | One workspace | Every workspace attached to the metastore | Every workspace attached to the metastore |
| Access control | Secret scope permissions (no secret-level access control) | Unity Catalog privileges on the secret, schema, or catalog | Unity Catalog privileges on the secret, schema, or catalog |
| To rotate | Update the value in Azure Databricks, or in Key Vault for a Key Vault-backed scope | Update the value in Azure Databricks | Rotate in Azure Key Vault. No change in Azure Databricks. |
Where you can use secrets
Workspace secret scopes and Unity Catalog secrets are supported in different places. Before you choose one, check that it works where your code runs.
| Surface | Workspace secret scope | Secret in Unity Catalog |
|---|---|---|
dbutils.secrets in notebooks and jobs |
Yes. See Read a secret. | Yes, on serverless environment version 4 and above or Databricks Runtime 17.3 LTS and above. See Read a secret. |
| REST API and Databricks CLI | Yes. Values returned by the API aren't redacted. | Yes. Values returned by the API aren't redacted. |
| Remote development, such as Databricks Connect | Yes, with dbutils.secrets in the Databricks Connect client. See Databricks Utilities with Databricks Connect for Python. |
No |
| Unity Catalog Python and Scala UDFs | No | Yes. Callers need only EXECUTE. See Use secrets in UDFs. |
| SQL warehouses | Yes, with the secret function | Only through a Unity Catalog Python or Scala UDF that declares the secret |
| Spark configuration properties and environment variables on compute | Yes. See Use a secret in a Spark configuration property or environment variable. | No |
| Model serving endpoint environment variables | Yes. See Add secrets-based environment variables. | No |
| Default package repositories | Yes, in the databricks-package-management scope. See Configure default package repositories. |
No |
Additional resources
| Resource | Description |
|---|---|
| Secrets in Unity Catalog | Create Unity Catalog secrets, grant access, and read them in notebooks, jobs, and UDFs. |
| Best practices for Unity Catalog secrets | Organize secrets by environment, application, or team, and let workloads use secrets without reading them. |