Enable OneLake catalog federation

This article shows how to read data in OneLake using catalog federation. This allows Unity Catalog queries to run directly against OneLake storage.

OneLake federation enables you to analyze data stored in your Lakehouse or Warehouse without copying it, bringing powerful analytics and AI/BI capabilities in Azure Databricks directly to your OneLake data. Data access is read-only.

Before you begin

You must meet the following requirements to run federated queries on OneLake using catalog federation:

Workspace requirements:

• Workspace enabled for Unity Catalog.

Compute requirements:

• Network connectivity from your compute resource to the target database systems. See Networking recommendations for Lakehouse Federation.
• Azure Databricks compute must use Databricks Runtime 18.0 or above and standard access mode.
• SQL warehouses must use 2025.40 or above.

Permissions required:

• To create a connection, you must be a metastore admin or a user with the CREATE CONNECTION and CREATE STORAGE CREDENTIAL privileges on the Unity Catalog metastore attached to the workspace.
• To create a foreign catalog, you must have the CREATE CATALOG permission on the metastore and be either the owner of the connection or have the CREATE FOREIGN CATALOG privilege on the connection.

Additional permission requirements are specified in each task-based section that follows.

• You must have permissions to create resources in Azure, configure access in Fabric, and manage Unity Catalog in Azure Databricks.
• Supported authentication methods:
  • Azure Managed Identity via an Access Connector for Azure Databricks
  • Azure service principal

Fabric tenant and workspace settings:

• A Fabric administrator must enable the Service principals can use Fabric APIs tenant setting. This allows managed identities and service principals to authenticate with Fabric. For more information, see Service principal tenant setting.
• A Fabric administrator must enable the Allow apps running outside of Fabric to access data via OneLake tenant setting. This allows Azure Databricks to connect to OneLake storage. For more information, see Allow apps running outside of Fabric to access data via OneLake.
• In the Fabric workspace that contains your data, Authenticate with OneLake user-delegated SAS tokens must be enabled. Navigate to Workspace settings > Delegated settings > OneLake settings to enable this setting.

The following Fabric data items are supported:

• Fabric Lakehouse
• Fabric Warehouse

Set up catalog federation

The following steps guide you through creating the connection and foreign catalog for OneLake federation.

Step 1: Set up Azure authentication

OneLake federation supports two authentication methods.

• Azure Managed Identity (recommended): Uses a Databricks Access Connector with a managed identity.
• Azure Service Principal: Uses a Microsoft Entra ID application with client credentials. This method supports cross-tenant authentication, allowing Databricks to access Fabric workspaces located in a different Azure tenant.

Option A: Create an access connector (Managed Identity)

The Databricks Access Connector creates a managed identity that Azure Databricks uses to authenticate with OneLake.

1. In the Azure Portal, search for and create a new Access Connector for Azure Databricks resource.

2. Follow the prompts to create the connector. This resource creates a system-assigned Managed Identity.

3. Record the Resource ID of the newly created connector. You need this ID when creating the Unity Catalog storage credential.

   The resource ID is in the format:

   /subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.Databricks/accessConnectors/<connector-name>
   

For more information about creating access connectors and using managed identities, see Use Azure managed identities in Unity Catalog to access storage.

Option B: Register a service principal

If you prefer to use a service principal instead of a managed identity:

1. In the Azure Portal, navigate to Microsoft Entra ID > App registrations and register a new application (or use an existing one).
2. Record the Application (client) ID and the Directory (tenant) ID.
3. Under Certificates & secrets, create a new client secret and record the secret value.

Step 2: Grant permissions in Fabric

Grant the managed identity or service principal permission to read the Fabric data.

1. In the Fabric portal, navigate to the workspace that contains your Lakehouse or Warehouse data items.
2. In the workspace, click the Workspace settings gear icon, then click Manage access.
3. Click Add people or groups.
4. Search for and select the Managed Identity or service principal. For Managed Identity, the name should match the access connector you created earlier. For a service principal, search for the application name you registered.
5. Assign the identity the Member role at minimum. You can also assign Contributor or Admin roles.
6. Click Add.
7. Verify that the identity appears in the access list with the appropriate role. Permissions on individual Lakehouse and Warehouse items are inherited from the workspace-level role.

Step 3: Create a storage credential

Create a storage credential in Unity Catalog that references the identity you configured in Step 1.

1. In your Azure Databricks workspace, click Catalog.
2. At the top of the Catalog pane, click the plus icon and select Create a credential from the menu.

If you're using a Managed Identity:

1. In the Create a new credential modal, for Credential Type, choose Azure Managed Identity.
2. For Credential name, enter a Name for the storage credential (for example, onelake_storage_cred).
3. For Access connector ID, enter the resource ID of the access connector you created earlier.
4. (Optional) Add a comment.
5. Click Create.

If you are using a Service Principal:

You cannot create a service principal storage credential using Catalog Explorer. You must be a Azure Databricks account admin and use the Storage Credentials API. For example:

curl -X POST -n \
https://<databricks-instance>/api/2.1/unity-catalog/storage-credentials \
-d '{
   "name": "<storage-credential-name>",
   "read_only": true,
   "azure_service_principal": {
      "directory_id": "<directory-id>",
      "application_id": "<application-id>",
      "client_secret": "<client-secret>"
   },
   "skip_validation": "false"
   }'

You can also create a storage credential by using Databricks Terraform provider and databricks_storage_credential.

Step 4: Create a Unity Catalog connection

Create a Unity Catalog connection that uses the storage credential to access OneLake.

Catalog Explorer

1. In your Azure Databricks workspace, click Catalog.
2. At the top of the Catalog pane, click the plus icon and select Create a connection from the menu.
3. On the Connection basics page, enter a Connection name (for example, onelake_connection).
4. Select a Connection type of OneLake.
5. (Optional) Add a comment.
6. Click Next.
7. On the Connection details page, for Credential, select the storage credential you created in the previous step (for example, onelake_storage_cred).
8. For Workspace, select the OneLake workspace.
9. Click Create connection.

SQL

Run the following command in a notebook or the Databricks SQL query editor. Replace the placeholder values:

• <connection-name>: A name for the connection.
• <workspace-id>: The GUID of your OneLake workspace.
• <storage-credential-name>: The name of the storage credential you created in step 3.

CREATE CONNECTION <connection-name> TYPE onelake
OPTIONS (
  workspace '<workspace-id>',
  credential '<storage-credential-name>'
);

Step 5: Create a foreign catalog

A foreign catalog links a specific Fabric data item to a catalog in Unity Catalog.

Catalog Explorer

1. In your Databricks workspace, click Catalog.
2. At the top of the Catalog pane, click the plus icon and select Create a catalog from the menu.
3. On the Create a new catalog dialog, enter a name for the catalog (for example, fabric_sales).
4. Select a Type of Foreign.
5. Select the Connection you created in Step 4 (for example, onelake_connection).
6. For Data item, select the Lakehouse or Warehouse from the list.
7. (Optional) For Item type, select Lakehouse or Warehouse. Defaults to Lakehouse.
8. (Optional) Select Create volume for lakehouse files to automatically create a volume that maps to the /Files folder of the Lakehouse. This is enabled by default and applies only to Lakehouse items. See Access unstructured data.
9. (Optional) Click Test connection to validate your configuration.
10. Click Create.

SQL

Run the following SQL command in a notebook or the Databricks SQL query editor. Items in brackets are optional. Replace the placeholder values:

• <catalog-name>: Name for the catalog in Azure Databricks.
• <connection-name>: The connection you created in step 4.
• <data-item-id>: GUID of the Fabric Lakehouse or Warehouse.

CREATE FOREIGN CATALOG [IF NOT EXISTS] <catalog-name> USING CONNECTION <connection-name>
OPTIONS (
  data_item '<data-item-id>'
);

You can also specify the following optional parameters:

CREATE FOREIGN CATALOG <catalog-name> USING CONNECTION <connection-name>
OPTIONS (
  data_item '<data-item-id>',
  item_type 'Lakehouse',
  create_volume_for_lakehouse_files 'true'
);

• item_type: Lakehouse (default) or Warehouse.
• create_volume_for_lakehouse_files: true (default) or false. When true, creates a volume for the /Files folder of the Lakehouse. Applies only to Lakehouse items. See Access unstructured data.

The catalog syncs automatically, making the Fabric tables available immediately.

Access unstructured data

OneLake federation supports read-only access to unstructured data stored in the /Files folder of a Lakehouse. This is achieved through Unity Catalog volumes.

• Requires Databricks Runtime 18.1 or above, or SQL warehouse version 2026.10 or above.

When you create a foreign catalog with create_volume_for_lakehouse_files set to true (the default), a volume is automatically created under the onelake-folders schema. This volume points to the /Files folder of the Lakehouse and provides read-only access to files stored there.

You can browse and read files using the volume path:

LIST '/Volumes/<catalog-name>/onelake-folders/files/';

df = spark.read.format("binaryFile").load("/Volumes/<catalog-name>/onelake-folders/files/")

You can also query structured files directly:

SELECT * FROM parquet.`/Volumes/<catalog-name>/onelake-folders/files/table.parquet`

Grant permissions on federated tables

After setting up catalog federation, users must have the appropriate Unity Catalog permissions to access federated tables:

• All users need USE CATALOG and USE SCHEMA permissions on the catalog and schema respectively.
• To read from the federated table, users need the SELECT permission.

For more information about Unity Catalog privileges and how to grant them, see Manage privileges in Unity Catalog.

Query OneLake data

After the setup is complete, you can find and query OneLake data in Unity Catalog.

Browse the catalog

1. In your Databricks workspace, navigate to Catalog Explorer.
2. Locate the catalog you created (for example, fabric_sales).
3. Expand the catalog to see the synchronized schemas and tables from the Fabric Lakehouse or Warehouse.

Run queries

Use the three-part naming convention (catalog.schema.table) in Databricks SQL or notebooks:

SELECT COUNT(*)
FROM fabric_sales.silver.customer_details;

SELECT
  customer_id,
  customer_name,
  total_purchases
FROM fabric_sales.silver.customer_details
WHERE total_purchases > 1000
ORDER BY total_purchases DESC
LIMIT 10;

Limitations

OneLake federation has the following limitations:

• Read-only access: Only SELECT queries are supported. Write operations are not available.
• Authentication: Azure Managed Identity and Azure service principal are the supported authentication methods.
• Supported data items: Only Fabric Lakehouse and Warehouse items are supported.
• Compute requirements: Dedicated access mode is not supported.
• Complex datatypes (arrays, maps, structs) are not supported.
• Materialized views and views are not supported.
• Unstructured data access (volumes) requires Databricks Runtime 18.1 or above, or SQL warehouse version 2026.10 or above.