Catalog Table Name Support (Unity Catalog, Fabric, Synapse)

LakeLogic supports 3-part table names for Unity Catalog, Fabric LakeDB, and Synapse Analytics. Table names like catalog.schema.table are automatically resolved to their underlying Delta Lake storage paths — no Spark required.

---

Quick Start

Unity Catalog (Databricks) — Fully Automatic

Unity Catalog table names are auto-detected by the processor when DATABRICKS_HOST is set:

from lakelogic import DataProcessor

# Automatically resolves main.default.customers → s3://bucket/.../customers/
processor = DataProcessor(engine="polars", contract="contracts/customers.yaml")
good_df, bad_df = processor.run_source("main.default.customers")

Requirements:

export DATABRICKS_HOST="https://your-workspace.cloud.databricks.com"
export DATABRICKS_TOKEN="dapi..."

Fabric LakeDB (Microsoft)

Fabric uses predictable OneLake paths — no API call needed for resolution. Use resolve_catalog_path() with platform="fabric":

from lakelogic.engines.unity_catalog import resolve_catalog_path

# Resolve table name → OneLake path
path = resolve_catalog_path("myworkspace.sales_lakehouse.customers", platform="fabric")
# → abfss://myworkspace@onelake.dfs.fabric.microsoft.com/sales_lakehouse.Lakehouse/Tables/customers/

# Then use with processor
from lakelogic import DataProcessor
processor = DataProcessor(engine="polars", contract="contracts/customers.yaml")
good_df, bad_df = processor.run_source(path)

Synapse Analytics (Azure)

Synapse uses predictable ADLS Gen2 paths — no API call needed for resolution. Use resolve_catalog_path() with platform="synapse":

import os
from lakelogic.engines.unity_catalog import resolve_catalog_path

os.environ["SYNAPSE_STORAGE_ACCOUNT"] = "mysynapsestorage"

# Resolve table name → ADLS path
path = resolve_catalog_path("salesdb.dbo.customers", platform="synapse")
# → abfss://salesdb@mysynapsestorage.dfs.core.windows.net/dbo/customers/

# Then use with processor
from lakelogic import DataProcessor
processor = DataProcessor(engine="polars", contract="contracts/customers.yaml")
good_df, bad_df = processor.run_source(path)

---

Auto-Detection Behavior

The processor calls resolve_catalog_path(path) without a platform parameter. The auto-detection logic in the code is:

Condition	Resolved As
DATABRICKS_HOST env var is set	Unity Catalog (API call)
No env vars set	Tries Unity Catalog, falls back to raw path
platform="fabric" passed explicitly	Fabric LakeDB (no API, predictable path)
platform="synapse" passed explicitly	Synapse Analytics (no API, predictable path)

Note: The processor currently only auto-detects Unity Catalog. For Fabric and Synapse, resolve the path first with resolve_catalog_path(..., platform="fabric"|"synapse"), then pass the resolved storage path to processor.run_source().

---

Permissions Model

LakeLogic uses a 2-step permission model:

1. Metadata Access (resolve table name → storage path)
2. Unity Catalog: requires Databricks API credentials

3. Fabric/Synapse: not required (predictable paths)

4. Data Access (read/write Delta Lake files)

5. Requires cloud storage permissions (S3, Azure Blob, GCS)
6. Used for all data operations

---

Required Permissions by Platform

Unity Catalog (Databricks)

Metadata Access: - Databricks SELECT, USE CATALOG, and USE SCHEMA permissions - Setup:

export DATABRICKS_HOST="https://your-workspace.cloud.databricks.com"
export DATABRICKS_TOKEN="dapi..."

- Or pass directly:

from lakelogic.engines.unity_catalog import UnityCatalogResolver
resolver = UnityCatalogResolver(
    host="https://your-workspace.cloud.databricks.com",
    token="dapi..."
)

Data Access (AWS S3):

from lakelogic.engines.delta_adapter import DeltaAdapter

adapter = DeltaAdapter(storage_options={
    "AWS_REGION": "us-west-2",
    "AWS_ACCESS_KEY_ID": "AKIA...",
    "AWS_SECRET_ACCESS_KEY": "..."
})
df = adapter.read("main.default.customers")

Required IAM: s3:GetObject, s3:ListBucket on the Unity Catalog storage bucket.

Data Access (Azure ADLS):

adapter = DeltaAdapter(storage_options={
    "AZURE_STORAGE_ACCOUNT_NAME": "your_account",
    "AZURE_STORAGE_ACCOUNT_KEY": "..."
})

Required role: Storage Blob Data Reader on the storage account.

---

Fabric LakeDB (Microsoft)

Metadata Access: Not required — paths follow a predictable pattern:

workspace.lakehouse.table
  → abfss://workspace@onelake.dfs.fabric.microsoft.com/lakehouse.Lakehouse/Tables/table/

Data Access (OneLake): - Requires Azure credentials and Contributor or Reader role on workspace - Setup:

from lakelogic.engines.delta_adapter import DeltaAdapter

adapter = DeltaAdapter(storage_options={
    "AZURE_STORAGE_ACCOUNT_NAME": "onelake",
    "AZURE_STORAGE_ACCOUNT_KEY": "..."
})
df = adapter.read("myworkspace.sales_lakehouse.customers")

Azure AD (recommended):

from azure.identity import DefaultAzureCredential

credential = DefaultAzureCredential()
token = credential.get_token("https://storage.azure.com/.default")

adapter = DeltaAdapter(storage_options={
    "AZURE_STORAGE_ACCOUNT_NAME": "onelake",
    "BEARER_TOKEN": token.token
})

---

Synapse Analytics (Azure)

Metadata Access: Not required — paths follow a predictable pattern:

database.schema.table
  → abfss://database@{STORAGE_ACCOUNT}.dfs.core.windows.net/schema/table/

Requires SYNAPSE_STORAGE_ACCOUNT env var:

export SYNAPSE_STORAGE_ACCOUNT="mysynapsestorage"

Data Access (ADLS Gen2): - Requires Storage Blob Data Reader role

from lakelogic.engines.delta_adapter import DeltaAdapter

adapter = DeltaAdapter(storage_options={
    "AZURE_STORAGE_ACCOUNT_NAME": "mysynapsestorage",
    "AZURE_STORAGE_ACCOUNT_KEY": "..."
})
df = adapter.read("salesdb.dbo.customers")

---

Permission Comparison

Platform	Metadata Access	Data Access	Credentials Required
Unity Catalog	Required (Databricks API)	Required (S3/Azure/GCS)	DATABRICKS_HOST + DATABRICKS_TOKEN + cloud creds
Fabric LakeDB	Not required (predictable paths)	Required (OneLake)	Azure credentials only
Synapse Analytics	Not required (predictable paths)	Required (ADLS Gen2)	SYNAPSE_STORAGE_ACCOUNT + Azure creds

---

Configuration Examples

Unity Catalog (Full Setup)

# .env file
DATABRICKS_HOST=https://your-workspace.cloud.databricks.com
DATABRICKS_TOKEN=dapi...
AWS_REGION=us-west-2
AWS_ACCESS_KEY_ID=AKIA...
AWS_SECRET_ACCESS_KEY=...

from lakelogic import DataProcessor

# Credentials loaded from environment — auto-resolves UC table names
processor = DataProcessor(engine="polars", contract="contracts/customers.yaml")
good_df, bad_df = processor.run_source("main.default.customers")

Fabric LakeDB (Full Setup)

# .env file
AZURE_STORAGE_ACCOUNT_NAME=onelake
AZURE_STORAGE_ACCOUNT_KEY=...

from lakelogic.engines.unity_catalog import resolve_catalog_path
from lakelogic import DataProcessor

path = resolve_catalog_path("myworkspace.sales_lakehouse.customers", platform="fabric")
processor = DataProcessor(engine="polars", contract="contracts/customers.yaml")
good_df, bad_df = processor.run_source(path)

Synapse Analytics (Full Setup)

# .env file
SYNAPSE_STORAGE_ACCOUNT=mysynapsestorage
AZURE_STORAGE_ACCOUNT_NAME=mysynapsestorage
AZURE_STORAGE_ACCOUNT_KEY=...

from lakelogic.engines.unity_catalog import resolve_catalog_path
from lakelogic import DataProcessor

path = resolve_catalog_path("salesdb.dbo.customers", platform="synapse")
processor = DataProcessor(engine="polars", contract="contracts/customers.yaml")
good_df, bad_df = processor.run_source(path)

---

Usage Examples

Contract YAML with Unity Catalog

version: 1.0.0
dataset: bronze_customers
server:
  type: delta
  path: main.default.customers  # Unity Catalog table name
  format: delta

model:
  fields:
    - name: id
      type: integer
    - name: name
      type: string

quality:
  row_rules:
    - not_null: id

from lakelogic import DataProcessor

processor = DataProcessor(engine="polars", contract="contracts/customers.yaml")
good_df, bad_df = processor.run_source()  # Uses path from contract

Explicit Platform Resolution

from lakelogic.engines.unity_catalog import resolve_catalog_path

# Fabric (auto-resolved, no API call)
path = resolve_catalog_path("myworkspace.sales_lakehouse.customers", platform="fabric")
# → abfss://myworkspace@onelake.dfs.fabric.microsoft.com/sales_lakehouse.Lakehouse/Tables/customers/

# Synapse (auto-resolved, no API call)
path = resolve_catalog_path("salesdb.dbo.customers", platform="synapse")
# → abfss://salesdb@mysynapsestorage.dfs.core.windows.net/dbo/customers/

DeltaAdapter for Ad-Hoc Access

from lakelogic.engines.delta_adapter import DeltaAdapter

adapter = DeltaAdapter()

# Read
df = adapter.read("main.default.customers")

# Write
adapter.write(df, "s3://bucket/output/", mode="append")

# Merge (upsert)
stats = adapter.merge(
    target_path="s3://bucket/table/",
    source_df=new_data,
    merge_key="id"
)

Cache Table Paths for Performance

from lakelogic.engines.unity_catalog import get_unity_catalog_resolver

resolver = get_unity_catalog_resolver()

# First call: API request to Databricks
path1 = resolver.resolve_table("main.default.customers")

# Second call: cached (no API request)
path2 = resolver.resolve_table("main.default.customers")

---

Troubleshooting

Unity Catalog table not found

ValueError: Failed to resolve Unity Catalog table main.default.customers: Table not found

1. Check credentials: echo $DATABRICKS_HOST && echo $DATABRICKS_TOKEN
2. Verify table exists: SHOW TABLES IN main.default;
3. Check permissions: SHOW GRANTS ON TABLE main.default.customers;

Fabric table access denied

AuthorizationPermissionMismatch: This request is not authorized

1. Check Azure credentials
2. Verify workspace access (Contributor or Reader role)
3. Use Azure AD authentication instead of account key

Synapse storage account not set

SYNAPSE_STORAGE_ACCOUNT environment variable not set

export SYNAPSE_STORAGE_ACCOUNT="mysynapsestorage"

---

Best Practices

1. Use environment variables for credentials — never hardcode secrets
2. Azure AD for Fabric/Synapse — more secure than storage account keys
3. Leverage caching — UnityCatalogResolver caches paths automatically
4. Use DeltaAdapter for ad-hoc work — the processor uses delta-rs automatically for pipeline runs

---

Summary

Platform	Table Name Format	Auto-Detected by Processor	Resolution Method
Unity Catalog	catalog.schema.table	✅ Yes (when DATABRICKS_HOST set)	Databricks API call
Fabric LakeDB	workspace.lakehouse.table	❌ No (use resolve_catalog_path(..., platform="fabric"))	Predictable path pattern
Synapse Analytics	database.schema.table	❌ No (use resolve_catalog_path(..., platform="synapse"))	Predictable path pattern

---

Last Updated: March 2026