LakeLogic Examples
Welcome to the LakeLogic examples! These executable Jupyter Notebooks run universally in Google Colab, Databricks, or locally. They demonstrate how to build End-to-End Lakehouse Pipelines, run data quality gates, and automate PII masking.
Quickstart & Medallion Pipelines
| Example | Description | Links |
|---|---|---|
| 0. Quickstart | Learn the fundamentals of Data Contracts, Pydantic validation, and Data Generation. |  ⬇️ Download Notebook |
| 1. Data Quality & Trust | End-to-end Medallion pipelines featuring SQL-first rules, schema drift protection, and 100% data reconciliation. | ⬇️ Download Notebook |
Enterprise Features
| Example | Description | Links |
|---|---|---|
| 2. Compliance & Governance | Automated PII masking with Presidio, Right-to-Erasure (GDPR) workflows, and immutable automatic lineage. | ⬇️ Download Notebook |
| 3. Engine & Scale | Engine-agnostic dimensional modeling (SCD2, upserts) and incremental execution across Polars, DuckDB, and Spark. | ⬇️ Download Notebook |
| 4. Developer Experience | Boost productivity with CI/CD testing, strict schema validation, and fully automated pipeline scaffolding. | ⬇️ Download Notebook |
| 5. Data Generation AI | AI-powered synthetic referential data, unstructured text extraction, and robust pipeline observability telemetry. | ⬇️ Download Notebook |
| 6. Integrations | Native integration with dbt, dlt (Data Load Tool), Data Catalog APIs, and multi-channel alerting webhooks. | ⬇️ Download Notebook |
Real-Time & Streaming
| Example | Description | Links |
|---|---|---|
| 9. Real-Time & Streaming | Resumable, engine-agnostic streaming: crash-safe Kafka offset resume, SSE Last-Event-ID feeds, memory-bounded keyset backfills, Spark foreachBatch for stateful layers, streaming governance lint, and effectively-once via merge — all runnable with no external infra. |
⬇️ Download Notebook |