7:50 · Music video
The Iceberg Open Lakehouse
Follow the stack from durable files through snapshots, catalogs, engines, governance, and agent-ready semantics.
Open videoOpen lakehouse architecture
Your data, in formats
anything can read.
An open lakehouse keeps analytical data in open formats on storage you control, and adds enough metadata that those files behave like tables. Any compliant engine can read them. No vendor sits between you and your own data.
This site is my corner of that world: how I think about each layer, where to find my work, and my books. For neutral definitions, OpenDataLakehouse.comis the reference.
Featured music video
The open lakehouse in motion
A visual and musical walkthrough of the Iceberg architecture I write, teach, and build around.
Alex's work in the open lakehouse
This site follows the decisions I make and explain: why an open table format matters when engines change, where catalogs enforce access, and how to give AI systems a stable semantic contract. For the general, vendor-neutral reference architecture, start at OpenDataLakehouse.com; for my books and talks, browse speaking topics or the book catalog.
Start with the vocabulary
Most disagreements about the lakehouse turn out to be two people using one word for different things. These entries fix the terms first.
- 01What is an open lakehouseA data architecture that keeps analytical data in open formats on object storage you control, readable by any compliant engine.
- 02Object storageThe durable, cheap byte store at the bottom of the stack, and the constraints it imposes on everything above it.
- 03File formatHow records are arranged inside a single file, which decides how much of it a query has to read.
- 04Table formatThe metadata layer that turns a collection of files into a table with atomic commits, schema evolution, and history.
- 05CatalogThe service that resolves table names, performs the atomic commit, enforces access, and hands out scoped credentials.
- 06Semantic layerWritten definitions of what the data means, so people and machines compute the same metric the same way.
The projects that carry the weight
Five open projects do most of the work in a modern lakehouse. One page each, covering what it is, where it fits, and where it stops.
Apache ParquetThe columnar file format most lakehouse data sits in, built to be small on disk and cheap to scan selectively.Apache ArrowA standard way to lay out tabular data in memory so processes and languages can share it without converting it first.Apache IcebergThe table format that turns files in object storage into a real table, with atomic commits, schema evolution, and history.Apache PolarisAn open catalog for Iceberg tables that resolves names, enforces access, and vends scoped credentials.Apache OssieAn incubating project defining a vendor-neutral standard for semantic metadata, so metrics are defined once and used everywhere.
Where to start reading
A few pieces from the Dremio blog that cover the ground this site is about.
- ArchitectureOpen Source and the Data LakehouseWhy the openness of each layer has to be assessed separately, rather than as one claim about a platform.
- Table formatsWhat Are Table Formats and Why Were They Needed?The path from raw files to table formats, and the specific problems that made them necessary.
- CatalogsApache Polaris: The Catalog Standard for Lakehouses and AIHow an open catalog standard keeps tables interoperable across engines and clouds.