Why it matters

Parquet's portability is its biggest value. A table in Parquet can be read by every major analytical engine without conversion. This is what enables data lakes with multiple compute engines against the same storage.

Advertisement

The architecture

A Parquet file is divided into row groups (typically 128 MB). Each row group contains column chunks; each column chunk contains pages (data page, dictionary page, index page). Data pages are encoded per-column with schemes chosen at write time.

File footer holds metadata: row group locations, column statistics, page statistics.

Parquet structureRow group (128 MB)column chunksColumn chunkpages + statsFile footerrow group + statsColumn chunks stored contiguously enable column pruning; page indexes skip within chunks
Parquet columnar layout.
Advertisement

How it works end to end

Column pruning is automatic like ORC. Predicate pushdown uses row group stats (min/max per column) to skip entire row groups.

Dictionary encoding is the biggest single win for low-cardinality columns. A country column with 200 unique values becomes tiny.

Page indexes let readers skip within a column chunk based on column min/max per page. Newer readers use this for fine-grained skipping.