This lesson's interactive features are locked, please to keep using them
Parquet
Apache Parquet is a columnar, binary file format. It's designed for storage efficiency and fast analytics on large datasets.
Click to play video
Columnar
"Columnar" means the data is organized by column instead of by row.
Imagine a table like this:
product |
inventory |
unit_price |
|---|---|---|
| t-shirt | 10 | 19.99 |
| lamp | 67 | 9.99 |
A row-based format stores each record's data together. A columnar format instead stores the product values together, then the inventory values, then the unit_price values:
| column | stored values |
|---|---|
product |
t-shirt, lamp |
inventory |
10, 67 |
unit_price |
19.99, 9.99 |
This matters because analytics questions often need only a few columns. If you want the average unit_price, a columnar format can often avoid reading unrelated columns entirely!
Columnar formats are usually better for analytics workloads. Row-based formats are usually better for transactional workloads.
Binary
Unlike CSV, JSON, or YAML, Parquet is a binary format. Those other formats are text-based, meaning you can open them in a text editor and read them.
Parquet files are not human-readable. That's annoying when you want to inspect one manually, but it's great for machines: Parquet stores different data types natively, compresses well, and can be much faster to read and write.