This lesson's interactive features are locked, please to keep using them

Parquet With Polars

Polars has great Parquet support. Use pl.read_parquet() to read a Parquet file into a DataFrame:

df = pl.read_parquet("sales_data.parquet")

You can also use lazy mode with Parquet for even better performance:

df = (
    pl.scan_parquet("large_data.parquet")  # Lazy scan
    .filter(pl.col("year") == 2024)
    .select(["product", "revenue"])
    .collect()
)

The pl.scan_parquet() function builds a lazy query. Polars can use that query to read only the columns and row groups that it needs.

Writing Parquet

Polars can write Parquet too:

df.write_parquet("filtered_sales.parquet")

Use write_parquet() when you want to save a processed DataFrame back to disk.

Parquet files are usually compressed. Polars supports several compression options, including the modern zstd algorithm:

df.write_parquet("filtered_sales.parquet", compression="zstd")

Our browser-based Python environment has limited support for Polars' native Parquet writer, so this lesson's assignment includes a small write_parquet() helper for you to call. You won't need it in the real world.

Assignment

SnackStack's telemetry archive is getting large, so the analytics team wants to build smaller Parquet extracts for focused analysis.

Complete the export_device_type_report function. It accepts the source_path of an input Parquet file; an output_path to write results to a new Parquet file; a device_type; and a list of columns.

Read the input Parquet file, passing source_path as the file path. Also pass the argument use_pyarrow=True (this is necessary for our current browser Python environment).
Keep only rows matching the given device_type.
Select only the requested list of columns.
Call the provided write_parquet() helper, passing the filtered DataFrame and output_path.
Return the DataFrame.