This lesson's interactive features are locked, please to keep using them

Removing Duplicates

Duplicate rows can sneak into your dataset from joins, imports, or simple data entry errors. They can:

Inflate totals or averages
Create confusion in charts and tables
Slow down refresh performance

Cleaning them ensures that every record represents a unique piece of data, so it's best to clean them up early in Power Query.

When to Remove Duplicates

I like to remove duplicates after importing but before merging or appending any data. This keeps your transformations clean and avoids spreading duplicate data further down the pipeline.

We Have Duplicate Data

Take a good look at our sales.csv data – it has duplicate rows! We have multiple rows with an id of 6, and there are likely others besides! Let's fix it!

Assignment

In Power Query, with the sales query selected and the sale_id column selected, click "Remove Rows" from the top ribbon of the Home tab.
Select "Remove Duplicates."

When removing duplicates, make sure that column(s) you select truly identify a unique record, otherwise, you might accidentally remove useful data.
Click "Close & Apply" in the ribbon to save your changes and close Power Query.

Save the project and, from the course directory, submit the CLI tests.