Remove duplicates from a list
Paste any list, or drop a text or CSV file, and get it back with every duplicate line removed and the original order kept. Choose whether case and surrounding spaces should count, see how many duplicates were found, and do it all without the list leaving your device.
Drop files here or click to browse
Results
How to remove duplicates from a list
- Paste your list into the box, or drop a .txt or .csv file.
- Choose whether to ignore case, trim whitespace and drop blank lines.
- Run. The result shows how many lines came in and how many duplicates were removed.
- Download the cleaned list.
The most common data cleanup job
Lists accumulate duplicates. Two exports get pasted together, a form is submitted twice, a mailing list is merged with an older one, or a product catalogue is re-imported. The result is repeated entries that skew counts, waste money on duplicate mailings and, in the case of email lists, get you flagged for spam. Removing them is the first step of almost every cleanup, and it should not require a formula or a script.
How this tool works
Paste your list into the box, or drop a text or CSV file. Each line is treated as one entry. The tool keeps the first occurrence of every entry and drops the rest, preserving the original order. Nothing is sorted unless you ask. The result tells you how many lines went in, how many came out, and how many duplicates were removed.
What counts as a duplicate
Two entries that are not byte-for-byte identical are often still duplicates to a human. The options control how strict the comparison is:
| Option | Effect | Example treated as duplicate |
|---|---|---|
| Ignore case | Upper and lower case compare equal | Ada@Example.com and ada@example.com |
| Trim whitespace | Leading and trailing spaces and tabs are ignored | "ada@example.com " and "ada@example.com" |
| Ignore blank lines | Empty lines are removed entirely | Every empty line |
For email lists, switch all three on. For product codes or usernames where case is meaningful, leave ignore case off.
Whole rows versus single columns
When you drop a CSV, each entire row is one entry, so two rows are duplicates only if every column matches. That is the right behaviour for removing accidentally repeated rows. If you want to deduplicate on one column, for instance keeping one row per email address even when the names differ, use the extract columns tool to pull out that column first, or sort the sheet by that column in a spreadsheet and remove duplicates there.
Getting the result
The cleaned list downloads as a text file, or as a CSV if that is what you dropped, with the same name and "-deduped" added. Copy it back into your spreadsheet, mailing tool or wherever it came from. Because everything happens in your browser, a customer list or a set of email addresses is never uploaded anywhere.
Related cleanup
If the list is email addresses, the clean email list tool does deduplication plus validation and normalisation in one pass. For merging several files before deduplicating, use the merge CSV tool first.
Frequently asked questions
Is my list uploaded?
No. Deduplication runs entirely in your browser, so email lists and customer data never leave your computer.
Does it keep the original order?
Yes. The first occurrence of each entry is kept in place and later repeats are removed. Nothing is sorted.
Are "Ada@example.com" and "ada@example.com" duplicates?
Only if ignore case is on. Switch it on for emails and names, off for codes where case matters.
How does it treat a CSV file?
Each full row is one entry, so rows are duplicates only when every column matches. To deduplicate on a single column, extract that column first.
Your files never leave your device
Every tool on Squishly Tools is 100% free and runs entirely inside your browser. Your images and documents are processed on your own computer and are never uploaded to a server. There are no accounts, no watermarks and no catch.