Remove duplicate lines
Find repeated lines with control over capitalization, whitespace, empty rows, and whether the first or last occurrence stays.
How to use remove duplicate lines
Paste any line-based list and get each entry once. The controls decide the harder questions: whether "Apple" and "apple" are the same entry, whether stray spaces count, and which copy — first or last — is the one that survives.
- 1Paste the list, one entry per line
Exports, email lists, keyword sets, log excerpts — anything where each line is a record. Empty lines are dropped along the way, so a list interrupted by blank rows still deduplicates cleanly.
- 2Set what counts as a duplicate
With Match case off, "Apple" and "apple" are one entry. With Ignore outer spaces on, leading and trailing spaces do not make " apple" distinct. Keep occurrence picks whether the first or the last copy of each entry is retained — Last is what you want when later rows supersede earlier ones.
- 3Clean and compare the counts
Select Clean text. The difference between the two panels’ counters is the number of duplicates removed. Copy the result, or Download it as a text file for re-import.
When this tool is useful
The matching rules, precisely
Two lines are duplicates when their comparison keys match, and the key is simply the line itself — optionally trimmed of outer whitespace, optionally lowercased. Those two switches are the entire matching model, so results are predictable: nothing fuzzy, no near-match guessing, and never any reordering.
Whichever occurrence is kept survives byte for byte, capitalisation included. With Keep occurrence set to First, the list keeps its original order with later repeats removed; set to Last, each entry sits where its final copy appeared — which matters for data that accumulated over time, where the newest row is the corrected one.
Spaces inside a line still count. "John Smith" with a doubled inner space is a different line from "John Smith", and no option here changes that — run the list through remove extra spaces first if inner whitespace should not distinguish entries.
Remove duplicate lines questions
No — deduplication only deletes repeats, it never reorders what remains. If you want an ordered result as well, run the output through sort lines afterwards.
By default yes: Match case is off, so comparison is case-insensitive, and the kept occurrence’s own capitalisation survives. Turn Match case on to keep both variants.
Keep occurrence decides. First keeps the earliest copy in its original position. Last keeps each entry’s final appearance — the usual choice when a list grew over time and newer rows are the trustworthy ones.
Empty rows are removed as part of deduplication: they carry no content, and they would otherwise all count as duplicates of one another. If blank lines separate groups you want to keep apart, deduplicate each group separately.
They differ invisibly — an inner double space, a non-breaking space, or a zero-width character. Run the list through the invisible character remover or remove extra spaces, then deduplicate again.