About the Remove Duplicate Lines
This duplicate line remover takes any list — email addresses, keywords, product SKUs, URLs, names or log lines — and returns each distinct line once, keeping the first occurrence and the original order. Paste the text, pick how strict the matching should be, and copy the cleaned result.
It is handy for marketers merging keyword or contact lists, developers cleaning up log output or config files, and anyone deduplicating a spreadsheet column without opening Excel. A summary shows how many lines were removed, and a table lists which lines were repeated and how often.
By default matching ignores letter case and surrounding spaces, so "Apple" and " apple" count as the same line. Turn case sensitivity on or trimming off when exact matching matters, for example with passwords, hashes or code. Everything runs in your browser; your text is not uploaded.
How to use the remove duplicate lines
- 1Paste your list or text into the box, one item per line.
- 2Decide whether "Apple" and "apple" should count as the same line and set case sensitivity.
- 3Keep trimming on to ignore stray spaces at the start or end of lines.
- 4Choose whether to drop blank lines.
- 5Copy the unique lines and check the table to see which entries were repeated.
Formula and method
The text is split into lines at every line break (Windows \r\n, old Mac \r and Unix \n are all recognised). Each line is optionally trimmed of leading and trailing spaces, and blank lines are optionally discarded. A comparison key is built from each line — lower-cased unless case-sensitive matching is on.
Lines are read top to bottom and a line is kept only the first time its key is seen; later lines with the same key are counted as duplicates and dropped. This preserves the original order, unlike sorting-based deduplication, and runs in a single pass so even long lists process instantly.
Worked examples
Fruit list with mixed case and stray spaces
The 8 input lines include "Apple" (a case variant of apple), a second "banana", " cherry" with leading spaces and one blank line. With case-insensitive matching and trimming, 3 duplicates and 1 blank line are removed, leaving 4 unique lines in their original order.
Case-sensitive email list
With case-sensitive matching, "[email protected]" and "[email protected]" are treated as different lines, so only the exact repeat of [email protected] is removed: 5 lines become 4.
Keyword list with repeated terms
"seo tools" appears three times (once capitalised) and "keyword research" twice. Ignoring case, 3 of the 6 lines are duplicates, leaving 3 unique keywords.
Frequently asked questions
Does removing duplicates change the order of my list?+
No. The first occurrence of each line is kept in its original position and later repeats are dropped, so the order of your list is preserved. Use the alphabetical sorter if you also want it sorted.
How do I remove duplicates but keep different capitalisation?+
Turn on case-sensitive matching. "Apple", "apple" and "APPLE" will then be treated as three different lines and all kept, while exact repeats are still removed.
Why were lines that look identical not removed?+
They probably differ by invisible characters such as trailing spaces, tabs or non-breaking spaces. Keep "Trim spaces" on to ignore leading and trailing whitespace; differences inside the line still count.
Can I remove duplicates from an Excel or Google Sheets column?+
Yes. Copy the column, paste it here (each cell becomes one line), remove the duplicates, then copy the result and paste it back into the sheet as a single column.
Is my text uploaded anywhere?+
No. Deduplication runs entirely in your browser using JavaScript, so lists of emails or customer data never leave your device.