Remove Duplicate Lines
Remove duplicates from a list and keep each line once, in its original order. Ignore capitalization and extra spaces, keep the first or the last copy, sort the result, or keep only the lines that appear once. Every removed line is listed with its line number, so you can check what went.
- Keeps your order
- Shows what was removed
- Stays on your device
- Lines
- 0
- Duplicates
- 0
- Lines in the result
- 0
- Empty lines removed
- 0
How to remove duplicate lines
Delete duplicate lines from any list in three steps with this duplicate line remover.
-
Paste your list
One item per line: emails, names, keywords, URLs or anything else. You can also open a .txt or .csv file.
-
Choose what to keep
Keep One copy of each line, Only lines that appear once or Only the repeated lines. Tick Ignore capitalization if Apple and apple are the same to you.
-
Copy the unique lines
Copy or download the result. Open lines left out under the tool to see each removed line and its line number.
The sample list has 6 lines. Exact matching removes 2 duplicates (lines 3 and 6) and keeps apple, banana, cherry, Banana; with Ignore capitalization, Banana on line 5 goes too.
Duplicate line remover features
Remove duplicates the way you need, and see exactly what changed.
-
1→1
Original order
Lines stay in the order you pasted them; only the extra copies are removed.
-
Aa
Ignore case and spaces
Treat Apple and apple, or a line with a trailing space, as the same line.
-
#
Removed lines listed
Each removed line with its line number, so nothing disappears unnoticed.
-
3
Three ways to keep
One copy of each line, only the lines that appear once, or only the repeated lines.
Duplicate line removal example
The sample list *apple, banana, apple, cherry, Banana, apple* (one per line) with different options:
| Options | Result | Removed |
|---|---|---|
| One copy of each line | apple, banana, cherry, Banana | Lines 3 and 6 |
| One copy, ignore capitalization | apple, banana, cherry | Lines 3, 5 and 6 |
| One copy, ignore capitalization, keep the last one | cherry, Banana, apple | Lines 1, 2 and 3 |
| Only lines that appear once, ignore capitalization | cherry | Every copy of apple and banana |
| Only the repeated lines, ignore capitalization | apple, banana | cherry and the extra copies |
Empty lines are kept as they are unless you tick Remove empty lines; they are never counted as duplicates.
Remove duplicates without changing the order
Many tools sort a list before they remove duplicates, so your list comes back in a different order. This one keeps the order: the first copy of each line stays where it was and the later copies are removed. Choose The last one under Which copy stays to keep the most recent copy instead, which is useful for logs and lists where newer lines matter more.
If you do want the list sorted, choose A to Z or Z to A under Order. To sort words rather than lines, use the Word Sorter.
Ignore capitalization and extra spaces
Two lines only count as duplicates when they match exactly, so Apple and apple, or apple and apple␣ with a space at the end, are different lines by default.
- Tick Ignore capitalization to treat capital and small letters as the same. The copy that stays keeps its own capitals.
- Ignore spaces at the start and end of lines is on by default, so a stray space doesn’t stop a match. Spaces inside a line still count.
For a list of email addresses, turn on Ignore capitalization: Anna@Example.com and anna@example.com reach the same inbox.
Remove duplicate lines in Excel and Google Sheets
- Excel: select the column and choose Data › Data Tools › Remove Duplicates. Excel compares what is shown in each cell, so the same date formatted two ways counts as two values.
- Google Sheets: select the range and choose Data › Data cleanup › Remove duplicates. Sheets treats cells that differ only in capitals or formatting as duplicates.
Both change your sheet in place. To dedupe a list without touching the original, copy the column, remove duplicate lines online here, and paste the unique lines back.
Duplicate lines or duplicate words?
This tool compares whole lines. It won’t remove a word that is repeated inside a sentence, like the the. For that, use the Duplicate Word Finder, which highlights repeated words, or the Unique Word Counter to list every different word once.
Remove duplicate lines FAQ
Does it remove every copy of a repeated line?
No, one copy is kept by default. Choose Only lines that appear once to remove every copy of the lines that repeat.
Can I remove duplicates without sorting my list?
Yes. The original order is kept unless you choose A to Z or Z to A under Order.
Are uppercase and lowercase lines treated as duplicates?
Not by default. Tick Ignore capitalization to treat Apple and apple as the same line.
Do spaces affect matching?
Spaces at the start and end of lines are ignored by default. Spaces inside a line always count.
Can I see which lines were removed?
Yes. Open the lines left out list under the tool: every removed line is shown with its line number.
Which copy is kept?
The first one, unless you choose The last one under Which copy stays.
Does it remove repeated words inside a sentence?
No, it compares whole lines. Use the Duplicate Word Finder for repeated words.
Can I process a long list?
Yes. Lists of tens of thousands of lines are handled in your browser; very long lists take a moment on slower phones.
Is my list uploaded?
No. Nothing you paste leaves your device.
Sources and references
- Filter for unique values or remove duplicate values, Microsoft Support
- Split text, remove duplicates, or trim whitespace, Google Docs Editors Help
More text editing tools
- A–Z Word Sorter Alphabetize words or lists, or sort them by length or frequency. Open tool
- Count Line Counter Count total, blank and duplicate lines in text, lists, code or CSV. Open tool
- Repeats Duplicate Word Finder Find doubled words like “the the” and words repeated too close together. Open tool
- ␣ Remove Extra Spaces Remove double spaces, tabs, invisible spaces and extra blank lines. Open tool