Text Tools

Remove Duplicates

All tools

Paste a list with repeats and get back each line once, in its original order.

How it works

Four simple steps

1

Paste the list

One item per line: emails, URLs, keywords or log entries.

2

Pick matching rules

Toggle Ignore case and Trim spaces to decide what counts as a repeat.

3

Duplicates drop out

The first occurrence stays, and the status shows lines before and after.

4

Copy or download

Copy the unique lines or save them as unique-lines.txt.

Remove Duplicates

About Remove Duplicates

You merged three exports of newsletter signups and the same addresses appear two or three times, some with stray spaces and different capitalization. Paste the combined list here, tick Ignore case, and copy a clean set of unique lines.

Each line is compared with the ones before it, and only the first occurrence is kept. Trim spaces is on by default, so " apple" and "apple " count as the same line and the output is trimmed. Ignore case treats "Cherry" and "cherry" as duplicates while keeping the spelling of the first one seen. Sort result orders the unique lines alphabetically with natural number ordering, so "item 2" comes before "item 10". The status line shows how many lines went in and how many came out.

Leave Preserve order switched on unless you have a reason not to. With it off, the tool keeps the text of the last duplicate it saw, and lines that are pure numbers jump to the top of the list. Blank lines are treated as lines too, so several empty rows shrink to a single one rather than disappearing.

To get rid of empty rows entirely, use the Filter non-empty action in Line Tools. For more sorting control, such as descending order, open Sort Lines. Deduplication runs in the page, so customer lists are not copied to a server.

Keyboard shortcut

Ctrl Enter - Run tool
FAQ

Frequently asked questions

See all FAQs

Answers for this tool. For site-wide help, open the FAQ hub.

With Preserve order on (the default), the first occurrence is kept in its original position and later repeats are removed.

No. It compares whole lines, so a line only counts as a duplicate when the entire line matches another one after trimming and any case folding.

Blank lines are compared like any other line, so after the first empty line the rest count as duplicates. Use Line Tools to remove the remaining one if needed.

View more FAQs

Related tools

View all

Uppercase

Convert text to UPPERCASE.

Open

Lowercase

Convert text to lowercase.

Open

Sentence Case

Capitalize first letter of sentences.

Open

Title Case

Capitalize each word.

Open

Explore 321 free tools

Everything you need as a developer, marketer or creator - in one beautiful place.

Browse all tools