# Duplicate line remover

URL: https://shahid09.is-a.dev/tools/duplicate-line-remover/
Author: Muhammad Shahid (https://shahid09.is-a.dev/)
Free tool · runs in your browser

Paste a list and remove repeated lines in one step, keeping the original order.

**Quick answer:** Paste your list with one item on each line. The tool keeps the first copy of each line, removes the rest and preserves the original order, showing how many duplicates it removed. You can ignore capital letters, ignore spaces at line ends, keep the last copy instead, and remove blank lines.

## How to use it

1. **Paste your list.** Emails, names, keywords or any text with one item per line.
2. **Set the rules.** Decide whether capitals and end spaces make lines different, and which copy to keep.
3. **Copy the result.** Duplicates are removed as you type, with a count.

## Example

The second "apple" is removed and the order stays the same:

Input:

```
apple
banana
apple
cherry
```

Output:

```
apple
banana
cherry
```

## Common errors and how to fix them

- **Duplicates are still there**: The lines differ somehow, usually in capitals or hidden spaces. Turn on "Ignore upper and lower case", and clean the text with the [remove extra spaces tool](https://shahid09.is-a.dev/tools/remove-extra-spaces/) if needed.
- **Everything is on one line**: The list is separated by commas rather than line breaks. Replace the commas with line breaks first.

## Frequently asked questions

### Does it keep the original order?

Yes. Lines stay in their first-seen order. To sort them as well, use the [text sorter](https://shahid09.is-a.dev/tools/text-sorter/).

### Are "Apple" and "apple" duplicates?

Only if you turn on "Ignore upper and lower case". By default they are treated as different.

### What does "keep the last copy" do?

When a line repeats, the final copy stays in its position and the earlier ones go. This is useful for logs where the latest entry matters.

### Are blank lines removed?

Not unless you ask, so paragraph spacing stays intact. Turn on "Remove blank lines" to clear them.

### How do I remove duplicate words instead of lines?

Put each word on its own line first, for example by replacing spaces with line breaks in a text editor, then paste the list here.

### Can it clean an email list?

Yes. Turn on "Ignore upper and lower case", because email addresses are effectively not case-sensitive, and paste one address per line.

### How large a list can it handle?

Hundreds of thousands of lines on a typical computer. The limit is your device memory because nothing runs on a server.

### Is my list private?

Yes. It stays in your browser, so it is safe for customer lists and other private data.

### Can I see how many lines were repeated?

Yes. "Duplicates removed" and "Unique lines" update as you type, so you can compare them with the lines before.

### Why are lines with different capitals kept?

By default the match is exact. Turn on the option to ignore case, or standardise the text first with the [case converter](https://shahid09.is-a.dev/tools/case-converter/).

## Related

- [Text sorter](https://shahid09.is-a.dev/tools/text-sorter/)
- [Remove extra spaces](https://shahid09.is-a.dev/tools/remove-extra-spaces/)
- [Case converter](https://shahid09.is-a.dev/tools/case-converter/)

## Sources

- [MDN: Set](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Set)