Sorting data whose order carries meaning
Logs, priority queues, and review trails often depend on chronology. Leave sorting off so the first occurrence order remains intact.
Remove Duplicate Lines | Keep First Occurrences
TextRuns in Your Browser (No Uploads)
Remove repeated lines from IDs, URLs, logs, and pasted lists while keeping the first original line. Choose exact case, trimmed comparison, or locale-aware natural sorting; bounded processing stays in a browser Worker.
What to do next
Continue with a related workflow or open the next tool that usually follows this task.
Workflow guide
Deduplicate lines from lists and logs without losing order
A practical workflow for turning repeated one-item-per-line data into an auditable unique list while preserving the first useful occurrence.
OpenRelated toolLine Ending Detector for CRLF, LF, and Mixed Text Files
Count original CRLF, LF, CR, and Unicode line separators in a local file.
OpenRelated toolRemove Empty Lines | Remove or Collapse Blank Rows
Remove every blank row or collapse repeated gaps, with exact counts and line-ending diagnostics.
OpenRelated toolRemove Line Numbers Without Damaging Code or Inline Values
Remove line-start numbering while preserving inline values, indentation, and exact line endings.
OpenHow to Use This Tool
Preserve the source before deduplicating when first-occurrence order, surrounding whitespace, or separator bytes are part of the contract.
Load one-item-per-line text from the editor or a local TXT, CSV, TSV, log, or Markdown file, staying within the visible byte and row caps.
Enable exact case only when case changes create distinct values.
Enable trimmed comparison only when surrounding whitespace should not distinguish rows.
Leave sorting off to preserve first-occurrence order, or enable it for a reviewable natural order.
Run the tool, compare source and unique counts, read any option or line-ending warning, then copy or download the result.
When to Use This Tool
Imported ID lists
Keep one copy of each identifier before a bulk admin action or spreadsheet lookup.
Log signal review
Collapse repeated log messages while retaining the first point at which each message appeared.
Configuration lists
Merge URL, hostname, allowlist, or denylist fragments and inspect exactly how many rows were removed.
Contact and label cleanup
Remove repeated email addresses or labels after deciding whether case and surrounding whitespace are meaningful.
Review-ready unique output
Produce a stable naturally sorted list for human review when the original order has no semantic value.
Common Mistakes
Ignoring case in machine identifiers
Identifiers may be case-sensitive even when human labels are not. Turn on exact case before deduplicating keys such as AbC1 and abc1.
Expecting trim comparison to rewrite output
Trimmed comparison changes only the comparison key. The tool keeps the first line exactly, including its surrounding spaces.
Assuming visually identical Unicode is identical
Composed é and decomposed e plus a combining mark remain separate because this tool does not apply Unicode normalization.
Skipping the line-ending result
Mixed LF, CRLF, and CR input is rewritten with the predominant source ending. Review the line-ending notice before a byte-sensitive import.
Examples
Keep the first imported ID
With case-insensitive comparison and sorting off, the first spelling and source order are retained.
Input
API-104
api-104
API-208
API-104Output
API-104
API-208Compare trimmed hostnames
Enable trimmed comparison to merge surrounding-space variants without modifying the kept first line.
Input
alpha.example.com
alpha.example.com
beta.example.comOutput
alpha.example.com
beta.example.comComparison, ordering, and line-ending contract
Line scanning preserves LF, CRLF, and CR separator evidence and treats a terminal separator as an ending marker rather than an extra value to deduplicate.
Each comparison key is stored in a Set-like map, giving average O(n) deduplication while retaining the first original line for each key.
Case-insensitive mode uses JavaScript lowercase conversion. Trimmed mode removes surrounding whitespace only from the comparison key; output content is not trimmed.
No Unicode normalization is applied. Code-point sequences that render alike can remain distinct unless their comparison keys are otherwise equal.
Natural sorting is applied only after deduplication through Intl.Collator for the active locale with numeric ordering; equal keys retain source order.
Inputs are capped at 1 MiB and 200,000 lines. Processing runs in a module Worker with a three-second deadline, and analytics records counts and options rather than source text.
Frequently Asked Questions
Does it preserve the first occurrence?
Yes. Sorting off preserves the first occurrence of every comparison key and the order in which those kept lines appeared.
How are case differences handled?
Comparison is case-insensitive by default. Enable exact case when Alpha and alpha must remain separate.
Does Ignore surrounding whitespace alter my text?
It trims only the temporary comparison key. The first kept line is returned byte-for-byte except when mixed line endings must be normalized.
Does it normalize Unicode before comparing?
No. NFC, NFD, NFKC, and NFKD are not applied, so canonically equivalent Unicode strings can remain distinct.
What happens to line endings?
For a uniform source, retained rows use its original LF, CRLF, or CR style and keep a final ending. Mixed sources choose the most common separator, with the earliest observed type resolving a tie.
How does Sort unique lines work?
Sorting runs after deduplication with the page locale, numeric ordering, and stable source order for equal sort keys. It is not a bytewise sort.
What are the processing and privacy limits?
A deduplication run is capped at 1,048,576 UTF-8 bytes and 200,000 source lines. It runs in a Worker with a three-second deadline and sends no line content in analytics.
How This Tool Was Verified
Maintained and tested by SimpleWebUtilsReviewed
Method: To check “Preserve first-seen issue IDs”, we used Remove Duplicate Lines with the guide's exact source data and applied “Choose the case rule”. The output had to match the documented result; evidence for “Using case-insensitive matching for opaque IDs” and “Expecting trim comparison to clean the kept row” was reviewed before recording the check.
Expected result: Case-insensitive comparison reduced six issue-ID rows to four in first-seen order and reported two removals without sorting the surviving values.
Open the tested workflowRelated workflow guides
Use these focused guides when you need a practical workflow before opening the tool.
Workflow guide
Deduplicate lines from lists and logs without losing order
A practical workflow for turning repeated one-item-per-line data into an auditable unique list while preserving the first useful occurrence.
Workflow guideSort unique lines naturally before an import
A review-first workflow for producing a unique, naturally ordered list of IDs, names, URLs, tags, or labels without confusing sorting with normalization.
Workflow guideRemove blank lines from pasted text without losing structure
A reviewable workflow for cleaning empty and whitespace-only rows while preserving the structure, order, and line-ending details that still matter.
Workflow guideClean lists, logs, and copied text with a repeatable pipeline
Text cleanup is safest when it is treated as a sequence of explicit, reversible transformations. Preserve the source, identify line and delimiter boundaries, make one structural change at a time, promote only reviewed output to the next step, and compare the final text with the destination contract before publishing, importing, or sharing it.
Related Tools
Continue with another maintained workflow
Line Ending Detector for CRLF, LF, and Mixed Text Files
Count original CRLF, LF, CR, and Unicode line separators in a local file.
Text
Remove Empty Lines | Remove or Collapse Blank Rows
Remove every blank row or collapse repeated gaps, with exact counts and line-ending diagnostics.
Text
Remove Line Numbers Without Damaging Code or Inline Values
Remove line-start numbering while preserving inline values, indentation, and exact line endings.
Text
Find and Replace Text | Literal and Regex Replacement
Replace literal or regex matches with Unicode boundaries, capture tokens, exact counts, and Worker limits.
Text
Word Counter | Words, Graphemes, Code Points and UTF-8 Bytes
Count word-like segments, graphemes, code points, UTF-16 units, UTF-8 bytes, lines and reading time locally.
Text