Calling ASCII-compatible bytes a confirmed encoding
Printable bytes below 80 hex are valid ASCII, UTF-8, and many legacy encodings. The original charset cannot be recovered from those bytes alone.
Detect Text File Encoding from Byte Evidence
EncodingRuns in Your Browser (No Uploads)
Inspect a file's exact bytes before import. Confirm UTF-8, UTF-16, or UTF-32 from BOM signatures, validate strict UTF-8 structure, identify ASCII-compatible bytes, estimate BOM-less Unicode from NUL placement, and label ambiguous legacy text honestly.
What to do next
Continue with a related workflow or open the next tool that usually follows this task.
Workflow guide
Remove a BOM from a CSV or text file without re-encoding it
Use this workflow when an invisible leading signature changes the first CSV header or script token. Work from the original bytes, distinguish UTF-32 LE from its UTF-16 LE prefix, and do not remove UTF-16 or UTF-32 byte-order evidence until the destination is known.
OpenRelated toolRemove BOM from UTF-8, UTF-16, and UTF-32 Files
Remove one supported leading BOM from exact file bytes without re-encoding the remaining content.
OpenRelated toolCSV to JSON and JSON to CSV Converter | Local and Loss-Aware
Convert CSV and JSON locally with text-safe typing, strict structure checks, and protected spreadsheet output.
OpenRelated toolLine Ending Detector for CRLF, LF, and Mixed Text Files
Count original CRLF, LF, CR, and Unicode line separators in a local file.
OpenHow to Use This Tool
Keep the untouched source file; opening and saving it in an editor may rewrite the bytes you need to inspect.
Select one file up to 25 MiB, or load the built-in UTF-16LE BOM CSV sample.
Run the inspection manually so choosing a file does not silently classify it.
Read the classification together with evidence strength: confirmed, strong, compatible, heuristic, or undetermined.
Check the leading hex bytes, strict UTF-8 result, Unicode structure, and NUL or control-byte counts.
Configure a small destination import with the candidate encoding and verify known text before converting the full source.
When to Use This Tool
Choose a CSV import encoding
Inspect the untouched export before a spreadsheet, database, or ETL process changes headers or multilingual fields.
Audit text files before conversion
Verify whether source, config, subtitle, or translation files contain strict UTF-8 before a repository migration.
Review a BOM-related parser failure
Record exact BOM and structure evidence before deciding whether the destination requires the signature to stay or be removed.
Triage an unknown attachment
Separate a plausible legacy text stream from likely binary data before trying controlled candidate decoders.
Diagnose mojibake from original bytes
Use known names, headers, and delimiters to verify a decoder after the byte report narrows the possibilities.
Common Mistakes
Assuming a BOM proves the whole file is valid
A BOM identifies an intended Unicode encoding scheme, but the remaining stream can still have an odd length, an unpaired surrogate, or an invalid scalar value.
Naming a legacy charset from one high byte
Windows-1252, ISO-8859 variants, Shift_JIS, and other legacy encodings can overlap. Invalid UTF-8 is not enough evidence to name one of them.
Treating a NUL-pattern estimate as confirmation
NUL placement is useful for ASCII-heavy UTF-16 or UTF-32, but CJK-heavy or short BOM-less files may not produce a distinctive lane pattern.
Saving garbled text before inspecting the source
Mojibake is already-decoded text. Inspect the untouched source bytes first; resaving the broken preview can destroy the evidence needed to choose the correct decoder.
Examples
Validate multilingual UTF-8 bytes
The non-ASCII byte sequences are well-formed under the strict UTF-8 ranges and no BOM is present.
Input
EC 84 9C EC 9A B8 2C 5A C3 BC 72 69 63 68Output
UTF-8 | strong evidence | no BOMDistinguish UTF-32LE from UTF-16LE
The four-byte signature must be checked before its FF FE UTF-16LE prefix.
Input
FF FE 00 00 69 00 00 00 64 00 00 00Output
UTF-32 LE | confirmed BOM | valid structureHow evidence-based charset inspection works
The detector recognizes EF BB BF, FE FF, FF FE, 00 00 FE FF, and FF FE 00 00 at byte zero. UTF-32LE is tested before UTF-16LE because its four-byte signature begins with the same FF FE prefix.
Strict UTF-8 validation follows the well-formed sequence ranges: C0 and C1 overlong leads are rejected, E0 and F0 lower bounds are enforced, ED surrogate encodings are rejected, and F4 cannot exceed U+10FFFF.
A BOM result is signature evidence, not blanket validation. The remaining UTF-16 stream must have complete two-byte units and paired surrogates; UTF-32 units must be complete Unicode scalar values outside D800-DFFF and at most 10FFFF.
Without a BOM, the detector considers UTF-32 before UTF-16 and requires both a repeated endian-specific NUL-lane pattern and valid code-unit structure. Short or CJK-heavy streams may remain undetermined because bytes alone do not provide enough evidence.
Printable ASCII-range data is reported as ASCII-compatible because the same bytes are valid UTF-8 and overlap many legacy encodings. Invalid UTF-8 with few control bytes becomes an undetermined legacy or single-byte candidate, not a named charset.
NUL bytes and suspicious C0 control bytes that do not form a credible Unicode lane pattern raise a binary warning. The 26,214,400-byte limit bounds browser memory and processing; normal inspection remains local and analytics excludes names and content.
Frequently Asked Questions
Is the selected file uploaded?
No. The selected file is read as an ArrayBuffer in your browser. Aggregate analytics may record byte count, result class, evidence type, BOM kind, and warning count, but not the file name or file bytes.
Does an ASCII-compatible result prove the file was saved as ASCII?
No. A valid ASCII-only stream is also valid UTF-8 and can be valid in many single-byte encodings. The result deliberately says ASCII-compatible rather than inventing an origin charset.
What does strict UTF-8 validation check?
It rejects unexpected continuation bytes, truncated sequences, overlong forms, UTF-8 encodings of surrogate code points, and values above U+10FFFF. A non-ASCII stream that passes gets strong UTF-8 evidence.
Which BOM signatures are recognized?
EF BB BF identifies UTF-8, FE FF UTF-16BE, FF FE UTF-16LE, 00 00 FE FF UTF-32BE, and FF FE 00 00 UTF-32LE. Four-byte signatures are tested first to avoid prefix collisions.
Can UTF-16 or UTF-32 be detected without a BOM?
Only heuristically. For sufficiently long ASCII-heavy data, repeated NUL positions can suggest LE or BE serialization and the matching UTF-16 or UTF-32 structure must also be valid. The result is never labelled confirmed.
Can it identify every legacy encoding?
No. The tool can show that bytes are not strict UTF-8 and look more like single-byte text than binary data, but it does not claim Windows-1252, ISO-8859-1, Shift_JIS, or another exact legacy encoding without external context.
Should I import immediately from this result?
No. Use the result to choose candidate decoders, then verify known headers, names, delimiters, and replacement characters in the destination. Keep the original file until that round-trip is confirmed.
How This Tool Was Verified
Maintained and tested by SimpleWebUtilsReviewed
Method: In Detect Text File Encoding from Byte Evidence, the “Distinguish a UTF-32LE export from its UTF-16LE prefix” fixture was run without repairing or simplifying its input. We verified the transition from “Inspect exact bytes manually” to “Keep ambiguous results ambiguous”, compared the final artifact or values, and reviewed “Treating ASCII-compatible as ASCII origin” plus “Accepting any continuation-shaped UTF-8” as non-success paths.
Expected result: The four-byte FF FE 00 00 signature and code-unit layout identified UTF-32LE; the detector did not stop at the shorter UTF-16LE prefix.
Sources and standards
- Encoding StandardWHATWG
Related workflow guides
Use these focused guides when you need a practical workflow before opening the tool.
Workflow guide
Remove a BOM from a CSV or text file without re-encoding it
Use this workflow when an invisible leading signature changes the first CSV header or script token. Work from the original bytes, distinguish UTF-32 LE from its UTF-16 LE prefix, and do not remove UTF-16 or UTF-32 byte-order evidence until the destination is known.
Workflow guideDetect a text file's encoding before import
Use the original file bytes to distinguish confirmed Unicode signatures from structural evidence and conservative heuristics. Then verify a candidate decoder against known headers and multilingual values before a full import.
Workflow guideRecover garbled text from the original file bytes
Character-set detection narrows decoder candidates; it does not reverse already-corrupted text by itself. Preserve the source, identify where the wrong decode happened, compare controlled previews, and convert once only after known values round-trip correctly.
Workflow guideConvert CSV to JSON for spreadsheet data cleanup
Use this workflow when a spreadsheet export needs to become API-ready JSON, a fixture, or a reviewable dataset while preserving the text that the sheet actually contains.
Workflow guideFix CSV and JSON conversion errors
Use this workflow when a spreadsheet export, API fixture, or CSV/JSON handoff fails because columns shift, headers disappear, characters look broken, or JSON will not parse.
Workflow guideChoose and verify a binary-to-text encoding for data handoffs
Use this workflow when an API, database field, DNS label, email body, configuration file, or legacy transport asks for Base32, Base58, Base62, Base85, hexadecimal, entities, Punycode, quoted-printable, UTF-16, or UUEncode but does not fully describe the variant.
Workflow guideConvert CRLF to LF for Git and shell scripts safely
Use this incident workflow when a Windows-edited script fails on Linux, Git shows a whole-file line-ending diff, or generated text mixes CRLF and LF. Inspect first, normalize only confirmed text, then enforce the decision with repository settings.
Workflow guideNormalize Unicode before comparing text across systems
A reliable Unicode-normalization workflow preserves the original representation, confirms the target system's contract, distinguishes canonical normalization from compatibility folding, inspects exact code points and hidden characters, compares the result, and validates the completed value where it will be stored or matched.
Related Tools
Continue with another maintained workflow
Remove BOM from UTF-8, UTF-16, and UTF-32 Files
Remove one supported leading BOM from exact file bytes without re-encoding the remaining content.
Text
CSV to JSON and JSON to CSV Converter | Local and Loss-Aware
Convert CSV and JSON locally with text-safe typing, strict structure checks, and protected spreadsheet output.
Converter
Line Ending Detector for CRLF, LF, and Mixed Text Files
Count original CRLF, LF, CR, and Unicode line separators in a local file.
Text
Unicode Normalizer | Compare NFC, NFD, NFKC and NFKD
Normalize Unicode with code-point evidence and explicit compatibility-loss warnings.
Encoding
Inspect and Remove Zero-Width Characters Safely
Detect six zero-width controls, inspect their positions, and remove only approved types.
Text