Document Workflow
Choose and verify a binary-to-text encoding for data handoffs
Translate a receiver's vague encoding request into an explicit charset, alphabet, padding, byte-order, and container contract, then verify it without losing the original evidence.
Written and tested by SimpleWebUtilsPublished: Reviewed:
How this workflow was checked
The review loaded the unmodified “Verify RFC 4648 Base32 for a UTF-8 text field” sample into Encoding Workbench. We exercised the path from “Write the receiver contract before converting” to “Test the actual receiver in both directions”, checked the displayed or downloaded result against the example, and separately examined “Choosing from the shortest output” and “Assuming every Base62 or Base85 is identical”.
UTF-8 “café!” encoded to the canonical padded Base32 vector MNQWNQ5JEE====== and decoded back to the same bytes under the stated RFC 4648 contract.
Problem
An instruction such as “send it as Base62,” “escape it for XML,” or “convert it to Unicode” leaves out the decisions that determine the bytes. The source may be text or arbitrary binary, the text needs a charset, Base62 needs an alphabet and leading-zero rule, UTF-16 needs byte order, and entities depend on an output context. A locally reversible result can still fail in production when the receiver follows a different contract. This workflow makes each assumption visible and tests the real boundary before the original is discarded.
When to use this
- An API or data field names Base32, Base58, Base62, Base85, binary, or hexadecimal without supplying a complete example and variant.
- A text import involves unknown UTF-8, Latin-1, Windows-1252, UTF-16, BOM, or byte-order assumptions.
- HTML or XML text is being escaped and the destination context, named-entity policy, or sanitizer boundary is unclear.
- An IDN, quoted-printable email body, Unicode escape sequence, or UUEncode block works locally but fails in another parser.
- A small image must become Base64 or a data URI and the receiver has size, MIME, CSP, or container requirements.
Steps
- Step 1
Write the receiver contract before converting
Record whether the input is text or arbitrary bytes; for text, record its charset. Add the exact representation name, alphabet or named standard, padding policy, line wrapping, byte order, BOM policy, filename metadata, MIME wrapper, and maximum accepted size. Ask for one authoritative input-output vector rather than inferring a variant from a short label.
- Step 2
Preserve the source and remove shareable secrets
Keep an untouched private source and a checksum or length where appropriate. Make a sanitized fixture for screenshots and collaboration, because local processing prevents an upload by this tool but cannot make copied output confidential. Do not transcode an unknown file as text merely because a text box accepts pasted characters.
- Step 3
Choose the matching workbench mode and options
Open Encoding Workbench and select the named representation. Match Bitcoin Base58, the documented 0-9/A-Z/a-z Base62 alphabet, Adobe ASCII85 or ZeroMQ Z85, RFC 4648 Base32 padding, UTF-16 byte order, HTML entity style, XML quote policy, or UUEncode filename and octal permission to the written contract. Use Charset Detector first when the source charset is genuinely unknown.
- Step 4
Run a small known vector and inspect every notice
Start with the receiver's vector or a fixture containing ASCII, a non-ASCII character, a leading zero byte when relevant, and a boundary such as padding or a line break. Confirm input bytes, output bytes, output characters, normalization notices, and the round-trip result. A notice about non-standard Base62, omitted padding, line endings, Latin-1, or BOM is part of the contract, not decoration.
- Step 5
Test the actual receiver in both directions
Send only the sanitized fixture to a controlled instance of the real API, parser, library, DNS validator, email stack, or browser context. Decode its output locally and ask it to decode the local output. Compare bytes, not only visible text, and include leading zeros, empty values, supplementary Unicode, line endings, padding, filenames, and maximum-size boundaries.
- Step 6
Document the vector and keep failure evidence
Store the exact standard or alphabet, options, source-byte description, expected result, receiver version, and one passing vector beside the integration. When results differ, preserve both raw strings and byte dumps before normalizing whitespace or retrying. Only replace the original source after production-equivalent tests prove the handoff is reversible where reversibility is required.
Example
Verify RFC 4648 Base32 for a UTF-8 text field
Input
Contract: UTF-8 text, RFC 4648 Base32 alphabet, uppercase, canonical padding. Fixture text: café!Output
MNQWNQ5JEE======; decoding with the same contract recovers café!, and the receiver must produce or accept the same padded vectorCommon mistakes
Choosing from the shortest output
Output length is only one property. The receiver's standard, allowed character set, checksum needs, copy channels, line rules, and existing libraries determine the correct representation.
Calling arbitrary bytes UTF-8 text
A strict text decoder can reject valid binary data. Preserve bytes and use a file or byte-oriented encoder when the payload is not defined as text.
Assuming every Base62 or Base85 is identical
Base62 alphabet order and leading-zero behavior vary. ASCII85 and Z85 have different alphabets, delimiters, and block requirements. Record the variant explicitly.
Using entity encoding as sanitization
HTML and XML escaping depends on text, attribute, URL, style, script, and parser contexts. Character conversion alone is not an allowlist sanitizer or XML security policy.
Stopping after a local round trip
A local inverse check tests one implementation with one option set. Cross-test the real receiver, its version, size limit, line handling, MIME wrapper, and error behavior.
FAQ
Should I choose Base64, Base32, Base58, or Base85?
Follow the receiver's documented standard first. Base64 is common and compact but uses +, /, and padding; Base32 uses a smaller case-insensitive-friendly alphabet; Bitcoin Base58 omits visually confusing characters; Base85 is denser but has more variant and block constraints. None encrypts the data.
Why does Base62 need an alphabet in the contract?
There is no single Base62 standard. Systems can order digits and letter cases differently and can treat leading zero bytes differently. The workbench uses 0-9, A-Z, then a-z and labels that choice as non-standard.
Does round-trip success prove that no information was lost?
It proves recovery under this implementation and the selected documented normalizations. Morse loses letter case, quoted-printable normalizes line endings for interchange, and another implementation may use different conventions. Compare the required semantic or byte identity explicitly.
Can I convert an unknown text file directly to Latin-1 or UTF-16?
Not reliably. Determine the source encoding from a trusted specification, BOM, metadata, or evidence first. Latin-1 and Windows-1252 differ, and UTF-16 needs byte order. Keep the original bytes while testing.
When is an image data URI appropriate?
Use it for a small, explicitly accepted fixture or embedded asset after checking receiver size and CSP rules. Base64 expands the data, complicates caching, and exposes all source bytes. A signature check does not scan image content or remove metadata.
Does the workbench send my source or encoded result anywhere?
No source transformation requires a network request. Work runs in a dedicated browser Worker. Operational analytics contain the selected mode, safe option names, timing, byte counts, and aggregate notice counts, not source text, image bytes, filenames, domains, or output.