SimpleWebUtils
ENKO

Document Workflow

Convert CRLF to LF for Git and shell scripts safely

Diagnose original line-ending bytes, choose a repository policy, convert CRLF to LF without hiding encoding problems, and verify scripts and diffs before committing.

Written and tested by Published: Reviewed:

How this workflow was checked

This check paired the exact “Repair a Windows-edited deployment script” input with Line Ending Detector for CRLF, LF, and Mixed Text Files. We followed “Preserve the failing file and inspect its original bytes”, captured the result after “Choose the target from the execution and repository policy”, and compared it with the example while treating “Pasting the file into a textarea before detection” and “Removing CRLF but leaving a BOM before the shebang” as explicit boundary questions.

The repaired script changed three CRLF sequences to LF, retained four final line terminators, and restored the byte-zero shebang without altering command text.

Open Line Ending Detector for CRLF, LF, and Mixed Text Files

Problem

CRLF and LF are both valid newline conventions, but hidden changes between them can turn a small edit into a whole-file Git diff or leave a carriage return in a Linux shebang. Blind replacement is risky when the file is binary, uses UTF-16/32, contains a BOM, or is intentionally checked out as CRLF. Pasting into a browser textarea can erase the distinction before diagnosis, while core.autocrlf can make the working tree differ from the index. The repair therefore needs the original file bytes, separate encoding and BOM decisions, an explicit target chosen from the actual consumer, a reviewed .gitattributes policy, and a post-conversion run on the target platform.

When to use this

  • Git reports every line as changed after a small edit on another operating system.
  • A shell script fails with bad interpreter, command not found, or a visible ^M near the shebang.
  • A Docker entrypoint, CI script, Makefile fragment, or environment file works locally but fails on Linux.
  • A CSV, log, or generated text file contains both CRLF and LF record endings.
  • A repository is adopting .gitattributes and needs a measured before-and-after normalization.

Steps

  1. Step 1

    Preserve the failing file and inspect its original bytes

    Keep an untouched copy or use the exact failing Git revision. Run the line ending detector on the file itself, not pasted text, because textarea values normalize line breaks to LF. Record CRLF, LF, CR, transition lines, and the final terminator.

  2. Step 2

    Confirm that the input is text and review its encoding

    Stop if binary data is suspected. For UTF-16/32, require a BOM or review the detector's heuristic warning. If the first line has a BOM, treat BOM removal as a separate decision from line-ending conversion.

  3. Step 3

    Choose the target from the execution and repository policy

    Use LF for shell scripts, Docker entrypoints, and most cross-platform source repositories. Keep CRLF only when a Windows-specific consumer requires it. Inspect .gitattributes plus core.autocrlf and editor settings before deciding.

  4. Step 4

    Normalize the confirmed text in one controlled pass

    Use the text cleanup workbench's line-ending operation to convert the complete file to LF. Do not run a raw byte replacement on UTF-16/32 or a file that has not been confirmed as text.

  5. Step 5

    Check the first line and final newline separately

    For executable scripts, verify that the shebang starts at byte zero and contains no BOM. Confirm whether the project expects a final LF, then add it deliberately rather than assuming conversion created one.

  6. Step 6

    Re-run detection and inspect the Git diff

    The follow-up report should show LF only, no unexpected Unicode separators, and the intended final terminator. Use git diff --ignore-space-at-eol only as a diagnostic comparison; review the normal diff before committing.

  7. Step 7

    Enforce the policy and run the real consumer

    Add a scoped .gitattributes rule such as *.sh text eol=lf when appropriate, align editor settings, and execute the script or CI job on its target platform. Avoid a repository-wide renormalization in the same commit unless it was planned and reviewed.

Example

Repair a Windows-edited deployment script

Input

Original bytes begin: 23 21 2F 62 69 6E 2F 62 61 73 68 0D 0A
Detector: CRLF 3, LF 1, first mixed line 3
Linux: /bin/bash^M: bad interpreter

Output

Decision: repository policy requires LF for *.sh
After conversion: CRLF 0, LF 4, final terminator LF
Verification: shebang begins at byte 0, normal Git diff reviewed, script runs in Linux CI

Common mistakes

Pasting the file into a textarea before detection

The browser can normalize CRLF and CR to LF in the textarea API value, destroying the evidence you meant to inspect. Select the original file.

Running byte replacement on an unknown encoding

UTF-16 and UTF-32 represent CR and LF across multi-byte code units. Confirm the encoding and use a text-aware conversion path instead of replacing isolated 0D or 0A bytes.

Removing CRLF but leaving a BOM before the shebang

A script may still fail if UTF-8 BOM bytes precede #!. Verify byte zero separately after line-ending conversion.

Changing core.autocrlf globally as the only fix

A developer-wide setting can hide repository intent and behave differently across machines. Prefer a reviewed .gitattributes rule for project policy.

Mixing code changes with a repository-wide normalization

A huge mechanical diff hides real edits and complicates rollback. Isolate planned normalization and verify it with the target build or runtime.

FAQ

Should every Git repository store LF?

Most cross-platform source repositories use LF in the index, but generated files and Windows-specific consumers can have different requirements. Encode the chosen rule explicitly in .gitattributes.

Why does /bin/bash^M appear?

A CR from CRLF remains after the interpreter path in the shebang, so Linux looks for an interpreter whose path effectively includes that carriage return.

Does git diff --ignore-space-at-eol fix the file?

No. It only changes how a comparison is displayed. Normalize the intended text, review the ordinary diff, and test the resulting file.

Can I normalize UTF-16 files the same way as UTF-8?

Use a decoder and encoder that preserve the intended UTF-16 byte order and BOM policy. Do not replace individual newline bytes as though the file were UTF-8.

Should a text file end with a newline?

Many source and POSIX-oriented workflows expect one, but not every data format requires it. Check the repository and consumer contract, then verify the detector's final-terminator result.

When should I use core.autocrlf?

Treat it as a workstation checkout preference, not a substitute for repository policy. .gitattributes gives collaborators a shared, path-specific rule.