Skip to content

fix(core): detect mixed line endings instead of flagging CRLF on a single match - #28983

Open
shoemoney wants to merge 2 commits into
google-gemini:mainfrom
shoemoney:fix/detect-line-ending-mixed-eol
Open

fix(core): detect mixed line endings instead of flagging CRLF on a single match#28983
shoemoney wants to merge 2 commits into
google-gemini:mainfrom
shoemoney:fix/detect-line-ending-mixed-eol

Conversation

@shoemoney

Copy link
Copy Markdown

Summary

detectLineEnding() in packages/core/src/utils/textUtils.ts classifies a
file as CRLF if it contains even one \r\n anywhere in the content, no
matter how many plain LF lines surround it:

export function detectLineEnding(content: string): '\r\n' | '\n' {
  return content.includes('\r\n') ? '\r\n' : '\n';
}

Both edit.ts and write-file.ts use that single verdict to decide whether
to rewrite every line ending in the whole file on write (edit.ts line
~937: const useCRLF = (!editData.isNewFile && editData.originalLineEnding === '\r\n') || ..., followed by finalContent.replace(/\r?\n/g, '\r\n')).

Concretely: a file with 99 LF lines and one stray CRLF line (a pasted
Windows snippet, or a partial line-ending conversion someone missed) gets
classified as a CRLF file. The next time an unrelated single-line edit
touches that file, every LF line is silently rewritten to CRLF - a 1-line
intended change turns into a 100-line diff, with nothing telling you it
happened.

Details

Made detection purity-based: '\r\n' is only returned when CRLF is the
only newline style present in the content.

export function detectLineEnding(content: string): '\r\n' | '\n' {
  const crlfCount = (content.match(/\r\n/g) || []).length;
  const totalNewlines = (content.match(/\n/g) || []).length;
  return totalNewlines > 0 && crlfCount === totalNewlines ? '\r\n' : '\n';
}

I did not touch edit.ts or write-file.ts. Both call useCRLF with no
else branch - when it's false the content is written through unmodified.
So '\n' is already the safe, non-destructive verdict; this change only
affects which files get classified that way. A file that's actually pure
CRLF still round-trips as CRLF (no regression to the existing behavior
that PR #16087 added).

This is a residual gap in the fix from #16087 / issue #9604, which added
detectLineEnding in the first place to stop line endings from being
"unpredictably converted." It didn't cover mixed-EOL files. I'm not
claiming to fix #9604 or #16087 - just closing a gap they left open.

The existing line-endings.test.ts only had pure-CRLF and pure-LF
fixtures, which is exactly why this passed CI. I added:

  • a mostly-LF file with one stray CRLF is now classified as LF
  • a pure-CRLF file still round-trips as CRLF (no regression)
  • an end-to-end test through EditTool: editing one line in a
    mostly-LF file that has a stray CRLF line no longer forces the whole
    file to CRLF

Related Issues

Related to #9604 (not fixing it directly - that issue covers the broader
"line endings unpredictably converted" problem; this closes one specific
gap in the detectLineEnding heuristic that #16087 introduced).

How to Validate

From packages/core:

npx vitest run src/tools/line-endings.test.ts

To see the bug reproduce, revert the change in textUtils.ts only (keep
the new tests) and rerun - two tests fail, and the failing edit-path test
shows the whole file getting rewritten to CRLF:

expected 'line1\r\nline2\r\nline3\r\nmodified\r\n' to be 'line1\nline2\nline3\nmodified\n'

I ran the fix through this cycle (tests red against unpatched code, green
against the fix, red again after reverting the fix) before opening this PR.

I also ran the broader adjacent suite to check for regressions:

npx vitest run src/utils/textUtils.test.ts src/tools/edit.test.ts src/tools/write-file.test.ts src/tools/line-endings.test.ts

All 169 tests pass. Also ran eslint and prettier --check on both
changed files, clean.

Pre-Merge Checklist

  • Updated relevant documentation and README (if needed)
  • Added/updated tests (if needed)
  • Noted breaking changes (if any)
  • Validated on required platforms/methods:
    • MacOS
      • npm run
      • npx
      • Docker
      • Podman
      • Seatbelt
    • Windows
      • npm run
      • npx
      • Docker
    • Linux
      • npm run
      • npx
      • Docker

…ngle match 🧵

detectLineEnding() returned '\r\n' if the content contained even one
CRLF sequence anywhere, no matter how many plain LF lines surrounded
it. edit.ts and write-file.ts both use that single verdict to decide
whether to rewrite every line ending in the whole file on write.

A file that is mostly LF but picked up one stray CRLF line (a pasted
Windows snippet, a partial conversion) would get entirely converted to
CRLF the next time any unrelated single-line edit touched it - a
1-line intended change silently becoming a file-wide diff.

Make detection purity-based: only report CRLF when every newline in
the content is part of a CRLF pair. Mixed content now reports '\n',
which is the safe branch - `if (useCRLF)` has no else in either
consumer, so when it's false the content is written through unchanged
rather than being force-normalized.

Added mixed-EOL coverage to line-endings.test.ts, which previously
only exercised pure-CRLF and pure-LF fixtures.
@shoemoney
shoemoney requested review from a team as code owners August 23, 2026 20:43
@github-actions github-actions Bot added the size/m A medium sized PR label Aug 23, 2026
@github-actions

github-actions Bot commented Aug 23, 2026

Copy link
Copy Markdown

📊 PR Size: size/M

  • Lines changed: 88
  • Additions: +84
  • Deletions: -4
  • Files changed: 2

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses an issue where the line ending detection logic was overly aggressive, causing files with mixed line endings to be incorrectly classified as CRLF. By refining the detection heuristic to require purity, the system now avoids unnecessary and destructive wholesale rewriting of line endings during minor file edits, improving the reliability of file modifications.

Highlights

  • Improved Line Ending Detection: Updated the detectLineEnding utility to use a purity-based heuristic, ensuring files are only classified as CRLF if they exclusively contain CRLF line endings.
  • Prevention of Unintended Line Ending Conversion: Prevented files with mixed line endings (e.g., mostly LF with a single stray CRLF) from being incorrectly identified as CRLF, which previously caused unrelated edits to silently rewrite the entire file's line endings.
  • Enhanced Test Coverage: Added new test cases to verify correct handling of mixed-EOL files and confirmed that pure CRLF files continue to be correctly identified.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the line ending detection logic in detectLineEnding to ensure that files with mixed line endings (e.g., mostly LF with a single stray CRLF) are classified as LF rather than CRLF, preventing unintended wholesale conversions during edits. It also adds comprehensive tests to verify this behavior. The reviewer suggested an excellent performance optimization to replace the double string scanning and array allocations with a single regex test using a negative lookbehind assertion.

Comment thread packages/core/src/utils/textUtils.ts Outdated
Comment on lines +103 to +105
const crlfCount = (content.match(/\r\n/g) || []).length;
const totalNewlines = (content.match(/\n/g) || []).length;
return totalNewlines > 0 && crlfCount === totalNewlines ? '\r\n' : '\n';

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Using content.match(/\r\n/g) and content.match(/\n/g) scans the entire string twice and allocates two arrays containing all matches. For large files, this can lead to significant memory overhead and garbage collection pressure.

We can optimize this to $O(N)$ time and $O(1)$ space with zero allocations by checking if the content contains \n and does not contain any \n that is not preceded by \r (using a negative lookbehind assertion (?<!\r)\n). Since Node.js >= 20 is used, lookbehind assertions are fully supported.

Suggested change
const crlfCount = (content.match(/\r\n/g) || []).length;
const totalNewlines = (content.match(/\n/g) || []).length;
return totalNewlines > 0 && crlfCount === totalNewlines ? '\r\n' : '\n';
return content.includes('\n') && !/(?<!\r)\n/.test(content) ? '\r\n' : '\n';

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, took the suggestion as written. Pushed in decad6b.

Checked equivalence with the counting version across the cases that matter: pure CRLF -> '\r\n', pure LF -> '\n', mixed (any lone \n) -> '\n', empty/no-newline -> '\n'. Also checked a lone-CR input (old-Mac \r with no \n at all) since the lookbehind reads differently from a naive glance at the counting code - that one falls through the includes('\n') check and returns '\n', same as before. Added a unit test for it.

Reran the same red/green/red cycle from the PR description against this change: the two regression tests in line-endings.test.ts still fail if you revert just textUtils.ts back to the pre-fix content.includes('\r\n') heuristic, and pass with either the counting version or this one. Full suite (textUtils.test.ts, edit.test.ts, write-file.test.ts, line-endings.test.ts) is 170/170 now (169 + the new lone-CR case). eslint/prettier clean.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 to bot comments

@gemini-cli gemini-cli Bot added priority/p2 Important but can be addressed in a future release. area/core Issues related to User Interface, OS Support, Core Functionality labels Aug 23, 2026
…nding ⚡

Replace the match()-based CRLF/LF counting in detectLineEnding with a
single includes() check plus a negative-lookbehind test, per review on
google-gemini#28983. Same purity semantics (CRLF only when every \n is part of a
CRLF pair), no allocations, single pass. Added a lone-CR unit case to
lock in that a bare \r with no \n at all still falls back to \n.
@gemini-cli

gemini-cli Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Hi there! Thank you for your interest in contributing to Gemini CLI.

To ensure we maintain high code quality and focus on our prioritized roadmap, we only guarantee review and consideration of pull requests for issues that are explicitly labeled as 'help wanted'.

This PR will be closed in 7 days if it remains without that designation. We encourage you to find and contribute to existing 'help wanted' issues in our backlog! Thank you for your understanding.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/core Issues related to User Interface, OS Support, Core Functionality priority/p2 Important but can be addressed in a future release. size/m A medium sized PR status/pr-nudge-sent

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: File line endings are unpredictably converted

2 participants