Skip to content
Utilboxes

Why two texts look identical but a comparison says they differ

A diff flags lines that look the same. Trailing spaces, tabs, line endings, curly quotes and invisible characters are usually why. How to find and fix them.

Published

Tools for this job: Text Compare · Text Cleaner · Character Counter

When a comparison flags lines that look identical, the difference is in characters you cannot see: trailing spaces, tabs instead of spaces, different line endings, curly versus straight quotes, non-breaking spaces or zero-width characters. Ignore spacing in the comparison, or clean both texts the same way first, and the real differences stand out.

The invisible culprits

  • Trailing spaces. “Total: 40 ” and “Total: 40” differ by a space at the end of the line that no editor shows.
  • Tabs vs spaces. A tab and four spaces can look identical; they are different characters.
  • Line endings. Windows ends lines with two characters (carriage return and line feed); macOS and Linux use one. Text that travelled between systems can carry either.
  • Non-breaking spaces. Copying from web pages and Word often brings in non-breaking spaces, which look exactly like ordinary ones.
  • Zero-width characters. Zero-width spaces and joiners take up no room at all but still count as characters.
  • Look-alike punctuation. Curly quotes (“ ” ‘ ’) versus straight ones (" '), and hyphens versus en and em dashes (- – —).
  • Look-alike letters. A Cyrillic “а” or Greek “ο” looks like a Latin a or o. Rare in normal text, but it happens in copied data and is a known trick in phishing addresses.
  • Case. Easy to miss in long lines: “ID” vs “Id”.

How to see what is really different

  1. Paste both versions into Text Compare. Lines that were added or removed are marked with a plus or a minus.
  2. Turn on the option to ignore spacing and blank lines. If a flagged line disappears, the difference was whitespace.
  3. Turn on ignore case. If more lines disappear, those differed only in capital letters.
  4. For anything still flagged, paste each version of the line into the Character Counter. If the counts differ, or the byte count is higher than the character count for plain English text, there is a hidden or special character.

The comparison runs in your browser, so contracts and code can be compared without uploading them.

Clean both texts the same way

When the invisible differences do not matter — comparing two drafts of an article, say — normalise both before comparing. The Text Cleaner collapses extra spaces, straightens curly quotes and strips invisible characters. Apply exactly the same options to both versions, then compare again; only real wording changes remain.

When the differences do matter — code, configuration files, data, passwords, legal text where a character can change meaning — keep them visible and fix the source instead.

Comparing documents rather than plain text

Comparing two Word documents or PDFs by pasting their text adds its own noise: line breaks fall in different places when a paragraph reflows, page headers and numbers get mixed into the text, and hyphenated words may be split. Ignoring spacing hides most of that. For PDFs specifically, comparing two versions of a PDF explains how to deal with layout differences.

Why the hidden characters got there

  • Copying from PDFs, web pages, email and chat apps.
  • Word processors replacing quotes and dashes as you type.
  • Files edited on different operating systems or by different editors.
  • Spreadsheets exporting cells with trailing spaces.

When it causes real problems

  • Searches and lookups fail because “Smith ” with a trailing space is not “Smith”.
  • Duplicate removal misses duplicates for the same reason; see removing duplicate lines.
  • Passwords and codes pasted with a trailing space are rejected even though they look right.
  • Version-control diffs show whole files changed when only line endings changed.

For a full list of invisible characters and how to remove them, see finding and removing invisible characters.

Checklist

  • Compared with spacing ignored, then with case ignored?
  • Remaining differences checked character by character?
  • Both versions cleaned with identical settings, if hidden characters do not matter?

Tools used in this guide

More tools for this job

All guides