Cleaning a Word file without breaking its layout
A .docx file is not text with some formatting bolted on.
Open one in an archive tool instead of Word and you will find a zip
containing a folder of XML files: document.xml holds the body text,
separate parts hold headers, footers, comments, and footnotes, and a
relationships file ties them together.
Cleaning the words in that document means walking into that structure,
finding every part that holds visible text, fixing it in place, and leaving
everything else, meaning styles, images, and layout, exactly as it was.
That second half is the part that is easy to get wrong.
Fixing the text is the easy 90 percent
Once you have located a run of visible text inside document.xml, applying
the same rules a plain-text cleaner already applies is straightforward: walk
the string, find invisible characters, curly quotes, stray em dashes, and
fix them the same way, character for character, with the same offsets
approach that keeps highlight() and clean() reading from a single pass
over the text.
Nothing about being inside a <w:t> element changes what an invisible
character is or when a dash should turn into a hyphen instead of a spaced
dash.
The part that is genuinely hard: runs
Word does not store a paragraph as one clean string.
It stores a sequence of "runs", each wrapped in its own <w:r> element with
its own formatting, and it splits a sentence across as many runs as its
internal editing history happens to produce.
This is normal, documented Word behavior, not a corrupted file: changing the
case of a word, applying bold to half a sentence, or even just editing text
over time can leave a single sentence looking like this in the underlying
XML:
<w:r><w:t>The plan</w:t></w:r>
<w:r><w:t>—which no</w:t></w:r>
<w:r><w:t>body read—</w:t></w:r>
<w:r><w:t>failed.</w:t></w:r>
Four runs, one sentence, and the em dashes sit at run boundaries in the middle of the text a reader sees as continuous. This is not a hand-picked worst case: it is the ordinary result of a document being edited over time, and it is the reason this project treats run-splitting as a real design problem rather than an edge case to hand-wave past.
This matters because dash resolution in this project's engine is
context-sensitive by design.
An en dash between two names with no spaces, like Bose and Einstein
joined directly by the dash character, becomes a tight hyphen.
An em dash used as a sentence interrupter, like the plan example above,
becomes a spaced dash instead.
Deciding which is which means looking at the character immediately before
and after the dash.
Concatenating the four runs above back into one string reproduces the
sentence correctly: The plan—which nobody read—failed.
But a per-run pass never does that concatenation.
It resolves each run's dash using only the text inside that one run, and
that is where it goes wrong: run three, body read—, ends with a dash and
nothing after it within that run, so a per-run pass resolves it as if it
sat at the very end of the text, producing -- instead of a spaced
interrupter, because failed. lives in the next run and is invisible to
this pass.
Run two, —which no, starts with a dash and nothing before it within that
run, so a per-run pass can resolve it as a line-initial list marker instead
of a sentence interrupter, because The plan lives in the previous run.
The same problem applies to the invisible-character logic. Deciding whether a zero-width joiner between two characters is meaningful punctuation in a joining script or leftover paste junk requires seeing both neighbours. Split those neighbours across a run boundary and the decision is being made on incomplete information, silently, with no error to signal that anything went wrong.
Why this is not shipped yet
The correct fix is to reconstruct the logical text of a paragraph across all of its runs, run the existing rules against that reconstructed text using the same offset-based approach already used for plain text, and then map each fix back to the specific run and position it came from, splitting or merging runs as needed without disturbing the formatting attached to them. That mapping step, going from an offset in reconstructed paragraph text back to a specific byte range inside a specific run, while preserving every run's own formatting properties, is a real design problem, not a small addition to the existing rule table. Getting it wrong in either direction, losing formatting or leaving dash and joiner decisions wrong at run boundaries, would make the tool worse than useless: worse, because it would look like it worked and quietly corrupt a subset of edits every time.
That is why .docx support is not part of this tool yet rather than shipped
half-working.
The plain-text pipeline already does the character-level work correctly.
What is missing is the layer above it that reassembles a Word document's
runs into something that pipeline can see whole, and ships the fix back out
without touching anything else in the file.