Bug description
The doc_parse markdown normalizer deletes real coordinate text inside multi-line HTML table cells. Its documented safety rule says content inside <table>...</table> must be preserved.
Reproduction
Current main: f2d2ec989c3b96dcc46a1f434e1032fd12c459d8. From the repository root, Python 3.12.13 (macOS arm64; no model/GPU required):
from inference.utils.output_utils import normalize_doc_parse_markdown
text = "<table>\n<tr>\n<td>\n(2,4),(3,3)\n</td>\n</tr>\n</table>"
actual, stats = normalize_doc_parse_markdown(text)
print(repr(actual), stats)
assert actual == text
Actual: the (2,4),(3,3) cell content is removed; stats contains U_coord_bare: 1.
Expected: preserve the table cell exactly, while still removing layout-coordinate suffixes outside tables.
Root cause and impact
apply_pattern_U skips individual lines containing table tags, but does not track whether a tag-free line is inside a table. This affects shared doc_parse postprocessing used by vLLM, transformers, and llama.cpp clients, and can silently corrupt recognized table data.
Scope
Track table boundaries across lines, including uppercase and nested table tags, without changing inference or the other normalization passes. Regression coverage will exercise the public normalizer. No model-quality claim is involved.
Bug description
The doc_parse markdown normalizer deletes real coordinate text inside multi-line HTML table cells. Its documented safety rule says content inside
<table>...</table>must be preserved.Reproduction
Current main:
f2d2ec989c3b96dcc46a1f434e1032fd12c459d8. From the repository root, Python 3.12.13 (macOS arm64; no model/GPU required):Actual: the
(2,4),(3,3)cell content is removed; stats containsU_coord_bare: 1.Expected: preserve the table cell exactly, while still removing layout-coordinate suffixes outside tables.
Root cause and impact
apply_pattern_Uskips individual lines containing table tags, but does not track whether a tag-free line is inside a table. This affects shared doc_parse postprocessing used by vLLM, transformers, and llama.cpp clients, and can silently corrupt recognized table data.Scope
Track table boundaries across lines, including uppercase and nested table tags, without changing inference or the other normalization passes. Regression coverage will exercise the public normalizer. No model-quality claim is involved.