Skip to content

Commit 8c462dc

Browse files
Krishcalinclaude
andcommitted
Fix the source count in two chapters, and catch that class in both editions
The architecture guide states 139 logical sources in its summary and in chapter 10, and 135 in two sentences of chapter 12 — including "you supplied 41 of 135 sources", in the passage that teaches a reader how to judge whether an export is thorough enough to trust. That is the worst place on the page for the denominator to be wrong. Its own figure test passed throughout, because it asks only whether each number appears SOMEWHERE on the page. 139 did appear, correctly, four chapters earlier. The summary-sentence test that was added after the last drift covers exactly one sentence. Neither could see this. test_no_stale_denominator_survives_anywhere_in_the_prose now checks every number that qualifies the word "sources", anywhere in the document. A partial is free — "41 of 139" is a legitimate illustration — but the total is not. It runs against BOTH editions separately. Written the way every other test in the file is written, taking the `page` fixture, it passed with the Markdown edition mutated: the same false pass that let the figure drift, reproduced inside the test written to stop it. The two files are hand-maintained side by side and either can rot alone, so each is now its own parametrised case, and each was confirmed to fail on its own mutation. Left alone deliberately: EXPORT_GUIDE.md's per-collector counts (14 sources for sapcontrol, 16 for icf). Verifying them needs a live SAP host, and a number changed without measuring it is the thing this commit is about. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent bf0fef8 commit 8c462dc

3 files changed

Lines changed: 47 additions & 4 deletions

File tree

docs/ARCHITECTURE.html

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1230,7 +1230,7 @@ <h4>The job</h4>
12301230

12311231
<h4>The unusual decision: absence is normal</h4>
12321232
<p>In most software, asking for something that is not there is an error. Here it is not. If <code>ral_config.csv</code> was never supplied, the slot called <code>ral_config</code> exists and is empty, and every check that needs it quietly stands aside.</p>
1233-
<p>This is the only sensible behaviour when nobody ever supplies all 135 sources — but it creates a serious danger, which the next chapter is entirely about. A system that skips silently produces a short, clean report and looks like good news.</p>
1233+
<p>This is the only sensible behaviour when nobody ever supplies all 139 sources — but it creates a serious danger, which the next chapter is entirely about. A system that skips silently produces a short, clean report and looks like good news.</p>
12341234

12351235
<div class="call crim">
12361236
<span class="k">The trap, stated once so you recognise it later</span>
@@ -1273,7 +1273,7 @@ <h4>The failure mode</h4>
12731273
<span class="plbl">With a coverage statement</span>
12741274
<span class="ph">“12 findings. 2 critical.”</span>
12751275
<ul>
1276-
<li>“You supplied 41 of 135 sources.”</li>
1276+
<li>“You supplied 41 of 139 sources.”</li>
12771277
<li>“21 areas were not assessed.”</li>
12781278
<li>“1 source cannot be obtained in RISE at all.”</li>
12791279
</ul>

docs/ARCHITECTURE.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -695,7 +695,7 @@ The DataLoader holds a table matching each expected filename to a named slot. Wh
695695

696696
In most software, asking for something that is not there is an error. Here it is not. If `ral_config.csv` was never supplied, the slot called `ral_config` exists and is empty, and every check that needs it quietly stands aside.
697697

698-
This is the only sensible behaviour when nobody ever supplies all 135 sources — but it creates a serious danger, which the next chapter is entirely about. A system that skips silently produces a short, clean report and looks like good news.
698+
This is the only sensible behaviour when nobody ever supplies all 139 sources — but it creates a serious danger, which the next chapter is entirely about. A system that skips silently produces a short, clean report and looks like good news.
699699

700700
> **The trap, stated once so you recognise it later**
701701
>
@@ -733,7 +733,7 @@ Nothing in that report is a lie. It is nonetheless the most dangerous document s
733733
- In truth 21 of 33 subject areas were never examined, because the files they need were not supplied. Every sentence is accurate. The document is still false.
734734

735735
**With a coverage statement** — “12 findings. 2 critical.”
736-
- “You supplied 41 of 135 sources.”
736+
- “You supplied 41 of 139 sources.”
737737
- “21 areas were not assessed.”
738738
- “1 source cannot be obtained in RISE at all.”
739739
- The reader concludes: we have looked at a third of the building.

tests/test_architecture_doc.py

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -181,6 +181,49 @@ def test_the_summary_sentence_attaches_each_figure_to_its_own_label(page):
181181
% ("%d %s" % (value, label)))
182182

183183

184+
def _source_totals_stated_in(text):
185+
"""Every number that qualifies the word "sources", as a total.
186+
187+
`41 of 139 sources` yields 139 — the partial is free, the denominator is
188+
not."""
189+
return [(m.group(0), int(m.group(2) or m.group(1)))
190+
for m in re.finditer(
191+
r"(\d+)(?:\s+of\s+(\d+))?\s+(?:logical\s+)?sources\b", text)]
192+
193+
194+
@pytest.mark.parametrize("edition", ["html", "markdown"])
195+
def test_no_stale_denominator_survives_anywhere_in_the_prose(edition):
196+
"""PRESENCE IS NOT ENOUGH, part two — the same hole, one paragraph over.
197+
198+
The test above pins the closing summary and the one above that only asks
199+
whether each figure appears SOMEWHERE. Between them, two sentences in the
200+
body went on saying "nobody ever supplies all 135 sources" and "you supplied
201+
41 of 135 sources" for as long as 139 appeared once, correctly, elsewhere on
202+
the page. Both passed. A reader meets the wrong denominator in the chapter
203+
that teaches them how to judge coverage, which is the worst place on the page
204+
for it to be wrong — the number is the one they use to decide whether an
205+
export is thorough enough to trust.
206+
207+
So every number that qualifies the word "sources" is checked, not just the
208+
one in the summary. `41 of 139` is fine: the partial is free, the total is
209+
not.
210+
211+
BOTH EDITIONS, SEPARATELY. Almost every test in this file takes the `page`
212+
fixture, which is the HTML. Written that way, this one passed with the
213+
Markdown edition mutated — the same false pass that let the figure drift in
214+
the first place, reproduced in the test written to stop it. The two files are
215+
maintained by hand side by side; either can rot alone.
216+
"""
217+
text = (DOC if edition == "html" else MD).read_text(encoding="utf-8")
218+
total = _derived()["logical sources"]
219+
bad = [phrase for phrase, stated in _source_totals_stated_in(text)
220+
if stated != total]
221+
assert not bad, (
222+
"the %s edition states a source total that is not %d: %s. A partial "
223+
"(\"41 of %d\") is fine; the denominator is not."
224+
% (edition, total, bad, total))
225+
226+
184227
def test_the_guide_does_not_hedge_a_count_it_can_state_exactly(page):
185228
""""~620 checks" was true only while the runtime families were unenumerated.
186229
They are enumerable now, so the tilde would be false modesty — and a hedged

0 commit comments

Comments
 (0)