Memoization: aggregate corpus evidence + thread-vs-memo isolation on typical words

Extends the corpus harness to report count-based and wall-clock aggregates
separately (a bimodal corpus otherwise hides the tradeoff behind one
scalar). Adds real measurements: a 70-word Sena slice shows 77% of words
faster under memo with a 2.42x aggregate wall-clock win (conservative,
since the one timed-out word is excluded); a repeated Indonesian
comparison isolates that the typical-word slowdown is mostly -- not
purely -- lost parallelism, since forcing the memo off entirely widens the
slowdown further. Also records the H2 shortfall as a plausible-but-
unconfirmed deep-clone-cost signal, not an established conclusion.

Author: johnml1135

memoization.md

@@ -327,6 +327,61 @@ words) are unmeasured at this depth — a natural follow-up, not a blocker,
 given the confirmed pattern (soundness holds, memo dominates the win) on
 the one word measured to completion.
 
+**Aggregate corpus evidence: count-based and wall-clock, reported
+separately (see the harness's own two-line report format).** A Sena
+sub-corpus (words 1-70, 200s per-side timeout — long enough for the one
+tractable heavy word in this range to complete on both sides, per §5's H1
+measurement above) gives, of the 69 words that completed within budget:
+**53/69 (77%) faster under memo, 16/69 (23%) slower** — count-based, this
+is the "yes, some words are slower" half of the picture. **Wall-clock:
+memo-on total 114.6s vs memo-off total 277.8s across those same 69
+words — 2.42× faster in aggregate** — this is the "but a lot are faster,
+and it nets out strongly positive" half. The one word that timed out at
+200s (a second tractable-but-slow heavy word, distinct from both H1 and
+H2) is excluded from both numbers; since it timed out even with the memo
+on, its memo-off side is presumably at least as slow, so **this 2.42×
+figure is a conservative lower bound**, not an inflated one — a longer
+timeout could only raise it. Both memo tables fired heavily and
+non-vacuously on this slice (mrule: tens of thousands of positive/nogood
+hits; template: tens of thousands positive, near-zero nogood).
+
+**Is the typical-word slowdown "just" losing a thread? Mostly, not
+purely.** Isolated on Indonesian (no template redundancy — the cleanest
+instrument for this, since the template memo never replays there) via
+three repeated runs per configuration to average out run-to-run JIT/GC
+noise (single-run absolute times varied by as much as 60% between
+otherwise-identical runs of the unchanged parallel baseline — only
+within-run memo-on-vs-memo-off ratios are trustworthy here, not
+across-run absolute milliseconds):
+
+| Configuration vs parallel-default | Aggregate ratio (3 reps) | Implied slowdown |
+|---|---|---|
+| sequential + memo (shipped opt-in) | 0.80×, 0.80×, 0.78× | ~21-26% |
+| sequential, memo forced off (mutation-tested) | 0.69×, 0.80×, 0.68× | ~25-45% |
+
+The sequential-*no-memo* slowdown is consistently at or above the
+sequential-*memo* slowdown, never below it. So most of the typical-word
+regression genuinely is the lost thread (both configurations are slower
+than parallel-default, by a similar order of magnitude) — but it is not
+*purely* that: even on a grammar where the template memo never once
+replays, the mrule-cascade's own cache still fires (64 positive / 157
+nogood hits on this corpus) and measurably claws back some of the
+thread-loss penalty rather than adding to it. Three reps each is enough to
+see the direction of the effect, not enough to state a precise recovered
+percentage with confidence — a larger repeated-trial run would sharpen
+this if the exact number ever matters for the default-flip decision.
+
+**Is the H2 shortfall actually the deep-clone tax?** Plausible, not
+confirmed — resist stating it as established. The one piece of direct
+evidence is that H2's memo fired heavily before timing out (tens of
+thousands of positive hits on both tables, §5 above) — the memo is
+clearly not idle on this word, which favors "replay/clone cost per hit is
+what's expensive" over "the memo doesn't engage." But that is a
+directional tell, not a measurement: distinguishing "expensive replay" from
+"just a lot more raw expansion than H1" requires profiling where memo-on
+wall time actually goes (`Clone`/`Freeze`/`ReplayOnto` vs raw rule
+expansion) on H2 specifically. Scoped as a follow-up, not undertaken here.
+
 ## 6. Expected outcome and honest caveats
 
 - Pathological template-heavy words: prototype showed ~5× (with COW clones). Master's
@@ -342,8 +397,11 @@ the one word measured to completion.
   word in 26.9s. If a future heavy-set measurement shows replay cost dominating after
   all, this is the evidence that would motivate porting `Shape` COW as its own
   follow-up — do not fold it into this PR pre-emptively.
-- Typical words (Indonesian-class) see little change — the memo pays off only where
-  order-variant redundancy exists. The corpus gate proves "no regression" there.
+- Typical words (Indonesian-class) get measurably *slower* under sequential+memo
+  (~21-26% in aggregate, §5 addendum) — mostly the cost of losing a thread, not the memo
+  itself (isolated via a sequential-no-memo comparison, same addendum). The corpus gate
+  proves "no divergence" there, not "no regression" — this is exactly why the memo ships
+  off by default rather than flipping the library's default cascade mode.
 - The memo is inert for `Ordered`-strata-only grammars and for `ParallelCombinationRuleCascade`
   (no natural subtree-completion moment in its breadth-first walk — same reason the
   prototype left it unmemoized).

tests/SIL.Machine.Morphology.HermitCrab.Tests/MemoCorpusVerification.cs

@@ -100,6 +100,38 @@ public void MemoOnMatchesMemoOff_AnalysisSetIdentical_OnRealCorpus()
         TestContext.Out.WriteLine($"words attempted: {words.Count}, timed out (>{timeoutMs}ms): {timedOut.Count}");
         TestContext.Out.WriteLine($"words with no parse on both sides: {noParseBoth}");
         TestContext.Out.WriteLine($"aggregate wall: {totalMs:F1} ms, p50: {p50:F1} ms, p95: {p95:F1} ms");
+
+        // Count-based vs wall-clock aggregates, reported SEPARATELY on purpose: a corpus is typically
+        // bimodal (many cheap words the memo makes slightly slower by losing a thread; a few pathological
+        // words the memo makes drastically faster) -- collapsing that into one ratio hides which regime a
+        // reader is in. "Most words are a bit slower" and "the corpus finishes much faster in total" are
+        // both true simultaneously and neither contradicts the other.
+        int fasterCount = perWordTimes.Count(x => x.OnMs < x.OffMs);
+        int slowerCount = perWordTimes.Count(x => x.OnMs > x.OffMs);
+        int tiedCount = perWordTimes.Count - fasterCount - slowerCount;
+        double totalOnMs = perWordTimes.Sum(x => x.OnMs);
+        double totalOffMs = perWordTimes.Sum(x => x.OffMs);
+        TestContext.Out.WriteLine(
+            $"count-based: {fasterCount}/{perWordTimes.Count} words faster under memo, "
+                + $"{slowerCount}/{perWordTimes.Count} slower, {tiedCount} tied"
+        );
+        TestContext.Out.WriteLine(
+            $"wall-clock: memo-on total {totalOnMs:F1} ms vs memo-off total {totalOffMs:F1} ms "
+                + $"({(totalOnMs > 0 ? totalOffMs / totalOnMs : 0):F2}x)"
+        );
+        if (timedOut.Count > 0)
+        {
+            // The wall-clock ratio above is a CONSERVATIVE lower bound on the memo's real aggregate
+            // benefit, not an inflated one: every excluded word timed out with memo ON (see the loop
+            // above -- the memo-on side is attempted first and a timeout on it skips memo-off entirely),
+            // so each excluded word is presumably at least as hard for memo-off, which would only pull
+            // totalOffMs (and thus the ratio) further up if it were included and able to finish at all.
+            TestContext.Out.WriteLine(
+                $"(the {timedOut.Count} timed-out word(s) above are excluded from both aggregates; "
+                    + "since each timed out even with the memo on, including them -- if memo-off could "
+                    + "finish them at all -- would only raise the wall-clock ratio further)"
+            );
+        }
         // Per-heavy-word attribution (memoization.md's own methodological rule: an aggregate can be
         // dominated by cheap words while hiding what pathological words actually do -- report both).
         // memo-on is sequential+memo; memo-off is the untouched parallel default, so this is the