Commit 260b62b
* fix(#29): an unclassified EBD outcome is not an acceptance
`_render_ebd_flowchart` derived both a node's label and its mermaid class from the
branch's result text, and the class fell through to `accept` whenever that text
was absent:
if result and "ablehnung" in result.lower(): ry:::reject
else: ry:::accept
So an outcome nobody classified was drawn green. That is not hypothetical: 77
nodes in the shipped dataset are in exactly that state today, and a regeneration
that leaves more result fields empty (Hochfrequenz/makorele#68 measured 1726 of
2214 branches) turns rejections green wholesale.
The classes now follow `ebd_clusters.cluster_to_kind`, i.e. the EBD PDF's own
`Cluster:` prefix, which is the only authority on what an answer code means —
rather than a substring test that this module kept for itself. Two new classes
make the distinction visible: `info` for an outcome that is neither, and
`unknown` (grey) for one the source did not classify.
Re-rendering every shipped process from output/yaml: reject 930 -> 930,
accept 77 -> unknown 77. No rejection loses its styling; the 77 false
acceptances stop claiming approval.
Written test-first: the four tests for the new behaviour fail on the previous
implementation, and the two guarding the old behaviour (a "Ablehnung" result
stays red, a "Zustimmung" result stays green) passed before and after. One more
asserts every `:::class` used has a `classDef`, since a missing one renders
unstyled and silently.
* fix(#29): classify an outcome from its cluster, not only its result text
Copilot's review point, and it turns the fix from "mark it unknown" into
"classify it correctly": `if_*_cluster` is the `Cluster:` prefix lifted out of
the EBD's own Hinweis cell, so it is the authority — and **91** branches in the
committed EBD data carry one while their result text is empty. Those are exactly
the outcomes that were being drawn as approvals.
`_outcome_kind` now consults the cluster first, the result text second (1405
branches have one and no cluster), and yields `unknown` only when neither is
present. `_outcome_label` does the same, so a branch the EBD did classify no
longer degrades to a bare answer code.
Effect on the shipped corpus, re-rendering all 194 output/yaml files:
before: reject 930 | accept 77 | bare "A0x" labels: 77
after : reject 1007 | bare "A0x" labels: 0
All 77 false approvals turn out to be rejections the cluster knew about, and
their labels read "A01: Ablehnung auf Kopfebene" instead of "A01". A regenerated
dataset, where the cluster is empty wherever the result is (makorele#68), gets
`unknown` — grey and honest — rather than green.
Test-first again: the three cluster tests fail on the previous commit, and the
result-text and both-absent cases are pinned so the fallback order cannot be
reordered silently.
* fix(#29): classify only from the cluster, and ratchet against the verified EBDs
The result text does not classify anything: it carries the same constant for
every code of an EBD, so E_0488's A01 reads "Ablehnung" while the source says
`Cluster: Zustimmung` — and that repo's own answer_codes.yaml agrees it is an
approval. `_outcome_kind` therefore uses the cluster only (resolved field first,
else the `Cluster:` prefix of the Hinweis via extract_cluster) and answers
`unknown` when there is none. Neither green nor red is asserted without evidence.
Measured against Hochfrequenz/machine-readable_entscheidungsbaumdiagramme, where
every EBD is verified from an independent source (ebdamame + rebdhuhn), over the
1407 answer codes shared with this pipeline's data:
the result text, by substring ("ablehnung") 822/1407 58.4 % <- shipped
cluster, falling back to the result text 1045/1437 72 %
cluster only, else unknown 1304/1407 92.7 % <- this
The fallback is what cost 20 points: it invents a rejection for codes the
verified data calls unknown. It also cut both ways before — 98 verified
approvals were drawn as rejections and 91 verified rejections as approvals.
100 % needs more than this module: part of the residue is data missing upstream
(makorele#68) and part is Formatversion skew between the two corpora. So the
guarantee added here is that it cannot get worse — test_verified_ebd_agreement
ratchets on the match count (raise it, never lower it), and asserts separately
that no verified approval renders as a rejection and no verified rejection as an
approval. The fixture pins the verified data by permalink at commit 834fd748,
FV2604, so a reader of the test can look any case up.
* fix(#29): pin the ratchet exactly and correct every measured claim
A review of this PR found three false numbers of mine and a hole in the ratchet.
The hole: MINIMUM_AGREEMENT had 6 codes of slack, and the reviewer constructed a
mutation that used it — dropping `erfolgreich` from the cluster vocabulary greys
out 4 verified approvals and still scored 1300 >= 1298, with the whole suite
green, because the direction tests only catch approval<->rejection crossings and
not a kind draining into `unknown`. Now pinned at the measured 1304, plus per-kind
floors (approval 92, rejection 822, info 116). That mutation now fails two tests.
The false numbers, all re-derived from the committed fixture:
* "98 verified approvals were drawn as rejections" -> 92.
* "91 verified rejections were drawn as approvals" -> 0; the old rule never did
that. The 91 were *unclassified* outcomes drawn as approvals, a different
error. The test guarding that crossing stays, but as a guard rather than a
historical claim.
* the docstring statistics were from an earlier vintage: 1437 shared codes,
829/1045/1328, "386 invented rejections" -> 1407 codes, 822/1030/1304
(58.4/73.2/92.7 %), 372 invented rejections.
Also noted and now stated: the 92.7 % that answer_codes.yaml also scores is not
independent corroboration — it uses the same vocabulary.
---------
Co-authored-by: hf-kklein <konstantin.klein+claude@hochfrequenz.de>
1 parent f063572 commit 260b62b
4 files changed
Lines changed: 12994 additions & 10 deletions
File tree
- src/makoralle/serialization
- unittests
- fixtures
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
| 10 | + | |
10 | 11 | | |
11 | 12 | | |
12 | 13 | | |
| |||
37 | 38 | | |
38 | 39 | | |
39 | 40 | | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
40 | 110 | | |
41 | 111 | | |
42 | 112 | | |
| |||
46 | 116 | | |
47 | 117 | | |
48 | 118 | | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
49 | 124 | | |
50 | 125 | | |
51 | 126 | | |
| |||
59 | 134 | | |
60 | 135 | | |
61 | 136 | | |
62 | | - | |
| 137 | + | |
| 138 | + | |
63 | 139 | | |
64 | | - | |
65 | | - | |
66 | | - | |
67 | | - | |
| 140 | + | |
68 | 141 | | |
69 | 142 | | |
70 | 143 | | |
71 | 144 | | |
72 | 145 | | |
73 | 146 | | |
74 | 147 | | |
75 | | - | |
| 148 | + | |
| 149 | + | |
76 | 150 | | |
77 | | - | |
78 | | - | |
79 | | - | |
80 | | - | |
| 151 | + | |
81 | 152 | | |
82 | 153 | | |
83 | 154 | | |
| |||
0 commit comments