Skip to content

Fix Ukrainian translation quality across OncoWiki (render engine, Onco Wiki, site, KB content) - #631

Open
romeo111 wants to merge 13 commits into
masterfrom
claude/oncowiki-translation-review-d1ddb6
Open

Fix Ukrainian translation quality across OncoWiki (render engine, Onco Wiki, site, KB content)#631
romeo111 wants to merge 13 commits into
masterfrom
claude/oncowiki-translation-review-d1ddb6

Conversation

@romeo111

@romeo111 romeo111 commented Jul 9, 2026

Copy link
Copy Markdown
Owner

Summary

User audit ("клацнув по сайту — переклад поганий, незрозумілий") found that Ukrainian-language pages across openonco.info were frequently showing raw English text. Root cause traced to three independent bug classes, fixed across the whole stack, plus a full-corpus content-quality pass and two follow-up schema fixes.

Render engine (knowledge_base/engine/render.py, plan.py, _actionability.py, _translation_overrides.py, schemas/plan.py)

  • The render layer produces Ukrainian by default (English is derived via post-hoc string substitution), so any hardcoded English in the default path leaked onto every treatment plan. ~60 confirmed instances fixed:
    • _UI_STRINGS had ~34 entries whose "uk" value was itself raw English (e.g. monitoring/timeline/sources_cited).
    • Several call sites (primary/alternative track headings, dt labels for Indication/Regimen/Reason/Supportive care, MDT talk-tree, evidence-lane labels, the no-plan-generated fallback page) hardcoded English literals that bypassed the _t() lookup entirely.
    • VariantActionabilityHit.evidence_summary had no schema channel for a translated companion to reach the render layer at all — added evidence_summary_ua.
    • track.selection_reason sentences were hardcoded English templates in plan.py; rewritten to author Ukrainian by default with EN mirrors registered in _translation_overrides.py.
    • Regimen names showed the raw English name instead of name_ua in four places.

Onco Wiki (scripts/build_kb_wiki.py)

  • Same bug class in a separate code path (loads YAML directly, not through the engine): 463 biomarker-actionability pages and 636 red-flag pages were silently ignoring their _ua companion field and showing English regardless of locale.
  • A literal Python str(True)/str(False) was leaking as the word "True" into the "Registered in Ukraine" column instead of "Так"/"Ні".

Site chrome + examples (scripts/build_site.py, scripts/site_cases.py)

  • 15 spots where an English literal survived while sibling strings in the same list/dict were already translated (About page title/h1, try.html buttons/errors, specs.html labels, landing carousel).
  • 286 of 295 case-gallery card titles had label_ua byte-identical to label_en — never translated (e.g. "NSCLC - ALK fusion - ESCAT IA / CIViC evidence" shown verbatim). Translated the ordinary connector words while preserving gene symbols/drug INNs/trial acronyms/disease-abbreviation shorthand in Latin script per the project's stay_latin glossary convention.

KB content-quality sweep (knowledge_base/hosted/content/**)

  • Parallel review across every entity type (diseases, biomarkers, biomarker_actionability, redflags, regimens, drugs, indications, algorithms, procedures, radiation_courses): 2,982 files checked, 1,919 edited. Fixed EN bleed, garbled/mixed-script transliterations, drug-name misspellings, grammar agreement errors, dropped content, and glossary-locked-phrase substitutions — while preserving gene symbols, drug INNs, trial acronyms, and classification codes in Latin script per knowledge_base/translation/ua_terminology.yaml.
  • All edited entities keep ukrainian_review_status: pending_clinical_signoff with draft_revision bumped — a language-quality pass, not a clinical content change.

Two follow-up schema fixes (surfaced while investigating pre-existing test failures)

  1. BiomarkerClinicalContext enum — 15 prevention-persona screening biomarker files consistently used a richer 8-value taxonomy than the ratified 5-value enum. Extended the enum with the 4 missing values (content was ahead of the schema, not wrong) and dropped a redundant duplicate token. Resolved all 15 pre-existing KB validator schema errors, which had cascaded into 59 test failures.
  2. MonitoringSchedule — the one entity type with zero schema support for a Ukrainian companion field despite the render layer displaying its content on every plan's Monitoring/Timeline sections. Added window_ua/checkpoints_ua/visits_ua/notes_ua, wired the renderer to prefer them, and translated all 20 monitoring-schedule files.

Test plan

  • Full pytest suite: 2337 passed, 23 failed, 9 skipped, 0 errors (down from a confirmed pre-existing baseline of 40 failed / 85 errors before the schema fix). All 23 remaining failures independently confirmed pre-existing and unrelated to this change (bundle-size ceilings, a Windows bash path-separator issue in test_tasktorrent_scripts_smoke.py, stale date assertions in test_ukraine_registration.py, a redflag coverage baseline) — verified via git stash isolation and cross-checking untouched files.
  • knowledge_base/validation/ua_quality.py error count: 684 → 372 over the KB content sweep (EN_BLEED 351 → 50).
  • Site rebuilt (scripts/build_site.py) and spot-checked rendered docs/ukr/ pages after each wave of fixes.
  • KB loader/schema validation passes cleanly for all touched entity types.

Notes for reviewers

  • claude/amazing-brattain-b7353b (unmerged) has an independently-authored render.py change for the same MonitoringSchedule UA feature (commit f035f3bd5d) — worth reconciling on next integration to avoid divergence.
  • Translation content is machine-drafted and marked pending_clinical_signoff throughout, per CHARTER §6.1 / §8.3 — needs Clinical Co-Lead review before any further promotion, same as prior translation waves.
  • .github/PULL_REQUEST_TEMPLATE.md is scoped to a different workflow (benchmark cancer-report submissions) and doesn't apply to this engineering change, so it wasn't used verbatim.

🤖 Generated with Claude Code

romeo111 and others added 13 commits July 8, 2026 14:58
The render layer produces Ukrainian by default (EN is derived via
post-hoc string substitution), so any English text baked into the
default path leaked straight onto every treatment plan. Root-caused
and fixed three independent classes of the bug:

- _UI_STRINGS had ~34 entries whose "uk" value was itself raw English
  (e.g. monitoring/timeline/sources_cited); corrected them.
- Several call sites (primary/alternative track headings, dt labels
  for Indication/Regimen/Reason/Supportive care, default badge,
  sources-cited heading, MDT talk-tree, evidence-lane labels, the
  no-plan-generated fallback page) hardcoded English literals that
  bypassed the _t()/_UI_STRINGS lookup entirely.
- VariantActionabilityHit.evidence_summary had no channel at all for
  a translated companion to reach the render layer, so the ESCAT
  evidence-summary column was always English regardless of what the
  source YAML's evidence_summary_ua contained. Added the field to the
  schema, threaded it through _actionability.py, and made render.py
  prefer it when present.
- MonitoringSchedule phase/window/checkpoint text has no _ua
  companion in the schema (a genuine content gap, not invented here);
  a prior _h_t() call was silently a no-op on this English text (and
  would have mistranslated it had a live client ever been wired up).
  Replaced with an honest plain-escape plus a render-layer lookup for
  the closed set of phase-name identifiers.
- plan.py's track.selection_reason sentences were hardcoded English
  templates; rewritten to author Ukrainian by default with EN mirrors
  registered in _translation_overrides.py, matching the established
  pattern already used by mdt_orchestrator.py.

Updated the three affected tests to assert the corrected (intended)
Ukrainian-by-default output instead of the previously-leaked English.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Same bug class as the render engine, in a separate code path since
build_kb_wiki.py loads YAML directly rather than through the engine's
Pydantic models. The Ukrainian entity page for a biomarker/BMA/redflag
was always showing the English evidence_summary/definition/notes field
verbatim, silently ignoring the _ua companion sitting right next to it
in the same YAML file — confirmed across 463 BMA and 636 redflag pages.

Also fixed:
- Disease-search result subtitles ("fill 88% | verified 0% | 73
  indications...") were hardcoded English on the Ukrainian search page;
  now locale-aware with correct Ukrainian pluralization.
- A raw Python str(True)/str(False) was leaking as the literal word
  "True" into the "Registered in Ukraine" table cell instead of
  Так/Ні.
- A reversed word order in a "+N more" suffix ("... N ще" instead of
  the natural "... ще N").

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
scripts/build_site.py: 15 spots where the site's own bilingual
convention was inconsistently applied — an English literal survived
in one branch while sibling strings in the same list/dict were
already translated (diseases.html h1, try.html button/errors/dt,
ask.html label, specs.html labels, about.html title/h1/principles,
landing-page carousel eyebrows).

scripts/site_cases.py: 286 of 295 case-gallery card titles had
label_ua byte-identical to label_en — never actually translated,
e.g. "NSCLC - ALK fusion - ESCAT IA / CIViC evidence" shown verbatim
on the Ukrainian gallery. Translated the ordinary connector words
(Relapsed/High Risk/Newly Diagnosed/Suspect/etc.) while preserving
gene symbols, drug INNs, trial acronyms, and disease-abbreviation
shorthand in Latin script per the project's stay_latin glossary
convention. The 9 untranslated remainders are pure technical-code
strings (disease abbreviation + regimen + trial name only) with no
translatable content.

docs/: full site rebuild reflecting these fixes plus the render-engine
fixes (previous commit) and the in-progress KB content translation
sweep (next commit).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Parallel review of _ua companion fields (notes_ua, description_ua,
evidence_summary_ua, definition_ua, name_ua, and regimen watchpoint
trigger_ua/action_ua) across diseases, biomarkers,
biomarker_actionability, redflags, and most of regimens/drugs.

Fixed, per file, whichever of the following applied: raw untranslated
English sentences/phrases remaining in the UA text; garbled or
mixed-script transliterations (e.g. Latin letters hidden inside
otherwise-Cyrillic words); drug-name misspellings inconsistent with
the corpus-standard transliteration used elsewhere for the same INN;
grammar errors (case/number/gender agreement); dropped content where
the UA text summarized away detail present in the English source;
glossary-locked phrases replaced with non-standard synonyms; and
missing ukrainian_review_status/ukrainian_drafted_by tracking fields.

Left in Latin script throughout, per the project's terminology
glossary (knowledge_base/translation/ua_terminology.yaml): gene/
biomarker symbols, drug INNs already given standard transliterations,
trial/regimen acronyms, and classification-system codes — this is a
direct response to the explicit instruction not to translate English
terms that are normal usage in Ukrainian clinical practice.

All edited entities keep ukrainian_review_status: pending_clinical_signoff
(no agent promoted review status) and have draft_revision bumped,
per the established Phase-1 translation-wave convention. This is a
language-quality pass, not a clinical-content change.

Coverage: diseases, biomarkers, biomarker_actionability, and redflags
fully reviewed. regimens ~89%, drugs ~27%. indications, algorithms,
procedures, and radiation_courses not yet covered — a Claude usage
rate limit interrupted the review mid-run; the remainder will land in
a follow-up commit.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
track.regimen_data.get("name") was used unconditionally for the
"Схема лікування" field, the sequencing-candidates table, the
timeline's induction-phase label, and the patient-mode regimen line
— none of them checked for the regimen's name_ua companion, so every
plan showed the English regimen name (e.g. "Sofosbuvir/Velpatasvir 12
weeks") even though the YAML had a Ukrainian name_ua right next to it.
Also translated the "N-day cycles × M" timeline caption, which was
hardcoded English regardless of target_lang.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Continuation of the parallel content-quality review: fully covers
regimens and drugs, plus 349 of 664 indications (the review was
interrupted twice by Claude usage limits — remaining indications,
algorithms, procedures, and radiation_courses land in a follow-up
commit). Same fix categories as the first wave: EN bleed, garbled/
mixed-script transliterations, drug-name misspellings, grammar
agreement errors, dropped content, and glossary-locked-phrase
substitutions, while preserving gene symbols/drug INNs/trial acronyms/
classification codes in Latin per the project glossary.

Includes one per-agent glossary term proposal file
(glossary_proposals_drugs43.jsonl) for maintainer review, following
the established convention from the original translation wave.

All edited entities keep ukrainian_review_status: pending_clinical_signoff
with draft_revision bumped — a language-quality pass, not a clinical
content change.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Reflects the regimen name_ua fix and the second wave of KB content
translation quality fixes (previous two commits).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Completes full-corpus coverage of the parallel content-quality review:
indications (663/664), algorithms, procedures, and radiation_courses —
the entity types interrupted by usage limits in the previous two
waves. Same fix categories as before (EN bleed, garbled
transliterations, drug-name misspellings, grammar agreement, dropped
content, glossary-locked-phrase substitutions), preserving gene
symbols/drug INNs/trial acronyms/classification codes in Latin script.

With this commit, every entity type in knowledge_base/hosted/content/
has been reviewed for Ukrainian translation quality: diseases,
biomarkers, biomarker_actionability, redflags, regimens, drugs,
indications, algorithms, procedures, and radiation_courses.
knowledge_base/validation/ua_quality.py error count dropped from 684
to 372 over the three waves (EN_BLEED 351 -> 50), with the remainder
concentrated in already-abbreviated notes_ua summaries that predate
this review (a content-completeness characteristic, not something
this pass introduced — confirmed by diffing individual files against
their pre-edit state).

All edited entities keep ukrainian_review_status: pending_clinical_signoff
with draft_revision bumped. Language-quality pass only, no clinical
content changes.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Reflects the completed full-corpus KB content translation review
(previous commit) — indications, algorithms, procedures, and
radiation_courses now render with reviewed Ukrainian text.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
BiomarkerClinicalContext had 5 values, but 15 prevention-persona
screening/surveillance biomarker YAMLs (FIT, Pap cytology, dermoscopy,
LungRADS, p16/p53 IHC, Ki-67, MGMT methylation, etc.) consistently
used a richer 8-value taxonomy that was never ratified into the enum
— the content was authored against an intended design the schema
hadn't caught up to, not written incorrectly. Extended the enum with
the 4 missing values (precursor_lesion, dysplasia_grading,
hereditary_surveillance, diagnostic_workup) and documented them in
KNOWLEDGE_SCHEMA_SPECIFICATION.md.

Separately, 7 of those files also listed a bare "screening" alongside
the valid "screening_surveillance" — always as a duplicate of the
latter, never standalone. That token belongs to a different enum
(IndicationIntent) entirely; dropped it as content-author error.

Resolves all 15 pre-existing KB validator schema errors, which had
cascaded into 59 test failures across test_loader, test_engine_nszu_smoke,
test_legacy_regimen_normalization, test_clinical_signoff_smoke,
test_engine, test_solid_tumor_2l_coverage, and test_workup_catalog —
all now pass.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
MonitoringSchedule was the one entity type with zero schema support
for a Ukrainian companion field, despite the render layer already
displaying its window/checkpoint text on every treatment plan's
Monitoring and Timeline sections. Added window_ua (str),
visits_ua/checkpoints_ua (index-aligned lists), and notes_ua (both
MonitoringPhase and MonitoringSchedule level) following the
established `<field>_ua` convention used by every other entity type.
Documented in KNOWLEDGE_SCHEMA_SPECIFICATION.md §12.2.

Wired render.py's monitoring-phase table and timeline strip to prefer
the `_ua` fields when target_lang == "uk" and populated, falling back
to English otherwise (same pattern as Regimen.name_ua).

Note for a future integration pass: claude/amazing-brattain-b7353b
has an unmerged, independently-authored render.py change for this
same feature (commit f035f3b) — worth reconciling when that branch
lands, since it may diverge from this implementation.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Fills the window_ua/checkpoints_ua/visits_ua/notes_ua fields added in
the previous commit across every MonitoringSchedule entity:

- 8 regimen-linked schedules (mon_*): BR, VRd, CLL-BTKi, FL surveillance,
  HCL-cladribine, MF-systemic, R-CHOP, rituximab-mono.
- 12 prevention/hereditary-surveillance schedules (ms_*): Barrett's
  esophagus, BRCA1/2 carrier breast + ovarian, cancer-survivor
  long-term follow-up, FAP colon, HCC surveillance, IBD colonoscopy,
  Li-Fraumeni, Lynch carrier CRC + endometrial, MEN2, VHL.

6 of the 12 ms_* schedules already had window_ua/checkpoints_ua from
an earlier merged commit (5f9d528) predating formal schema support
(tolerated via extra="allow") — that existing translated text was
left untouched; only the genuinely missing visits_ua/notes_ua/tracking
fields were added to those files. The remaining 14 files had no prior
UA content and were translated in full.

Preserves numbers, lab-value thresholds, and dosing exactly; keeps
gene symbols, drug INNs, trial acronyms, and classification-system
codes in Latin script per knowledge_base/translation/ua_terminology.yaml.
All files carry ukrainian_review_status: pending_clinical_signoff.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Reflects the MonitoringSchedule window_ua/checkpoints_ua/visits_ua
schema fields, render.py wiring, and full 20-file translation
(previous two commits) plus the clinical_context enum fix. Verified:
the Monitoring and Timeline sections on treatment-plan pages now show
translated window/checkpoint text (e.g. auto-hcv_mzl.html) instead of
raw English.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant