Fix Ukrainian translation quality across OncoWiki (render engine, Onco Wiki, site, KB content) - #631
Open
romeo111 wants to merge 13 commits into
Open
Fix Ukrainian translation quality across OncoWiki (render engine, Onco Wiki, site, KB content)#631romeo111 wants to merge 13 commits into
romeo111 wants to merge 13 commits into
Conversation
The render layer produces Ukrainian by default (EN is derived via post-hoc string substitution), so any English text baked into the default path leaked straight onto every treatment plan. Root-caused and fixed three independent classes of the bug: - _UI_STRINGS had ~34 entries whose "uk" value was itself raw English (e.g. monitoring/timeline/sources_cited); corrected them. - Several call sites (primary/alternative track headings, dt labels for Indication/Regimen/Reason/Supportive care, default badge, sources-cited heading, MDT talk-tree, evidence-lane labels, the no-plan-generated fallback page) hardcoded English literals that bypassed the _t()/_UI_STRINGS lookup entirely. - VariantActionabilityHit.evidence_summary had no channel at all for a translated companion to reach the render layer, so the ESCAT evidence-summary column was always English regardless of what the source YAML's evidence_summary_ua contained. Added the field to the schema, threaded it through _actionability.py, and made render.py prefer it when present. - MonitoringSchedule phase/window/checkpoint text has no _ua companion in the schema (a genuine content gap, not invented here); a prior _h_t() call was silently a no-op on this English text (and would have mistranslated it had a live client ever been wired up). Replaced with an honest plain-escape plus a render-layer lookup for the closed set of phase-name identifiers. - plan.py's track.selection_reason sentences were hardcoded English templates; rewritten to author Ukrainian by default with EN mirrors registered in _translation_overrides.py, matching the established pattern already used by mdt_orchestrator.py. Updated the three affected tests to assert the corrected (intended) Ukrainian-by-default output instead of the previously-leaked English. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Same bug class as the render engine, in a separate code path since
build_kb_wiki.py loads YAML directly rather than through the engine's
Pydantic models. The Ukrainian entity page for a biomarker/BMA/redflag
was always showing the English evidence_summary/definition/notes field
verbatim, silently ignoring the _ua companion sitting right next to it
in the same YAML file — confirmed across 463 BMA and 636 redflag pages.
Also fixed:
- Disease-search result subtitles ("fill 88% | verified 0% | 73
indications...") were hardcoded English on the Ukrainian search page;
now locale-aware with correct Ukrainian pluralization.
- A raw Python str(True)/str(False) was leaking as the literal word
"True" into the "Registered in Ukraine" table cell instead of
Так/Ні.
- A reversed word order in a "+N more" suffix ("... N ще" instead of
the natural "... ще N").
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
scripts/build_site.py: 15 spots where the site's own bilingual convention was inconsistently applied — an English literal survived in one branch while sibling strings in the same list/dict were already translated (diseases.html h1, try.html button/errors/dt, ask.html label, specs.html labels, about.html title/h1/principles, landing-page carousel eyebrows). scripts/site_cases.py: 286 of 295 case-gallery card titles had label_ua byte-identical to label_en — never actually translated, e.g. "NSCLC - ALK fusion - ESCAT IA / CIViC evidence" shown verbatim on the Ukrainian gallery. Translated the ordinary connector words (Relapsed/High Risk/Newly Diagnosed/Suspect/etc.) while preserving gene symbols, drug INNs, trial acronyms, and disease-abbreviation shorthand in Latin script per the project's stay_latin glossary convention. The 9 untranslated remainders are pure technical-code strings (disease abbreviation + regimen + trial name only) with no translatable content. docs/: full site rebuild reflecting these fixes plus the render-engine fixes (previous commit) and the in-progress KB content translation sweep (next commit). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Parallel review of _ua companion fields (notes_ua, description_ua, evidence_summary_ua, definition_ua, name_ua, and regimen watchpoint trigger_ua/action_ua) across diseases, biomarkers, biomarker_actionability, redflags, and most of regimens/drugs. Fixed, per file, whichever of the following applied: raw untranslated English sentences/phrases remaining in the UA text; garbled or mixed-script transliterations (e.g. Latin letters hidden inside otherwise-Cyrillic words); drug-name misspellings inconsistent with the corpus-standard transliteration used elsewhere for the same INN; grammar errors (case/number/gender agreement); dropped content where the UA text summarized away detail present in the English source; glossary-locked phrases replaced with non-standard synonyms; and missing ukrainian_review_status/ukrainian_drafted_by tracking fields. Left in Latin script throughout, per the project's terminology glossary (knowledge_base/translation/ua_terminology.yaml): gene/ biomarker symbols, drug INNs already given standard transliterations, trial/regimen acronyms, and classification-system codes — this is a direct response to the explicit instruction not to translate English terms that are normal usage in Ukrainian clinical practice. All edited entities keep ukrainian_review_status: pending_clinical_signoff (no agent promoted review status) and have draft_revision bumped, per the established Phase-1 translation-wave convention. This is a language-quality pass, not a clinical-content change. Coverage: diseases, biomarkers, biomarker_actionability, and redflags fully reviewed. regimens ~89%, drugs ~27%. indications, algorithms, procedures, and radiation_courses not yet covered — a Claude usage rate limit interrupted the review mid-run; the remainder will land in a follow-up commit. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
track.regimen_data.get("name") was used unconditionally for the
"Схема лікування" field, the sequencing-candidates table, the
timeline's induction-phase label, and the patient-mode regimen line
— none of them checked for the regimen's name_ua companion, so every
plan showed the English regimen name (e.g. "Sofosbuvir/Velpatasvir 12
weeks") even though the YAML had a Ukrainian name_ua right next to it.
Also translated the "N-day cycles × M" timeline caption, which was
hardcoded English regardless of target_lang.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Continuation of the parallel content-quality review: fully covers regimens and drugs, plus 349 of 664 indications (the review was interrupted twice by Claude usage limits — remaining indications, algorithms, procedures, and radiation_courses land in a follow-up commit). Same fix categories as the first wave: EN bleed, garbled/ mixed-script transliterations, drug-name misspellings, grammar agreement errors, dropped content, and glossary-locked-phrase substitutions, while preserving gene symbols/drug INNs/trial acronyms/ classification codes in Latin per the project glossary. Includes one per-agent glossary term proposal file (glossary_proposals_drugs43.jsonl) for maintainer review, following the established convention from the original translation wave. All edited entities keep ukrainian_review_status: pending_clinical_signoff with draft_revision bumped — a language-quality pass, not a clinical content change. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Reflects the regimen name_ua fix and the second wave of KB content translation quality fixes (previous two commits). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Completes full-corpus coverage of the parallel content-quality review: indications (663/664), algorithms, procedures, and radiation_courses — the entity types interrupted by usage limits in the previous two waves. Same fix categories as before (EN bleed, garbled transliterations, drug-name misspellings, grammar agreement, dropped content, glossary-locked-phrase substitutions), preserving gene symbols/drug INNs/trial acronyms/classification codes in Latin script. With this commit, every entity type in knowledge_base/hosted/content/ has been reviewed for Ukrainian translation quality: diseases, biomarkers, biomarker_actionability, redflags, regimens, drugs, indications, algorithms, procedures, and radiation_courses. knowledge_base/validation/ua_quality.py error count dropped from 684 to 372 over the three waves (EN_BLEED 351 -> 50), with the remainder concentrated in already-abbreviated notes_ua summaries that predate this review (a content-completeness characteristic, not something this pass introduced — confirmed by diffing individual files against their pre-edit state). All edited entities keep ukrainian_review_status: pending_clinical_signoff with draft_revision bumped. Language-quality pass only, no clinical content changes. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Reflects the completed full-corpus KB content translation review (previous commit) — indications, algorithms, procedures, and radiation_courses now render with reviewed Ukrainian text. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
BiomarkerClinicalContext had 5 values, but 15 prevention-persona screening/surveillance biomarker YAMLs (FIT, Pap cytology, dermoscopy, LungRADS, p16/p53 IHC, Ki-67, MGMT methylation, etc.) consistently used a richer 8-value taxonomy that was never ratified into the enum — the content was authored against an intended design the schema hadn't caught up to, not written incorrectly. Extended the enum with the 4 missing values (precursor_lesion, dysplasia_grading, hereditary_surveillance, diagnostic_workup) and documented them in KNOWLEDGE_SCHEMA_SPECIFICATION.md. Separately, 7 of those files also listed a bare "screening" alongside the valid "screening_surveillance" — always as a duplicate of the latter, never standalone. That token belongs to a different enum (IndicationIntent) entirely; dropped it as content-author error. Resolves all 15 pre-existing KB validator schema errors, which had cascaded into 59 test failures across test_loader, test_engine_nszu_smoke, test_legacy_regimen_normalization, test_clinical_signoff_smoke, test_engine, test_solid_tumor_2l_coverage, and test_workup_catalog — all now pass. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
MonitoringSchedule was the one entity type with zero schema support for a Ukrainian companion field, despite the render layer already displaying its window/checkpoint text on every treatment plan's Monitoring and Timeline sections. Added window_ua (str), visits_ua/checkpoints_ua (index-aligned lists), and notes_ua (both MonitoringPhase and MonitoringSchedule level) following the established `<field>_ua` convention used by every other entity type. Documented in KNOWLEDGE_SCHEMA_SPECIFICATION.md §12.2. Wired render.py's monitoring-phase table and timeline strip to prefer the `_ua` fields when target_lang == "uk" and populated, falling back to English otherwise (same pattern as Regimen.name_ua). Note for a future integration pass: claude/amazing-brattain-b7353b has an unmerged, independently-authored render.py change for this same feature (commit f035f3b) — worth reconciling when that branch lands, since it may diverge from this implementation. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Fills the window_ua/checkpoints_ua/visits_ua/notes_ua fields added in the previous commit across every MonitoringSchedule entity: - 8 regimen-linked schedules (mon_*): BR, VRd, CLL-BTKi, FL surveillance, HCL-cladribine, MF-systemic, R-CHOP, rituximab-mono. - 12 prevention/hereditary-surveillance schedules (ms_*): Barrett's esophagus, BRCA1/2 carrier breast + ovarian, cancer-survivor long-term follow-up, FAP colon, HCC surveillance, IBD colonoscopy, Li-Fraumeni, Lynch carrier CRC + endometrial, MEN2, VHL. 6 of the 12 ms_* schedules already had window_ua/checkpoints_ua from an earlier merged commit (5f9d528) predating formal schema support (tolerated via extra="allow") — that existing translated text was left untouched; only the genuinely missing visits_ua/notes_ua/tracking fields were added to those files. The remaining 14 files had no prior UA content and were translated in full. Preserves numbers, lab-value thresholds, and dosing exactly; keeps gene symbols, drug INNs, trial acronyms, and classification-system codes in Latin script per knowledge_base/translation/ua_terminology.yaml. All files carry ukrainian_review_status: pending_clinical_signoff. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Reflects the MonitoringSchedule window_ua/checkpoints_ua/visits_ua schema fields, render.py wiring, and full 20-file translation (previous two commits) plus the clinical_context enum fix. Verified: the Monitoring and Timeline sections on treatment-plan pages now show translated window/checkpoint text (e.g. auto-hcv_mzl.html) instead of raw English. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
User audit ("клацнув по сайту — переклад поганий, незрозумілий") found that Ukrainian-language pages across openonco.info were frequently showing raw English text. Root cause traced to three independent bug classes, fixed across the whole stack, plus a full-corpus content-quality pass and two follow-up schema fixes.
Render engine (
knowledge_base/engine/render.py,plan.py,_actionability.py,_translation_overrides.py,schemas/plan.py)_UI_STRINGShad ~34 entries whose"uk"value was itself raw English (e.g.monitoring/timeline/sources_cited).dtlabels for Indication/Regimen/Reason/Supportive care, MDT talk-tree, evidence-lane labels, the no-plan-generated fallback page) hardcoded English literals that bypassed the_t()lookup entirely.VariantActionabilityHit.evidence_summaryhad no schema channel for a translated companion to reach the render layer at all — addedevidence_summary_ua.track.selection_reasonsentences were hardcoded English templates inplan.py; rewritten to author Ukrainian by default with EN mirrors registered in_translation_overrides.py.nameinstead ofname_uain four places.Onco Wiki (
scripts/build_kb_wiki.py)_uacompanion field and showing English regardless of locale.str(True)/str(False)was leaking as the word "True" into the "Registered in Ukraine" column instead of "Так"/"Ні".Site chrome + examples (
scripts/build_site.py,scripts/site_cases.py)label_uabyte-identical tolabel_en— never translated (e.g. "NSCLC - ALK fusion - ESCAT IA / CIViC evidence" shown verbatim). Translated the ordinary connector words while preserving gene symbols/drug INNs/trial acronyms/disease-abbreviation shorthand in Latin script per the project'sstay_latinglossary convention.KB content-quality sweep (
knowledge_base/hosted/content/**)knowledge_base/translation/ua_terminology.yaml.ukrainian_review_status: pending_clinical_signoffwithdraft_revisionbumped — a language-quality pass, not a clinical content change.Two follow-up schema fixes (surfaced while investigating pre-existing test failures)
BiomarkerClinicalContextenum — 15 prevention-persona screening biomarker files consistently used a richer 8-value taxonomy than the ratified 5-value enum. Extended the enum with the 4 missing values (content was ahead of the schema, not wrong) and dropped a redundant duplicate token. Resolved all 15 pre-existing KB validator schema errors, which had cascaded into 59 test failures.MonitoringSchedule— the one entity type with zero schema support for a Ukrainian companion field despite the render layer displaying its content on every plan's Monitoring/Timeline sections. Addedwindow_ua/checkpoints_ua/visits_ua/notes_ua, wired the renderer to prefer them, and translated all 20 monitoring-schedule files.Test plan
test_tasktorrent_scripts_smoke.py, stale date assertions intest_ukraine_registration.py, a redflag coverage baseline) — verified viagit stashisolation and cross-checking untouched files.knowledge_base/validation/ua_quality.pyerror count: 684 → 372 over the KB content sweep (EN_BLEED 351 → 50).scripts/build_site.py) and spot-checked rendereddocs/ukr/pages after each wave of fixes.Notes for reviewers
claude/amazing-brattain-b7353b(unmerged) has an independently-authored render.py change for the same MonitoringSchedule UA feature (commitf035f3bd5d) — worth reconciling on next integration to avoid divergence.pending_clinical_signoffthroughout, per CHARTER §6.1 / §8.3 — needs Clinical Co-Lead review before any further promotion, same as prior translation waves..github/PULL_REQUEST_TEMPLATE.mdis scoped to a different workflow (benchmark cancer-report submissions) and doesn't apply to this engineering change, so it wasn't used verbatim.🤖 Generated with Claude Code