Clear and concise description of the problem
As a developer using AIRI with Japanese TTS, I want assistant messages to retain natural Japanese orthography in chat while allowing the speech pipeline to receive an explicit pronunciation form, so that ambiguous kanji, names, and domain-specific terms are pronounced correctly.
Today the same streamed literal is used for both the visible assistant message and TTS. Japanese TTS backends may infer the wrong reading for kanji or proper nouns. Writing the whole response in kana avoids the pronunciation error but makes the chat transcript less natural and less readable.
Suggested solution / Ideas
Introduce separate display and speech projections for assistant text. One possible input convention is silent ruby metadata:
Raw model output: ｜約束《やくそく》は｜約束《やくそく》です。
Display text: 約束は約束です。
Speech text: やくそくはやくそくです。
The exact syntax is flexible; the important contract is that:
- display text preserves the original kanji;
- TTS receives the explicit reading;
- pronunciation metadata is never displayed, persisted as conversational content, or pronounced;
- parsing remains safe across arbitrary streaming chunk boundaries;
- replay/read-aloud uses the resolved speech text or cached audio;
- unannotated text continues to work unchanged.
A general displayText / speechText projection in the core-agent-to-speech boundary may be more reusable than implementing this in individual TTS providers.
Alternative
- Ask the model to output all Japanese in kana: corrects some readings but degrades transcript readability.
- Perform automatic G2P in each TTS adapter: adds backend-specific behavior and still struggles with names and fictional/domain terminology.
- Make a second LLM call to generate kana: increases latency, cost, and synchronization complexity.
Additional context
This is especially useful for Japanese, but the display/speech split could also support abbreviations, symbols, mathematical notation, and pronunciation overrides in other languages.
Validations completed:
- Followed the Code of Conduct and read the contributing guidance.
- Searched existing open and closed issues for TTS pronunciation, ruby/furigana, and separate display/speech text; no duplicate was found.
Clear and concise description of the problem
As a developer using AIRI with Japanese TTS, I want assistant messages to retain natural Japanese orthography in chat while allowing the speech pipeline to receive an explicit pronunciation form, so that ambiguous kanji, names, and domain-specific terms are pronounced correctly.
Today the same streamed literal is used for both the visible assistant message and TTS. Japanese TTS backends may infer the wrong reading for kanji or proper nouns. Writing the whole response in kana avoids the pronunciation error but makes the chat transcript less natural and less readable.
Suggested solution / Ideas
Introduce separate display and speech projections for assistant text. One possible input convention is silent ruby metadata:
The exact syntax is flexible; the important contract is that:
A general
displayText/speechTextprojection in the core-agent-to-speech boundary may be more reusable than implementing this in individual TTS providers.Alternative
Additional context
This is especially useful for Japanese, but the display/speech split could also support abbreviations, symbols, mathematical notation, and pronunciation overrides in other languages.
Validations completed: