Self Checks
Describe the problem
The "Delimiter for text" field on the dataset configuration page (and the matching "Delimiter for text" field on child chunks) accepts a single string in parser_config.delimiter / parser_config.children_delimiter, but the tooltip does not explain how that string is parsed into a list of delimiters. Users have to discover by trial-and-error that backticks are the meta-delimiter.
The current English tooltip (web/src/locales/en.ts, knowledgeDetails.delimiterTip / knowledgeDetails.childrenDelimiterTip) reads:
A delimiter or separator can consist of one or multiple special characters. If it is multiple characters, ensure they are enclosed in backticks(``). For example, if you configure your delimiters like this: \n##;, then your texts will be separated at line breaks, double hash symbols (##), and semicolons.
Problems with this wording:
- It never names backticks as the separator of delimiters — the user has to infer it from the example.
- The example is asymmetric:
\n has no backticks, ## does, ; doesn't. A reader can easily read that as a typo.
- The example does not make clear that characters outside backticks are each treated as their own single-character delimiter (so
!?; is three delimiters, not one three-character delimiter).
- There is no link to the backend parser, which makes the rule feel arbitrary.
This has led to repeated user confusion — see e.g. #7436 ("Multiple delimiters for Delimiter for text"), #4704 ("Delimiter setting not working"), and several related threads.
Proposed solution
Documentation-only change: rewrite knowledgeDetails.delimiterTip and knowledgeDetails.childrenDelimiterTip (and the parallel strings in other locales) so that they explicitly state the parsing rule. The rule, which is implemented in:
rag/nlp/__init__.py — get_delimiters()
deepdoc/parser/txt_parser.py — parser_txt()
deepdoc/parser/markdown_parser.py — MarkdownElementExtractor.get_delimiters()
is:
The field is a string. Backticks (`) group characters into a single multi-character delimiter. Any character outside backticks is itself a delimiter. Multi-character delimiters must be wrapped in backticks; single-character delimiters do not need to be. Resulting delimiters are matched longest-first.
Example: \n`##`; parses to three delimiters — newline (\n), double hash (##), semicolon (;) — and your text will be split at any of them.
A shorter second example covering only single chars (!?; → !, ?, ;) would also help.
Scope
Files to change
web/src/locales/en.ts — knowledgeDetails.delimiterTip, knowledgeDetails.childrenDelimiterTip
web/src/locales/<other>.ts — the matching keys in other locales, where the same ambiguity exists
Acceptance criteria
- The tooltip names backticks as the delimiter-of-delimiters.
- The tooltip states the single-character vs. multi-character rule.
- The example clearly shows which characters are single-char delimiters and which are backtick-wrapped multi-char delimiters.
childrenDelimiterTip is updated in lockstep with delimiterTip.
Self Checks
Describe the problem
The "Delimiter for text" field on the dataset configuration page (and the matching "Delimiter for text" field on child chunks) accepts a single string in
parser_config.delimiter/parser_config.children_delimiter, but the tooltip does not explain how that string is parsed into a list of delimiters. Users have to discover by trial-and-error that backticks are the meta-delimiter.The current English tooltip (
web/src/locales/en.ts,knowledgeDetails.delimiterTip/knowledgeDetails.childrenDelimiterTip) reads:Problems with this wording:
\nhas no backticks,##does,;doesn't. A reader can easily read that as a typo.!?;is three delimiters, not one three-character delimiter).This has led to repeated user confusion — see e.g. #7436 ("Multiple delimiters for Delimiter for text"), #4704 ("Delimiter setting not working"), and several related threads.
Proposed solution
Documentation-only change: rewrite
knowledgeDetails.delimiterTipandknowledgeDetails.childrenDelimiterTip(and the parallel strings in other locales) so that they explicitly state the parsing rule. The rule, which is implemented in:rag/nlp/__init__.py—get_delimiters()deepdoc/parser/txt_parser.py—parser_txt()deepdoc/parser/markdown_parser.py—MarkdownElementExtractor.get_delimiters()is:
A shorter second example covering only single chars (
!?;→!,?,;) would also help.Scope
Files to change
web/src/locales/en.ts—knowledgeDetails.delimiterTip,knowledgeDetails.childrenDelimiterTipweb/src/locales/<other>.ts— the matching keys in other locales, where the same ambiguity existsAcceptance criteria
childrenDelimiterTipis updated in lockstep withdelimiterTip.