fix: optimize stdout#2562
Open
StartE wants to merge 8 commits into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
问题背景
C++
InputContainerStdio处理 Docker JSON 格式容器日志时,Docker 运行时将超 16KB 的行拆成多条 partial log:JSON 多行模式创建 JsonLogFileReader,会对日志进行错误\0分割,导致后续完全错乱
修复的缺陷
缺陷 1: Partial Log 不标记
根因:
ParseDockerJsonLogLine只去掉末尾\n,没有利用"有无\n"这一信息设置 PartLogFlag。修复:
缺陷 2: JSON 多行模式创建 JsonLogFileReader
根因:
CreateInnerProcessors中SetRequiringJsonReaderFlag(true)导致使用JsonLogFileReader,其RemoveLastIncompleteLog在容器格式剥离前做{}配对,破坏 CRI/Docker 头部。修复:移除
SetRequiringJsonReaderFlag(true),改为 Processor 层 MergeByJson 处理。缺陷 3:
DockerJsonFileParser无条件 fullLine=true根因:
parseLine不检测 log 字段有无\n;checkContainerType给 DockerJsonFileParser 分配 buffer=0。修复:
parseLine:fullLine = hasNewline(有\n为 Full,无为 Partial)checkContainerType: buffer 分配LogFileReader::BUFFER_SIZEGetLastLine: 添加 Phase 2 P 行合并逻辑 +mergeLines方法新功能: MergeByJson(Phase 4)
在容器格式剥离 + P/F 合并之后,通过
{}大括号配对将多行 JSON 日志合并为一条事件。Pipeline 变化:
核心实现 — 大括号状态机:
测试覆盖(14 个新增测试方法)
Phase 1: Docker JSON Partial Log 标记与合并【processor层】
TestDockerJsonPartialLogBasicTestDockerJsonPartialLogWithSplitTestDockerJsonPartialLogWithSplitAndRegexPhase 3: DockerJsonFileParser P/F 行支持【fileReade层】
TestLastDockerJsonFileSingleLineTestLastDockerJsonFileMergePhase 2: JSON 多行模式修复【reader + processor】
TestCreateInnerProcessorsJsonMultilineNoJsonReaderPhase 4: MergeByJson 大括号状态机
TestMergeJsonInitTestMergeJsonSingleLineBlockTestMergeJsonMultiLineBlockTestMergeJsonMultipleBlocksTestMergeJsonBraceInString{}不影响计数TestMergeJsonEscapedQuote\"不影响引号状态TestMergeJsonOversizedTestMergeJsonIncompleteAtEnd