feat: feed intelligence sources into analysis contexts - #1709
Conversation
github-actions
Bot
added
ai
documentation
🤖 自动审查报告
📁 修改的文件
🧠 AI 代码审查意见结论Ready to Merge 结构化审查结果必要性通过 关联性通过 类型判定类型： 描述完整性完整
风险级别中
必改项无。 建议项
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e1d3dcce84
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if url and url in seen_urls: | ||
| continue | ||
| seen_urls.add(url) | ||
| merged.append({ |
There was a problem hiding this comment.
When normal market-news search returns six or more items (the current profiles issue three queries with up to three results each), these persisted items are appended after the search results, but _build_review_prompt() only reads news[:6] and build_market_review_payload() only keeps news[:8]. In that common path the configured local intelligence never reaches the prompt and may be absent from the structured payload, contradicting the new evidence-injection behavior; insert or reserve persisted items ahead of the slice window after deduping.
Useful? React with 👍 / 👎.
ZhuLinsen
left a comment
ZhuLinsen
left a comment
There was a problem hiding this comment.
评审结论
- 必要性：通过，本 PR 将 #1707 的本地资讯池接入个股、Agent 和大盘分析链路，属于已有功能闭环的必要增强。
- 是否有对应 issue：有，描述中提到 issue #1707，但未使用
Refs/Fixes/Closes #1707形式建立正式关联。 - PR 类型：feat，新增运行时分析上下文注入能力，并配套文档、CHANGELOG 和集成测试。
- description 完整性：不完整，已有 Summary 和验证命令，但缺少正式 issue 关联、风险/回滚说明；结构化检测提示外部模型/API 或运行时配置相关风险，也未在描述中澄清是否实际涉及及其兼容验证。
- 是否可直接合入：不可，需先处理大盘复盘本地资讯可能被搜索结果截断掉的问题，并澄清结构化检测到的外部模型/API 或运行时配置风险是否为真实变更。
主要问题
- [Correctness blocker]
src/market_analyzer.py中_merge_persisted_market_intelligence将本地 market 资讯追加在普通搜索新闻之后；当前讨论中的自动审查也指出，当搜索新闻已返回较多条目时，后续生成 prompt、结构化 payload 或报告 news 字段如果只取前 N 条，本地资讯会被排在截断范围外。风险是本 PR 声称“大盘复盘会把同市场 market 级资讯合并到市场新闻列表，Prompt、结构化 payload 和报告 news 字段都能看到来源链接”，但在常见非空搜索结果场景下可能实际不可见。现有测试只覆盖news=[]，建议补覆盖已有搜索新闻达到上限时本地资讯仍进入最终消费路径的回归测试，并调整合并/排序策略。 - [Process blocker] 结构化事实提示检测到外部模型/API 兼容风险和运行时配置迁移风险，但 PR 描述与当前摘要未说明是否实际改动了模型名、provider、Base URL、默认模型或保存前清理逻辑。若确有相关变更，需要补官方来源、当前依赖/运行时兼容验证、旧配置迁移/回退路径；若只是检测误触发，也需要在 PR 描述中明确本 PR 不涉及这些配置语义，避免合入判断低估兼容风险。
🤖 此回复由 OpenReview Bot 自动生成，仅供参考。如有疑问请 @维护者。
…d Validate redirect targets
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 807839a69d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| {"scope_type": "symbol", "scope_value": code, "market": market}, | ||
| {"scope_type": "market", "market": market}, | ||
| ): | ||
| payload = service.list_items(days=days, page=1, page_size=limit, **filters) |
There was a problem hiding this comment.
When a feed item is fetched recently but has an old or missing publication date, this query still returns it because IntelligenceRepository.list_items(days=...) filters on fetched_at, and the new formatter later falls back to fetched_at when published_at is absent. That lets stale or undated RSS/Atom entries enter the stock prompt as recent evidence despite NEWS_MAX_AGE_DAYS/the prompt's recency rules; the same pattern was added in src/market_analyzer.py for market reviews. Please filter by published_at for analysis evidence, or keep missing publish dates unknown and exclude them from recent-news injection.
Useful? React with 👍 / 👎.
…eam feeds before enforcing
ZhuLinsen
left a comment
ZhuLinsen
left a comment
There was a problem hiding this comment.
评审结论
- 必要性：通过。本 PR 将 #1707 已落库的资讯池接入个股、Agent 和大盘复盘上下文，属于该能力闭环的必要增强。
- 是否有对应 issue：有，描述中提到 issue #1707，但未使用
Refs/Fixes/Closes #1707建立正式关联。 - PR 类型：feat。新增运行时分析上下文注入能力，并补充文档、CHANGELOG 和集成测试。
- description 完整性：不完整。已有 Summary 和验证命令，但缺少正式 issue 关联、风险/回滚说明；结构化检测提示外部模型/API 与运行时配置风险，虽然当前 diff 和文档说明未实际改变模型、provider、base URL 或回退策略，PR 描述仍应明确澄清该点。
- 是否可直接合入：不可。需要先处理本地资讯按
fetched_at而非实际发布时间进入分析上下文的正确性风险，并补齐 PR 描述中的关联 issue、风险与最小回滚说明。
主要问题
- [Correctness blocker]
src/core/pipeline.py与src/market_analyzer.py通过IntelligenceService.list_items(days=...)读取本地资讯，但该查询语义是按入库/拉取时间窗口筛选。这样最近刚抓取到的旧新闻，或缺失published_at的历史条目，会被当作最近资讯注入个股分析、Agentnews_context和大盘复盘 payload。风险是 LLM 把陈旧事件当作当前催化因素，影响用户可见分析结论。本 PR 扩大了既有资讯池查询语义到分析链路，建议在消费侧按published_at做二次过滤或明确降级规则，并补充覆盖“旧发布时间但近期 fetched_at”的回归测试。 - [Process blocker] PR 描述未按仓库规范补齐正式 issue 关联、兼容性/风险说明和回滚方式。尤其本次触及用户可见分析上下文与报告 news/payload 结构，
docs/intelligence-sources.md已写明不改模型/provider/base URL 与回滚方式，但 PR body 还没有同步说明；建议至少补充Refs #1707、说明无模型/API 配置迁移、回滚为 revert 本 PR 或关闭/清退本地资讯接入。
🤖 此回复由 OpenReview Bot 自动生成，仅供参考。如有疑问请 @维护者。
…ce, or keep missing publish
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ac89316857
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| {"scope_type": "symbol", "scope_value": normalize_stock_code(code), "market": market}, | ||
| {"scope_type": "symbol", "scope_value": code, "market": market}, |
There was a problem hiding this comment.
When a symbol intelligence source was created with an equivalent but different stock-code spelling, such as scope_value=SH600519 or 600519.SH, a normal analysis run for 600519 will never load those items because source creation stores scope_value verbatim while this lookup only tries the normalized request code and the raw request code. This makes configured symbol feeds silently disappear for common exchange-prefixed/suffixed or case variants; normalize/canonicalize symbol scope values at write time or query all accepted aliases before falling back to market-level items.
Useful? React with 👍 / 👎.
…m and Normalize nullable
ZhuLinsen
left a comment
ZhuLinsen
left a comment
There was a problem hiding this comment.
评审结论
- 必要性：通过。本 PR 将 #1707 已落库的本地资讯池接入个股分析、Agent 分析和大盘复盘上下文，补齐“采集入库后被分析消费”的闭环。
- 是否有对应 issue：有，描述中提到 issue #1707，但未使用
Refs/Fixes/Closes #1707建立正式关联。 - PR 类型：feat。新增运行时分析上下文注入能力，并补充集成测试、专题文档和
docs/CHANGELOG.md。 - description 完整性：不完整。已有 Summary 和验证命令，但缺少正式 issue 关联、风险/回滚说明；不过
docs/intelligence-sources.md已说明本次不改模型名、provider/base URL、回退策略或运行时配置语义，当前 diff 也未看到实际外部模型/API 或运行时配置迁移改动。 - 是否可直接合入：可。当前 CI success，mergeable_state=clean；此前关于本地资讯排序、发布时间过滤、股票代码等价匹配的风险在当前 diff 中已有对应实现和测试覆盖，未发现阻断性正确性问题。
🤖 此回复由 OpenReview Bot 自动生成，仅供参考。如有疑问请 @维护者。
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b2e6616913
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| """Load locally persisted intelligence as fail-open evidence context.""" | ||
| try: | ||
| service = IntelligenceService() | ||
| days = max(1, int(getattr(self.config, "news_max_age_days", 3) or 3)) |
There was a problem hiding this comment.
When NEWS_MAX_AGE_DAYS is higher than the selected NEWS_STRATEGY_PROFILE window (for example max age 30 with the default short profile), normal search uses the effective min window, but this new persisted-intelligence path filters by the raw news_max_age_days. That lets older local feed items enter stock prompts even though the configured search recency contract would exclude them; the same cutoff pattern is used in market review. Fresh evidence: the new published_days filter now checks publish time, but the cutoff is still computed from the raw max-age here rather than Config.get_effective_news_window_days().
Useful? React with 👍 / 👎.
| trimmed_digits = digits.lstrip("0") or digits | ||
| add_case_variants(normalized_upper) | ||
| add_case_variants(f"{trimmed_digits}.HK") | ||
| add_case_variants(f"{digits}.HK") | ||
| return values |
There was a problem hiding this comment.
When a Hong Kong source is saved with the common plain code scope_value=00700, analyses launched with canonical or suffixed forms such as HK00700 or 00700.HK never load it: this branch adds HK00700, 700.HK, and 00700.HK, then returns without adding the plain five-digit alias, while source creation stores scope_value verbatim. Fresh evidence: the new alias helper added in this commit still omits that HK plain-code alias, so configured local evidence silently disappears for one accepted HK spelling.
Useful? React with 👍 / 👎.
…uplicate races and Reject
…idence and add the plain
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9ad6229559
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if len(collected) >= limit: | ||
| break |
There was a problem hiding this comment.
When equivalent symbol feeds exist under multiple accepted spellings and the first alias queried already returns limit rows, this break exits before checking the remaining aliases. For example, six recent 600519 items will prevent a newer item stored as 600519.SH/SH600519 (or six HK00700 items preventing 00700) from ever reaching the prompt, so alias-scoped intelligence can still silently disappear; collect/query all alias matches, de-dupe, sort by publish time, then apply the limit.
Useful? React with 👍 / 👎.
ZhuLinsen
left a comment
ZhuLinsen
left a comment
There was a problem hiding this comment.
评审结论
- 必要性：通过。本 PR 将 #1707 的本地资讯/情报池接入个股、Agent 与大盘复盘分析上下文，补齐采集入库后的消费链路。
- 是否有对应 issue：有，描述中提到 issue #1707，但未使用
Refs/Fixes/Closes #1707建立正式关联。 - PR 类型：feat。新增运行时分析上下文注入能力，并补充集成测试、专题文档和
docs/CHANGELOG.md。 - description 完整性：不完整。已有 Summary 和验证命令，但缺少正式 issue 关联、风险/回滚说明；另外结构化检测提示外部模型/API 与运行时配置风险，PR 描述未明确说明该提示是否为误报、未列出核验依据。
- 是否可直接合入：不可。当前 CI 为 success 且 mergeable_state=clean，但在结构化风险提示未被 PR 描述澄清前，不建议直接合入。
主要问题
- [Process blocker] PR 描述未覆盖结构化检测出的外部模型/API 兼容风险和运行时配置迁移风险。当前 diff 主要集中在本地 intelligence evidence 注入，
docs/intelligence-sources.md已说明“不改模型名、provider/base URL、回退策略或运行时配置语义”，但 PR 描述没有同步说明该检测结论为何不适用于本次改动，也没有给出对应核验依据。按仓库 AGENTS.md 对配置语义、模型/provider、运行时配置变更的稳定性要求，建议在 PR 描述中明确：本 PR 是否实际修改模型名/provider/Base URL/默认配置；若没有，说明检测为关键词误报；若有，则补充官方来源、依赖/运行时兼容验证、旧配置迁移与回退路径。 - [Nice to have] PR 描述未使用
Refs #1707/Fixes #1707/Closes #1707形式关联 issue，也缺少最小回滚说明。对本类功能 PR，建议补充“revert 本 PR 或移除本地资讯接入入口/清退本地资讯源配置数据”级别说明即可。
🤖 此回复由 OpenReview Bot 自动生成，仅供参考。如有疑问请 @维护者。
ZhuLinsen
left a comment
ZhuLinsen
left a comment
There was a problem hiding this comment.
评审结论
- 必要性：通过。本 PR 将 #1707 已落库的本地资讯/情报池接入个股分析、Agent 分析和大盘复盘上下文，补齐采集后消费链路。
- 是否有对应 issue：有，描述中提到 #1707，但未使用
Refs/Fixes/Closes #1707建立正式关联。 - PR 类型：feat。新增运行时分析上下文注入能力，并补充集成测试、专题文档和
docs/CHANGELOG.md。 - description 完整性：不完整。已有 Summary、针对性 py_compile/pytest 验证和 stacked PR 说明，但缺少正式 issue 关联、风险点与回滚方式；相关兼容性/回滚说明已写入
docs/intelligence-sources.md和docs/CHANGELOG.md，但 PR body 未同步。 - 是否可直接合入：不可。当前未发现代码正确性阻断；CI 为 success，
mergeable_state=unknown不单独构成阻断。但按本仓库AGENTS.md，合入前需要补齐 PR 描述里的风险/回滚说明，建议同时补Refs #1707。
主要问题
- [Process blocker] PR 描述缺少风险点与回滚方式。证据摘要：当前 PR body 只有 Summary、Verification 和 stacked-on 说明；虽然文档已说明本次不改模型名/provider/base URL/回退策略或运行时配置迁移，回滚为
revert/停用本地资讯接入，但这些关键信息没有进入 PR 描述。风险是合入记录无法直接反映兼容性判断和回滚路径，不符合AGENTS.md对用户可见能力变更的交付要求。建议将文档中的兼容性结论、回滚方式和Refs #1707同步补到 PR body。
🤖 此回复由 OpenReview Bot 自动生成，仅供参考。如有疑问请 @维护者。
ZhuLinsen
left a comment
ZhuLinsen
left a comment
There was a problem hiding this comment.
评审结论
- 必要性：通过。本 PR 补齐 #1707 本地资讯/情报池从入库到个股、Agent 与大盘复盘分析消费的链路。
- 是否有对应 issue：有，Refs #1707。
- PR 类型：feat，新增运行时分析上下文注入能力，并补充集成测试与专题文档。
- description 完整性：完整，已包含 Summary、issue 关联、兼容性/风险、回滚和验证说明。
- 是否可直接合入：不可。当前 merge 状态存在冲突，且本地资讯别名聚合仍有遗漏风险，需要先修复后再合入。
主要问题
- [Correctness blocker]
src/core/pipeline.py：_symbol_scope_lookup_values()与_load_persisted_intelligence_context()仍会遗漏等价代码下的本地资讯。典型场景是分析入口使用港股纯代码00700时，当前只查询00700，不会补查HK00700/00700.HK；另外按别名逐个查询并在达到limit后提前break，会让后续等价别名下更新的资讯没有机会参与全局排序。风险是传统分析和 Agent 分析静默丢失本应命中的本地 evidence，尤其与仓库已有港股等价代码匹配语义不一致。建议把等价别名双向展开后统一合并、去重、按发布时间排序，再截断，并补充“纯 HK 分析代码命中 canonical/suffixed scope”和“多个别名全局排序后截断”的回归测试。 - [Process blocker] 当前 PR 结构化状态显示存在冲突，虽然 CI 为 success，但分支当前不能直接合并。需要先解决冲突，并在冲突解决后的 head 上重新确认相关测试/CI。
🤖 此回复由 OpenReview Bot 自动生成，仅供参考。如有疑问请 @维护者。
ZhuLinsen
left a comment
ZhuLinsen
left a comment
There was a problem hiding this comment.
评审结论
- 必要性：通过。本 PR 目标是补齐 #1707 本地资讯/情报池在个股、Agent 与大盘复盘中的消费链路，方向成立。
- 是否有对应 issue：有，Refs #1707。
- PR 类型：feat，新增本地资讯 evidence 注入分析上下文与大盘复盘 payload 的运行时能力，并补充测试和文档。
- description 完整性：不完整。已有 Summary、issue 关联、兼容性/风险、回滚和验证说明，但未说明当前 diff 同时回退了资讯源 URL 安全边界、错误脱敏、去重/数据库 schema 语义和批量拉取分页行为；验证说明也未覆盖这些回归路径。
- 是否可直接合入：不可。当前 CI 成功且 merge 状态 clean，但存在安全与正确性阻断问题，需要修复后再合入。
主要问题
- [Correctness blocker]
src/services/intelligence_service.py回退了资讯源 URL 安全防护：移除了 hostname DNS 解析校验、请求阶段 DNS 再校验、禁用环境代理、手动重定向目标预校验和流式大小限制；同时_validate_url()允许no-url:intel:作为源 URL。风险是自定义 RSS/Atom 源可能经 DNS 漂移、代理或重定向访问内网/metadata 地址，或在大小限制前把大响应读入内存。对应的安全回归测试也被删除，需要恢复等价防护与测试。 - [Correctness blocker]
api/v1/endpoints/intelligence.py将内部异常str(exc)直接拼进 500 响应并完整记录日志，且_sanitize_error()覆盖范围变窄，原先验证token/api-key/secret不泄露的 API 测试被删除。包含 token 的 feed URL 在 fetch 失败时可能暴露给 API 调用方或日志。 - [Correctness blocker]
src/storage.py/src/repositories/intelligence_repo.py将intelligence_items去重从 source/scope/market 维度收窄成全局 URL，并在命中同 URL 时覆盖现有行的scope_type/scope_value/market。同一新闻 URL 被多个市场或标的复用时会丢失或串改 evidence 归属；同时既有 SQLite 表不会因 ORM 变更自动从旧的非空scope_value/复合唯一约束迁移，市场级scope_value=None在旧表上可能被静默丢弃。 - [Correctness blocker]
src/services/intelligence_service.py的fetch_enabled_sources()只读取第一页最多 100 个 enabled source，删除了原来的分页遍历。超过 100 个资讯源时后续源会被静默跳过，但接口仍返回 ok，属于批量拉取语义回归。 - [Validation gap] 结构化检测提示外部模型/API 兼容风险；从当前代码 diff 看未发现模型名、provider 或 Base URL 的运行时变更，但
tests/test_config_env_compat.py删除了证明NEWS_INTEL_*不影响 LLM/OpenAI 配置的回归测试。PR 描述继续声明“不改模型/provider/base URL/运行时配置迁移”时，需要保留或迁移这类兼容性证据。
🤖 此回复由 OpenReview Bot 自动生成，仅供参考。如有疑问请 @维护者。
…源 URL 安全防护：移除了 hostname DNS
ZhuLinsen
left a comment
ZhuLinsen
left a comment
There was a problem hiding this comment.
评审结论
- 必要性：通过。本 PR 将已落库的本地资讯/情报池接入个股分析、Agent 分析和大盘复盘上下文，补齐 #1707 的消费链路。
- 是否有对应 issue：有，Refs #1707。
- PR 类型：feat，新增运行时分析上下文注入能力，并补充集成测试、专题文档和
docs/CHANGELOG.md。 - description 完整性：完整，已包含 Summary、issue 关联、兼容性/风险、回滚方式和验证命令；未运行
ci_gate.sh，但当前 CI 为 success，且已有针对本次改动面的 pytest/py_compile 说明。 - 是否可直接合入：可。当前 mergeable_state=clean、CI success；结合本地 diff，未发现当前 head 中仍存在阻断性正确性问题。
🤖 此回复由 OpenReview Bot 自动生成，仅供参考。如有疑问请 @维护者。
ZhuLinsen
left a comment
ZhuLinsen
left a comment
There was a problem hiding this comment.
评审结论
- 必要性：通过，本 PR 补齐 #1707 本地资讯/情报池在个股、Agent 与大盘复盘中的消费链路。
- 是否有对应 issue：有，Refs #1707。
- PR 类型：feat，新增 RSS/Atom 资讯源配置、拉取、入库、查询以及分析上下文注入能力。
- description 完整性：完整，已包含 Summary、issue 关联、兼容性/风险、回滚和验证命令；本地 diff 未发现实际模型/provider/Base URL/默认模型策略或保存前清理语义变更，结构化模型/API 风险目前更像关键词误报。
- 是否可直接合入：不可，当前存在 merge conflict，且资讯源真实拉取路径有运行时错误需要先修复。
主要问题
- [Correctness blocker]
src/services/intelligence_service.py：_fetch_feed_entries()直接调用requests.get(..., trust_env=False, ...)，但当前requests>=2.31.0的Session.request不接受trust_env参数；真实执行/sources/test或/sources/{id}/fetch会在发起请求前抛TypeError。现有测试把requests.get整体 mock 掉，因此 CI 没覆盖真实 requests 调用签名。风险是本 PR 新增的 RSS/Atom 测试、拉取和入库主能力在运行时不可用。 - [Correctness blocker]
src/core/pipeline.py：港股 symbol lookup 仍遗漏HK700/hk700这类未补零 HK 前缀别名。仓库已有多处等价股票代码语义把HK700、HK00700、00700.HK、700.HK视为同一标的；如果资讯源以scope_value=HK700保存，分析时用HK00700或00700.HK不会加载该 evidence，导致本地资讯静默缺失。建议补齐别名并加回归测试。 - [Process blocker] 结构化事实显示当前 merge 状态存在冲突；即使 CI 当前为 success，也需要先解决冲突后才能合入。
🤖 此回复由 OpenReview Bot 自动生成，仅供参考。如有疑问请 @维护者。
…e-analysis-1707 # Conflicts: # .env.example # api/v1/endpoints/intelligence.py # api/v1/schemas/intelligence.py # docs/intelligence-sources.md # src/repositories/intelligence_repo.py # src/services/intelligence_service.py # src/storage.py # tests/test_intelligence_api.py # tests/test_intelligence_service.py
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: aab08db855
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| except IntelligenceServiceError: | ||
| raise | ||
| except Exception as exc: | ||
| raise IntelligenceServiceError(f"fetch failed: {exc}") from exc |
There was a problem hiding this comment.
When a configured feed URL carries a query secret (for example ?token=...) and requests/raise_for_status() raises an error that includes the URL, this wraps the upstream exception as IntelligenceServiceError. The API endpoints catch that as a 400 via _bad_request and return str(exc) directly, so /sources/{id}/fetch or /sources/test can expose the token in the response; keep these as sanitized/internal fetch failures or sanitize IntelligenceServiceError messages before returning them.
Useful? React with 👍 / 👎.
ZhuLinsen
left a comment
ZhuLinsen
left a comment
There was a problem hiding this comment.
评审结论
- 必要性：通过，本 PR 补齐 #1707 本地资讯/情报池从采集入库到个股分析、Agent 分析和大盘复盘消费的链路，方向成立。
- 是否有对应 issue：有，Refs #1707。
- PR 类型：feat，新增分析上下文注入、RSS/Atom 模板后端入口，并补充集成/回归测试与文档。
- description 完整性：完整，已包含 Summary、issue 关联、兼容性/风险、回滚方式和验证命令；本地 diff 未发现实际模型/provider/Base URL/默认模型策略或运行时配置清理语义变更，结构化模型/API 风险提示目前更像关键词误报。
- 是否可直接合入：不可，需要先修复 fetch 失败链路可能泄露上游 URL query secret 的问题。当前 CI 为 success，
mergeable_state=blocked可视为分支保护状态，不单独构成阻断。
主要问题
- [Correctness blocker]
api/v1/endpoints/intelligence.py/src/services/intelligence_service.py/tests/test_intelligence_api.py：当前 fetch 错误脱敏回归只覆盖了 endpoint 层IntelligenceService.fetch_source抛RuntimeError("token=secret ...")的路径，但 diff 中移除了此前针对requests.HTTPError携带完整源 URL（如?token=secret）的 API 回归。真实 RSS/Atom 拉取中，requests/raise_for_status()的异常消息可能包含完整 URL；如果 service 将该异常包装后作为业务错误返回或记录，用户配置在 feed URL query 中的 token/API key 仍可能进入 API 响应或日志。风险是新增资讯源安全边界不完整，且与 PR 描述中的 harden/sanitized 目标不一致。建议在 service 层对上游 fetch 异常统一脱敏或返回固定错误文案，并补回覆盖raise_for_status()/HTTPError含敏感 URL 的回归测试。
🤖 此回复由 OpenReview Bot 自动生成，仅供参考。如有疑问请 @维护者。
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ef5c22b6ac
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (market or "").strip().lower() != "cn": | ||
| return values |
There was a problem hiding this comment.
Fresh evidence: when the analysis is launched with an accepted plain HK ticker such as 00700, normalize_stock_code() leaves normalized_upper as 00700, so the HK branch above is skipped and this non-CN early return leaves only the plain value in the lookup list. Any source saved under the equally accepted HK00700 or 00700.HK spelling is then not loaded for that run; handle market == "hk" plus plain digits before returning.
Useful? React with 👍 / 👎.
| "source_type": "rss", | ||
| "url": "https://feeds.content.dowjones.io/public/rss/mw_topstories", | ||
| "scope_type": "market", | ||
| "market": "global", |
There was a problem hiding this comment.
When users create this new global-marketwatch template, fetched rows are persisted with market="global", but the new stock and market-review consumers query only the concrete analysis market (cn/hk/us), and MarketAnalyzer never runs with a global region. That means this advertised built-in source can be fetched and stored but will not reach any normal analysis context; either include global market items in those lookups or do not offer it as an analysis template yet.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 45554c1889
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| logger.info("Rebuilding intelligence_items table to align composite uniqueness constraints.") | ||
| with self._engine.begin() as connection: | ||
| connection.execute(text(f'DROP TABLE IF EXISTS "{temporary_table}"')) | ||
| tmp_table.create(connection) |
There was a problem hiding this comment.
When upgrading a SQLite database that still has the earlier intelligence_items.url unique index, _ensure_intelligence_items_unique_index() enters this rebuild path, but tmp_table.create(connection) creates the copied table with the same ORM index names (ix_intel_item_scope_time, ix_intel_item_fetch_time, and column indexes) while the original table is still present. SQLite index names are database-wide, so creation fails with index ... already exists before the old table is dropped, blocking DB initialization for existing installs; create the temp table without indexes or rename/drop/recreate indexes after the swap.
Useful? React with 👍 / 👎.
ZhuLinsen
left a comment
ZhuLinsen
left a comment
There was a problem hiding this comment.
评审结论
- 必要性：通过，本 PR 补齐 #1707 本地资讯/情报池进入个股分析、Agent 分析和大盘复盘上下文的消费链路，需求方向成立。
- 是否有对应 issue：有，Refs #1707。
- PR 类型：feat，新增资讯源模板、NewsNow 默认源初始化、分析上下文注入和相关回归测试。
- description 完整性：完整，已包含 Summary、issue 关联、兼容性与风险、回滚方式和验证命令；同时说明未运行完整
ci_gate.sh，但当前阻断型 CI 已通过。 - 是否可直接合入：不可，仍有一个数据库迁移兼容性风险需要先确认并修复。
主要问题
- [Correctness blocker]
src/storage.py：当前新增的intelligence_items唯一索引迁移/重建逻辑需要覆盖旧库中仍存在早期intelligence_items.url唯一索引的场景。已有自动 review 指出 rebuild 临时表路径可能复用或碰撞 SQLite index 名称；如果用户从旧版本数据库升级，会在启动或迁移阶段失败，影响整个资讯池功能落地。该风险属于兼容性/迁移风险，不能只依赖全新测试库通过。建议补充旧 schema 升级回归测试，并确保临时表创建、索引清理/重建过程不会与既有 SQLite index 名冲突。
🤖 此回复由 OpenReview Bot 自动生成，仅供参考。如有疑问请 @维护者。
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5898c0df8b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| return HTTPException(status_code=500, detail={"error": "internal_error", "message": message}) | ||
| sanitized_error = sanitize_diagnostic_text(str(exc), max_length=300) or "internal intelligence error" | ||
| logger.error("%s: %s", message, sanitized_error) | ||
| return HTTPException(status_code=500, detail={"error": "internal_error", "message": f"{message}: {sanitized_error}"}) |
There was a problem hiding this comment.
When an unexpected intelligence endpoint failure includes internal details that the best-effort diagnostic sanitizer does not cover, this now returns those details directly in the HTTP 500 body. For example, DB/runtime exceptions can include SQL fragments, DSNs, hostnames, or filesystem paths outside the limited path-redaction prefixes, whereas the previous behavior only exposed the generic operation message; keep the sanitized text in logs and return a stable generic message to clients.
Useful? React with 👍 / 👎.
ZhuLinsen
left a comment
ZhuLinsen
left a comment
There was a problem hiding this comment.
评审结论
- 必要性：通过，本 PR 补齐 #1707 本地资讯池进入个股分析、Agent 分析和大盘复盘上下文的链路，方向成立。
- 是否有对应 issue：有，Refs #1707。
- PR 类型：feat，新增资讯源模板、NewsNow 默认源初始化、分析上下文注入和相关回归测试。
- description 完整性：不完整，Summary、回滚和验证信息基本齐全，但新增
NEWSNOW_BASE_URL外部 API 默认值仍缺少官方来源链接或当前实例/API 契约的运行时兼容验证说明。 - 是否可直接合入：不可，CI 已通过且当前 merge 状态本身不构成阻断，但仍有 API 500 响应泄露内部错误细节和 NewsNow 外部默认 API 证据不足两个待修复/待澄清点。
主要问题
- [Correctness blocker]
api/v1/endpoints/intelligence.py：_internal_error()现在把sanitize_diagnostic_text(str(exc))拼进 HTTP 500 响应体。这个 sanitizer 只能覆盖已知 secret/path 模式，不能保证任意内部异常文本都适合暴露给 API 客户端；当前测试也只覆盖了token/api_key这类命中规则的用例。建议 500 body 保持通用错误文案，脱敏后的异常细节只进日志，并补一个未命中敏感规则的异常回归测试。 - [Compatibility / verification blocker]
src/config.py、src/services/intelligence_service.py、.env.example、docs/intelligence-sources.md：本 PR 新增默认NEWSNOW_BASE_URL=https://newsnow.busiyi.world并内置多个 NewsNow source id，这是用户可执行的外部 API 默认行为。文档已说明可改自建实例，但目前缺少官方/上游来源链接、当前 API shape 的非 mock 验证证据，且POST /sources/defaults不传enabled=false时会创建启用状态的外部源。建议补齐来源与兼容性证据，或调整默认启用语义/文档，使公开示例实例不可用时的用户预期更明确。
🤖 此回复由 OpenReview Bot 自动生成，仅供参考。如有疑问请 @维护者。
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cdfd42b7ae
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| logger.debug("[大盘] %s action=load_local_intelligence status=failed error=%s", self._log_context(), exc) | ||
| # Keep local market intelligence at the front so market-review prompt/payload slicing | ||
| # still includes local links when search news already reaches the top-N limit. | ||
| return merged_local + search_news |
There was a problem hiding this comment.
When the local market-intelligence pool has six recent items, this prepends all six ahead of search_news, but _build_review_prompt() only consumes news[:6]. In that common case the live market-news results fetched immediately above never reach the LLM prompt, so the recap can ignore current search evidence whenever local feeds are populated; reserve or interleave only part of the slice for local items instead of front-loading the entire local page.
Useful? React with 👍 / 👎.
ZhuLinsen
left a comment
ZhuLinsen
left a comment
There was a problem hiding this comment.
评审结论
- 必要性：通过，本 PR 补齐 #1707 本地资讯/情报池进入个股分析、Agent 分析和大盘复盘上下文的链路，需求方向成立。
- 是否有对应 issue：有，Refs #1707。
- PR 类型：feat，新增 NewsNow/default source/API 模板能力，并把本地资讯证据接入分析上下文。
- description 完整性：不完整，已包含 Summary、回滚、验证和 CHANGELOG/配置文档同步，但新增
NEWSNOW_BASE_URL默认外部实例/API 契约仍缺少官方来源或非 mock 运行验证；风险段里“不改 base URL”也需要明确限定为 LLM/provider base URL，避免和本次新增 NewsNow Base URL 语义冲突。 - 是否可直接合入：不可，CI 已通过且当前
mergeable_state=blocked本身不构成阻断，但仍有 NewsNow 错误脱敏、大盘复盘新闻截断回归和外部默认 API 兼容性证据缺口需要先处理。
主要问题
- [Correctness blocker]
src/services/intelligence_service.py/api/v1/endpoints/intelligence.py：RSS 拉取错误已统一返回fetch failed: upstream request failed，但 NewsNow 分支在_fetch_newsnow_entries()中会把非IntelligenceServiceError包装成fetch failed: {exc}，API 的_bad_request()又会直接返回str(exc)。如果用户配置的 NewsNow URL 带?token=...等查询密钥，response.raise_for_status()产生的 HTTPError 仍可能把完整 URL 返回给/sources/test或/sources/{id}/fetch调用方。建议让 NewsNow 上游请求错误复用同样的通用错误文案，并补source_type=newsnow的 API 回归测试。 - [Correctness blocker]
src/market_analyzer.py：_merge_persisted_market_intelligence()最多读取 6 条本地 market intelligence 并整体前置，而_build_review_prompt()只消费news[:6]。当本地池正好有 6 条近期资讯时，原有search_market_news()的搜索新闻会完全从 LLM 复盘 prompt 中消失，结构化 payload 的news[:8]也只剩很少搜索新闻。这会让本 PR 在扩展证据池的同时回退既有大盘复盘时效性。建议为搜索新闻保留固定槽位或做交错合并，并补 6 条本地资讯 + 多条搜索新闻的回归测试。 - [Process blocker]
.env.example/src/config.py/docs/intelligence-sources.md：本 PR 新增默认NEWSNOW_BASE_URL=https://newsnow.busiyi.world和GET /api/s?id=<source_id>可执行契约；现有通用文档已补充配置和核验建议，但 PR 验证只覆盖 mock JSON，未提供该公开实例/NewsNow API 形态的官方来源链接或当前非 mock 兼容性验证。按仓库对新增配置、外部依赖和用户可见 API 行为的要求，需要补齐来源与运行时兼容证据，或避免把公开示例实例作为默认可执行 Base URL。
🤖 此回复由 OpenReview Bot 自动生成，仅供参考。如有疑问请 @维护者。
ZhuLinsen
left a comment
ZhuLinsen
left a comment
There was a problem hiding this comment.
必要性：通过。本 PR 补齐 #1707 本地资讯池进入个股分析、Agent 分析和大盘复盘上下文的消费链路，方向成立。
是否有对应 issue：有，Refs #1707。
PR 类型：feat，新增资讯源模板、NewsNow 默认源初始化、分析上下文注入和相关回归测试。
description 完整性：基本完整。Summary、回滚、验证和 CHANGELOG 同步均已说明，但对新增 NEWSNOW_BASE_URL 外部依赖的兼容性和官方来源验证说明不足。
是否可直接合入：不可。需在合入前确认并修复外部 API 兼容性风险。
主要问题
-
[Compatibility blocker] NewsNow Base URL 外部依赖兼容性未验证 — 文件：
.env.example,src/config.py,docs/CHANGELOG.md
PR 引入NEWSNOW_BASE_URL配置项且使用公开示例实例作为默认值，但缺少官方来源链接或非 mock 环境下的运行时兼容验证。虽然 PR 描述提到生产环境建议改为自建，但：- 用户可能误用公开实例，导致限流或可用性问题；
- 无官方文档链接支撑当前默认 URL 的可靠性；
- 仅通过单元测试 mock 无法证明公开实例 API 契约在生产环境的兼容性。
建议：补齐官方 NewsNow 实例链接文档或 API 契约确认，明确指出公开实例风险（需结合当前部署指南文档，更新.env.example注释和docs/相关说明），确保用户可见。
-
[Verification gap] 完整 CI 门控未运行 — 文件：PR 描述
PR 说明未运行./scripts/ci_gate.sh和 Web build，虽然自动 CI 显示backend-gate:success，但不能替代本地完整 gate 验证。
建议：在合入前运行./scripts/ci_gate.sh确认所有后端检查通过。 -
[Nice to have] HTTP 500 响应隐私处理确认 — 文件：
api/v1/endpoints/intelligence.py
Codex review 指出当上游服务错误含敏感词汇时可能暴露。diff 中已见"Fetch intelligence source failed: internal intelligence service error"的通用消息，但建议确认所有 500 路径都已清理敏感信息，不在响应体中包含原始异常文本或 URL 参数。 -
[Nice to have] 市场分析上下文中资讯项排序冲突 — 文件：
src/market_analyzer.py,src/core/pipeline.py
当本地资讯池有多条记录时，市场分析上下文中 local intelligence 可能挤掉search_news。虽然不是功能错误，但需确认是否有对应的 prompt 或截断逻辑来保证关键新闻不被挤出，特别是 6+ 条本地资讯时的行为。
🤖 此回复由 OpenReview Bot 自动生成，仅供参考。如有疑问请 @维护者。
…inks - Add official NewsNow GitHub deployment guide link to .env.example - Include curl-based API contract verification example for production validation - Update CHANGELOG.md with official repository reference and deployment recommendation - Clarify risk of public example instance and necessity for self-hosted in production - All HTTP 500 responses already use sanitize_diagnostic_text for privacy Ref #1707 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…结合当前部署指南文档，更新 .env.example
ZhuLinsen
left a comment
ZhuLinsen
left a comment
There was a problem hiding this comment.
必要性：通过，本 PR 补齐 #1707 本地资讯池进入个股分析、Agent 分析和大盘复盘上下文的消费链路，需求方向成立。
是否有对应 issue：有，Refs #1707。
PR 类型：feat，新增资讯源模板、NewsNow 默认源初始化、分析上下文注入和相关回归测试。
description 完整性：完整，已包含 Summary、兼容性与风险、回滚方式、验证命令，同时说明未运行完整 ci_gate.sh，但当前阻断型 CI 已通过。
是否可直接合入：可，当前 CI 成功，结构化主要风险已得到处理。
主要问题：未发现阻断性问题
简要评注：
-
外部 API 兼容性：PR 新增
NEWSNOW_BASE_URL默认实例，CHANGELOG 已明确提供官方部署指南链接（https://github.com/qqhann/newsnow）并建议生产环境自建，降级兼容策略清晰。 -
数据库迁移：新增
test_legacy_intelligence_items_url_unique_index_rebuilds_without_collision测试覆盖了 SQLite 旧索引升级路径，验证了临时表碰撞处理；本地_ensure_intelligence_items_unique_index()已处理索引冲突和数据保留。 -
运行时配置：PR 描述明确声明"不改模型名/provider/base URL/默认模型策略/回退策略"，新增仅为本地资讯消费路径和模板初始化，fail-open 降级机制保证不阻断既有链路。
-
安全性修复：CHANGELOG 已说明 HTTP 500 响应现统一返回通用错误信息，敏感细节仅入日志；新增测试用例验证错误信息脱敏（
test_fetch_source_internal_error_is_sanitized等）。 -
测试覆盖：57 个测试通过，覆盖了源管理、NewsNow JSON 摄取、URL 安全校验、重复源处理、配置兼容性、资讯注入分析上下文等关键路径。
-
版本兼容：本地验证已执行
python -m py_compile和pytest，Web 端无改动（如 PR 描述所述）。
🤖 此回复由 OpenReview Bot 自动生成，仅供参考。如有疑问请 @维护者。
|
补充实测结论（2026-06-19）： 我补做了外部在线 smoke，不只看本地 mock/CI。
结论：后端主链路已有 CI/targeted pytest 证据，外部 NewsNow 默认实例和内置 RSS 模板当前在线契约也实测通过。残余风险主要是公开 NewsNow 实例、Cloudflare 策略和上游 RSS 稳定性不受项目控制；其中 NewsNow 成功依赖本 PR 中的专用浏览器 UA。 |
* feat: add intelligence source ingestion baseline * feat: feed local intelligence into analysis contexts * fix(review-feedback-1708): Reject DNS names that resolve privately and Validate redirect targets * fix(review-feedback-1709): 处理大盘复盘本地资讯可能被搜索结果截断掉的问题，并澄清结构化检测到的外部模型/API 或运行时配置风险是否为真实变更 * fix(review-feedback-1708): Pin DNS resolution before fetching and Stream feeds before enforcing * fix(review-feedback-1709): filter by published at for analysis evidence, or keep missing publish * fix(review-feedback-1708): Sanitize fetch errors before returning them and Normalize nullable * fix(review-feedback-1709): Normalize symbol scope before lookup * fix(review-feedback-1708): Avoid rolling back prior item inserts on duplicate races and Reject * fix(review-feedback-1709): Use the effective news window for local evidence and add the plain * fix(review-feedback-1708): 落地可配置 RSS/Atom 情报源、存储、查询、retention 和基础安全边界 * fix(review-feedback-1709): 补充“revert 本 PR 或移除本地资讯接入入口/清退本地资讯源配置数据”级别说明即可 * fix(review-feedback-1708): 落地 RSS/Atom 情报源的存储、拉取、查询、retention 和基础安全边界 * fix(review-feedback-1709): 补 Refs 1707 * fix(review-feedback-1709): 解决冲突后再合入 * fix(review-feedback-1709): src/services/intelligence service.py 回退了资讯源 URL 安全防护：移除了 hostname DNS * fix: harden intelligence source ingestion * fix(review-feedback-1709): Sanitize upstream fetch errors before returning them * feat: add NewsNow intelligence sources * fix(review-feedback-1709): 确认并修复 * fix(review-feedback-1709): 补一个未命中敏感规则的异常回归测试 * fix(review-feedback-1709): 处理 * docs: enhance NEWSNOW_BASE_URL compatibility guidance with official links - Add official NewsNow GitHub deployment guide link to .env.example - Include curl-based API contract verification example for production validation - Update CHANGELOG.md with official repository reference and deployment recommendation - Clarify risk of public example instance and necessity for self-hosted in production - All HTTP 500 responses already use sanitize_diagnostic_text for privacy Ref ZhuLinsen#1707 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(review-feedback-1709): 补齐官方 NewsNow 实例链接文档或 API 契约确认，明确指出公开实例风险（需结合当前部署指南文档，更新 .env.example --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 participant
Summary
news_context.requestsusage.source_type=newsnow,NEWSNOW_BASE_URL, andPOST /api/v1/intelligence/sources/defaultsto initialize mainstream finance sources: 财联社热门、雪球热门股票、华尔街见闻快讯、金十数据、格隆汇事件.Refs #1707
Compatibility & Risk
NEWSNOW_BASE_URL使用公开示例实例；生产环境建议改为自建 NewsNow 实例，避免公开实例不可用、限流或源侧调整影响。origin/main合并最新基线并解决冲突；PR 当前不再堆叠在 feat: add intelligence source ingestion baseline #1708 分支上。Rollback
revert this PR。Verification
python -m py_compile src/services/intelligence_service.py src/repositories/intelligence_repo.py src/config.py api/v1/schemas/intelligence.py api/v1/endpoints/intelligence.py tests/test_intelligence_service.py tests/test_intelligence_api.py tests/test_config_env_compat.py tests/test_intelligence_analysis_integration.pypython -m pytest tests/test_intelligence_service.py tests/test_intelligence_api.py tests/test_config_env_compat.py tests/test_intelligence_analysis_integration.py -q— 57 passedgit diff --checkNot run:
./scripts/ci_gate.shand Web build were not run locally; no Web files are changed by this PR after conflict resolution.