fix: improve conversation list performance and prevent stale updates#9226
Open
Sisyphbaous-DT-Project wants to merge 3 commits into
Open
Conversation
- 对话列表改为摘要查询，避免读取和传输完整聊天历史，同时保持详情、导出及插件默认行为兼容 - 统一筛选、排除和稳定分页逻辑，补充 SQLite 索引并保留全文搜索能力 - 使用组件级请求编号和取消控制阻止迟到响应覆盖最新状态，修复删除、筛选防抖与分页竞态 - 增加摘要查询、索引、API 响应和参数清洗回归测试
dosubot
Bot
added
area:core
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request optimizes the dashboard conversation list by retrieving summaries without loading the full conversation history, adding database indexes, and refactoring the frontend loading states to prevent race conditions. The review feedback highlights three key improvements: robustly handling quoted and unquoted table names when forcing SQLite indexes, safely defaulting optional string filters to empty strings to prevent AttributeError during splitting, and adding a null check in the frontend's platform filter mapping to avoid a TypeError when selections are cleared.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Hey - I've found 2 issues, and left some high level feedback:
- The
fetchConversationsmethod inConversationPage.vuehas grown fairly complex with request ID tracking and snapshot logic; consider extracting the filter snapshotting and request lifecycle (abort, state transitions) into a small helper/composable to keep the component logic easier to follow and maintain. - In
SQLiteDatabase.get_filtered_conversations, the string replacement on compiled SQL to injectINDEXED BY ix_conversations_created_at_inner_idis somewhat brittle; it may be safer to guard this with assertions or a more structured check on the compiled SQL to ensure the replacement only happens when the expectedFROM conversationsclause is present and unchanged.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- The `fetchConversations` method in `ConversationPage.vue` has grown fairly complex with request ID tracking and snapshot logic; consider extracting the filter snapshotting and request lifecycle (abort, state transitions) into a small helper/composable to keep the component logic easier to follow and maintain.
- In `SQLiteDatabase.get_filtered_conversations`, the string replacement on compiled SQL to inject `INDEXED BY ix_conversations_created_at_inner_id` is somewhat brittle; it may be safer to guard this with assertions or a more structured check on the compiled SQL to ensure the replacement only happens when the expected `FROM conversations` clause is present and unchanged.
## Individual Comments
### Comment 1
<location path="astrbot/core/db/sqlite.py" line_range="401-410" />
<code_context>
+ if len(platforms) > 1 or len(platform_ids or []) > 1:
</code_context>
<issue_to_address>
**suggestion (bug_risk):** The `INDEXED BY` injection via `str.replace` on compiled SQL is brittle and may break with SQLAlchemy/SQLite changes.
This approach depends on the exact text of the generated SQL (table name, casing, aliases, spacing). Any change in SQLAlchemy, the SQLite dialect, or future query refactors could make the replacement fail or generate invalid SQL.
If you need to force an index, consider a more robust mechanism (e.g., a view/alias whose DDL encodes the index hint, SQLite planner pragmas, or separate query shapes for global vs per-platform indexes) instead of post-processing SQL text.
If you keep this pattern, at least assert that the target substring is present (and log when it is not) so failures are explicit rather than silently running a different plan or malformed query.
Suggested implementation:
```python
compiled = result_query.compile(
dialect=sqlite_dialect(paramstyle="named"),
compile_kwargs={"render_postcompile": True},
)
sql = compiled.string
target = "FROM conversations"
if target not in sql:
logger.warning(
"Expected '%s' in compiled conversations SQL; got: %s",
target,
sql,
)
assert target in sql, f"Expected '{target}' in compiled conversations SQL"
indexed_sql = sql.replace(
target,
"FROM conversations INDEXED BY "
```
1. Ensure there is a module-level `logger` defined, e.g.:
- `import logging` at the top of the file, and
- `logger = logging.getLogger(__name__)`.
2. If the file already uses a different logging convention (e.g., a shared logger), reuse that instead of introducing a new one.
</issue_to_address>
### Comment 2
<location path="astrbot/core/conversation_mgr.py" line_range="82" />
<code_context>
user_id=conv_v2.user_id,
cid=conv_v2.conversation_id,
- history=json.dumps(conv_v2.content or []),
+ history=json.dumps(conv_v2.content or []) if include_history else "",
title=conv_v2.title,
persona_id=conv_v2.persona_id,
</code_context>
<issue_to_address>
**issue (bug_risk):** Returning an empty string for `history` when `include_history=False` may break consumers expecting JSON.
With this change, `history` is no longer guaranteed to be valid JSON: when `include_history=False` it becomes `""`, which will cause `json.loads(conversation.history)` to raise `JSONDecodeError` for existing callers. To preserve v1 compatibility and avoid breaking consumers, consider instead returning a valid empty JSON structure (e.g. `'[]'`), keeping `history` as `None` when omitted, or introducing a separate flag to indicate history omission/truncation while keeping `history` itself valid JSON.
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
- 摘要查询未加载历史时返回空 JSON 数组字符串 - 增加管理器摘要结果可解析性的回归测试
Contributor
Author
|
@sourcery-ai review |
Contributor
There was a problem hiding this comment.
Hey - I've found 1 issue, and left some high level feedback:
- The SQLite multi‑platform ordering hack relies on string replacement of compiled SQL ("FROM conversations" → indexed variant), which is brittle if the query shape changes; consider using explicit index hints or separate query construction rather than manipulating the compiled SQL string.
- The
listLoadStatestate machine only renders UI forloading,empty, anderror; make sure the initialidlestate is explicitly handled in the template or set toloadingbefore the first fetch to avoid an unintentional blankno-dataarea on initial load.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- The SQLite multi‑platform ordering hack relies on string replacement of compiled SQL ("FROM conversations" → indexed variant), which is brittle if the query shape changes; consider using explicit index hints or separate query construction rather than manipulating the compiled SQL string.
- The `listLoadState` state machine only renders UI for `loading`, `empty`, and `error`; make sure the initial `idle` state is explicitly handled in the template or set to `loading` before the first fetch to avoid an unintentional blank `no-data` area on initial load.
## Individual Comments
### Comment 1
<location path="dashboard/src/views/ConversationPage.vue" line_range="74-81" />
<code_context>
:items-per-page-options="pageSizeOptions" show-select return-object
- :disabled="loading" @update:options="handleTableOptions">
+ @update:options="handleTableOptions">
+ <template v-slot:top v-if="listLoadState === 'error'">
+ <v-alert type="error" variant="tonal" density="compact" class="ma-3" closable>
+ {{ tm('messages.fetchError') }}
+ <template v-slot:append>
+ <v-btn size="small" variant="text" @click="fetchConversations">
+ {{ tm('history.refresh') }}
+ </v-btn>
+ </template>
+ </v-alert>
+ </template>
</code_context>
<issue_to_address>
**suggestion:** Error state is rendered both in the table header and in the no-data slot, which can lead to duplicated error UI.
When `listLoadState === 'error'` and there are no rows, both the `top` slot alert and the `no-data` error block will render. This duplication can be confusing. Consider showing the error in only one place (header or no-data), or conditionally hiding one based on whether there are items.
Suggested implementation:
```
:items-per-page-options="pageSizeOptions" show-select return-object
@update:options="handleTableOptions">
<template v-slot:top v-if="listLoadState === 'error' && conversations && conversations.length">
<v-alert type="error" variant="tonal" density="compact" class="ma-3" closable>
{{ tm('messages.fetchError') }}
<template v-slot:append>
<v-btn size="small" variant="text" @click="fetchConversations">
{{ tm('history.refresh') }}
</v-btn>
</template>
</v-alert>
</template>
```
To fully avoid duplicate error UI, the `no-data` slot (likely further down in the same `<v-data-table>` block) should be updated to only render the error state when `listLoadState === 'error' && (!conversations || !conversations.length)`. If it currently checks only `listLoadState === 'error'`, adjust its condition accordingly so that the header error appears only when there are rows, and the no-data error appears only when there are none.
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
- 仅在保留旧列表数据时显示顶部错误横幅 - 空列表失败时继续使用表格内错误与重试状态
Contributor
Author
|
@sourcery-ai review |
Contributor
There was a problem hiding this comment.
Hey - I've left some high level feedback:
- In
SQLiteDatabase.get_filtered_conversations, the manualcompiled.string.replaceto injectINDEXED BY ix_conversations_created_at_inner_idis quite brittle against SQLAlchemy query shape changes; consider using explicittext()with named index or a dedicated helper so it fails loudly rather than silently if the generated SQL changes. - The new
invalidateConversationListRequestsetslistLoadingandlistLoadState = 'loading'on every filter change even before a debounced fetch starts, which may cause a brief loading/empty UI flicker when rapidly typing; you might want to separate “pending filters” from “active request” so the list stays in its previous state untilfetchConversationsactually runs.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- In `SQLiteDatabase.get_filtered_conversations`, the manual `compiled.string.replace` to inject `INDEXED BY ix_conversations_created_at_inner_id` is quite brittle against SQLAlchemy query shape changes; consider using explicit `text()` with named index or a dedicated helper so it fails loudly rather than silently if the generated SQL changes.
- The new `invalidateConversationListRequest` sets `listLoading` and `listLoadState = 'loading'` on every filter change even before a debounced fetch starts, which may cause a brief loading/empty UI flicker when rapidly typing; you might want to separate “pending filters” from “active request” so the list stays in its previous state until `fetchConversations` actually runs.Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
Contributor
Author
|
@Soulter 求光顾 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AstrBot 的对话数据页面在记录较多、单条历史较长或通过公网访问时，普通列表加载、分页、每页数量切换以及群聊/私聊筛选都会读取并返回完整聊天历史。列表响应会随历史正文持续增大，造成不必要的数据库读取、JSON 序列化和网络传输开销。
此外，页面在连续筛选或慢网络环境下存在请求竞态：较早发出的请求可能晚于新请求返回，并覆盖最新列表或加载状态；筛选防抖也可能覆盖紧接着进行的分页操作。删除成功时，如果删除前的列表请求稍后返回，已删除记录还可能短暂重新出现在页面中。
改动点
后端列表性能
include_content，并为对话管理器增加默认开启的include_history，保持插件及内部调用默认获取完整Conversation的兼容行为。defer(content)避免普通列表读取完整历史正文。umo_info，不再在列表响应中包含history；对话详情、历史编辑和导出仍读取完整历史。exclude_ids前缀排除及exclude_platforms平台排除逻辑，并清理逗号参数中的空项和多余空格。created_at DESC, inner_conversation_id DESC作为稳定排序，避免相同创建时间下分页重复或遗漏。前端请求一致性
将列表请求控制移入
ConversationPage组件实例，使用递增请求编号保证只有最新请求能够更新列表、分页、错误信息和 loading 状态。保留
AbortController以减少无用传输，同时确保取消失败时仍由请求编号阻止旧响应覆盖新结果。筛选变化时立即使旧请求失效，并保留 300ms 防抖；分页、每页数量和手动刷新会取消尚未执行的筛选防抖，避免页面被重新拉回第一页。
首次加载显示 loading，刷新时保留旧列表；仅在最新请求成功且结果为空时显示“暂无记录”，失败时显示列表级错误和重试入口。
将列表 loading 与详情、编辑、删除和导出操作状态分离，避免不同操作互相关闭 loading。
单条删除成功后重新获取服务器最新列表，防止删除前的迟到响应把已删除记录重新显示。
组件卸载时取消防抖、终止请求并使请求编号失效，避免离开页面后的迟到更新。
这不是破坏性变更。现有 HTTP 路径和查询参数保持不变，插件默认行为以及详情、编辑、导出流程保持兼容。
运行截图或测试结果
专项 SQLite 和 API 验证中，将历史正文扩大到约 200 万字符后，摘要列表响应仍保持为 5327 字节，响应大小不再随正文增长。
后端完整测试：
代码格式与静态检查：
Dashboard 验证：
真实浏览器专项验证已覆盖：
AbortController.abort()失效时仍只应用最新请求。检查清单
Summary by Sourcery
Optimize dashboard conversation listing to avoid loading full histories, improve database filtering and ordering, and harden frontend list requests against stale or competing updates.
New Features:
Bug Fixes:
Enhancements:
Tests: