Skip to content

Commit 70fa64e

Browse files
committed
feat: add LightRAG integration with settings and chat view enhancements
- Introduced LightRAGTab for configuring LightRAG settings including server URL, API key, and sync options. - Updated settings types to include LightRAG configurations. - Enhanced ChatView to support LightRAG responses, including concise and retrieved answers. - Implemented LightRAG query handling with streaming responses and context retrieval. - Improved UI styles for LightRAG status and records display. - Added functionality for managing LightRAG references and source links in chat messages.
1 parent cb1c487 commit 70fa64e

11 files changed

Lines changed: 2234 additions & 19 deletions

File tree

README.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ Chat with your Obsidian vault using configurable LLM providers, embeddings, loca
1515
- Local vector storage in the vault.
1616
- Optional PostgreSQL + pgvector storage for larger indexes.
1717
- Per-file chunking overrides and chunk preview in the index manager.
18+
- Optional LightRAG backend for graph-based RAG answers, Markdown vault sync, processing status, and source references.
1819

1920
## Requirements
2021

@@ -75,6 +76,16 @@ For easiest local development, keep this repository inside:
7576
6. Run **Index all vault notes** or index individual files from the index manager.
7677
7. Open **Copilot Chat** and ask questions about your vault.
7778

79+
### LightRAG setup
80+
81+
1. Open **Settings → Obsidian RAG Integration → LightRAG**.
82+
2. Enable LightRAG and set the server URL, for example `http://192.168.50.209:9621`.
83+
3. Select **Test connection** to verify `/health`.
84+
4. Keep **Auto sync Markdown** enabled or select **Start sync** to sync immediately.
85+
5. In **Vault chat** mode, the default answer mode calls LightRAG `/query/data` for retrieval, then uses the plugin's selected chat model to generate the final answer. You can switch to direct LightRAG `/query/stream` generation in the LightRAG query settings.
86+
87+
LightRAG sync only sends Markdown files. New or changed files are submitted in batches of up to 3 texts per `/documents/texts` request. When a synced file changes, the plugin deletes the previously tracked LightRAG document and inserts the updated text because the current LightRAG API does not expose an in-place document update endpoint. The plugin listens for vault file events, scans once on startup, and can periodically rescan the vault by mtime to catch missed events. The LightRAG exclude list supports exact paths, folder prefixes such as `private/`, and simple glob rules such as `archive/**/*.md`; excluded files are not synced and previously synced excluded files are removed from LightRAG on the next scan. LightRAG chat defaults to `/query/data` retrieval plus plugin-model generation, strips generated inline References sections, and shows a limited de-duplicated source list based on retrieved chunks. The LightRAG settings tab refreshes pipeline and tracked document status automatically while it is open.
88+
7889
## Commands
7990

8091
- **Open Copilot chat**
@@ -90,6 +101,7 @@ Depending on your settings, it may send the following data to external services:
90101
- Chat messages and retrieved note context to your selected chat model provider.
91102
- Note chunks and user queries to your selected embedding provider.
92103
- Candidate retrieval results to your selected rerank provider, if reranking is enabled.
104+
- If LightRAG is enabled, Markdown file contents and vault-chat retrieval queries are sent to your configured LightRAG server. In the default answer mode, retrieved chunks are then sent to your selected chat provider so the plugin's model can generate the final answer. If you switch to direct LightRAG generation, LightRAG uses its own configured LLM for the final answer.
93105

94106
API keys are stored in Obsidian plugin settings data. Only configure providers you trust. If you use local vector storage, vectors are stored inside your vault. If you use pgvector, vectors and chunk text are stored in your configured PostgreSQL database.
95107

src/lightrag/LightRAGClient.ts

Lines changed: 210 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,210 @@
1+
import { LightRAGConfig } from '../types/settings';
2+
import { Logger } from '../logger/Logger';
3+
import {
4+
LightRAGDeleteResponse,
5+
LightRAGHealth,
6+
LightRAGInsertResponse,
7+
LightRAGPaginatedDocuments,
8+
LightRAGPipelineStatus,
9+
LightRAGQueryDataResponse,
10+
LightRAGQueryRequest,
11+
LightRAGReference,
12+
LightRAGStatusCounts,
13+
LightRAGTrackStatus,
14+
} from './types';
15+
16+
interface StreamCallbacks {
17+
onReferences?: (references: LightRAGReference[]) => void;
18+
onChunk: (text: string) => void;
19+
}
20+
21+
export class LightRAGClient {
22+
constructor(private config: LightRAGConfig) {}
23+
24+
updateConfig(config: LightRAGConfig): void {
25+
this.config = config;
26+
}
27+
28+
async health(): Promise<LightRAGHealth> {
29+
return this.requestJson<LightRAGHealth>('/health');
30+
}
31+
32+
async getPipelineStatus(): Promise<LightRAGPipelineStatus> {
33+
return this.requestJson<LightRAGPipelineStatus>('/documents/pipeline_status');
34+
}
35+
36+
async getStatusCounts(): Promise<LightRAGStatusCounts> {
37+
return this.requestJson<LightRAGStatusCounts>('/documents/status_counts');
38+
}
39+
40+
async getDocumentsPage(page = 1, pageSize = 50): Promise<LightRAGPaginatedDocuments> {
41+
return this.requestJson<LightRAGPaginatedDocuments>('/documents/paginated', {
42+
method: 'POST',
43+
body: JSON.stringify({
44+
page,
45+
page_size: pageSize,
46+
sort_field: 'updated_at',
47+
sort_direction: 'desc',
48+
}),
49+
});
50+
}
51+
52+
async insertTexts(texts: string[], fileSources: string[]): Promise<LightRAGInsertResponse> {
53+
return this.requestJson<LightRAGInsertResponse>('/documents/texts', {
54+
method: 'POST',
55+
body: JSON.stringify({
56+
texts,
57+
file_sources: fileSources,
58+
}),
59+
});
60+
}
61+
62+
async deleteDocuments(docIds: string[], deleteLlmCache = true): Promise<LightRAGDeleteResponse> {
63+
return this.requestJson<LightRAGDeleteResponse>('/documents/delete_document', {
64+
method: 'DELETE',
65+
body: JSON.stringify({
66+
doc_ids: docIds,
67+
delete_file: false,
68+
delete_llm_cache: deleteLlmCache,
69+
}),
70+
});
71+
}
72+
73+
async getTrackStatus(trackId: string): Promise<LightRAGTrackStatus> {
74+
return this.requestJson<LightRAGTrackStatus>(`/documents/track_status/${encodeURIComponent(trackId)}`);
75+
}
76+
77+
async queryData(request: LightRAGQueryRequest): Promise<LightRAGQueryDataResponse> {
78+
return this.requestJson<LightRAGQueryDataResponse>('/query/data', {
79+
method: 'POST',
80+
body: JSON.stringify(request),
81+
});
82+
}
83+
84+
async streamQuery(
85+
request: LightRAGQueryRequest,
86+
callbacks: StreamCallbacks,
87+
signal: AbortSignal
88+
): Promise<void> {
89+
const response = await fetch(this.buildUrl('/query/stream'), {
90+
method: 'POST',
91+
headers: this.headers(),
92+
body: JSON.stringify(request),
93+
signal,
94+
});
95+
if (!response.ok) {
96+
throw new Error(await this.formatError(response));
97+
}
98+
await this.parseNdjsonStream(response, callbacks, signal);
99+
}
100+
101+
private async requestJson<T>(path: string, init: RequestInit = {}): Promise<T> {
102+
const response = await fetch(this.buildUrl(path), {
103+
...init,
104+
headers: {
105+
...this.headers(),
106+
...(init.headers ?? {}),
107+
},
108+
});
109+
if (!response.ok) {
110+
throw new Error(await this.formatError(response));
111+
}
112+
const text = await response.text();
113+
if (!text.trim()) return {} as T;
114+
return JSON.parse(text) as T;
115+
}
116+
117+
private buildUrl(path: string): string {
118+
const baseUrl = this.config.baseUrl.trim().replace(/\/+$/, '');
119+
if (!baseUrl) throw new Error('LightRAG Server URL 未配置');
120+
const normalizedPath = path.replace(/^\/+/, '');
121+
const url = new URL(`${baseUrl}/${normalizedPath}`);
122+
const apiKey = this.config.apiKey.trim();
123+
if (apiKey) {
124+
url.searchParams.set('api_key_header_value', apiKey);
125+
}
126+
return url.toString();
127+
}
128+
129+
private headers(): Record<string, string> {
130+
return {
131+
'Content-Type': 'application/json',
132+
'Accept': 'application/json, application/x-ndjson',
133+
};
134+
}
135+
136+
private async formatError(response: Response): Promise<string> {
137+
const text = await response.text().catch(() => '');
138+
if (!text) return `LightRAG 请求失败: HTTP ${response.status}`;
139+
try {
140+
const json = JSON.parse(text) as { detail?: unknown; message?: unknown };
141+
return `LightRAG 请求失败: ${String(json.detail ?? json.message ?? text)}`;
142+
} catch {
143+
return `LightRAG 请求失败: HTTP ${response.status} ${text.slice(0, 300)}`;
144+
}
145+
}
146+
147+
private async parseNdjsonStream(
148+
response: Response,
149+
callbacks: StreamCallbacks,
150+
signal: AbortSignal
151+
): Promise<void> {
152+
const reader = response.body?.getReader();
153+
if (!reader) {
154+
const json = await response.json() as { response?: string; references?: LightRAGReference[]; error?: string };
155+
this.handleStreamObject(json, callbacks);
156+
return;
157+
}
158+
159+
const decoder = new TextDecoder();
160+
let buffer = '';
161+
let chunks = 0;
162+
const startedAt = performance.now();
163+
164+
try {
165+
while (true) {
166+
if (signal.aborted) return;
167+
const { done, value } = await reader.read();
168+
if (done) break;
169+
buffer += decoder.decode(value, { stream: true });
170+
const lines = buffer.split(/\r?\n/);
171+
buffer = lines.pop() ?? '';
172+
for (const line of lines) {
173+
const handled = this.handleStreamLine(line, callbacks);
174+
if (handled) chunks++;
175+
}
176+
}
177+
if (buffer.trim()) {
178+
const handled = this.handleStreamLine(buffer, callbacks);
179+
if (handled) chunks++;
180+
}
181+
} finally {
182+
reader.releaseLock();
183+
}
184+
185+
Logger.info(`[LightRAG] stream done: ${chunks} chunks, ${Math.round(performance.now() - startedAt)}ms`);
186+
}
187+
188+
private handleStreamLine(line: string, callbacks: StreamCallbacks): boolean {
189+
const text = line.trim();
190+
if (!text) return false;
191+
let json: { response?: string; references?: LightRAGReference[]; error?: string };
192+
try {
193+
json = JSON.parse(text) as { response?: string; references?: LightRAGReference[]; error?: string };
194+
} catch (err) {
195+
Logger.warn('[LightRAG] ignored malformed stream line', err);
196+
return false;
197+
}
198+
this.handleStreamObject(json, callbacks);
199+
return true;
200+
}
201+
202+
private handleStreamObject(
203+
json: { response?: string; references?: LightRAGReference[]; error?: string },
204+
callbacks: StreamCallbacks
205+
): void {
206+
if (json.error) throw new Error(json.error);
207+
if (Array.isArray(json.references)) callbacks.onReferences?.(json.references);
208+
if (typeof json.response === 'string') callbacks.onChunk(json.response);
209+
}
210+
}

0 commit comments

Comments
 (0)