基於 Rails 8 開發的 RAG (Retrieval-Augmented Generation) 應用程式,使用 RubyLLM 整合 OpenAI API,搭配 PostgreSQL pgvector 提供智慧文件檢索和對話功能。
本專案是一個教學與演示用途的 RAG 應用程式,主要目標為:
- 認識 Ruby 生態系支援:展示使用 Ruby on Rails 及相關套件,實作完整 RAG 流程的可行性。
- 探索社群熱門 LLM 工具:使用近期 Ruby 社群中討論度高的 RubyLLM 套件。
- 實踐核心概念:學習 RAG 的相關技術與流程(文件切塊、向量檢索、LLM 整合)。
考量教學時間與主題聚焦,本專案不包含以下內容:
- 對話歷史截斷:完整的對話歷史會隨著輪次增加一併送出,沒有 token budget 管理。長對話可能逼近模型的 context window 上限並增加 API 成本,正式產品需自行實作截斷策略(如滑動視窗、摘要壓縮等)
- Prompt Injection 的進階議題:本專案僅實作基礎 prompt,以展示 RAG 流程為主。Prompt Injection 安全性議題不在本次教學範疇內。
- UI/UX 設計:介面以基本功能展示為主,未進行使用者體驗改善。
- 部署設定:不涵蓋部署到雲端平台的相關設定與最佳實踐。
- 雲端儲存整合:目前僅使用 ActiveStorage 本地儲存,未配置 S3/Azure/GCS。
本專案適合作為學習 RAG 概念、Ruby LLM 整合的起點,若要用於正式產品,建議根據實際需求自行補強。
- 根據上傳的文件內容回答問題
- 即時串流回應
- 完整的對話歷史記錄
- 上傳 Markdown 文件到知識庫
- 支援多語言文件(中文、英文、日文、韓文等)
- 自動文件切塊 (Chunking) 和索引
- 支援批次上傳和索引
- 支援文件重新索引
- 文件狀態追蹤(建立中、已索引、錯誤等)
- 開發環境使用本地儲存
- 向量相似度搜尋
- 支援 MMR (Maximal Marginal Relevance) 多樣性檢索
- 兩階段搜尋策略:
- 系統自動搜尋
- LLM 工具輔助搜尋(Agentic RAG with Function Calling)
- 自動標記回答來源
- 顯示相關文件片段和相似度分數
- 文件階層路徑
- 可追溯回原始文件
- Ruby 3.3.11
- Rails 8.1.3
- PostgreSQL 15 + pgvector 擴充功能
- neighbor gem(pgvector 整合)
- Puma Web Server
- ActiveStorage(本地檔案儲存)
- RubyLLM - 核心 LLM 整合框架
- 統一的 LLM API 介面
- Function Calling 工具整合
- Streaming Response 即時回應
- Rails 深度整合
- 快速指令建立 AI 聊天介面
- OpenAI API
- 預設 GPT-4o-mini(對話生成)
- 預設 text-embedding-3-small(文件向量化)
- 支援 Function Calling
- tiktoken_ruby - Token 計算與管理
- pragmatic_segmenter - 多語言句子邊界切分
- Hotwire (Turbo + Stimulus)
- Tailwind CSS 4
- ImportMap (JavaScript 管理)
使用 Rails 8 的 Solid 家族,所有狀態共用同一個 PostgreSQL 資料庫:
- Solid Queue(工作佇列):已啟用。排程分為
high_priority(聊天回應、Turbo broadcast)與default(文件索引)兩個 queue,確保大量索引時聊天仍保持流暢。 - Solid Cable(即時通訊):已啟用。WebSocket 廣播透過資料庫轉發,支援較大的訊息 payload。
- Solid Cache(快取):尚未啟用,目前沿用 Rails 預設的記憶體快取。
- Docker
- VS Code + Dev Containers 擴充功能
專案使用 Docker Compose 管理以下服務:
- 容器名稱:
ruby-on-rag-development - Port:
33000→ 容器內3000 - 說明:主要的 Rails 應用程式
- 容器名稱:
ruby-on-rag-postgres - 映像檔:
pgvector/pgvector:pg15 - Port:
35432→ 容器內5432 - 說明:PostgreSQL 資料庫,已啟用 pgvector 擴充功能
- 自動建立的資料庫:由
POSTGRES_DB決定,預設為ruby_on_rag
- 容器名稱:
ruby-on-rag-pgadmin - 映像檔:
dpage/pgadmin4:latest - Port:
30080→ 容器內80 - 網址:http://localhost:30080
- 登入資訊:請查看
.env中的PGADMIN_DEFAULT_EMAIL和PGADMIN_DEFAULT_PASSWORD - 自動配置:PostgreSQL 連線設定會在啟動時自動載入,登入後第一次連線需輸入資料庫密碼(
POSTGRES_PASSWORD)
注意:
pgadmin-servers.json的MaintenanceDB欄位設定為ruby_on_rag。若你修改了.env中的POSTGRES_DB,請同步更新此檔案,或是在 pgAdmin 頁面自行設定資料庫連線。
cp .env.example .env接著編輯 .env:
# 必填:設定 OpenAI API Key
OPENAI_API_KEY=sk-...
# 視系統調整 UID/GID
# macOS 通常是 501/20,Linux 通常是 1000/1000 (查詢方式:執行 `id -u` 和 `id -g`)
# windows 通常保持 1000/1000 即可
USER_UID=1000
USER_GID=1000# 用 VS Code 開啟專案
code .VS Code 通常會在右下角跳出「Reopen in Container」提示,點擊即可進入 Dev Container。
若提示沒有出現,可以手動觸發:按 Cmd+Shift+P(Windows/Linux 為 Ctrl+Shift+P)開啟指令面板,輸入 Reopen in Container 後按 Enter。
Dev Container 會自動:
- 啟動 PostgreSQL 資料庫(含 pgvector 擴充)
- 自動建立資料庫(名稱由
POSTGRES_DB決定) - 啟動 pgAdmin(已預先載入連線設定)
Dev Container 啟動後,在終端機執行:
bin/setup如果只需要啟動伺服器:直接執行
bin/dev即可。
- 應用程式:http://localhost:33000
- pgAdmin:http://localhost:30080
- 登入資訊請查看
.env(PGADMIN_DEFAULT_EMAIL/PGADMIN_DEFAULT_PASSWORD) - 登入後點擊左側展開 Servers,可以看到「ruby_on_rag_postgres」伺服器,第一次連線會跳出需輸入密碼
POSTGRES_PASSWORD的對話框
- 登入資訊請查看
若需要重建開發環境,且資料庫內的資料可以捨棄,建議先執行以下指令停止並移除所有服務、並刪除 volume: 如此,重新啟動才會觸發資料庫建立、初始連線等初始化流程。
docker compose down -v再確認 volume 已完全移除:
docker volume ls| 參數 | 預設值 | 說明 |
|---|---|---|
OPENAI_API_KEY |
(必填) | OpenAI API 金鑰 |
DEFAULT_DOCUMENT_LANGUAGE |
zh | 文件預設語言(zh, en, ja, ko, es, fr, de) |
CHUNK_SIZE |
1500 | 每個文件切塊的最大 token 數 |
CHUNK_OVERLAP |
64 | 相鄰 chunk 之間的重疊 token 數 |
TOP_K_RESULTS |
7 | 檢索回傳的結果數量 |
SIMILARITY_THRESHOLD |
0.4 | 相似度門檻(越高越嚴格) |
TOOL_TRIGGER_THRESHOLD |
0.6 | 觸發 LLM 工具搜尋的門檻 |
詳細的參數說明、調整建議和使用情境請參考 .env.example。
-
使用者提問
-
初始向量檢索
- 將問題轉換成 embedding
- 在 pgvector 中搜尋相似文件片段
- 回傳 TOP_K_RESULTS 個最相關結果
-
品質評估
- 檢查最高相似度分數
- 如果完全沒找到結果:進入步驟 4
- 如果分數 < TOOL_TRIGGER_THRESHOLD:進入步驟 4(附帶初步結果)
- 如果分數 ≥ TOOL_TRIGGER_THRESHOLD:跳到步驟 5
-
LLM 工具輔助搜尋(Agentic RAG)(可選)
- 使用 Function Calling 機制,讓 LLM 主動呼叫搜尋工具
- LLM 分析初始結果品質並決定是否需要額外搜尋
- 生成改良的搜尋關鍵字(Query Rewriting)
- 約束執行額外搜尋次數
- 合併所有搜尋結果
-
生成回答
- LLM 根據檢索到的文件內容回答
- 嚴格限制:只能使用知識庫內容,不使用訓練資料
- 標記來源文件
-
顯示結果
- 串流輸出回答(使用 Turbo Streams)
- 顯示參考來源和相似度分數
- 根據餘弦相似度排序
- 速度相較快
- 可能回傳語義相似但內容重複的結果
- 平衡相關性與多樣性
- 減少重複內容
- 計算量較大
- 透過
USE_MMR=true啟用
上半場介紹的 NLP 概念在本專案的對應位置:
| 概念 | 程式碼位置 | 說明 |
|---|---|---|
| 文件切塊(Chunking) | app/models/concerns/document/chunkable.rb |
Markdown 結構解析、段落切分、滑動視窗、token 計算 |
| 向量化(Embedding) | app/models/concerns/document/embeddable.rb |
呼叫 OpenAI embedding API,批次生成向量 |
| 索引流程(Indexing) | app/models/concerns/document/indexable.rb |
組合 chunkable + embeddable,將 chunk 存入 DB |
| 向量檢索(Retrieval) | app/models/concerns/document_chunk/searchable.rb |
餘弦相似度搜尋、MMR 多樣性篩選、結果格式化 |
| LLM 工具搜尋(Function Calling) | app/tools/document_search.rb |
讓 LLM 主動呼叫的搜尋工具定義 |
| 完整問答流程 | app/jobs/chat_with_doc_response_job.rb |
串起向量檢索 → 品質評估 → 工具搜尋 → 生成回答的完整流程 |
.
├── .devcontainer/ # VS Code Dev Container 設定
│ ├── devcontainer.json
│ └── Dockerfile
├── app/
│ ├── agents/ # RubyLLM Agent
│ │ └── chat_with_doc_agent.rb
│ ├── controllers/ # 控制器
│ ├── jobs/ # 背景任務
│ │ ├── chat_response_job.rb # RubyLLM 預設模式參考實作(未啟用)
│ │ ├── chat_with_doc_response_job.rb
│ │ └── index_document_job.rb
│ ├── models/ # 資料模型
│ │ ├── concerns/
│ │ │ ├── document/
│ │ │ │ ├── chunkable.rb # 文件切塊邏輯
│ │ │ │ ├── embeddable.rb # Embedding 向量化邏輯
│ │ │ │ └── indexable.rb # 索引流程協調(組合 chunkable + embeddable)
│ │ │ └── document_chunk/
│ │ │ └── searchable.rb # 向量檢索、結果格式化邏輯
│ │ ├── chat.rb
│ │ ├── document.rb
│ │ ├── document_chunk.rb
│ │ ├── message.rb
│ │ ├── model.rb # RubyLLM 模型清單快取
│ │ └── tool_call.rb # Function Calling 工具呼叫記錄
│ ├── prompts/ # LLM Prompt 模板
│ │ └── chat_with_doc_agent/
│ │ └── instructions.txt.erb # Agent 系統提示詞
│ ├── tools/
│ │ └── document_search.rb # LLM Function Calling 工具
│ └── views/ # 視圖
├── config/
│ ├── initializers/
│ │ ├── rag_config.rb # 載入 rag.yml 與 openai.yml 設定
│ │ └── ruby_llm.rb # LLM 模型設定
│ ├── database.yml
│ ├── openai.yml # OpenAI 模型與 API 設定
│ ├── queue.yml
│ ├── rag.yml # RAG 參數設定(chunk size、top-k、相似度門檻等)
│ ├── storage.yml
│ └── cable.yml
├── db/
│ ├── migrate/
│ ├── schema.rb
│ ├── queue_schema.rb # Solid Queue schema
│ ├── cable_schema.rb # Solid Cable schema
│ └── cache_schema.rb # Solid Cache schema(備用)
├── storage/ # ActiveStorage 本地儲存目錄
├── docker-compose.yml # Dev Container 服務定義(application / postgres / pgAdmin)
├── pgadmin-servers.json # pgAdmin 連線預設設定
├── .env.example # 環境變數範例
├── Gemfile
└── README.md
MIT License
A RAG (Retrieval-Augmented Generation) application built with Rails 8, using RubyLLM to integrate the OpenAI API with PostgreSQL pgvector for intelligent document retrieval and conversational Q&A.
This project is a teaching and demonstration RAG application. Its main goals are:
- Showcase Ruby ecosystem support: demonstrate that a complete RAG pipeline is achievable with Ruby on Rails and related gems.
- Explore a trending Ruby LLM library: use RubyLLM, which has been gaining traction in the Ruby community.
- Practice core RAG concepts: learn about the techniques and workflow of document chunking, vector retrieval, and LLM integration.
Given the educational focus, the following are intentionally not included:
- Conversation history truncation: the full history is sent on every turn with no token-budget management. Long conversations may approach the model's context window limit and increase API costs. Production systems should implement truncation strategies (sliding window, summarization compression, etc.).
- Advanced Prompt Injection topics: only a basic prompt is implemented to illustrate the RAG flow. Prompt Injection security is out of scope for this tutorial.
- UI/UX design: the interface is minimal and focused on feature demonstration.
- Deployment configuration: cloud deployment setup and best practices are not covered.
- Cloud storage integration: only ActiveStorage local storage is configured; S3/Azure/GCS are not set up.
This project is a good starting point for learning RAG concepts and Ruby LLM integration. For production use, supplement it according to your requirements.
- Answer questions based on uploaded document content
- Real-time streaming responses
- Full conversation history
- Upload Markdown documents to the knowledge base
- Multilingual support (Chinese, English, Japanese, Korean, etc.)
- Automatic document chunking and indexing
- Batch upload and indexing
- Document re-indexing
- Document status tracking (creating, indexed, error, etc.)
- Local storage for development
- Vector similarity search (powered by pgvector)
- MMR (Maximal Marginal Relevance) diversity retrieval
- Two-stage search strategy:
- Automatic vector search
- LLM tool-assisted search (Agentic RAG with Function Calling)
- Automatic source attribution for answers
- Display of relevant document snippets and similarity scores
- Document hierarchy path
- Traceable back to the original document
- Ruby 3.3.11
- Rails 8.1.3
- PostgreSQL 15 + pgvector extension
- neighbor gem (pgvector integration)
- Puma Web Server
- ActiveStorage (local file storage)
- RubyLLM — core LLM integration framework
- Unified LLM API interface
- Function Calling / tool integration
- Streaming responses
- Deep Rails integration
- Quick scaffolding for AI chat interfaces
- OpenAI API
- GPT-4o-mini by default (chat generation)
- text-embedding-3-small by default (document vectorization)
- Function Calling support
- tiktoken_ruby — token counting and management
- pragmatic_segmenter — multilingual sentence boundary segmentation
- Hotwire (Turbo + Stimulus)
- Tailwind CSS 4
- ImportMap (JavaScript management)
Uses Rails 8's Solid family — all state shares the same PostgreSQL database:
- Solid Queue (job queue): enabled. Two queues:
high_priority(chat responses, Turbo broadcasts) anddefault(document indexing), keeping chat responsive during heavy indexing. - Solid Cable (real-time): enabled. WebSocket broadcasts relay through the database, supporting larger message payloads.
- Solid Cache (caching): not yet enabled; using Rails default in-memory cache.
- Docker
- VS Code + Dev Containers extension
The project uses Docker Compose to manage the following services:
- Container name:
ruby-on-rag-development - Port:
33000→ container3000
- Container name:
ruby-on-rag-postgres - Image:
pgvector/pgvector:pg15 - Port:
35432→ container5432 - Database created automatically: set by
POSTGRES_DB, defaults toruby_on_rag
- Container name:
ruby-on-rag-pgadmin - Image:
dpage/pgadmin4:latest - Port:
30080→ container80 - URL: http://localhost:30080
- Credentials: see
PGADMIN_DEFAULT_EMAILandPGADMIN_DEFAULT_PASSWORDin.env - Auto-configured: PostgreSQL connection settings are loaded on startup; the first connection requires the
POSTGRES_PASSWORD.
Note:
pgadmin-servers.jsonsetsMaintenanceDBtoruby_on_rag. If you changePOSTGRES_DBin.env, update this file accordingly, or configure the connection manually in pgAdmin.
cp .env.example .envThen edit .env:
# Required: set your OpenAI API Key
OPENAI_API_KEY=sk-...
# Adjust UID/GID to match your system
# macOS is typically 501/20, Linux is typically 1000/1000 (run `id -u` and `id -g` to check)
# Windows can usually stay at 1000/1000
USER_UID=1000
USER_GID=1000code .VS Code usually shows a "Reopen in Container" prompt in the bottom-right corner. Click it to enter the Dev Container.
If the prompt does not appear, trigger it manually: press Cmd+Shift+P (or Ctrl+Shift+P on Windows/Linux), type Reopen in Container, and press Enter.
The Dev Container will automatically:
- Start PostgreSQL (with pgvector extension)
- Create the database (name set by
POSTGRES_DB) - Start pgAdmin (with the connection pre-loaded)
Once inside the Dev Container, run:
bin/setupTo start the server only: run
bin/devdirectly.
- Application: http://localhost:33000
- Chat: http://localhost:33000/chats
- Document management: http://localhost:33000/documents
- pgAdmin: http://localhost:30080
- Credentials are in
.env(PGADMIN_DEFAULT_EMAIL/PGADMIN_DEFAULT_PASSWORD) - After login, expand Servers on the left to find
ruby_on_rag_postgres; the first connection will prompt forPOSTGRES_PASSWORD
- Credentials are in
If you need to rebuild the dev environment and can discard all data, stop and remove all services and volumes:
docker compose down -vThen confirm the volumes are gone:
docker volume ls| Variable | Default | Description |
|---|---|---|
OPENAI_API_KEY |
(required) | OpenAI API key |
DEFAULT_DOCUMENT_LANGUAGE |
zh | Default document language (zh, en, ja, ko, es, fr, de) |
CHUNK_SIZE |
1500 | Maximum tokens per document chunk |
CHUNK_OVERLAP |
64 | Overlapping tokens between adjacent chunks |
TOP_K_RESULTS |
7 | Number of retrieval results to return |
SIMILARITY_THRESHOLD |
0.4 | Similarity threshold (higher = stricter) |
TOOL_TRIGGER_THRESHOLD |
0.6 | Threshold for triggering LLM tool-assisted search |
For detailed parameter explanations, tuning advice, and usage scenarios, see .env.example.
-
User submits a question
-
Initial vector retrieval
- Convert the question to an embedding
- Search pgvector for similar document chunks
- Return the top
TOP_K_RESULTSresults
-
Quality assessment
- Check the highest similarity score
- No results found → go to step 4
- Score <
TOOL_TRIGGER_THRESHOLD→ go to step 4 (with preliminary results) - Score ≥
TOOL_TRIGGER_THRESHOLD→ skip to step 5
-
LLM tool-assisted search (Agentic RAG) (optional)
- Uses Function Calling to let the LLM actively invoke the search tool
- LLM evaluates initial result quality and decides whether additional search is needed
- Generates improved search keywords (Query Rewriting)
- Bounded number of additional search iterations
- Merges all search results
-
Generate answer
- LLM answers based strictly on retrieved document content
- Hard constraint: only knowledge-base content is used; training data is excluded
- Sources are annotated
-
Display result
- Stream the answer (via Turbo Streams)
- Show reference sources and similarity scores
- Ranked by cosine similarity
- Relatively fast
- May return semantically similar but repetitive results
- Balances relevance and diversity
- Reduces duplicate content
- Higher computational cost
- Enable with
USE_MMR=true
The NLP concepts from the first half of the session map to these files:
| Concept | File | Description |
|---|---|---|
| Chunking | app/models/concerns/document/chunkable.rb |
Markdown structure parsing, paragraph splitting, sliding window, token counting |
| Embedding | app/models/concerns/document/embeddable.rb |
Calls the OpenAI embedding API and generates vectors in batches |
| Indexing pipeline | app/models/concerns/document/indexable.rb |
Orchestrates chunkable + embeddable and persists chunks to the DB |
| Vector retrieval | app/models/concerns/document_chunk/searchable.rb |
Cosine similarity search, MMR diversity filtering, result formatting |
| LLM tool search (Function Calling) | app/tools/document_search.rb |
Search tool definition invoked by the LLM via Function Calling |
| Full Q&A flow | app/jobs/chat_with_doc_response_job.rb |
Wires together retrieval → quality check → tool search → answer generation |
.
├── .devcontainer/ # VS Code Dev Container config
│ ├── devcontainer.json
│ └── Dockerfile
├── app/
│ ├── agents/ # RubyLLM Agent
│ │ └── chat_with_doc_agent.rb
│ ├── controllers/
│ ├── jobs/
│ │ ├── chat_response_job.rb # RubyLLM default mode reference (disabled)
│ │ ├── chat_with_doc_response_job.rb
│ │ └── index_document_job.rb
│ ├── models/
│ │ ├── concerns/
│ │ │ ├── document/
│ │ │ │ ├── chunkable.rb # chunking logic
│ │ │ │ ├── embeddable.rb # embedding / vectorization logic
│ │ │ │ └── indexable.rb # indexing orchestration (chunkable + embeddable)
│ │ │ └── document_chunk/
│ │ │ └── searchable.rb # vector retrieval and result formatting
│ │ ├── chat.rb
│ │ ├── document.rb
│ │ ├── document_chunk.rb
│ │ ├── message.rb
│ │ ├── model.rb # RubyLLM model list cache
│ │ └── tool_call.rb # Function Calling record
│ ├── prompts/
│ │ └── chat_with_doc_agent/
│ │ └── instructions.txt.erb # Agent system prompt
│ ├── tools/
│ │ └── document_search.rb # LLM Function Calling tool
│ └── views/
├── config/
│ ├── initializers/
│ │ ├── rag_config.rb # loads rag.yml and openai.yml
│ │ └── ruby_llm.rb # LLM model config
│ ├── database.yml
│ ├── openai.yml # OpenAI model and API config
│ ├── queue.yml
│ ├── rag.yml # RAG parameters (chunk size, top-k, thresholds, etc.)
│ ├── storage.yml
│ └── cable.yml
├── db/
│ ├── migrate/
│ ├── schema.rb
│ ├── queue_schema.rb # Solid Queue schema
│ ├── cable_schema.rb # Solid Cable schema
│ └── cache_schema.rb # Solid Cache schema (reserved)
├── storage/ # ActiveStorage local storage directory
├── docker-compose.yml # Dev Container services (application / postgres / pgAdmin)
├── pgadmin-servers.json # pgAdmin default connection config
├── .env.example # Environment variable template
├── Gemfile
└── README.md
MIT License