Skip to content

rubytaiwan/ruby-on-rag-workshop

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Ruby on RAG - AI 知識庫問答應用程式

English version below

基於 Rails 8 開發的 RAG (Retrieval-Augmented Generation) 應用程式,使用 RubyLLM 整合 OpenAI API,搭配 PostgreSQL pgvector 提供智慧文件檢索和對話功能。

目錄


關於本專案

本專案是一個教學與演示用途的 RAG 應用程式,主要目標為:

專案目標

  • 認識 Ruby 生態系支援:展示使用 Ruby on Rails 及相關套件,實作完整 RAG 流程的可行性。
  • 探索社群熱門 LLM 工具:使用近期 Ruby 社群中討論度高的 RubyLLM 套件。
  • 實踐核心概念:學習 RAG 的相關技術與流程(文件切塊、向量檢索、LLM 整合)。

專案範圍說明

考量教學時間與主題聚焦,本專案不包含以下內容:

  • 對話歷史截斷:完整的對話歷史會隨著輪次增加一併送出,沒有 token budget 管理。長對話可能逼近模型的 context window 上限並增加 API 成本,正式產品需自行實作截斷策略(如滑動視窗、摘要壓縮等)
  • Prompt Injection 的進階議題:本專案僅實作基礎 prompt,以展示 RAG 流程為主。Prompt Injection 安全性議題不在本次教學範疇內。
  • UI/UX 設計:介面以基本功能展示為主,未進行使用者體驗改善。
  • 部署設定:不涵蓋部署到雲端平台的相關設定與最佳實踐。
  • 雲端儲存整合:目前僅使用 ActiveStorage 本地儲存,未配置 S3/Azure/GCS。

本專案適合作為學習 RAG 概念、Ruby LLM 整合的起點,若要用於正式產品,建議根據實際需求自行補強。


主要功能

智慧對話

  • 根據上傳的文件內容回答問題
  • 即時串流回應
  • 完整的對話歷史記錄

文件管理

  • 上傳 Markdown 文件到知識庫
  • 支援多語言文件(中文、英文、日文、韓文等)
  • 自動文件切塊 (Chunking) 和索引
  • 支援批次上傳和索引
  • 支援文件重新索引
  • 文件狀態追蹤(建立中、已索引、錯誤等)
  • 開發環境使用本地儲存

文本檢索

  • 向量相似度搜尋
  • 支援 MMR (Maximal Marginal Relevance) 多樣性檢索
  • 兩階段搜尋策略:
    1. 系統自動搜尋
    2. LLM 工具輔助搜尋(Agentic RAG with Function Calling)

來源追蹤

  • 自動標記回答來源
  • 顯示相關文件片段和相似度分數
  • 文件階層路徑
  • 可追溯回原始文件

使用到的技術

後端框架

  • Ruby 3.3.11
  • Rails 8.1.3
  • PostgreSQL 15 + pgvector 擴充功能
  • neighbor gem(pgvector 整合)
  • Puma Web Server
  • ActiveStorage(本地檔案儲存)

NLP / LLM

  • RubyLLM - 核心 LLM 整合框架
    • 統一的 LLM API 介面
    • Function Calling 工具整合
    • Streaming Response 即時回應
    • Rails 深度整合
    • 快速指令建立 AI 聊天介面
  • OpenAI API
    • 預設 GPT-4o-mini(對話生成)
    • 預設 text-embedding-3-small(文件向量化)
    • 支援 Function Calling
  • tiktoken_ruby - Token 計算與管理
  • pragmatic_segmenter - 多語言句子邊界切分

前端

  • Hotwire (Turbo + Stimulus)
  • Tailwind CSS 4
  • ImportMap (JavaScript 管理)

背景任務

使用 Rails 8 的 Solid 家族,所有狀態共用同一個 PostgreSQL 資料庫:

  • Solid Queue(工作佇列):已啟用。排程分為 high_priority(聊天回應、Turbo broadcast)與 default(文件索引)兩個 queue,確保大量索引時聊天仍保持流暢。
  • Solid Cable(即時通訊):已啟用。WebSocket 廣播透過資料庫轉發,支援較大的訊息 payload。
  • Solid Cache(快取):尚未啟用,目前沿用 Rails 預設的記憶體快取。

開發環境設置

前置需求

  • Docker
  • VS Code + Dev Containers 擴充功能

服務架構

專案使用 Docker Compose 管理以下服務:

1. Application (Ruby on Rails)

  • 容器名稱ruby-on-rag-development
  • Port33000 → 容器內 3000
  • 說明:主要的 Rails 應用程式

2. PostgreSQL (with pgvector)

  • 容器名稱ruby-on-rag-postgres
  • 映像檔pgvector/pgvector:pg15
  • Port35432 → 容器內 5432
  • 說明:PostgreSQL 資料庫,已啟用 pgvector 擴充功能
  • 自動建立的資料庫:由 POSTGRES_DB 決定,預設為 ruby_on_rag

3. pgAdmin

  • 容器名稱ruby-on-rag-pgadmin
  • 映像檔dpage/pgadmin4:latest
  • Port30080 → 容器內 80
  • 網址http://localhost:30080
  • 登入資訊:請查看 .env 中的 PGADMIN_DEFAULT_EMAILPGADMIN_DEFAULT_PASSWORD
  • 自動配置:PostgreSQL 連線設定會在啟動時自動載入,登入後第一次連線需輸入資料庫密碼(POSTGRES_PASSWORD

注意pgadmin-servers.jsonMaintenanceDB 欄位設定為 ruby_on_rag。若你修改了 .env 中的 POSTGRES_DB,請同步更新此檔案,或是在 pgAdmin 頁面自行設定資料庫連線。

快速開始

1. 複製設定檔

cp .env.example .env

接著編輯 .env

# 必填:設定 OpenAI API Key
OPENAI_API_KEY=sk-...

# 視系統調整 UID/GID
# macOS 通常是 501/20,Linux 通常是 1000/1000 (查詢方式:執行 `id -u` 和 `id -g`)
# windows 通常保持 1000/1000 即可
USER_UID=1000
USER_GID=1000

2. 在 VS Code 中開啟

# 用 VS Code 開啟專案
code .

VS Code 通常會在右下角跳出「Reopen in Container」提示,點擊即可進入 Dev Container。

若提示沒有出現,可以手動觸發:按 Cmd+Shift+P(Windows/Linux 為 Ctrl+Shift+P)開啟指令面板,輸入 Reopen in Container 後按 Enter。

Dev Container 會自動:

  • 啟動 PostgreSQL 資料庫(含 pgvector 擴充)
  • 自動建立資料庫(名稱由 POSTGRES_DB 決定)
  • 啟動 pgAdmin(已預先載入連線設定)

3. 初始化並啟動

Dev Container 啟動後,在終端機執行:

bin/setup

如果只需要啟動伺服器:直接執行 bin/dev 即可。

各步驟的細節請參考 bin/setupbin/dev

4. 存取應用程式

如果需要重置環境

若需要重建開發環境,且資料庫內的資料可以捨棄,建議先執行以下指令停止並移除所有服務、並刪除 volume: 如此,重新啟動才會觸發資料庫建立、初始連線等初始化流程。

docker compose down -v

再確認 volume 已完全移除:

docker volume ls

環境變數說明

參數 預設值 說明
OPENAI_API_KEY (必填) OpenAI API 金鑰
DEFAULT_DOCUMENT_LANGUAGE zh 文件預設語言(zh, en, ja, ko, es, fr, de)
CHUNK_SIZE 1500 每個文件切塊的最大 token 數
CHUNK_OVERLAP 64 相鄰 chunk 之間的重疊 token 數
TOP_K_RESULTS 7 檢索回傳的結果數量
SIMILARITY_THRESHOLD 0.4 相似度門檻(越高越嚴格)
TOOL_TRIGGER_THRESHOLD 0.6 觸發 LLM 工具搜尋的門檻

詳細的參數說明、調整建議和使用情境請參考 .env.example


本專案 RAG 基本流程

問答流程

  1. 使用者提問

  2. 初始向量檢索

    • 將問題轉換成 embedding
    • 在 pgvector 中搜尋相似文件片段
    • 回傳 TOP_K_RESULTS 個最相關結果
  3. 品質評估

    • 檢查最高相似度分數
    • 如果完全沒找到結果:進入步驟 4
    • 如果分數 < TOOL_TRIGGER_THRESHOLD:進入步驟 4(附帶初步結果)
    • 如果分數 ≥ TOOL_TRIGGER_THRESHOLD:跳到步驟 5
  4. LLM 工具輔助搜尋(Agentic RAG)(可選)

    • 使用 Function Calling 機制,讓 LLM 主動呼叫搜尋工具
    • LLM 分析初始結果品質並決定是否需要額外搜尋
    • 生成改良的搜尋關鍵字(Query Rewriting)
    • 約束執行額外搜尋次數
    • 合併所有搜尋結果
  5. 生成回答

    • LLM 根據檢索到的文件內容回答
    • 嚴格限制:只能使用知識庫內容,不使用訓練資料
    • 標記來源文件
  6. 顯示結果

    • 串流輸出回答(使用 Turbo Streams)
    • 顯示參考來源和相似度分數

搜尋策略

單純向量搜尋(預設)

  • 根據餘弦相似度排序
  • 速度相較快
  • 可能回傳語義相似但內容重複的結果

MMR 多樣性搜尋(選用)

  • 平衡相關性與多樣性
  • 減少重複內容
  • 計算量較大
  • 透過 USE_MMR=true 啟用

程式碼對應

上半場介紹的 NLP 概念在本專案的對應位置:

概念 程式碼位置 說明
文件切塊(Chunking) app/models/concerns/document/chunkable.rb Markdown 結構解析、段落切分、滑動視窗、token 計算
向量化(Embedding) app/models/concerns/document/embeddable.rb 呼叫 OpenAI embedding API,批次生成向量
索引流程(Indexing) app/models/concerns/document/indexable.rb 組合 chunkable + embeddable,將 chunk 存入 DB
向量檢索(Retrieval) app/models/concerns/document_chunk/searchable.rb 餘弦相似度搜尋、MMR 多樣性篩選、結果格式化
LLM 工具搜尋(Function Calling) app/tools/document_search.rb 讓 LLM 主動呼叫的搜尋工具定義
完整問答流程 app/jobs/chat_with_doc_response_job.rb 串起向量檢索 → 品質評估 → 工具搜尋 → 生成回答的完整流程

專案結構

.
├── .devcontainer/            # VS Code Dev Container 設定
│   ├── devcontainer.json
│   └── Dockerfile
├── app/
│   ├── agents/               # RubyLLM Agent
│   │   └── chat_with_doc_agent.rb
│   ├── controllers/          # 控制器
│   ├── jobs/                 # 背景任務
│   │   ├── chat_response_job.rb          # RubyLLM 預設模式參考實作(未啟用)
│   │   ├── chat_with_doc_response_job.rb
│   │   └── index_document_job.rb
│   ├── models/               # 資料模型
│   │   ├── concerns/
│   │   │   ├── document/
│   │   │   │   ├── chunkable.rb      # 文件切塊邏輯
│   │   │   │   ├── embeddable.rb     # Embedding 向量化邏輯
│   │   │   │   └── indexable.rb      # 索引流程協調(組合 chunkable + embeddable)
│   │   │   └── document_chunk/
│   │   │       └── searchable.rb     # 向量檢索、結果格式化邏輯
│   │   ├── chat.rb
│   │   ├── document.rb
│   │   ├── document_chunk.rb
│   │   ├── message.rb
│   │   ├── model.rb                  # RubyLLM 模型清單快取
│   │   └── tool_call.rb              # Function Calling 工具呼叫記錄
│   ├── prompts/              # LLM Prompt 模板
│   │   └── chat_with_doc_agent/
│   │       └── instructions.txt.erb  # Agent 系統提示詞
│   ├── tools/
│   │   └── document_search.rb        # LLM Function Calling 工具
│   └── views/                # 視圖
├── config/
│   ├── initializers/
│   │   ├── rag_config.rb             # 載入 rag.yml 與 openai.yml 設定
│   │   └── ruby_llm.rb               # LLM 模型設定
│   ├── database.yml
│   ├── openai.yml            # OpenAI 模型與 API 設定
│   ├── queue.yml
│   ├── rag.yml               # RAG 參數設定(chunk size、top-k、相似度門檻等)
│   ├── storage.yml
│   └── cable.yml
├── db/
│   ├── migrate/
│   ├── schema.rb
│   ├── queue_schema.rb       # Solid Queue schema
│   ├── cable_schema.rb       # Solid Cable schema
│   └── cache_schema.rb       # Solid Cache schema(備用)
├── storage/                  # ActiveStorage 本地儲存目錄
├── docker-compose.yml        # Dev Container 服務定義(application / postgres / pgAdmin)
├── pgadmin-servers.json      # pgAdmin 連線預設設定
├── .env.example              # 環境變數範例
├── Gemfile
└── README.md

授權

MIT License


English Version

回到中文版

A RAG (Retrieval-Augmented Generation) application built with Rails 8, using RubyLLM to integrate the OpenAI API with PostgreSQL pgvector for intelligent document retrieval and conversational Q&A.

Table of Contents (EN)


About

This project is a teaching and demonstration RAG application. Its main goals are:

Goals

  • Showcase Ruby ecosystem support: demonstrate that a complete RAG pipeline is achievable with Ruby on Rails and related gems.
  • Explore a trending Ruby LLM library: use RubyLLM, which has been gaining traction in the Ruby community.
  • Practice core RAG concepts: learn about the techniques and workflow of document chunking, vector retrieval, and LLM integration.

Out of Scope

Given the educational focus, the following are intentionally not included:

  • Conversation history truncation: the full history is sent on every turn with no token-budget management. Long conversations may approach the model's context window limit and increase API costs. Production systems should implement truncation strategies (sliding window, summarization compression, etc.).
  • Advanced Prompt Injection topics: only a basic prompt is implemented to illustrate the RAG flow. Prompt Injection security is out of scope for this tutorial.
  • UI/UX design: the interface is minimal and focused on feature demonstration.
  • Deployment configuration: cloud deployment setup and best practices are not covered.
  • Cloud storage integration: only ActiveStorage local storage is configured; S3/Azure/GCS are not set up.

This project is a good starting point for learning RAG concepts and Ruby LLM integration. For production use, supplement it according to your requirements.


Features

Smart Chat

  • Answer questions based on uploaded document content
  • Real-time streaming responses
  • Full conversation history

Document Management

  • Upload Markdown documents to the knowledge base
  • Multilingual support (Chinese, English, Japanese, Korean, etc.)
  • Automatic document chunking and indexing
  • Batch upload and indexing
  • Document re-indexing
  • Document status tracking (creating, indexed, error, etc.)
  • Local storage for development

Text Retrieval

  • Vector similarity search (powered by pgvector)
  • MMR (Maximal Marginal Relevance) diversity retrieval
  • Two-stage search strategy:
    1. Automatic vector search
    2. LLM tool-assisted search (Agentic RAG with Function Calling)

Source Tracking

  • Automatic source attribution for answers
  • Display of relevant document snippets and similarity scores
  • Document hierarchy path
  • Traceable back to the original document

Tech Stack

Backend

  • Ruby 3.3.11
  • Rails 8.1.3
  • PostgreSQL 15 + pgvector extension
  • neighbor gem (pgvector integration)
  • Puma Web Server
  • ActiveStorage (local file storage)

NLP / LLM

  • RubyLLM — core LLM integration framework
    • Unified LLM API interface
    • Function Calling / tool integration
    • Streaming responses
    • Deep Rails integration
    • Quick scaffolding for AI chat interfaces
  • OpenAI API
    • GPT-4o-mini by default (chat generation)
    • text-embedding-3-small by default (document vectorization)
    • Function Calling support
  • tiktoken_ruby — token counting and management
  • pragmatic_segmenter — multilingual sentence boundary segmentation

Frontend

  • Hotwire (Turbo + Stimulus)
  • Tailwind CSS 4
  • ImportMap (JavaScript management)

Background Jobs

Uses Rails 8's Solid family — all state shares the same PostgreSQL database:

  • Solid Queue (job queue): enabled. Two queues: high_priority (chat responses, Turbo broadcasts) and default (document indexing), keeping chat responsive during heavy indexing.
  • Solid Cable (real-time): enabled. WebSocket broadcasts relay through the database, supporting larger message payloads.
  • Solid Cache (caching): not yet enabled; using Rails default in-memory cache.

Development Setup

Prerequisites

  • Docker
  • VS Code + Dev Containers extension

Service Architecture

The project uses Docker Compose to manage the following services:

1. Application (Ruby on Rails)
  • Container name: ruby-on-rag-development
  • Port: 33000 → container 3000
2. PostgreSQL (with pgvector)
  • Container name: ruby-on-rag-postgres
  • Image: pgvector/pgvector:pg15
  • Port: 35432 → container 5432
  • Database created automatically: set by POSTGRES_DB, defaults to ruby_on_rag
3. pgAdmin
  • Container name: ruby-on-rag-pgadmin
  • Image: dpage/pgadmin4:latest
  • Port: 30080 → container 80
  • URL: http://localhost:30080
  • Credentials: see PGADMIN_DEFAULT_EMAIL and PGADMIN_DEFAULT_PASSWORD in .env
  • Auto-configured: PostgreSQL connection settings are loaded on startup; the first connection requires the POSTGRES_PASSWORD.

Note: pgadmin-servers.json sets MaintenanceDB to ruby_on_rag. If you change POSTGRES_DB in .env, update this file accordingly, or configure the connection manually in pgAdmin.

Quick Start

1. Copy the config file
cp .env.example .env

Then edit .env:

# Required: set your OpenAI API Key
OPENAI_API_KEY=sk-...

# Adjust UID/GID to match your system
# macOS is typically 501/20, Linux is typically 1000/1000 (run `id -u` and `id -g` to check)
# Windows can usually stay at 1000/1000
USER_UID=1000
USER_GID=1000
2. Open in VS Code
code .

VS Code usually shows a "Reopen in Container" prompt in the bottom-right corner. Click it to enter the Dev Container.

If the prompt does not appear, trigger it manually: press Cmd+Shift+P (or Ctrl+Shift+P on Windows/Linux), type Reopen in Container, and press Enter.

The Dev Container will automatically:

  • Start PostgreSQL (with pgvector extension)
  • Create the database (name set by POSTGRES_DB)
  • Start pgAdmin (with the connection pre-loaded)
3. Initialize and start

Once inside the Dev Container, run:

bin/setup

To start the server only: run bin/dev directly.

See bin/setup and bin/dev for details.

4. Access the application

Resetting the environment

If you need to rebuild the dev environment and can discard all data, stop and remove all services and volumes:

docker compose down -v

Then confirm the volumes are gone:

docker volume ls

Environment Variables

Variable Default Description
OPENAI_API_KEY (required) OpenAI API key
DEFAULT_DOCUMENT_LANGUAGE zh Default document language (zh, en, ja, ko, es, fr, de)
CHUNK_SIZE 1500 Maximum tokens per document chunk
CHUNK_OVERLAP 64 Overlapping tokens between adjacent chunks
TOP_K_RESULTS 7 Number of retrieval results to return
SIMILARITY_THRESHOLD 0.4 Similarity threshold (higher = stricter)
TOOL_TRIGGER_THRESHOLD 0.6 Threshold for triggering LLM tool-assisted search

For detailed parameter explanations, tuning advice, and usage scenarios, see .env.example.


RAG Workflow

Q&A Flow

  1. User submits a question

  2. Initial vector retrieval

    • Convert the question to an embedding
    • Search pgvector for similar document chunks
    • Return the top TOP_K_RESULTS results
  3. Quality assessment

    • Check the highest similarity score
    • No results found → go to step 4
    • Score < TOOL_TRIGGER_THRESHOLD → go to step 4 (with preliminary results)
    • Score ≥ TOOL_TRIGGER_THRESHOLD → skip to step 5
  4. LLM tool-assisted search (Agentic RAG) (optional)

    • Uses Function Calling to let the LLM actively invoke the search tool
    • LLM evaluates initial result quality and decides whether additional search is needed
    • Generates improved search keywords (Query Rewriting)
    • Bounded number of additional search iterations
    • Merges all search results
  5. Generate answer

    • LLM answers based strictly on retrieved document content
    • Hard constraint: only knowledge-base content is used; training data is excluded
    • Sources are annotated
  6. Display result

    • Stream the answer (via Turbo Streams)
    • Show reference sources and similarity scores

Search Strategies

Standard vector search (default)
  • Ranked by cosine similarity
  • Relatively fast
  • May return semantically similar but repetitive results
MMR diversity search (optional)
  • Balances relevance and diversity
  • Reduces duplicate content
  • Higher computational cost
  • Enable with USE_MMR=true

Where to Find It in Code

The NLP concepts from the first half of the session map to these files:

Concept File Description
Chunking app/models/concerns/document/chunkable.rb Markdown structure parsing, paragraph splitting, sliding window, token counting
Embedding app/models/concerns/document/embeddable.rb Calls the OpenAI embedding API and generates vectors in batches
Indexing pipeline app/models/concerns/document/indexable.rb Orchestrates chunkable + embeddable and persists chunks to the DB
Vector retrieval app/models/concerns/document_chunk/searchable.rb Cosine similarity search, MMR diversity filtering, result formatting
LLM tool search (Function Calling) app/tools/document_search.rb Search tool definition invoked by the LLM via Function Calling
Full Q&A flow app/jobs/chat_with_doc_response_job.rb Wires together retrieval → quality check → tool search → answer generation

Project Structure

.
├── .devcontainer/            # VS Code Dev Container config
│   ├── devcontainer.json
│   └── Dockerfile
├── app/
│   ├── agents/               # RubyLLM Agent
│   │   └── chat_with_doc_agent.rb
│   ├── controllers/
│   ├── jobs/
│   │   ├── chat_response_job.rb          # RubyLLM default mode reference (disabled)
│   │   ├── chat_with_doc_response_job.rb
│   │   └── index_document_job.rb
│   ├── models/
│   │   ├── concerns/
│   │   │   ├── document/
│   │   │   │   ├── chunkable.rb      # chunking logic
│   │   │   │   ├── embeddable.rb     # embedding / vectorization logic
│   │   │   │   └── indexable.rb      # indexing orchestration (chunkable + embeddable)
│   │   │   └── document_chunk/
│   │   │       └── searchable.rb     # vector retrieval and result formatting
│   │   ├── chat.rb
│   │   ├── document.rb
│   │   ├── document_chunk.rb
│   │   ├── message.rb
│   │   ├── model.rb                  # RubyLLM model list cache
│   │   └── tool_call.rb              # Function Calling record
│   ├── prompts/
│   │   └── chat_with_doc_agent/
│   │       └── instructions.txt.erb  # Agent system prompt
│   ├── tools/
│   │   └── document_search.rb        # LLM Function Calling tool
│   └── views/
├── config/
│   ├── initializers/
│   │   ├── rag_config.rb             # loads rag.yml and openai.yml
│   │   └── ruby_llm.rb               # LLM model config
│   ├── database.yml
│   ├── openai.yml            # OpenAI model and API config
│   ├── queue.yml
│   ├── rag.yml               # RAG parameters (chunk size, top-k, thresholds, etc.)
│   ├── storage.yml
│   └── cable.yml
├── db/
│   ├── migrate/
│   ├── schema.rb
│   ├── queue_schema.rb       # Solid Queue schema
│   ├── cable_schema.rb       # Solid Cable schema
│   └── cache_schema.rb       # Solid Cache schema (reserved)
├── storage/                  # ActiveStorage local storage directory
├── docker-compose.yml        # Dev Container services (application / postgres / pgAdmin)
├── pgadmin-servers.json      # pgAdmin default connection config
├── .env.example              # Environment variable template
├── Gemfile
└── README.md

License

MIT License

About

Ruby Taiwan 舉辦的 Ruby on RAG (檢索式增強生成) 工作坊實作專案。 Hands-on project for the Ruby on RAG (Retrieval-Augmented Generation) Workshop organized by Ruby Taiwan.

Resources

License

Stars

1 star

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors