feat: background pipeline loading for instant startup

UI available immediately; /search returns loading fragment with
HTMX auto-retry until models are ready; /health returns 503 while
loading. Adds loading.html template and tests for loading state.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Author: antoine-de-daran

CLAUDE.md

@@ -31,9 +31,9 @@ French Bible RAG with two-stage retrieval:
 2. **`rag/ingest.py`** -- ingestion pipeline: reads `bible.db` SQLite, filters short/non-content verses, encodes with SentenceTransformer, builds FAISS IndexFlatIP, writes `data/index.faiss` + `data/mapping.json`
 3. **`rag/retrieve.py`** -- two-stage search: FAISS top-K (cosine via inner product on L2-normalized vectors), then cross-encoder reranking with sigmoid score normalization
 4. **`config.py`** -- all tunable parameters (paths, model names, thresholds, retrieval K values)
-5. **`app.py`** -- FastAPI server: loads pipeline once at startup via lifespan, serves Jinja2 HTML fragments to HTMX frontend. Query sanitization, input validation, contextual verse display with surrounding verses bounded by book_id. Root URL serves SPA, SEO routes (`/robots.txt`, `/sitemap.xml`), static asset cache middleware (24h), HF-to-custom-domain redirect middleware
+5. **`app.py`** -- FastAPI server: loads pipeline in a background thread at startup (UI available immediately, `/search` returns a loading fragment with HTMX auto-retry until ready, `/health` returns 503 while loading). Query sanitization, input validation, contextual verse display with surrounding verses bounded by book_id. Root URL serves SPA, SEO routes (`/robots.txt`, `/sitemap.xml`), static asset cache middleware (24h), HF-to-custom-domain redirect middleware
 
-Data flow: `bible.db` -> ingest -> `data/{index.faiss, mapping.json}` -> app startup loads into memory -> HTMX POST `/search` -> HTML fragment response. Root `/` serves the SPA entry point.
+Data flow: `bible.db` -> ingest -> `data/{index.faiss, mapping.json}` -> app startup spawns background thread to load into memory -> HTMX POST `/search` -> HTML fragment response (or loading fragment if pipeline not yet ready). Root `/` serves the SPA entry point.
 
 ## Frontend
 
@@ -44,6 +44,7 @@ Custom design system with warm parchment aesthetic (`#f5f0e8` background, `#2a2a
 - **`static/app.js`** -- component initializers inside `DOMContentLoaded`: `initPageHeader`, `initSearchBar`, `initStatusMessages`, `initCarousel`, `initCarouselNavigation`, `initHistorySidebar`, `initOfflineDetection`. Shared state via `window.appState`
 - **`static/service-worker.js`** -- cache-first for static assets, network-only for `/search` API
 - **`templates/results.html`** -- Embla Carousel structure (viewport > track > slides) with score badges and context verses
+- **`templates/loading.html`** -- loading state with HTMX `hx-trigger="load delay:2s"` auto-retry
 - **`templates/error.html`** -- error message with "Reessayer" retry button
 - **`templates/no_results.html`** -- simple no-results feedback
 
@@ -63,7 +64,8 @@ Custom design system with warm parchment aesthetic (`#f5f0e8` background, `#2a2a
 - Cross-encoder raw scores are sigmoid-normalized to [0, 1] (0.5 = decision boundary)
 - `data/` is gitignored -- regenerate with `make ingest` (requires `bible.db` in `data/`)
 - Tests use two markers: `unit` (fast, mocked, default) and `integration` (loads real models + data)
-- App tests use `mock_pipeline` fixture from `conftest.py` to avoid loading models
+- App tests use `mock_pipeline` fixture from `conftest.py` to avoid loading models; `mock_pipeline_loading` simulates the not-yet-ready state
+- `pipeline_ready` is a `threading.Event` in `app.py`; tests patch it with set/unset events
 - Docstrings follow numpy convention
 - Line length: 100 chars
 - Python 3.12+ (uses `X | Y` union syntax)

README.md

@@ -74,6 +74,7 @@ The entire system runs locally with no external API calls, no paid dependencies,
 - **Semantic search** -- find verses by meaning, not just keywords
 - **Two-stage retrieval** -- FAISS vector search for recall, cross-encoder reranking for precision
 - **Contextual results** -- each match includes surrounding verses for readable context
+- **Instant startup** -- background pipeline loading; UI available in < 1s, search auto-retries until models are ready
 - **Fast** -- sub-2s response times on CPU
 - **35,000+ verses** -- complete French Bible (AELF translation)
 - **PWA-ready** -- offline support via service worker, installable on mobile
@@ -107,7 +108,7 @@ Open [http://localhost:8000](http://localhost:8000) in your browser.
 |--------|----------------|------------------------------------------|
 | GET    | `/`            | Main SPA entry point                     |
 | POST   | `/search`      | Search (accepts `query` form field, returns HTML fragment) |
-| GET    | `/health`      | Health check (`{"status": "ok"}`)        |
+| GET    | `/health`      | Health check (200 `ok` or 503 `loading`) |
 | GET    | `/robots.txt`  | Robots.txt for crawlers                  |
 | GET    | `/sitemap.xml` | XML sitemap for crawlers                 |
 
@@ -145,15 +146,17 @@ flowchart LR
 
 ```
 config.py           # Central configuration (paths, models, thresholds)
-app.py              # FastAPI application + lifespan setup
+app.py              # FastAPI application + background pipeline loading
 rag/                # Core package
   embeddings.py     #   Model loading and text encoding
   ingest.py         #   Ingestion: filter, embed, index
   retrieve.py       #   Two-stage retrieval: FAISS + cross-encoder
-templates/          # Jinja2 HTML fragments
-  results.html      #   Search results (Embla Carousel)
-  error.html        #   Error display
-  no_results.html   #   No results feedback
+templates/              # Jinja2 HTML fragments
+  results.html          #   Search results (Embla Carousel)
+  context_verses.html   #   Surrounding context verses
+  loading.html          #   Loading state with HTMX auto-retry
+  error.html            #   Error display
+  no_results.html       #   No results feedback
 static/             # Frontend assets (no build step)
   index.html        #   SPA entry point (HTMX)
   styles.css        #   Design system (CSS custom properties)
@@ -175,6 +178,7 @@ data/               # Generated artifacts (gitignored)
 |-----------------|------------------------------------------------------|---------------------------------------|
 | Embeddings      | `paraphrase-multilingual-MiniLM-L12-v2`             | 384-dim multilingual sentence encoder |
 | Reranker        | `mmarco-mMiniLMv2-L12-H384-v1`                      | Cross-encoder for precision reranking |
+| Inference       | ONNX Runtime                                         | Optimized CPU inference backend       |
 | Vector index    | FAISS (`IndexFlatIP`)                                | Fast inner-product similarity search  |
 | Backend         | FastAPI + Uvicorn                                    | Async HTTP server                     |
 | Frontend        | HTMX + Embla Carousel + vanilla CSS/JS               | No-build interactive UI               |

app.py

@@ -2,6 +2,7 @@
 
 import logging
 import re
+import threading
 from collections.abc import AsyncGenerator
 from contextlib import asynccontextmanager
 from html import escape as html_escape
@@ -13,6 +14,7 @@
 from fastapi.responses import (
     FileResponse,
     HTMLResponse,
+    JSONResponse,
     PlainTextResponse,
     RedirectResponse,
     Response,
@@ -31,6 +33,7 @@
 
 # Module-level state set during lifespan
 pipeline: dict[str, Any] = {}
+pipeline_ready = threading.Event()
 
 templates = Jinja2Templates(directory=Path(__file__).parent / "templates")
 
@@ -153,24 +156,36 @@ def nl2br(value: str) -> Markup:
     return Markup(escaped.replace("\n", "<br>"))
 
 
+def _load_pipeline_background() -> None:
+    """Load the retrieval pipeline in a background thread."""
+    try:
+        index, mapping, embed_model, cross_encoder = _load_pipeline()
+        pipeline["index"] = index
+        pipeline["mapping"] = mapping
+        pipeline["embed_model"] = embed_model
+        pipeline["cross_encoder"] = cross_encoder
+
+        verse_idx: dict[tuple[str, str, str], int] = {}
+        for i, entry in enumerate(mapping):
+            key = (entry["book_title"], entry["chapter"], entry["verse"])
+            verse_idx[key] = i
+        pipeline["verse_index"] = verse_idx
+
+        pipeline["loaded"] = True
+        pipeline_ready.set()
+        logger.info("Pipeline loaded successfully")
+    except Exception:
+        logger.exception("Failed to load pipeline")
+
+
 @asynccontextmanager
 async def lifespan(_app: FastAPI) -> AsyncGenerator[None, None]:
-    """Load models once at startup."""
-    index, mapping, embed_model, cross_encoder = _load_pipeline()
-    pipeline["index"] = index
-    pipeline["mapping"] = mapping
-    pipeline["embed_model"] = embed_model
-    pipeline["cross_encoder"] = cross_encoder
-
-    verse_idx: dict[tuple[str, str, str], int] = {}
-    for i, entry in enumerate(mapping):
-        key = (entry["book_title"], entry["chapter"], entry["verse"])
-        verse_idx[key] = i
-    pipeline["verse_index"] = verse_idx
-
-    pipeline["loaded"] = True
+    """Spawn background pipeline loading so HTTP is available immediately."""
+    thread = threading.Thread(target=_load_pipeline_background, daemon=True)
+    thread.start()
     yield
     pipeline.clear()
+    pipeline_ready.clear()
 
 
 app = FastAPI(title="RAG Bible", lifespan=lifespan)
@@ -257,14 +272,23 @@ def sitemap_xml() -> Response:
 
 
 @app.get("/health")  # type: ignore[misc]
-def health() -> dict[str, str]:
-    """Health check endpoint."""
-    return {"status": "ok"}
+def health() -> Response:
+    """Health check endpoint. Returns 503 while pipeline is loading."""
+    if not pipeline_ready.is_set():
+        return JSONResponse({"status": "loading"}, status_code=503)
+    return JSONResponse({"status": "ok"})
 
 
 @app.post("/search", response_class=HTMLResponse)  # type: ignore[misc]
 def search_endpoint(request: Request, query: str = Form("")) -> HTMLResponse:
     """Search the Bible and return an HTML fragment."""
+    if not pipeline_ready.is_set():
+        return templates.TemplateResponse(
+            request=request,
+            name="loading.html",
+            context={"query": query},
+        )
+
     cleaned = sanitize_query(query)
 
     if not cleaned:

templates/loading.html

@@ -0,0 +1,6 @@
+<div class="status-message loading-message" role="status" aria-live="polite"
+     hx-post="/search" hx-trigger="load delay:2s"
+     hx-target="#results" hx-swap="innerHTML"
+     hx-vals='{"query": "{{ query }}"}'>
+  <p>Chargement des modeles en cours...</p>
+</div>

tests/conftest.py

@@ -202,8 +202,13 @@ def _mock_context(result: dict[str, Any], *_args: Any, **_kwargs: Any) -> list[d
 @pytest.fixture
 def mock_pipeline() -> Generator[None, None, None]:
     """Patch app pipeline so no models are loaded."""
+    import threading
+
+    ready = threading.Event()
+    ready.set()
     with (
         patch("app.pipeline", {"loaded": True, "mapping": [], "verse_index": {}}),
+        patch("app.pipeline_ready", ready),
         patch("app._run_search", side_effect=lambda q: _mock_search_results(relevant=True)),
         patch("app.get_verse_context", side_effect=_mock_context),
     ):
@@ -213,14 +218,32 @@ def mock_pipeline() -> Generator[None, None, None]:
 @pytest.fixture
 def mock_pipeline_low_scores() -> Generator[None, None, None]:
     """Patch app pipeline with only low-score results."""
+    import threading
+
+    ready = threading.Event()
+    ready.set()
     with (
         patch("app.pipeline", {"loaded": True, "mapping": [], "verse_index": {}}),
+        patch("app.pipeline_ready", ready),
         patch("app._run_search", side_effect=lambda q: _mock_search_results(relevant=False)),
         patch("app.get_verse_context", side_effect=_mock_context),
     ):
         yield
 
 
+@pytest.fixture
+def mock_pipeline_loading() -> Generator[None, None, None]:
+    """Patch app pipeline in loading state (not ready)."""
+    import threading
+
+    not_ready = threading.Event()  # not set = still loading
+    with (
+        patch("app.pipeline", {}),
+        patch("app.pipeline_ready", not_ready),
+    ):
+        yield
+
+
 @pytest.fixture
 def client(mock_pipeline: None) -> TestClient:
     """FastAPI test client with mocked pipeline."""

tests/test_app.py

@@ -65,6 +65,26 @@ def test_returns_ok(self, client: TestClient) -> None:
         assert response.status_code == 200
         assert response.json() == {"status": "ok"}
 
+    def test_returns_503_while_loading(self, mock_pipeline_loading: None) -> None:
+        from app import app as fastapi_app
+
+        loading_client = TestClient(fastapi_app, raise_server_exceptions=False)
+        response = loading_client.get("/health")
+        assert response.status_code == 503
+        assert response.json() == {"status": "loading"}
+
+
+@pytest.mark.unit
+class TestSearchLoading:
+    def test_search_returns_loading_fragment(self, mock_pipeline_loading: None) -> None:
+        from app import app as fastapi_app
+
+        loading_client = TestClient(fastapi_app, raise_server_exceptions=False)
+        response = loading_client.post("/search", data={"query": "amour"})
+        assert response.status_code == 200
+        assert "Chargement" in response.text
+        assert "hx-post" in response.text
+
 
 @pytest.mark.unit
 class TestSearchEndpoint: