[Feature] Test contextual verse embeddings with surrounding context window

[antoine-de-daran]

Summary

Evaluate whether embedding each verse with its N surrounding verses (context window) instead of embedding verses atomically improves retrieval quality. The hypothesis is that richer input text at embedding time produces embeddings that carry more semantic meaning, leading to better FAISS recall and overall RAG performance.

Motivation

Bible verses are short, self-contained units that often lack the context needed to fully capture their meaning. The current pipeline embeds each verse's text field in isolation (ingest.py:144):

texts = [v["text"] for v in filtered]

This causes several known issues with short-text embedding:

• Semantic ambiguity: short verses like "Jesus wept" (Jean 11:35) or "Rejoice always" (1 Thess 5:16) produce generic embeddings that match too many unrelated queries
• Lost narrative continuity: a verse in the middle of a parable or argument carries meaning that depends on its neighbors, but the embedding sees none of that
• Low information density: the embedding model has little signal to work with, so many verse embeddings cluster together in the vector space, reducing retrieval discrimination

Contextual embedding is a well-documented technique in dense retrieval literature. The idea: embed context + separator + verse so the model produces a more informed vector, while still mapping one embedding to one verse in the FAISS index.

Proposed Solution

Approach

During ingestion, instead of encoding verse.text alone, construct a context string for each verse:

[prev_N verses joined] [SEP] target verse [SEP] [next_N verses joined]

The FAISS index and mapping remain 1:1 with verses -- only the input text to the embedding model changes. The context window size N becomes a new config parameter (e.g., EMBED_CONTEXT_WINDOW = 2).

Implementation steps

1. Add config parameter: EMBED_CONTEXT_WINDOW: int = 2 in config.py

2. Modify ingestion (rag/ingest.py):

   • After filter_verses(), group verses by book_id to ensure context never crosses book boundaries
   • For each verse at position i, build the embedding input by concatenating verses [i-N, ..., i-1, i, i+1, ..., i+N] within the same book, joined by a separator (e.g., " " or " | ")
   • Pass these context-enriched strings to encode_texts() instead of raw verse texts
   • The mapping still stores the original verse metadata (no change to mapping.json schema)

3. Leave retrieval unchanged: retrieve.py already encodes the user query atomically and searches the same FAISS index. The cross-encoder reranking stage already receives (query, verse.text) pairs, so no change needed there either

4. Evaluation: compare retrieval quality before/after on a set of test queries

Pseudocode

def build_contextual_texts(
    verses: list[dict],
    window: int = config.EMBED_CONTEXT_WINDOW,
) -> list[str]:
    by_book: dict[int, list[dict]] = {}
    for v in verses:
        by_book.setdefault(v["book_id"], []).append(v)

    # Preserve original ordering
    contextual = []
    for v in verses:
        book_verses = by_book[v["book_id"]]
        idx = next(i for i, bv in enumerate(book_verses) if bv["rowid"] == v["rowid"])
        start = max(0, idx - window)
        end = min(len(book_verses), idx + window + 1)
        context_parts = [bv["text"] for bv in book_verses[start:end]]
        contextual.append(" ".join(context_parts))

    return contextual

Technical Implications

Token length

The current model (paraphrase-multilingual-MiniLM-L12-v2) has a max sequence length of 128 tokens. With a context window of 2 (5 verses total), many inputs will exceed this limit and get silently truncated, potentially discarding the target verse itself if it appears late in the concatenation.

Mitigations to evaluate:

• Center the target verse in the concatenation so truncation clips context, not the verse itself
• Reduce window to 1 (3 verses) to stay closer to the token limit
• Test with a longer-context model (e.g., BGE-M3 with 8192 tokens) -- this pairs with the existing BGE-M3 evaluation issue

Index compatibility

The FAISS index dimensions and type (IndexFlatIP, 384d) remain unchanged. However, the index built with contextual embeddings is not interchangeable with the current atomic index -- the vectors occupy different regions of the embedding space. Switching strategies requires a full make ingest.

Ingestion performance

• Longer input strings increase encoding time. With window=2, average input length roughly triples
• The model's tokenizer still processes each text independently, so batch encoding via encode_texts() still works
• Grouping by book_id adds negligible overhead

Query asymmetry

This introduces an asymmetry: documents are embedded with context, but queries are embedded without. This is intentional and standard practice in dense retrieval (the query naturally carries its own context). However, it means the improvement depends on the model's ability to match a short query against a context-enriched document embedding -- worth validating empirically.

No impact on reranking

The cross-encoder reranking stage (retrieve.py:137-139) receives (query, verse.text) pairs using the original verse text from the mapping. This is unaffected by the embedding change, which is a clean separation of concerns.

Mapping schema

No change to mapping.json. The mapping still stores per-verse metadata. The contextual text is only used transiently during build_index() and is not persisted.

Acceptance Criteria

• EMBED_CONTEXT_WINDOW config parameter added
• build_contextual_texts() function in ingest.py with book-boundary enforcement
• Existing unit tests still pass (atomic embedding remains the default with window=0)
• Side-by-side retrieval comparison on evaluation queries (atomic vs window=1 vs window=2)
• Token truncation behavior documented for each window size
• Ingestion time and index size compared across configurations
• Decision on default window size (or keep atomic as default)

Alternatives Considered

• Sliding window chunking: merge consecutive verses into overlapping chunks, each chunk gets one embedding. This changes the 1:1 verse-to-embedding mapping and complicates the mapping schema and display logic
• Hierarchical embedding: embed at verse, chapter, and book level, then combine scores. Significantly more complex with unclear benefits for this corpus size
• Prepend book/chapter title as context: simpler than full surrounding verses but adds less semantic signal. Could be a quick complementary test
• Query expansion instead: enrich the query side rather than the document side. Orthogonal approach that could be combined later

Additional Context

Current ingestion path: bible.db has 35,480 verses (35,470 after filtering). Verses are ordered by book_id, chapter, verse in the database, so positional grouping by book_id naturally respects chapter/verse ordering.

This experiment pairs well with the BGE-M3 evaluation (see issue-bge-m3.md): BGE-M3's 8192-token context window would eliminate the truncation concern entirely, making contextual embeddings more viable.