[Feature] Test BGE-M3 embedding model as alternative to MiniLM

[antoine-de-daran]

Summary

Evaluate the BGE-M3 embedding model as a potential replacement for the current paraphrase-multilingual-MiniLM-L12-v2 (384d) model used for semantic search.

Motivation

• BGE-M3 supports multi-lingual, multi-granularity (dense, sparse, colbert) retrieval in a single model
• 1024-dimension dense embeddings may improve retrieval quality for French Bible text
• Built-in sparse retrieval could complement or replace the current cross-encoder reranking stage
• Strong multilingual performance on benchmarks (MIRACL, MKQA)

Proposed Solution

1. Benchmark setup: create a small evaluation set of French Bible queries with expected relevant verses
2. Integration: add BGE-M3 as a configurable embedding model in config.py (currently paraphrase-multilingual-MiniLM-L12-v2, 384d)
3. Ingestion: re-run make ingest with BGE-M3 and measure index size / build time differences
4. Evaluation: compare retrieval quality (precision@K, MRR) and latency against current pipeline
5. Resource impact: measure memory usage and inference time (model is ~2.2GB vs ~470MB for MiniLM)

Acceptance Criteria

• BGE-M3 loads and produces L2-normalized embeddings compatible with FAISS IndexFlatIP
• Ingestion completes successfully with the new model
• Side-by-side retrieval quality comparison on evaluation queries
• Latency and memory benchmarks documented
• Decision on whether to adopt, keep as option, or discard

Alternatives Considered

• multilingual-e5-large: another strong multilingual model but lacks BGE-M3's multi-granularity features
• Keep current model: MiniLM is lightweight and fast, may be sufficient for the use case

Additional Context

Current embedding config in config.py:

EMBEDDING_MODEL = "sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2"
EMBEDDING_DIMENSION = 384

BGE-M3 would require updating EMBEDDING_DIMENSION to 1024 and adjusting rag/embeddings.py if the loading interface differs from SentenceTransformers.