Skip to content

Commit 2151086

Browse files
author
Flamehaven CI
committed
docs: v1.5.2 changelog, readme, architecture sanity pass
- CHANGELOG: add [1.5.2] section (parse_cache, ContextExtractor, Backend Plugin) - README: v1.5.2 badge + What's New v1.5.2 section - Architecture.md: v1.5.2 header, engine/ file list updated, 443 tests, Backend Plugin dispatch - Document_Parsing.md: version header v1.5.0 -> v1.5.2 - pyproject.toml + __init__.py: 1.5.1 -> 1.5.2
1 parent 782671a commit 2151086

6 files changed

Lines changed: 83 additions & 18 deletions

File tree

CHANGELOG.md

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,49 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
---
99

10+
## [1.5.2] - 2026-04-19
11+
12+
### Added
13+
14+
- **Parse Cache** (`engine/parse_cache.py`): mtime-based file extraction cache.
15+
Algorithm absorbed from RAG-Anything `processor.py:_generate_cache_key()`.
16+
Cache key = MD5(resolved_path + mtime + parser_config). Path-indexed reverse
17+
map enables O(1) `invalidate()`. API: `get/put/invalidate/clear/stats`.
18+
`extract_text(use_cache=True)` integrates transparently — no API change.
19+
Score: **0.0 CLEAN**.
20+
21+
- **ContextExtractor** (`engine/context_extractor.py`): Sliding-window chunk
22+
context extractor for RAG result enrichment. Algorithm absorbed from
23+
RAG-Anything `modalprocessors.py:ContextExtractor`. Given `chunk_text()`
24+
output, `enrich_chunks()` adds a `context` key to each chunk containing
25+
surrounding neighbour text. `ContextConfig`: `window_size`, `max_context_chars`,
26+
`include_headings`. Zero external dependencies. Score: **0.0 CLEAN**.
27+
28+
- **Backend Plugin Architecture** (`engine/format_backends.py`): Format-family
29+
backends absorbed from Docling `abstract_backend.py` pattern.
30+
`AbstractFormatBackend` ABC with `supported_extensions` + `extract()`.
31+
`BackendRegistry` maps extensions to backend classes; new formats register
32+
without modifying the dispatcher. 11 concrete backends:
33+
`PDFBackend`, `DOCXBackend`, `DOCBackend`, `XLSXBackend`, `PPTXBackend`,
34+
`RTFBackend`, `HTMLBackend`, `VTTBackend`, `LaTeXBackend`, `CSVBackend`,
35+
`ImageBackend`, `PlainTextBackend`. Score: **12.2 clean**.
36+
37+
### Refactored
38+
39+
- **`engine/file_parser.py`** (75 lines, was 340): Rewritten as pure registry
40+
dispatcher — `_dispatch()` resolves backend via `BackendRegistry.get(ext)`
41+
then calls `backend.extract()`. Cyclomatic complexity 13 → 3.
42+
`function_clone_cluster` (5 structurally similar `_extract_*` functions)
43+
eliminated by moving each into its own Backend class. Score: **3.0 CLEAN**.
44+
45+
### Tests
46+
47+
- `tests/test_phase1_parse_cache_context.py`: 33 tests (parse_cache + ContextExtractor).
48+
- `tests/test_phase2_format_backends.py`: 50 tests (registry + backends + helpers).
49+
- Combined: **83 tests**, all passing. AI-Slop-Detector critical deficits: 0.
50+
51+
---
52+
1053
## [1.5.1] - 2026-04-18
1154

1255
### Removed

README.md

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77
### Self-hosted RAG search engine. Production-ready in 3 minutes.
88

99
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
10-
[![Version](https://img.shields.io/badge/version-1.5.1-blue.svg)](CHANGELOG.md)
10+
[![Version](https://img.shields.io/badge/version-1.5.2-blue.svg)](CHANGELOG.md)
1111
[![Python](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/)
1212
[![Docker](https://img.shields.io/badge/docker-ready-brightgreen.svg)](https://hub.docker.com/r/flamehaven/filesearch)
1313

@@ -60,6 +60,14 @@ Open source & MIT licensed</p>
6060
- **⚡ Ultra-Fast Vectors** - DSP v2.0 algorithm generates embeddings in <1ms without ML frameworks
6161
- **🎯 Source Attribution** - Every answer includes links back to source documents
6262

63+
### What's New in v1.5.2 (patch)
64+
65+
- **Parse Cache** (`engine/parse_cache.py`) — mtime-based file parse cache; `extract_text(use_cache=True)` skips re-parsing unchanged files
66+
- **ContextExtractor** (`engine/context_extractor.py`) — sliding-window chunk context enrichment for RAG pipelines (`enrich_chunks()`)
67+
- **Backend Plugin Architecture** (`engine/format_backends.py`) — 11 `AbstractFormatBackend` subclasses + `BackendRegistry`; new formats plug in without touching the dispatcher
68+
- **`file_parser.py`** refactored to 75 lines (was 340); cyclomatic complexity 13 → 3
69+
- 83 new tests; AI-Slop-Detector critical deficits: 0
70+
6371
### What's New in v1.5.1 (patch)
6472

6573
- **Dead code removed**`embedding_generator_legacy.py` deleted (306-line duplicate, unused)
@@ -485,7 +493,7 @@ Use the links below to jump to the most relevant guide.
485493
| [Document Parsing](docs/wiki/Document_Parsing.md) | Supported formats, internal parsers, RAG chunking |
486494
| [Framework Integrations](docs/wiki/Framework_Integrations.md) | LangChain, LlamaIndex, Haystack, CrewAI adapters |
487495
| [API Reference](docs/wiki/API_Reference.md) | REST endpoints, payloads, rate limits |
488-
| [Architecture](docs/wiki/Architecture.md) | How all layers fit together (v1.5.1) |
496+
| [Architecture](docs/wiki/Architecture.md) | How all layers fit together (v1.5.2) |
489497
| [Configuration Reference](docs/wiki/Configuration.md) | Full list of environment variables and config fields |
490498
| [Production Deployment](docs/wiki/Production_Deployment.md) | Docker, systemd, reverse proxy, scaling tips |
491499
| [Troubleshooting](docs/wiki/Troubleshooting.md) | Step-by-step debugging playbook |

docs/wiki/Architecture.md

Lines changed: 27 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,15 @@
11
# Architecture Overview
22

33
Flamehaven FileSearch balances simplicity with production-grade safeguards. This
4-
document describes the moving parts as of **v1.5.1**, featuring:
4+
document describes the moving parts as of **v1.5.2**, featuring:
55
- **Gravitas DSP Engine** (v1.3.1+)
66
- **Multimodal Search** (v1.4.0+)
77
- **pgvector with HNSW** (v1.4.0+)
88
- **Circuit Breaker & Health Checks** (v1.4.1+)
99
- **ABC base classes, ruff CI, Windows filename fix** (v1.4.2)
1010
- **Universal Document Parser, Internal Chunker, Framework Integrations** (v1.5.0)
1111
- **Dead code removal, critical complexity fixes, 360-test suite** (v1.5.1)
12+
- **Parse Cache, ContextExtractor, Backend Plugin Architecture** (v1.5.2)
1213

1314
---
1415

@@ -85,7 +86,7 @@ The new **Chronos-Grid** integration handles high-speed vector storage and simil
8586

8687
## 6. Testing & Quality (v1.4.2)
8788

88-
- **Test Framework**: `pytest`360 tests collected, all passing.
89+
- **Test Framework**: `pytest`443 tests collected, all passing (360 + 83 new).
8990
- **Lint**: `black` (format) + `ruff` (lint/unused imports) — both enforced in CI.
9091
- **Validation**: `validators.py` enforces security policies (Filename 200-char max, FileSize, SearchQuery XSS/SQLi checks).
9192
- **SIDRCE Certification**: Omega 0.9894 (S++) — AI-Slop-Detector P0-P5 clean.
@@ -118,19 +119,32 @@ The `engine/` sub-package contains the full parsing stack:
118119

119120
```
120121
engine/
121-
file_parser.py — Dispatcher: routes by extension, tries parsers in priority order
122-
format_parsers.py — Internal parsers: HTML, WebVTT, LaTeX, CSV, Image OCR
123-
text_chunker.py — Structure-aware + token-aware RAG chunker (stdlib only)
124-
embedding_generator.py — DSP v2.0 vectorizer
125-
chronos_grid.py — Vector index + metadata store
126-
gravitas_pack.py — Metadata compression
127-
intent_refiner.py — Query analysis + search mode selection
122+
file_parser.py — Dispatcher: BackendRegistry.get(ext) -> backend.extract()
123+
format_backends.py — 11 AbstractFormatBackend classes + BackendRegistry (v1.5.2)
124+
format_parsers.py — Internal parsers: HTML, WebVTT, LaTeX, CSV, Image OCR
125+
parse_cache.py — mtime-based parse result cache (v1.5.2)
126+
context_extractor.py — RAG chunk context window extractor (v1.5.2)
127+
text_chunker.py — Structure-aware + token-aware RAG chunker (stdlib only)
128+
embedding_generator.py — DSP v2.0 vectorizer
129+
chronos_grid.py — Vector index + metadata store
130+
gravitas_pack.py — Metadata compression
131+
intent_refiner.py — Query analysis + search mode selection
128132
```
129133

130-
**Extraction priority** (for each file):
131-
1. Per-format internal parser (HTML/VTT/LaTeX/CSV via `format_parsers.py`)
132-
2. Optional heavy parser (pymupdf, python-docx, openpyxl, python-pptx, striprtf)
133-
3. Plain UTF-8 read (last resort for unknown text formats)
134+
**Extraction dispatch** (v1.5.2 — Backend Plugin):
135+
136+
Each file extension resolves to an `AbstractFormatBackend` subclass via
137+
`BackendRegistry`. New formats plug in by subclassing and registering — no
138+
changes to `file_parser.py` required.
139+
140+
**Extraction stack per backend:**
141+
- PDF: `PDFBackend` — pymupdf → pypdf fallback
142+
- DOCX/DOC: `DOCXBackend` / `DOCBackend` — python-docx + antiword
143+
- XLSX: `XLSXBackend` — openpyxl multi-sheet
144+
- PPTX: `PPTXBackend` — python-pptx text + tables
145+
- HTML/VTT/LaTeX/CSV: stdlib-only backends (zero extra deps)
146+
- Images: `ImageBackend` — pytesseract ([vision] extra)
147+
- Unknown: `PlainTextBackend` — UTF-8 fallback
134148

135149
**Content-based embedding** (v1.5.0): The first 2000 characters of extracted
136150
content are used to generate the vector embedding, replacing the previous

docs/wiki/Document_Parsing.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Document Parsing (v1.5.0)
1+
# Document Parsing (v1.5.2)
22

33
FLAMEHAVEN FileSearch extracts plain UTF-8 text from 34 file extensions using
44
a layered stack of internal and optional parsers. No external document-AI

flamehaven_filesearch/__init__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66
Fast, simple, and transparent file search for developers
77
"""
88

9-
__version__ = "1.5.1"
9+
__version__ = "1.5.2"
1010
__author__ = "FLAMEHAVEN"
1111
__license__ = "MIT"
1212

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
44

55
[project]
66
name = "flamehaven-filesearch"
7-
version = "1.5.1"
7+
version = "1.5.2"
88
description = "FLAMEHAVEN FileSearch - Open source semantic document search with API authentication powered by Google Gemini"
99
readme = "README.md"
1010
requires-python = ">=3.8"

0 commit comments

Comments
 (0)