feat: mempalace_explain — entity-aware search with KG enrichment (decision archaeology)

[Kesshite]

Summary

Proposing a new MCP tool mempalace_explain that wires together four existing but currently isolated modules into a single intelligent query pipeline:

1. Entity Registry → extract_people_from_query() exists, tested, functional — but never called from the search path
2. Knowledge Graph → query_entity() returns temporal facts with source_closet — but the search path never consults it
3. source_closet → lives in the KG schema pointing to drawers — but is a dead reference (never traversed)
4. Wing scoping → the 34% retrieval boost from wing+room filtering — but tool_search never auto-scopes by detected entity

The result: when a user asks "why did we choose Postgres for Orion?", the AI runs a generic unscoped search against the entire palace. mempalace_explain would auto-detect "Orion" as a project, scope to wing_orion, enrich with KG facts, and filter by decision-type rooms — returning structured reasoning, not just loose chunks.

Motivation

From MISSION.md:

"remember when we talked about that idea... but in vague terms. Impossible with regular keyword search."

MemPalace is the only memory system that stores verbatim conversations including the reasoning, tradeoffs, and rejected alternatives behind decisions. Every competitor (Mem0, Zep, Letta) uses LLM extraction and discards this context. But today, accessing this unique advantage requires multiple manual tool calls — the modules that could power it are architecturally disconnected.

What competitors can answer:

• Mem0: "User chose Postgres" (extracted fact, no reasoning)
• Zep: (user, chose, Postgres) (triple, no verbatim)

What MemPalace could answer with mempalace_explain:

• The alternatives considered and why they were rejected
• Who advocated for what
• The full reasoning chain in verbatim quotes
• KG facts showing whether the decision is still current

"Other AI memory systems remember facts. MemPalace remembers reasoning."

Proposed Tool

{
  "name": "mempalace_explain",
  "description": "Entity-aware search with KG enrichment. Auto-detects entities in the query, scopes search to the correct wing, enriches with temporal KG facts, and filters by memory type.",
  "parameters": {
    "query": {"type": "string", "required": true},
    "wing": {"type": "string", "description": "Override auto-detected wing"},
    "memory_type": {"type": "string", "enum": ["decision", "milestone", "problem", "preference", "emotional"]},
    "include_kg": {"type": "boolean", "default": true},
    "include_tunnels": {"type": "boolean", "default": false}
  }
}

Pipeline

Query: "Why did we choose Postgres for Orion?"
  │
  ├─ entity_registry.extract_people_from_query(query)
  │  → detects: ["Orion"] (project), infers wing: "wing_orion"
  │
  ├─ knowledge_graph.query_entity("Postgres")
  │  → facts: [(Orion, uses, Postgres, valid_from=2025-11-03)]
  │  → source_closet → original drawer ID
  │
  ├─ searcher.search_memories(query, wing="wing_orion")
  │  → top-5 verbatim chunks, SCOPED to the correct wing
  │  → filtered to room="decisions" when memory_type specified
  │
  ├─ (optional) palace_graph.find_tunnels(wing_a="wing_orion")
  │  → rooms that exist in other wings (cross-references)
  │
  └─ Returns structured response:
     {
       "entities_detected": ["Orion"],
       "wing_scoped": "wing_orion",
       "kg_facts": [...temporal facts...],
       "verbatim_results": [...search chunks...],
       "related_rooms": [...tunnels...],
       "source_drawers": [...original drawer IDs...]
     }

What already exists (just needs wiring)

Component	File	Status
extract_people_from_query()	entity_registry.py	Shipped, tested, never called from search
query_entity(name, as_of)	knowledge_graph.py	Shipped, returns temporal facts
source_closet field	knowledge_graph.py schema	Stored, never traversed
search_memories(wing=...)	searcher.py	Shipped, 34% boost with wing scoping
find_tunnels()	palace_graph.py	Shipped, finds cross-wing rooms
Wing naming convention	wing_{entity_name}	Consistent across codebase
Decision/milestone rooms	general_extractor.py	Shipped, classifies memory types

No new files needed. This is an orchestration of existing functions in a new tool_explain handler in mcp_server.py.

Design Principles Alignment

CLAUDE.md Principle	How explain honors it
Verbatim always	Returns exact drawer text, never summarizes
Entity-first	Auto-detects entities and scopes by them — the stated principle finally expressed in the query path
Local-first, zero API	Pure Python orchestration of existing local modules
Privacy by architecture	No data leaves the machine

Graceful degradation

• KG empty? → Returns search results only (no enrichment, no error)
• No entity detected? → Falls back to unscoped search_memories (same as today)
• No tunnels? → Omits related_rooms from response
• source_closet missing? → Omits drawer cross-reference

The tool is strictly additive — it never produces worse results than a plain mempalace_search.

Related Issues/PRs

Ref	Title	Relationship
#595 / PR #596	Synapse Advanced Retrieval	Complementary — Synapse improves ranking AFTER search; explain improves SCOPING before search
#565	Cognition Engine	Complementary — Cognition finds deep connections; explain reconstructs specific decisions
#352	Era metadata	Complementary — Era adds "when"; explain adds "why"
#463 / #465	Temporal search	Complementary — useful but not required
#799	engram decision provider	Validation — external project already surfaces MemPalace decisions inline, proving demand

Scope

• ~80-120 lines in mcp_server.py
• ~50 lines of tests
• Zero new dependencies
• Zero new files

Happy to submit a PR if this direction aligns with the project's vision. I've already contributed security fixes (#811, #812, #814, #815, #817) and am familiar with the codebase conventions.

— @Kesshite