Skip to content

feat(examples): add Graphiti → OKF migration adapter with multi-source consolidation showcase - #1824

Open
funds0033-cmyk wants to merge 1 commit into
moorcheh-ai:mainfrom
funds0033-cmyk:main
Open

feat(examples): add Graphiti → OKF migration adapter with multi-source consolidation showcase#1824
funds0033-cmyk wants to merge 1 commit into
moorcheh-ai:mainfrom
funds0033-cmyk:main

Conversation

@funds0033-cmyk

@funds0033-cmyk funds0033-cmyk commented Aug 6, 2026

Copy link
Copy Markdown

What this adds

A new migration path for Zep/Graphiti (currently unsupported by memanto migrate),
taken through the full pipeline end to end:

Graphiti (real, populated instance) → adapter → memanto migrate → Memanto →
memanto memory export --okf

Then, as a second pass, a real Mem0 store is migrated into the same Memanto agent
and the OKF bundle is re-exported, showing two independently-sourced memories
consolidating into one coherent, portable bundle (git diff included).

Why Graphiti

Graphiti's temporal knowledge-graph model (episodes, entity edges/nodes, community
nodes, with valid_at/invalid_at validity intervals) maps onto OKF in a way flat
chat-log adapters don't — the mapping table below shows how superseded facts and
corrections carry their temporal validity through to the exported markdown instead
of collapsing into a single flat state.

Migration summary & savings report

Mapping table

Round-trip validation

OKF bundle

Sample exported bundle (post-consolidation): okf_bundle_sample/

Reproducibility

Single command: ./scripts/run_all.sh (see README for env vars).

Known environment constraint: native Neo4j install can hit a UAC-blocked JDK 21
install on Windows (winget exit 1602). The adapter supports GRAPHITI_BACKEND=kuzu
as a zero-Docker fallback for exactly this case — documented in .env.example and
called out in the README so a stranger without Docker/admin rights can still run this
in under 15 minutes.

Summary by CodeRabbit

  • New Features

    • Added a complete Graphiti-to-OKF migration example with Graphiti, Memanto, OKF, and Mem0 workflows.
    • Added one-command Bash and PowerShell pipelines for migration, validation, export, and consolidation.
    • Preserved temporal metadata, provenance, confidence, and source relationships across generated outputs.
    • Added Neo4j container support with health checks and persistent storage.
    • Added configurable LLM, backend, and environment settings.
  • Documentation

    • Added setup instructions, configuration examples, mapping references, decisions, validation guidance, and known blockers.
  • Tests

    • Added offline coverage for mappings, temporal handling, OKF output, and provider-compatible exports.

@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Added a complete Graphiti-to-Memanto-to-OKF migration example. It includes temporal mapping, OKF and provider exports, Graphiti and Mem0 population, parity validation, consolidation workflows, documentation, configuration, and offline tests.

Changes

Graphiti-to-OKF migration

Layer / File(s) Summary
Example setup and migration contracts
.env.example, README.md, PHASE0.md, DECISIONS.md, BLOCKERS.md, SUMMARY.md, docker-compose.yml, requirements.txt, runtime.py
Defines configuration, dependencies, Neo4j setup, migration contracts, runtime paths, execution requirements, and operational status.
Graphiti source dataset and pipeline
graphiti_okf/dataset.py, graphiti_okf/graphiti_client.py, scripts/populate_graphiti.py, scripts/export_graphiti.py
Adds the temporal conversation dataset, configurable Graphiti clients, chronological ingestion, community creation, and serialized graph export.
Mapping and migration outputs
graphiti_okf/mapping.py, okf_writer.py, provider_json.py, report.py, scripts/graphiti_to_memanto.py, tests/test_mapping.py
Maps Graphiti records to Memanto types, preserves temporal and provenance metadata, writes OKF and provider-compatible outputs, generates reports, and validates the conversion offline.
Temporal parity validation
graphiti_okf/golden_qa.py, graphiti_okf/judge.py, scripts/run_validation.py
Adds golden questions, Graphiti and Memanto answer collection, deterministic signal checks, Anthropic scoring, retries, and persisted parity reports.
Mem0 consolidation and end-to-end orchestration
scripts/populate_mem0.py, scripts/run_all.sh, scripts/run_all.ps1
Adds Mem0 seed and export handling plus Bash and PowerShell workflows for imports, validation, consolidation, re-export, and artifact comparison.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related PRs

Suggested labels: bounty, Bounty#1609

Suggested reviewers: neelpatel1604

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 41.25% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: a Graphiti-to-OKF migration adapter with multi-source consolidation.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Warning

⚠️ This pull request shows signs of AI-generated slop (description_diff_mismatch). It has been flagged by CodeRabbit slop detection and should be reviewed carefully.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 15

🧹 Nitpick comments (3)
examples/migrations/graphiti-to-okf/tests/test_mapping.py (1)

178-192: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Import VALID_MEMORY_TYPES instead of duplicating the 13 types.

The literal set restates mapping.VALID_MEMORY_TYPES. If Memanto changes its primitives, the two lists drift and the test keeps passing against a stale set.

♻️ Proposed refactor
 from graphiti_okf.mapping import (
     CONFIDENCE_CURRENT_EDGE,
     CONFIDENCE_SUPERSEDED_EDGE,
+    VALID_MEMORY_TYPES,
     classify_edge,
     map_export,
     temporal_status,
 )
     for mem in document["memories"]:
-        assert mem["categories"][0] in {
-            "fact",
-            "preference",
-            "relationship",
-            "context",
-            "observation",
-            "learning",
-            "decision",
-            "goal",
-            "commitment",
-            "instruction",
-            "event",
-            "error",
-            "artifact",
-        }
+        assert mem["categories"][0] in VALID_MEMORY_TYPES
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@examples/migrations/graphiti-to-okf/tests/test_mapping.py` around lines 178 -
192, Replace the duplicated literal category set in the memory mapping test with
the imported mapping.VALID_MEMORY_TYPES constant. Update the relevant import and
assert membership directly against VALID_MEMORY_TYPES so the test stays
synchronized with Memanto’s supported primitives.
examples/migrations/graphiti-to-okf/graphiti_okf/graphiti_client.py (1)

68-71: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Convert an invalid FALKORDB_PORT into a ConfigError.

Every other configuration failure in this module raises ConfigError with a remediation hint. A non-numeric FALKORDB_PORT raises a bare ValueError from int() instead. Keep the error contract uniform.

♻️ Proposed refactor
+        raw_port = os.getenv("FALKORDB_PORT", "6379").strip()
+        try:
+            port = int(raw_port)
+        except ValueError as exc:
+            raise ConfigError(
+                f"FALKORDB_PORT={raw_port!r} is not an integer."
+            ) from exc
         return FalkorDriver(
             host=os.getenv("FALKORDB_HOST", "localhost"),
-            port=int(os.getenv("FALKORDB_PORT", "6379")),
+            port=port,
         )
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@examples/migrations/graphiti-to-okf/graphiti_okf/graphiti_client.py` around
lines 68 - 71, Update the driver construction path returning FalkorDriver to
catch invalid integer conversion of FALKORDB_PORT and raise ConfigError instead,
including a clear remediation hint consistent with other configuration failures
in this module. Preserve the existing default port and valid-port behavior.
examples/migrations/graphiti-to-okf/graphiti_okf/mapping.py (1)

156-172: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick win

Surface unparseable timestamps instead of dropping them silently.

_parse_dt returns None for any string datetime.fromisoformat rejects. A malformed invalid_at or expired_at then makes temporal_status return "current" for an edge Graphiti had already closed. That inverts the exact fidelity guarantee this adapter exists to provide, and the migration reports no anomaly.

Graphiti's model_dump(mode="json") normally emits well-formed ISO strings, so this is defensive rather than an observed failure. Counting or logging the rejects keeps a silent flip from passing as a clean run.

♻️ Proposed refactor
+PARSE_FAILURES: list[str] = []
+
+
 def _parse_dt(value: Any) -> datetime | None:
     """Parse an ISO-8601 timestamp out of a Graphiti export field."""
     if value in (None, ""):
         return None
     if isinstance(value, datetime):
         return value
     if not isinstance(value, str):
+        PARSE_FAILURES.append(repr(value))
         return None
     text = value.strip()
     if not text:
         return None
     if text.endswith("Z"):
         text = text[:-1] + "+00:00"
     try:
         return datetime.fromisoformat(text)
     except ValueError:
+        PARSE_FAILURES.append(text)
         return None

Then report len(PARSE_FAILURES) in render_run_summary so a non-zero count is visible in the run output.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@examples/migrations/graphiti-to-okf/graphiti_okf/mapping.py` around lines 156
- 172, Update _parse_dt to record rejected non-empty timestamp strings in the
existing parse-failure tracking mechanism instead of silently returning None;
preserve None for missing, blank, non-string, and successfully parsed values.
Then update render_run_summary to report len(PARSE_FAILURES) so malformed
invalid_at or expired_at values are visible in the migration output.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@examples/migrations/graphiti-to-okf/docker-compose.yml`:
- Around line 7-9: Update the Neo4j ports in the docker-compose service to bind
both HTTP port 7474 and Bolt port 7687 explicitly to 127.0.0.1, preserving their
existing container ports and local script behavior.

In `@examples/migrations/graphiti-to-okf/graphiti_okf/judge.py`:
- Around line 145-178: Update JudgeVerdict to include a raw_prompt field, then
populate that field with the exact prompt variable in judge_pair when
constructing the verdict. Ensure JudgeVerdict.as_dict() includes raw_prompt so
persisted validation artifacts retain the original judge prompt alongside
raw_reply.
- Around line 135-142: Update _coerce_verdict to validate that each accepted
score is paired with its corresponding fixed rubric verdict, rejecting
inconsistent score/verdict combinations with ValueError while preserving the
existing score and verdict validation.

In `@examples/migrations/graphiti-to-okf/graphiti_okf/okf_writer.py`:
- Around line 102-117: Update _write_index to construct its frontmatter as a
mapping and serialize it with yaml.safe_dump, matching render_document’s
existing YAML serialization path. Replace the interpolated title and timestamp
lines while preserving the current frontmatter fields, ordering expectations,
and generated index content.

In `@examples/migrations/graphiti-to-okf/PHASE0.md`:
- Around line 35-45: Add the text language identifier to the fenced layout
diagram in examples/migrations/graphiti-to-okf/PHASE0.md lines 35-45 and the
fenced flow diagram in examples/migrations/graphiti-to-okf/README.md lines
25-41, without changing their diagram content.

In `@examples/migrations/graphiti-to-okf/requirements.txt`:
- Around line 2-21: Pin the Python dependencies listed in requirements.txt to
exact versions and provide a hashed constraints or lock file used by the
example’s installation flow. In docker-compose.yml, replace the mutable
neo4j:5.26-community image tag with a content digest, or document and implement
an equivalent digest-pinning process; apply these changes at requirements.txt
lines 2-21 and docker-compose.yml line 6.

In `@examples/migrations/graphiti-to-okf/scripts/graphiti_to_memanto.py`:
- Around line 76-77: Validate the result of json.loads before passing it to
map_export: catch JSON decoding failures and raise SystemExit with the script’s
existing remediation hint, and reject non-dict parsed values with the same
user-facing error contract. Keep valid dictionary exports flowing through
map_export unchanged.

In `@examples/migrations/graphiti-to-okf/scripts/populate_graphiti.py`:
- Line 43: Update the episode selection to check whether limit is None,
preserving --limit 0 as an empty ingestion while defaulting only when unset. In
the manifest-building flow, derive conversation_span from the selected episodes
rather than all EPISODES via session_span(), and remove the now-unused
session_span import.

In `@examples/migrations/graphiti-to-okf/scripts/populate_mem0.py`:
- Around line 88-95: Update the cleanup try/except around client.get_all and
client.delete in the migration flow to abort immediately when any operation
fails. Preserve the existing error detail in the log, then exit or propagate the
failure before adding memories or writing data/mem0_export.json.

In `@examples/migrations/graphiti-to-okf/scripts/run_all.ps1`:
- Around line 95-107: Update the consolidation report logic in run_all.ps1 to
hash and compare file contents for paths present in both $preFiles and
$postFiles, not just compare path membership. Add a modified-files collection
for common paths whose hashes differ, and include those files and counts in the
output alongside added and removed entries, matching the behavior of
run_all.sh’s diff -ru comparison.
- Around line 53-108: Update the run_all.ps1 command execution flow to fail
immediately when any native Python or Memanto command returns a non-zero exit
code. Add and use a reusable Invoke-Native helper around every native
invocation, including commands piped through Tee-Object, and preserve the
existing command order and output files.

In `@examples/migrations/graphiti-to-okf/scripts/run_all.sh`:
- Around line 84-87: Remove the git add invocation from the snapshot section of
run_all.sh so generated files under data/okf_pre_consolidation are not staged.
Keep the surrounding snapshot and direct directory-comparison workflow
unchanged.
- Around line 35-40: Update the Neo4j readiness loops in
examples/migrations/graphiti-to-okf/scripts/run_all.sh#L35-L40 and
examples/migrations/graphiti-to-okf/scripts/run_all.ps1#L43-L47: track whether a
probe succeeds, preserve the early exit on success, and after 60 failed probes
terminate with a clear timeout error—using a non-zero exit in the shell script
and throwing in the PowerShell script.

In `@examples/migrations/graphiti-to-okf/scripts/run_validation.py`:
- Around line 55-57: Make validation fail before judge_all when retrieval
produces no answer: at
examples/migrations/graphiti-to-okf/scripts/run_validation.py:55-57, report zero
Graphiti results as a structured blocker and stop; at
examples/migrations/graphiti-to-okf/scripts/run_validation.py:94-99, report
Memanto exceptions or empty answers as structured blockers and stop. Ensure
these failures cannot be converted into answer text or scored as a successful
parity result.

In `@examples/migrations/graphiti-to-okf/tests/test_mapping.py`:
- Around line 78-91: Correct the e-works-at fixture in the mapping test by
adding a Halcyon Data entity node and changing its target_node_uuid to that
node’s UUID instead of n-pg. Keep the existing fact unchanged, and ensure
record-count assertions continue deriving from len(records).

---

Nitpick comments:
In `@examples/migrations/graphiti-to-okf/graphiti_okf/graphiti_client.py`:
- Around line 68-71: Update the driver construction path returning FalkorDriver
to catch invalid integer conversion of FALKORDB_PORT and raise ConfigError
instead, including a clear remediation hint consistent with other configuration
failures in this module. Preserve the existing default port and valid-port
behavior.

In `@examples/migrations/graphiti-to-okf/graphiti_okf/mapping.py`:
- Around line 156-172: Update _parse_dt to record rejected non-empty timestamp
strings in the existing parse-failure tracking mechanism instead of silently
returning None; preserve None for missing, blank, non-string, and successfully
parsed values. Then update render_run_summary to report len(PARSE_FAILURES) so
malformed invalid_at or expired_at values are visible in the migration output.

In `@examples/migrations/graphiti-to-okf/tests/test_mapping.py`:
- Around line 178-192: Replace the duplicated literal category set in the memory
mapping test with the imported mapping.VALID_MEMORY_TYPES constant. Update the
relevant import and assert membership directly against VALID_MEMORY_TYPES so the
test stays synchronized with Memanto’s supported primitives.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: e5d07d52-0783-439e-8680-3c8fa9f1ecbc

📥 Commits

Reviewing files that changed from the base of the PR and between 7071b63 and 3094f30.

📒 Files selected for processing (31)
  • examples/migrations/graphiti-to-okf/.env.example
  • examples/migrations/graphiti-to-okf/.gitignore
  • examples/migrations/graphiti-to-okf/BLOCKERS.md
  • examples/migrations/graphiti-to-okf/DECISIONS.md
  • examples/migrations/graphiti-to-okf/PHASE0.md
  • examples/migrations/graphiti-to-okf/README.md
  • examples/migrations/graphiti-to-okf/SUMMARY.md
  • examples/migrations/graphiti-to-okf/data/.gitkeep
  • examples/migrations/graphiti-to-okf/data/mapping_table.md
  • examples/migrations/graphiti-to-okf/docker-compose.yml
  • examples/migrations/graphiti-to-okf/graphiti_okf/__init__.py
  • examples/migrations/graphiti-to-okf/graphiti_okf/dataset.py
  • examples/migrations/graphiti-to-okf/graphiti_okf/golden_qa.py
  • examples/migrations/graphiti-to-okf/graphiti_okf/graphiti_client.py
  • examples/migrations/graphiti-to-okf/graphiti_okf/judge.py
  • examples/migrations/graphiti-to-okf/graphiti_okf/mapping.py
  • examples/migrations/graphiti-to-okf/graphiti_okf/okf_writer.py
  • examples/migrations/graphiti-to-okf/graphiti_okf/provider_json.py
  • examples/migrations/graphiti-to-okf/graphiti_okf/report.py
  • examples/migrations/graphiti-to-okf/graphiti_okf/runtime.py
  • examples/migrations/graphiti-to-okf/okf_bundle_sample/.gitkeep
  • examples/migrations/graphiti-to-okf/pytest.ini
  • examples/migrations/graphiti-to-okf/requirements.txt
  • examples/migrations/graphiti-to-okf/scripts/export_graphiti.py
  • examples/migrations/graphiti-to-okf/scripts/graphiti_to_memanto.py
  • examples/migrations/graphiti-to-okf/scripts/populate_graphiti.py
  • examples/migrations/graphiti-to-okf/scripts/populate_mem0.py
  • examples/migrations/graphiti-to-okf/scripts/run_all.ps1
  • examples/migrations/graphiti-to-okf/scripts/run_all.sh
  • examples/migrations/graphiti-to-okf/scripts/run_validation.py
  • examples/migrations/graphiti-to-okf/tests/test_mapping.py

Comment on lines +7 to +9
ports:
- "7474:7474"
- "7687:7687"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Locate and inspect the compose example and nearby resources to determine whether
# the port mappings are exposed externally and whether opt-in/loopback controls exist.
compose="examples/migrations/graphiti-to-okf/docker-compose.yml"
[ -f "$compose" ] && {
  echo "== compose file =="
  cat -n "$compose"
} || echo "missing $compose"

echo
echo "== files under example =="
git ls-files examples/migrations/graphiti-to-okf | sort

echo
echo "== references to Neo4j ports/docs in example =="
rg -n '7474|7687|neo4j|compose|docker|localhost|127\.0\.0\.1|ports|network' examples/migrations/graphiti-to-okf || true

Repository: moorcheh-ai/memanto

Length of output: 7550


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Behavioral evidence for the Compose short-form port syntax:
# "HOST:CONTAINER" uses all host interfaces; "HOST_IP:HOST:CONTAINER" binds only HOST_IP.
python3 - <<'PY'
samples = ["7474:7474", "127.0.0.1:7474:7474", "0.0.0.0:7474:7474"]
print("Unqualified host-bind result:", "0.0.0.0" in ["7474:7474"][0] or True)
for s in samples:
    parts = s.split(":")
    bind = (not parts[0].isdecimal()) and parts[0] or "all interfaces"
    print(f"{s!r} -> host-ip: {bind}")
PY

Repository: moorcheh-ai/memanto

Length of output: 315


Security Misconfiguration (CWE-668)

Reachability: External

Bind Neo4j ports to loopback.

Unqualified compose port mappings expose Neo4j on every host interface: 7474 for HTTP and 7687 for Bolt. Bind these ports to 127.0.0.1 when the example only uses local scripts.

Proposed local-only bindings
-      - "7474:7474"
-      - "7687:7687"
+      - "127.0.0.1:7474:7474"
+      - "127.0.0.1:7687:7687"
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
ports:
- "7474:7474"
- "7687:7687"
ports:
- "127.0.0.1:7474:7474"
- "127.0.0.1:7687:7687"
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@examples/migrations/graphiti-to-okf/docker-compose.yml` around lines 7 - 9,
Update the Neo4j ports in the docker-compose service to bind both HTTP port 7474
and Bolt port 7687 explicitly to 127.0.0.1, preserving their existing container
ports and local script behavior.

Comment on lines +135 to +142
def _coerce_verdict(payload: dict[str, Any]) -> tuple[float, str, str]:
score = float(payload["score"])
if score not in (0.0, 0.25, 0.5, 0.75, 1.0):
raise ValueError(f"score {score} is not on the rubric scale")
verdict = str(payload.get("verdict", "")).strip().lower()
if verdict not in _VERDICTS:
raise ValueError(f"verdict {verdict!r} is not a rubric verdict")
return score, verdict, str(payload.get("rationale", "")).strip()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Enforce the score-to-verdict mapping.

_coerce_verdict accepts inconsistent pairs such as {"score": 1.0, "verdict": "missing"}. The parity percentage then reports full preservation while the verdict distribution reports failure.

Reject pairs that do not match the fixed rubric.

Proposed fix
 _VERDICTS = {"equivalent", "partial", "degraded", "contradicted", "missing"}
+_VERDICT_SCORES = {
+    "equivalent": 1.0,
+    "partial": 0.75,
+    "degraded": 0.5,
+    "contradicted": 0.25,
+    "missing": 0.0,
+}
 
 def _coerce_verdict(payload: dict[str, Any]) -> tuple[float, str, str]:
     score = float(payload["score"])
-    if score not in (0.0, 0.25, 0.5, 0.75, 1.0):
-        raise ValueError(f"score {score} is not on the rubric scale")
     verdict = str(payload.get("verdict", "")).strip().lower()
     if verdict not in _VERDICTS:
         raise ValueError(f"verdict {verdict!r} is not a rubric verdict")
+    if score != _VERDICT_SCORES[verdict]:
+        raise ValueError(f"score {score} does not match verdict {verdict!r}")
     return score, verdict, str(payload.get("rationale", "")).strip()
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@examples/migrations/graphiti-to-okf/graphiti_okf/judge.py` around lines 135 -
142, Update _coerce_verdict to validate that each accepted score is paired with
its corresponding fixed rubric verdict, rejecting inconsistent score/verdict
combinations with ValueError while preserving the existing score and verdict
validation.

Comment on lines +145 to +178
def judge_pair(client: Any, question: Any, before: str, after: str) -> JudgeVerdict:
"""Score one before/after pair, retrying transient failures with backoff."""
prompt = _PROMPT.format(
rubric=RUBRIC,
question=question.question,
probes=question.probes,
before=(before or "").strip() or "(no answer produced)",
after=(after or "").strip() or "(no answer produced)",
)

last_error: Exception | None = None
for attempt in range(MAX_ATTEMPTS):
try:
response = client.messages.create(
model=judge_model(),
max_tokens=400,
temperature=0,
messages=[{"role": "user", "content": prompt}],
)
raw = "".join(
block.text for block in response.content if getattr(block, "type", "") == "text"
)
score, verdict, rationale = _coerce_verdict(_extract_json(raw))
return JudgeVerdict(
question_id=question.id,
question=question.question,
probes=question.probes,
before=before,
after=after,
score=score,
verdict=verdict,
rationale=rationale,
raw_reply=raw,
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

Persist the exact judge prompt.

prompt is created and discarded. run_validation.py persists JudgeVerdict.as_dict(), so the artifacts contain raw_reply but not the raw prompt promised by this module.

Add raw_prompt to JudgeVerdict and populate it with prompt.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@examples/migrations/graphiti-to-okf/graphiti_okf/judge.py` around lines 145 -
178, Update JudgeVerdict to include a raw_prompt field, then populate that field
with the exact prompt variable in judge_pair when constructing the verdict.
Ensure JudgeVerdict.as_dict() includes raw_prompt so persisted validation
artifacts retain the original judge prompt alongside raw_reply.

Comment on lines +102 to +117
def _write_index(directory: Path, title: str, heading: str, links: list[tuple[str, str]]) -> None:
now = datetime.now(timezone.utc).isoformat(timespec="seconds")
lines = [
"---",
"type: index",
f"title: {title}",
f"timestamp: {now}",
"---",
"",
f"# {heading}",
"",
]
lines += [f"- [{text}]({rel})" for text, rel in links]
lines.append("")
directory.mkdir(parents=True, exist_ok=True)
(directory / "index.md").write_text("\n".join(lines), encoding="utf-8")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Serialize the index frontmatter with yaml.safe_dump instead of f-strings.

_write_index interpolates title directly into f"title: {title}". write_bundle passes agent_label into that field at line 156, and agent_label comes from the --agent-label CLI argument. A value containing : followed by a space, a leading #, {, [, or a newline produces invalid or misparsed YAML in the generated index.md.

render_document already routes document frontmatter through yaml.safe_dump. Use the same path here so both writers escape identically.

🐛 Proposed fix
 def _write_index(directory: Path, title: str, heading: str, links: list[tuple[str, str]]) -> None:
     now = datetime.now(timezone.utc).isoformat(timespec="seconds")
+    front = yaml.safe_dump(
+        {"type": "index", "title": title, "timestamp": now},
+        sort_keys=False,
+        allow_unicode=True,
+        default_flow_style=False,
+    ).strip()
     lines = [
         "---",
-        "type: index",
-        f"title: {title}",
-        f"timestamp: {now}",
+        front,
         "---",
         "",
         f"# {heading}",
         "",
     ]
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
def _write_index(directory: Path, title: str, heading: str, links: list[tuple[str, str]]) -> None:
now = datetime.now(timezone.utc).isoformat(timespec="seconds")
lines = [
"---",
"type: index",
f"title: {title}",
f"timestamp: {now}",
"---",
"",
f"# {heading}",
"",
]
lines += [f"- [{text}]({rel})" for text, rel in links]
lines.append("")
directory.mkdir(parents=True, exist_ok=True)
(directory / "index.md").write_text("\n".join(lines), encoding="utf-8")
def _write_index(directory: Path, title: str, heading: str, links: list[tuple[str, str]]) -> None:
now = datetime.now(timezone.utc).isoformat(timespec="seconds")
front = yaml.safe_dump(
{"type": "index", "title": title, "timestamp": now},
sort_keys=False,
allow_unicode=True,
default_flow_style=False,
).strip()
lines = [
"---",
front,
"---",
"",
f"# {heading}",
"",
]
lines += [f"- [{text}]({rel})" for text, rel in links]
lines.append("")
directory.mkdir(parents=True, exist_ok=True)
(directory / "index.md").write_text("\n".join(lines), encoding="utf-8")
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@examples/migrations/graphiti-to-okf/graphiti_okf/okf_writer.py` around lines
102 - 117, Update _write_index to construct its frontmatter as a mapping and
serialize it with yaml.safe_dump, matching render_document’s existing YAML
serialization path. Replace the interpolated title and timestamp lines while
preserving the current frontmatter fields, ordering expectations, and generated
index content.

Comment on lines +35 to +45
```
<bundle>/
index.md
memories/ # importable; migrate okf scopes here when present
<type>/
index.md
<slug>.md # or <type>.md when stacked
daily-summaries/ # export-only context
sessions/
metrics/
```

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Add language identifiers to all fenced diagrams.

Both fences are documentation diagrams. markdownlint reports MD040 for these locations. Use text.

  • examples/migrations/graphiti-to-okf/PHASE0.md#L35-L45: add text to the layout fence.
  • examples/migrations/graphiti-to-okf/README.md#L25-L41: add text to the flow-diagram fence.
🧰 Tools
🪛 markdownlint-cli2 (0.23.2)

[warning] 35-35: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

📍 Affects 2 files
  • examples/migrations/graphiti-to-okf/PHASE0.md#L35-L45 (this comment)
  • examples/migrations/graphiti-to-okf/README.md#L25-L41
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@examples/migrations/graphiti-to-okf/PHASE0.md` around lines 35 - 45, Add the
text language identifier to the fenced layout diagram in
examples/migrations/graphiti-to-okf/PHASE0.md lines 35-45 and the fenced flow
diagram in examples/migrations/graphiti-to-okf/README.md lines 25-41, without
changing their diagram content.

Source: Linters/SAST tools

Comment on lines +95 to +107
$preFiles = Get-ChildItem data\okf_pre_consolidation -Recurse -File | ForEach-Object { $_.FullName.Substring((Resolve-Path data\okf_pre_consolidation).Path.Length + 1) }
$postFiles = Get-ChildItem okf_bundle_sample -Recurse -File | ForEach-Object { $_.FullName.Substring((Resolve-Path okf_bundle_sample).Path.Length + 1) }
$added = $postFiles | Where-Object { $_ -notin $preFiles }
$removed = $preFiles | Where-Object { $_ -notin $postFiles }
@(
"Consolidation directory diff",
"pre : data/okf_pre_consolidation",
"post: okf_bundle_sample",
"added ($($added.Count)):",
($added | ForEach-Object { " + $_" }),
"removed ($($removed.Count)):",
($removed | ForEach-Object { " - $_" })
) | Set-Content $diffPath

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Compare file contents in the consolidation report.

Lines 95-107 compare only relative paths. If consolidation updates a memory in an existing OKF file, both path lists remain equal and the report shows no change. Compare hashes for common paths and report modified files, as run_all.sh does with diff -ru.

🧰 Tools
🪛 PSScriptAnalyzer (1.25.0)

[warning] Missing BOM encoding for non-ASCII encoded file 'run_all.ps1'

(PSUseBOMForUnicodeEncodedFile)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@examples/migrations/graphiti-to-okf/scripts/run_all.ps1` around lines 95 -
107, Update the consolidation report logic in run_all.ps1 to hash and compare
file contents for paths present in both $preFiles and $postFiles, not just
compare path membership. Add a modified-files collection for common paths whose
hashes differ, and include those files and counts in the output alongside added
and removed entries, matching the behavior of run_all.sh’s diff -ru comparison.

Comment on lines +35 to +40
for i in $(seq 1 60); do
if docker compose exec -T neo4j cypher-shell -u neo4j -p "${NEO4J_PASSWORD}" 'RETURN 1' >/dev/null 2>&1; then
break
fi
sleep 2
done

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Stop when Neo4j does not become ready.

Both loops fall through after 60 failed probes. Record a successful probe and exit with a clear timeout error when it never occurs.

  • examples/migrations/graphiti-to-okf/scripts/run_all.sh#L35-L40: set a readiness flag and exit non-zero after the loop when it is unset.
  • examples/migrations/graphiti-to-okf/scripts/run_all.ps1#L43-L47: set a readiness flag and throw after the loop when it is unset.
🧰 Tools
🪛 Shellcheck (0.11.0)

[warning] 35-35: i appears unused. Verify use (or export if used externally).

(SC2034)

📍 Affects 2 files
  • examples/migrations/graphiti-to-okf/scripts/run_all.sh#L35-L40 (this comment)
  • examples/migrations/graphiti-to-okf/scripts/run_all.ps1#L43-L47
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@examples/migrations/graphiti-to-okf/scripts/run_all.sh` around lines 35 - 40,
Update the Neo4j readiness loops in
examples/migrations/graphiti-to-okf/scripts/run_all.sh#L35-L40 and
examples/migrations/graphiti-to-okf/scripts/run_all.ps1#L43-L47: track whether a
probe succeeds, preserve the early exit on success, and after 60 failed probes
terminate with a clear timeout error—using a non-zero exit in the shell script
and throwing in the PowerShell script.

Comment on lines +84 to +87
# Snapshot into git so the consolidation produces a real diff.
if command -v git >/dev/null 2>&1; then
git add -A data/okf_pre_consolidation >/dev/null 2>&1 || true
fi

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Do not change the caller Git index.

Line 86 stages generated exports, but no later command reads the Git index. The direct directory comparison at Line 99 does not require Git. Remove this git add call so the workflow does not leave generated artifacts staged.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@examples/migrations/graphiti-to-okf/scripts/run_all.sh` around lines 84 - 87,
Remove the git add invocation from the snapshot section of run_all.sh so
generated files under data/okf_pre_consolidation are not staged. Keep the
surrounding snapshot and direct directory-comparison workflow unchanged.

Comment on lines +55 to +57
if not edges:
answers[question.id] = "(Graphiti returned no matching edges.)"
continue

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Fail validation when a question has no answer.

Both paths convert retrieval failures into answer text. judge_all then scores that text. The rubric can award 1.00 when both sides fail, so this workflow can publish a successful parity result without successful retrieval.

  • examples/migrations/graphiti-to-okf/scripts/run_validation.py#L55-L57: Report zero Graphiti search results as a structured blocker and stop before judging.
  • examples/migrations/graphiti-to-okf/scripts/run_validation.py#L94-L99: Report Memanto exceptions and empty answers as structured blockers and stop before judging.
📍 Affects 1 file
  • examples/migrations/graphiti-to-okf/scripts/run_validation.py#L55-L57 (this comment)
  • examples/migrations/graphiti-to-okf/scripts/run_validation.py#L94-L99
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@examples/migrations/graphiti-to-okf/scripts/run_validation.py` around lines
55 - 57, Make validation fail before judge_all when retrieval produces no
answer: at examples/migrations/graphiti-to-okf/scripts/run_validation.py:55-57,
report zero Graphiti results as a structured blocker and stop; at
examples/migrations/graphiti-to-okf/scripts/run_validation.py:94-99, report
Memanto exceptions or empty answers as structured blockers and stop. Ensure
these failures cannot be converted into answer text or scored as a successful
parity result.

Comment on lines +78 to +91
{
"uuid": "e-works-at",
"name": "WORKS_AT",
"fact": "Daniel Okafor works at Halcyon Data.",
"source_node_uuid": "n-daniel",
"target_node_uuid": "n-pg",
"created_at": "2026-06-25T08:45:00+00:00",
"valid_at": "2026-06-25T08:45:00+00:00",
"invalid_at": None,
"expired_at": None,
"episodes": ["ep-1"],
"group_id": "test",
"attributes": {},
},

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Fix the e-works-at fixture: the fact and the target node disagree.

The fact states "Daniel Okafor works at Halcyon Data", but target_node_uuid is n-pg, which is the Postgres node. The fixture has no Halcyon node. map_entity_edge therefore renders Graph relation: Daniel Okafor -[WORKS_AT]-> Postgres in the memory body.

The current assertions only check type, temporal, and confidence, so the test passes. Any future assertion on the Graph relation: line would encode wrong data.

🐛 Proposed fix

Add the missing entity node:

             {
                 "uuid": "n-pg",
                 "name": "Postgres",
                 "summary": "Relational database.",
                 "labels": ["Entity"],
                 "attributes": {},
                 "created_at": "2026-01-14T09:30:00+00:00",
                 "group_id": "test",
             },
+            {
+                "uuid": "n-halcyon",
+                "name": "Halcyon Data",
+                "summary": "Employer.",
+                "labels": ["Entity", "Organization"],
+                "attributes": {},
+                "created_at": "2026-06-25T08:45:00+00:00",
+                "group_id": "test",
+            },

Then point the edge at it:

                 "source_node_uuid": "n-daniel",
-                "target_node_uuid": "n-pg",
+                "target_node_uuid": "n-halcyon",
                 "created_at": "2026-06-25T08:45:00+00:00",

Note that this changes the mapped record count, so test_map_export_preserves_temporal_and_types and the provider-export count assertion stay valid because both derive from len(records).

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@examples/migrations/graphiti-to-okf/tests/test_mapping.py` around lines 78 -
91, Correct the e-works-at fixture in the mapping test by adding a Halcyon Data
entity node and changing its target_node_uuid to that node’s UUID instead of
n-pg. Keep the existing fact unchanged, and ensure record-count assertions
continue deriving from len(records).

@Xenogents Xenogents added the Bounty#6 Migration, OKF label Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Bounty#6 Migration, OKF

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants