Skip to content

Commit f3098ce

Browse files
jpheinclaude
andcommitted
docs: add scripts/check-docs.sh + fix typo
- New scripts/check-docs.sh: passive drift detection for fork docs. Checks (1) test count in README matches `pytest --collect-only`, (2) every fork commit hash referenced in CLAUDE.md / README.md / FORK_CHANGELOG.md resolves via `git cat-file -e`, (3) every #NNNN PR reference has a state matching what the doc claims (skipped gracefully if `gh` isn't authenticated). Cross-repo URL stripping + multi-PR-on-one-line skip avoid false positives. Documented as Layer 1 in the doc-update conversation. - Fix typo `552d0d5` → `552a0d7` in README (the actual fork commit cherry-pick of upstream MemPalace#976's HNSW num_threads pin) — caught by the new check-docs.sh on first run. Layer 3 (canonical-source rendering for FORK_CHANGELOG / README fork-change-queue table / CLAUDE.md row inventory / promises tracker) is the next commit; this lands the lint first so the canonical renderer can use it as part of its own validation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 54e6de8 commit f3098ce

2 files changed

Lines changed: 173 additions & 1 deletion

File tree

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -377,7 +377,7 @@ Two fork commits made the migration concrete:
377377
- **Marker guard ([#1177](https://github.com/MemPalace/mempalace/pull/1177)):** `.blob_seq_ids_migrated` sentinel file skips `sqlite3.connect()` on already-migrated palaces — opening sqlite against a live ChromaDB 1.5.x WAL database corrupts the next `PersistentClient`. Closes #1090.
378378
- **Search BM25 None guard ([#1198](https://github.com/MemPalace/mempalace/pull/1198), filed 2026-04-24):** `_tokenize` short-circuits to `[]` for `None` documents — closes the gap upstream's #999 None-metadata audit left in BM25 helpers.
379379

380-
Felipe's `hnsw:num_threads: 1` pin from #976 (cherry-picked into the fork as commit `552d0d5` and now natively merged via the 2026-04-25 develop sync) is the actual root-cause fix for the parallel-HNSW race — applied at collection-creation metadata + via `_pin_hnsw_threads()` on every `get_collection` (ChromaDB 1.5.x doesn't persist the modified config across reopens). The daemon serializes around that fix at a higher layer; the two compose cleanly.
380+
Felipe's `hnsw:num_threads: 1` pin from #976 (cherry-picked into the fork as commit `552a0d7` and now natively merged via the 2026-04-25 develop sync) is the actual root-cause fix for the parallel-HNSW race — applied at collection-creation metadata + via `_pin_hnsw_threads()` on every `get_collection` (ChromaDB 1.5.x doesn't persist the modified config across reopens). The daemon serializes around that fix at a higher layer; the two compose cleanly.
381381

382382
**Postgres + pgvector — long-term option, no immediate move.** RFC 001's backend seam is merged (#413, #995) and the registry already advertises `mempalace_postgres` as the canonical entry-point example. @skuznetsov's [#665](https://github.com/MemPalace/mempalace/pull/665) ships the actual PostgreSQL backend implementation (`pg_sorted_heap` preferred path, `pgvector` fallback); @malakhov-dmitrii's [#1072](https://github.com/MemPalace/mempalace/pull/1072) wires `palace._DEFAULT_BACKEND` through the registry so `MEMPALACE_BACKEND=postgres` actually takes effect. When both land, switching is `pip install mempalace-postgres && export MEMPALACE_BACKEND=postgres`. Postgres would eliminate the entire ChromaDB 1.5.x failure class natively (MVCC, no HNSW drift, no Rust-binding segfaults), but with the daemon now serializing access cleanly, the migration cost (151K+ drawers off ChromaDB via `export_palace()` + a Postgres importer) isn't justified by current pain. Re-evaluate if the daemon proves unstable, or once bensig's TypeScript rewrite picks its own storage layer.
383383

scripts/check-docs.sh

Lines changed: 172 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,172 @@
1+
#!/usr/bin/env bash
2+
# check-docs.sh — sanity-check that fork docs are still in sync with reality.
3+
#
4+
# What it checks:
5+
# 1. Test count in README matches `pytest --collect-only -q` reality.
6+
# 2. Every fork commit hash referenced in CLAUDE.md / README.md /
7+
# FORK_CHANGELOG.md actually resolves via `git cat-file -e`.
8+
# 3. Every upstream PR mentioned (#NNNN) has a state matching what the
9+
# doc claims (OPEN / MERGED / CLOSED). Uses `gh pr view`; skipped
10+
# gracefully if `gh` isn't authenticated.
11+
#
12+
# Exit codes:
13+
# 0 — clean
14+
# 1 — at least one drift detected
15+
# 2 — internal error (e.g., not in a git repo)
16+
#
17+
# Usage:
18+
# scripts/check-docs.sh # interactive run
19+
# scripts/check-docs.sh --quiet # only print failures
20+
# STRICT_PR_STATE=1 scripts/check-docs.sh # warn → error on PR-state drift
21+
22+
set -uo pipefail
23+
shopt -s nullglob
24+
25+
REPO_ROOT="$(git rev-parse --show-toplevel 2>/dev/null)" || {
26+
echo "✗ not a git repo" >&2
27+
exit 2
28+
}
29+
cd "$REPO_ROOT"
30+
31+
quiet=0
32+
[ "${1:-}" = "--quiet" ] && quiet=1
33+
34+
step() { (( quiet )) || printf '\n\033[1m▸ %s\033[0m\n' "$1"; }
35+
ok() { (( quiet )) || printf ' \033[32m✓\033[0m %s\n' "$1"; }
36+
warn() { printf ' \033[33m!\033[0m %s\n' "$1" >&2; }
37+
fail() { printf ' \033[31m✗\033[0m %s\n' "$1" >&2; ((failures++)); }
38+
39+
failures=0
40+
41+
# ── 1. test count ────────────────────────────────────────────────────────
42+
step "1/3 test count in README"
43+
readme_count=$(grep -oE '^[0-9]+ tests pass on `main`' README.md | grep -oE '^[0-9]+' || echo "")
44+
if [ -z "$readme_count" ]; then
45+
warn "README has no '<N> tests pass on \`main\`' line — skipping"
46+
else
47+
actual_count=$(python -m pytest --collect-only -q 2>/dev/null | grep -E "^[0-9]+ tests collected" | head -1 | awk '{print $1}' || echo "")
48+
if [ -z "$actual_count" ]; then
49+
warn "could not run pytest --collect-only (venv inactive?) — skipping"
50+
elif [ "$readme_count" != "$actual_count" ]; then
51+
fail "README says $readme_count, pytest collects $actual_count"
52+
else
53+
ok "README $readme_count == pytest $actual_count"
54+
fi
55+
fi
56+
57+
# ── 2. commit hash references ────────────────────────────────────────────
58+
step "2/3 commit hashes referenced in docs resolve"
59+
docs=(README.md CLAUDE.md FORK_CHANGELOG.md)
60+
# Strip cross-repo URLs first so we only check hashes that should resolve
61+
# in *this* fork. Pattern: anything inside (https://github.com/<other>/<repo>/commit/HASH)
62+
# where <other>/<repo> is not jphein/mempalace.
63+
# For each line, skip the line entirely if it mentions a sibling repo
64+
# (palace-daemon / multipass-structural-memory-eval) — we can't tell which
65+
# hashes on that line are fork-mempalace vs cross-repo without parsing
66+
# linked URLs by repo. Treating the whole line as cross-repo is the
67+
# conservative under-call: false negatives (missing a real bad hash
68+
# adjacent to a sibling-repo mention) but no false positives.
69+
mapfile -t hashes < <(
70+
for d in "${docs[@]}"; do
71+
grep -v -E 'palace-daemon|multipass-structural-memory-eval|/jphein/[a-z-]+/commit/' "$d" 2>/dev/null
72+
done | grep -hoE '`[0-9a-f]{7,40}`' | tr -d '`' | sort -u
73+
)
74+
unresolved=0
75+
for h in "${hashes[@]}"; do
76+
if ! git cat-file -e "$h" 2>/dev/null; then
77+
fail "commit hash \`$h\` referenced in docs but does not resolve in this repo"
78+
((unresolved++))
79+
fi
80+
done
81+
if (( unresolved == 0 )) && (( ${#hashes[@]} > 0 )); then
82+
ok "all ${#hashes[@]} fork hash references resolve"
83+
fi
84+
85+
# ── 3. upstream PR states ────────────────────────────────────────────────
86+
step "3/3 upstream PR states match doc claims"
87+
if ! command -v gh >/dev/null 2>&1; then
88+
warn "gh not on PATH — skipping PR state check"
89+
elif ! gh auth status >/dev/null 2>&1; then
90+
warn "gh not authenticated — skipping PR state check"
91+
else
92+
# Pull every #NNNN reference from the docs, dedupe.
93+
mapfile -t pr_numbers < <(grep -hoE '#[0-9]{2,5}' "${docs[@]}" 2>/dev/null \
94+
| grep -oE '[0-9]+' | sort -u)
95+
drift=0
96+
for n in "${pr_numbers[@]}"; do
97+
# Heuristic: only check PRs (not issues). gh handles either; on
98+
# state==null we assume it's an issue and skip.
99+
state=$(gh pr view "$n" --repo MemPalace/mempalace --json state \
100+
--jq '.state' 2>/dev/null || echo "")
101+
[ -z "$state" ] && continue
102+
# Pull all doc lines mentioning this PR for context comparison.
103+
# We don't try to parse exhaustively; just flag when a doc says
104+
# MERGED but gh says OPEN, or vice versa.
105+
doc_says_merged=0; doc_says_open=0; doc_says_closed=0
106+
for d in "${docs[@]}"; do
107+
line=$(grep -E "(#$n|/$n)" "$d" 2>/dev/null | head -1 | tr A-Z a-z)
108+
[[ "$line" == *"merged"* ]] && doc_says_merged=1
109+
[[ "$line" == *"open"* ]] && doc_says_open=1
110+
[[ "$line" == *"closed"* ]] && doc_says_closed=1
111+
done
112+
# Skip narrative paragraphs that mention multiple PRs — words
113+
# like "merged" / "open" usually refer to *other* PRs on the
114+
# same line, not the one we're checking. Only check lines that
115+
# mention this PR alone.
116+
for d in "${docs[@]}"; do
117+
line=$(grep -E "(#$n[^0-9]|/$n[^0-9])" "$d" 2>/dev/null | head -1)
118+
other_prs=$(echo "$line" | grep -oE '#[0-9]{2,5}' | grep -v "^#$n$" | wc -l)
119+
if (( other_prs > 0 )); then
120+
doc_says_merged=0; doc_says_open=0; doc_says_closed=0
121+
fi
122+
done
123+
# If both states appear, it's commentary too.
124+
if (( doc_says_merged )) && (( doc_says_open )); then
125+
continue
126+
fi
127+
case "$state" in
128+
MERGED)
129+
if (( doc_says_open )) && (( ! doc_says_merged )); then
130+
if [ "${STRICT_PR_STATE:-0}" = "1" ]; then
131+
fail "PR #$n is MERGED upstream, docs still say OPEN"
132+
else
133+
warn "PR #$n is MERGED upstream, docs still say OPEN"
134+
fi
135+
((drift++))
136+
fi
137+
;;
138+
OPEN)
139+
if (( doc_says_merged )) && (( ! doc_says_open )); then
140+
if [ "${STRICT_PR_STATE:-0}" = "1" ]; then
141+
fail "PR #$n is OPEN upstream, docs say MERGED"
142+
else
143+
warn "PR #$n is OPEN upstream, docs say MERGED"
144+
fi
145+
((drift++))
146+
fi
147+
;;
148+
CLOSED)
149+
if (( doc_says_open )) && (( ! doc_says_closed )); then
150+
if [ "${STRICT_PR_STATE:-0}" = "1" ]; then
151+
fail "PR #$n is CLOSED (not merged), docs say OPEN"
152+
else
153+
warn "PR #$n is CLOSED (not merged), docs say OPEN"
154+
fi
155+
((drift++))
156+
fi
157+
;;
158+
esac
159+
done
160+
if (( drift == 0 )) && (( ${#pr_numbers[@]} > 0 )); then
161+
ok "all ${#pr_numbers[@]} PR references match upstream state"
162+
fi
163+
fi
164+
165+
# ── summary ──────────────────────────────────────────────────────────────
166+
if (( failures == 0 )); then
167+
(( quiet )) || printf '\n\033[1;32m✦ docs clean\033[0m\n'
168+
exit 0
169+
else
170+
printf '\n\033[1;31m✗ %d issue(s) found\033[0m\n' "$failures" >&2
171+
exit 1
172+
fi

0 commit comments

Comments
 (0)