Entity extraction is nondeterministic at temperature 0 — same input, same container, different entity sets

[outerheaven666]

摘要：即使把建群相关的温度全部设为 0，同一份输入在同一个容器内连续两次 ontology/generate + graph/build，仍会产出不同的实体集。这使得 #751 建议的「把温度做成可配置」不足以获得可复现的人群。此外，抽取出的非人名实体（包括 MiroFish 本身）会被提升为仿真人格。

---

This is a follow-up to #751 and, I think, the layer underneath it. #751 asks for the four cohort-construction temperatures to be configurable so that reruns build the same crowd. Having set all four to 0, reruns still build different crowds — so temperature was not the only source.

Reproduction

Same container, same image, same input file, all cohort temperatures at 0. Two consecutive builds:

POST /api/graph/ontology/generate   (packet + participant roster)
POST /api/graph/build

Then compare Entity nodes by graph_id:

MATCH (e:Entity {graph_id: $gid}) RETURN e.name ORDER BY e.name;

	entities
build 1	27
build 2	26
shared	24

Only in build 1: Person, bear case, breaking headlines, sourced color, thread summaries of what each side believes
Only in build 2: Client, HedgeFundManager

The 18 real person names are identical across both builds. Every difference is in non-person entities.

One caveat on method: the build task reports completed while entities are still being written. Reading too early gave 8 entities against 26–27 once settled. The counts above were taken after the entity count stopped changing.

The non-person entities are sentence fragments, and they become personas

That list is not a stray edge case. Entities extracted from the same input include:

• edges
• None
• market event
• Posts 3-5 times daily: cross-asset takes, historical analogies ('I've seen this movie before'), decay-of-panic arguments.
• supply chain detail, management call parsing, unit economics
• MiroFish — the project's own name, extracted from the input text

And these are promoted to simulated agents. In one of our runs the persona roster of 20 contained market event and MiroFish as two of the twenty "people". They were interviewed and they answered.

The project's own simulation_requirement states the constraint explicitly:

Agent personas MUST be individual people (Person entities), NOT companies; companies mentioned in the event are discussion topics, not participants.

So the filter is not merely intended — it is written into the prompt that drives cohort construction. MiroFish and market event became personas anyway.

Why this matters beyond tidiness

The count of extracted entities determines the persona count (26 entities → 19 personas, 28 → 20 in our runs), and the entity set feeds the graph context that persona generation reads. So a varying entity set produces:

• a varying persona count, and
• different persona descriptions under identical names — across four builds of one input, the same 18–20 names came back with the persona field differing 19/19, 19/19, 19/19, 3/20, 3/20 and 0/20 across the six pairs

The downstream effect is large and very uneven. Mapping each run's interview results to a probability vector and taking pairwise L2 distances over those four runs:

0.0393   0.2481   0.2828   0.3165   0.5572   0.5877

A fifteen-fold spread between the closest and the furthest pair.

A correction to what I wrote in #751. I said there that three of four runs produced "byte-identical persona rosters". That was a hash of persona names only. Comparing the full profiles, only one pair of runs (3 and 4) had identical persona text; two other pairs differed on 3 of 20. And the relationship is not monotone at this sample size: the pair with identical personas differs by 0.2481 in the final reading, while the closest pair (0.0393) had three personas differing. Identical names do not imply identical people, and identical people did not produce identical readings.

What I have not established

I do not know the cause. Temperature is 0 at all four cohort sites, verified inside the running container before each run. Concurrency or chunk ordering during extraction is the obvious suspect — the build is chunked at 500 with 50 overlap — but I have not tested it and am not asserting it.

Nor is this only an extraction-layer issue: on a fixed graph_id, two prepare calls still produced persona differing 20/20 and interested_topics 19/20. I am keeping that out of this report to stay on one topic, and can open it separately if useful.

Suggestions, in order of value

1. Filter non-person entities before persona promotion. Independent of the determinism question, MiroFish and market event should not be interviewable agents.
2. Make extraction reproducible, or document that it is not. If the fix in Cohort construction is non-deterministic: the same input builds a different persona population on every run #751 ships as "reproducible cohorts", this will make that claim false in practice.
3. If neither is quick: document that a stable graph_id does not imply a stable cohort, so downstream users do not pin the wrong thing.

Happy to test a patch — the reproduction above takes about five minutes per attempt.