LLM Stochastic Bias

Testing whether LLMs can sample from probability distributions or just pick the most likely option. 5.6M API calls across GPT-5.4, 5.4-mini, 5-mini, and 4.1. Spoiler: they can't.

[View the interactive results dashboard]

The Question

When you tell an LLM "pick a state with 20% probability," does it actually produce that state ~20% of the time across many independent calls? Or does it just pick the most likely option every time?

Results at a Glance

Model	State 6 picks (expected 20%)	Verdict
GPT-5.4-mini	99.94%	Near-collapse
GPT-5.4	55.75%	Only model with spread
GPT-5-mini	100.00%	Total collapse
GPT-4.1	100.00%	Total collapse

Each model was tested with 1,000,000 independent API calls (temperature=1.0).

Two Experiments

Part 1: Flat Distribution (10 states)

The model is given a probability distribution over 10 states and asked to "pick one" per call. No code execution, no random library -- just its own sense of probability.

stateDiagram-v2
    [*] --> S1: 3.0%
    [*] --> S2: 5.0%
    [*] --> S3: 8.5%
    [*] --> S4: 11.5%
    [*] --> S5: 15.0%
    [*] --> S6: 20.0%
    [*] --> S7: 15.0%
    [*] --> S8: 10.0%
    [*] --> S9: 7.0%
    [*] --> S10: 5.0%

    S1: State 1 - Query Parsing
    S2: State 2 - Token Weighting
    S3: State 3 - Contextual Retrieval
    S4: State 4 - Logic Branching
    S5: State 5 - Variable Assignment
    S6: State 6 - Pattern Matching
    S7: State 7 - Constraint Validation
    S8: State 8 - Syntax Refinement
    S9: State 9 - Probability Check
    S10: State 10 - Output Rendering

    S1 --> [*]
    S2 --> [*]
    S3 --> [*]
    S4 --> [*]
    S5 --> [*]
    S6 --> [*]
    S7 --> [*]
    S8 --> [*]
    S9 --> [*]
    S10 --> [*]

Loading

Part 2: Multi-Layer State Transitions (3-layer tree, 5 root states)

Can models follow conditional probability distributions across a hierarchical state machine? Each iteration traces a path from root to leaf. Two modes: Chained (3 API calls, one per layer) and Single-shot (1 call, full tree shown).

stateDiagram-v2
    [*] --> S1: 15%
    [*] --> S2: 30%
    [*] --> S3: 25%
    [*] --> S4: 20%
    [*] --> S5: 10%

    state S1 {
        [*] --> S1_1: 40%
        [*] --> S1_2: 35%
        [*] --> S1_3: 25%
    }

    state S2 {
        [*] --> S2_1: 20%
        [*] --> S2_2: 30%
        [*] --> S2_3: 35%
        [*] --> S2_4: 15%
    }

    state S3 {
        [*] --> S3_1: 45%
        [*] --> S3_2: 30%
        [*] --> S3_3: 25%
    }

    state S4 {
        [*] --> S4_1: 45%
        [*] --> S4_2: 25%
        [*] --> S4_3: 18%
        [*] --> S4_4: 12%
    }

    state S5 {
        [*] --> S5_1: 55%
        [*] --> S5_2: 45%
    }

    S1: S1 Input
    S2: S2 Compute
    S3: S3 Validate
    S4: S4 Optimize
    S5: S5 Output

Loading

Surprising finding: models don't always pick the argmax in Part 2. GPT-5.4-mini favored S3 (25%) over S2 (30%) in chained mode. Prompt structure matters more than probability values.

Key Findings

1. Understanding a distribution and sampling from one are different capabilities. All models understand probability perfectly. None can act as a sampler.
2. GPT-5.4 is the only model with any distributional spread -- chi-squared 4.5x lower than others.
3. A "probability floor" exists around 10%. States below this threshold are ignored by all models.
4. Prompt structure changes everything. The same model produces wildly different distributions in chained vs single-shot mode.

Model Behavior Taxonomy

Model	Behavior	Description
GPT-5.4	Approximate distributer	Distributes across all states with heavy mode bias. Best probabilistic intuition.
GPT-5.4-mini	Non-argmax collapser	Collapses to a single state but NOT always the highest probability one.
GPT-4.1	Strict argmax	Reliably picks the highest-probability option. Deterministic.
GPT-5-mini	Strict argmax / variable	Argmax in chained mode, more variety in single-shot.

Running the Experiments

# Setup
pip install uv
uv init && uv venv
uv add openai python-dotenv jinja2
echo "OPENAI_API_KEY=sk-..." > .env

# Part 1: Flat distribution
uv run python experiment.py --llm-iterations 1000 --model gpt-4.1
uv run python experiment.py --llm-iterations 1000000 --model gpt-5.4 gpt-4.1

# Part 2: State transitions
uv run python experiment_transitions.py --iterations 1000 --model gpt-4.1
uv run python experiment_transitions.py --iterations 100000 --model gpt-5.4 gpt-4.1

Output

All results in results/ with timestamped filenames:

• experiment_results_*.csv / transitions_paths_*.csv -- raw data
• summary_stats_*.csv / transitions_summary_*.csv -- aggregate metrics
• report_*.html / report_transitions_*.html -- per-model interactive dashboards
• model_comparison.html -- cross-model comparison dashboard

Full Analysis

See docs/conclusions.md for detailed write-ups of every run.

Total API Calls

Experiment	Calls
Part 1 (4 models x 1M)	4,000,000
Part 2 (4 models x 100K x 2 modes)	~1,600,000
Test runs	~10,000
Total	~5,610,000