Skip to content

Shard Joshua test admission for large ensembles#159

Open
tclinkenbeard-oai wants to merge 2 commits into
FoundationDB:mainfrom
tclinkenbeard-oai:dev/tclinkenbeard/joshua-concurrent-claims
Open

Shard Joshua test admission for large ensembles#159
tclinkenbeard-oai wants to merge 2 commits into
FoundationDB:mainfrom
tclinkenbeard-oai:dev/tclinkenbeard/joshua-concurrent-claims

Conversation

@tclinkenbeard-oai

Copy link
Copy Markdown
Contributor

Summary

  • add opt-in per-ensemble claim shards for large finite ensembles
  • keep legacy ensembles on the existing exact admission path
  • record the chosen shard with each in-progress seed so dead-agent reclaim returns the same shard slot
  • document the rolling-upgrade gate for claim_shard_count

Why

try_starting_test() currently reads the aggregate started counter
serializably and then atomically increments that same key. Thousands of agents
claiming together conflict on one key, retry as a herd, and can time out before
doing useful work.

With --property claim_shard_count=10000, each claim reads and increments one
shard whose quota is an exact slice of max_runs; the aggregate started
counter remains an atomic status counter. The shard quotas sum to max_runs,
so the existing started <= max_runs invariant is preserved.

The protocol is opt-in so operators can drain old agents before enabling it for
new ensembles; legacy agents do not write shard claims.

Validation

  • python3 -m py_compile joshua/joshua_model.py test_joshua_model.py
  • pytest -q test_joshua_model.py -k 'concurrent_claims_preserve_max_runs or dead_claim_releases_shard or legacy_claims_keep_exact_max_runs'

The focused tests ran against a temporary FoundationDB 7.1.57 arm64 package and
passed: 3 passed, 15 deselected.

@tclinkenbeard-oai tclinkenbeard-oai marked this pull request as ready for review July 11, 2026 19:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant