[BFTree] Add RangeIndex cluster migration

[tiagonapoli]

Summary

Adds end-to-end cluster migration for RangeIndex keys, supporting both MIGRATE SLOTS and MIGRATE KEYS paths. RangeIndex keys are backed by a native BfTree whose on-disk state (data.bftree) lives outside Tsavorite — shipping just the 51-byte stub is insufficient. This change streams the entire BfTree snapshot file alongside the stub over the existing migration transport.

Architecture

• Serializer (RangeIndexChunkedSerializer): Pure state machine over Span<byte> — no I/O. Takes file data as input via MoveNext(dest, fileData, out consumed).
• Migration Reader (RangeIndexMigrationReader): Async wrapper that reads the snapshot file and feeds bytes to the serializer.
• Deserializer (RangeIndexChunkedDeserializer): Sync state machine that writes received file data to a temp file, validates xxHash64 checksum, recovers the native BfTree, and publishes the stub to the store.
• Factory (RangeIndexManager.SnapshotRangeIndexAndCreateReader): Snapshots the BfTree under an exclusive lock to a temp file, then creates the reader.

Wire format

Single MigrationRecordSpanType.SerializedRangeIndexStream (tag 4). Stream format across chunks:

[4B keyLen][key bytes][8B fileSize][file bytes][8B xxHash64][4B stubLen][stub]

Key and file bytes may span chunks; all other elements must fit within a single chunk.

SLOTS path

1. Scan detects RI keys via RecordType == 2, captures to MigrateOperation.RangeIndexes
2. After scan completes, MigrateRangeIndexKeysAsync runs a sketch-protected batch cycle:
   • Add all RI keys to sketch (INITIALIZING)
   • TRANSMITTING + epoch barrier — blocks writes during snapshot+transmit
   • Transmit each key via TransmitRangeIndexAsync
   • DELETING + epoch barrier — blocks reads+writes during delete
   • Delete each key
   • finally: clear sketch (unblocks clients)

KEYS path

1. GetRangeIndexKeysForMigration discovers RI keys via RIGET
2. TransmitKeysAsync skips RI keys (in rangeIndexKeysToIgnore)
3. Each RI key transmitted via TransmitRangeIndexAsync, then marked in sketch for DeleteKeysAsync

Bug fixes

• Remove RI+cluster startup guard (GarnetServer.cs)
• Fix round-trip migration: Publish deletes existing data file before move
• Fix Publish registration: accept InPlaceUpdated/CopyUpdated status
• Fix serializer empty-buffer bug in FileData phase
• TransmitRangeIndexAsync catches all exceptions (never throws)
• Sketch protection for RI keys in SLOTS path (previously unprotected)

Tests

• 26 unit tests for serializer/deserializer (round-trip, checksums, error states, buffer boundaries, chunk sizes)
• 11 cluster integration tests: SingleBySlot, ByKeys, ManyBySlot, WhileModifying, MigrateBack, LargeTree (1KB/256KB chunks), ChunkSize variants, StressAsync (Explicit)

TODO

• Vector Set index keys have the same sketch protection gap (documented in code)
• AOF replication of migrated trees to destination replicas

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[vazois]

Missing RangeIndex feature-enabled guards

This PR doesn't consistently check whether the RangeIndex feature is enabled (i.e., whether RangeIndexManager is non-null) before using it. Two spots are at risk of NullReferenceException:

1. Source side — MigrateSession.RangeIndex.cs:29

var rangeIndexManager = clusterProvider.storeWrapper.DefaultDatabase.RangeIndexManager;
// No null check before calling .SnapshotRangeIndexAndCreateReader(...)

The KEYS path correctly uses ?.GetRangeIndexKeysForMigration(...) with a fallback to an empty dictionary, but TransmitRangeIndexAsync assumes the manager is always available. If the SLOTS scan somehow discovers an RI key on a node where the feature is disabled, this will crash.

2. Receive side — RespClusterMigrateCommands.cs:181

if (!rangeIndexMigrationState.ProcessRecord(...))

rangeIndexMigrationState is set to null when RangeIndexManager is null (line 70 in ClusterSession.cs), but the dispatch at line 181 doesn't guard against that. If a receiving node has RangeIndex disabled and the sender transmits a SerializedRangeIndexStream record, this will throw.

Suggestion: Add a null check on the receive side (e.g., log a warning and skip/error the record), and consider a defensive null check in TransmitRangeIndexAsync as well.

[vazois]

Is it possible to combine these two instead of having to maintain two dictionaries?

[vazois]

libs/cluster/Server/Migration/MigrateScanFunctions.cs:53 — maybe not for this PR, but can't we just get these values from an enum?

[vazois]

can we avoid the copy here and just iterate over the container?

[vazois]

libs/cluster/Server/Migration/MigrateSessionKeys.cs:43 — seems really wasteful to maintain a separate dictionary just for skipping keys. We can potentially store the info for the key type inline and skip the key once we read it from the sketch list

[vazois]

Ensure that the reset does not race with back to back Migration calls or parallel sessions on different slots

[tiagonapoli]

Each ClusterSession has a single RangeIndexMigrationReceiveSession - within a single ClusterSession, processing is single threaded right?