Skip to content

feat(event-sourcing): add journaled state log consistency provider#10108

Draft
ReubenBond wants to merge 6 commits into
dotnet:mainfrom
ReubenBond:rebond/issue-10100-event-sourcing-journaled-state
Draft

feat(event-sourcing): add journaled state log consistency provider#10108
ReubenBond wants to merge 6 commits into
dotnet:mainfrom
ReubenBond:rebond/issue-10100-event-sourcing-journaled-state

Conversation

@ReubenBond

@ReubenBond ReubenBond commented May 16, 2026

Copy link
Copy Markdown
Member

Problem

Orleans.EventSourcing currently has storage-backed log consistency providers, but no provider that stores the relevant grain's event log in its own Orleans.Journaling state machine.

Solution

Add a journaled-state-backed log consistency provider which registers durable state for the event log and write-vector metadata. The provider wires journaling registration through silo builder extensions, uses Orleans binary journaling for arbitrary event payloads, performs explicit durable rereads for recovery, and clears only provider-owned journaled state so unrelated grain state is preserved.

Rationale

Using the grain's journaled state keeps event-sourcing events colocated with the activation's journaling infrastructure while preserving the existing log consistency protocol semantics. The explicit read API avoids relying on initialization for ambiguous write recovery, and the clear behavior avoids deleting unrelated durable state owned by the same grain.

Notes

The new provider depends on the experimental Orleans.Journaling APIs, so warning suppressions are kept local to the new integration points.

Microsoft Reviewers: Open in CodeFlow

@ReubenBond ReubenBond marked this pull request as draft May 16, 2026 15:58
@ReubenBond ReubenBond force-pushed the rebond/issue-10100-event-sourcing-journaled-state branch from fe36814 to 3bcc650 Compare May 16, 2026 16:02
@ReubenBond ReubenBond requested a review from Copilot May 16, 2026 16:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Introduces a new log-consistency provider, Orleans.EventSourcing.JournaledState, that stores the event log and write-vector for log-consistent grains inside the grain's own Orleans.Journaling state machine. To support the explicit-reread recovery flow used by the provider, IJournaledStateManager gains a new ReadStateAsync method backed by a ReadStateWorkItem in the journaling work loop, and a related migration-snapshot fix avoids double-appending during format migration. New silo builder extensions, public-API surface updates, and end-to-end tests (including a grain variant with auxiliary durable state) round out the change.

Changes:

  • Add IJournaledStateManager.ReadStateAsync and the matching work-loop handler; suppress an append during the migration-snapshot path; honor cancellation on Initialize/Delete/Read via WaitAsync.
  • Add the JournaledState log-consistency provider (LogConsistencyProvider, LogViewAdaptor<TLogView, TLogEntry>) and AddJournaledStateBasedLogConsistencyProvider[AsDefault] silo-builder extensions, registering durable list/value for the event log and write-vector.
  • Add tests for the new provider (persistence across activation, Clear preserving auxiliary journaled state), a journaling read-state test, a JSON-to-binary migration "no-append" test, plus interface plumbing in test grains and fixtures; update generated API surfaces; swap storedEtag/currentEtag arguments in AzureBlobJournalStorage.CreateInconsistentWalStateException.
Show a summary per file
File Description
src/Orleans.EventSourcing/JournaledState/LogConsistencyProvider.cs New ILogViewAdaptorFactory that constructs the journaled-state log view adaptor.
src/Orleans.EventSourcing/JournaledState/LogViewAdaptor.cs Core adaptor that persists event log and write-vector via IJournaledStateManager, with read/write/recovery loops.
src/Orleans.EventSourcing/Hosting/JournaledStateSiloBuilderExtensions.cs Silo builder extensions to register the provider; sets JournalFormatKey to orleans-binary.
src/Orleans.EventSourcing/Orleans.EventSourcing.csproj Adds project reference to Orleans.Journaling.
src/Orleans.Journaling/IJournaledStateManager.cs Adds ReadStateAsync interface method.
src/Orleans.Journaling/JournaledStateManager.cs Implements ReadStateAsync/ReadStateWorkItem, applies WaitAsync for cancellation, and avoids appending during migration snapshots.
src/Azure/Orleans.Journaling.AzureStorage/AzureBlobJournalStorage.cs Swaps the storedEtag/currentEtag arguments when creating InconsistentStateException.
src/api/Orleans.EventSourcing/Orleans.EventSourcing.cs Public-API surface updates for new provider and extensions.
src/api/Orleans.Journaling/Orleans.Journaling.cs Public-API surface update for ReadStateAsync.
test/Orleans.EventSourcing.Tests/EventSourcingTests/EventSourcingClusterFixture.cs Registers the new provider and a volatile journal storage in the test cluster.
test/Orleans.EventSourcing.Tests/EventSourcingTests/LogTestGrainClearTests.cs Adds tests for Clear, persistence, and preservation of auxiliary durable state.
test/Grains/TestGrainInterfaces/ILogTestGrain.cs Adds ILogTestGrainWithAuxiliaryState interface.
test/Grains/TestGrains/LogTestGrainVariations.cs Adds journaled-state grain variants, including one with auxiliary durable state.
test/Orleans.Journaling.Tests/JournalBatchTests.cs Adds test verifying ReadStateAsync discards unflushed changes.
test/Orleans.Journaling.Tests/DurableListDirectWriteTests.cs Implements new ReadStateAsync on test fake.
test/Orleans.Journaling.Tests/DurableCollectionDirectWriteTests.cs Implements new ReadStateAsync on test fake.
test/Orleans.Journaling.Json.Tests/CodecRecoveryTests.cs Adds test that JSON→binary migration replaces without appending, plus a CountingStorage helper.

Copilot's findings

  • Files reviewed: 17/17 changed files
  • Comments generated: 2

ArgumentException.ThrowIfNullOrWhiteSpace(name);

services.AddLogConsistencyProtocolServicesFactory();
services.Configure<JournaledStateManagerOptions>(options => options.JournalFormatKey = OrleansBinaryJournalFormatKey);
Comment on lines +148 to +185
while (true)
{
try
{
await stateManager.WriteStateAsync(CancellationToken.None);

Services.Log(LogLevel.Debug, "write ({0} updates) success v{1}", updates.Length, eventLog.Count);

UpdateConfirmedViewFromJournal();
LastPrimaryIssue.Resolve(Host, Services);

exit_operation("WriteAsync");
return updates.Length;
}
catch (Exception exception)
{
LastPrimaryIssue.Record(new UpdateJournaledStateFailed { Exception = exception }, Host, Services);
}

Services.Log(LogLevel.Debug, "write failed {0}", LastPrimaryIssue);

await LastPrimaryIssue.DelayBeforeRetry();
await RecoverAfterWriteFailureAsync();

if (writeBit == GetWriteVectorBit())
{
Services.Log(LogLevel.Debug, "last write ({0} updates) was actually a success v{1}", updates.Length, eventLog.Count);

UpdateConfirmedViewFromJournal();
LastPrimaryIssue.Resolve(Host, Services);

exit_operation("WriteAsync");
return updates.Length;
}

exit_operation("WriteAsync");
return 0;
}
@ReubenBond ReubenBond force-pushed the rebond/issue-10100-event-sourcing-journaled-state branch from 10d751f to 4e8eb63 Compare May 16, 2026 23:39
ReubenBond and others added 6 commits May 17, 2026 08:29
Add an Orleans.EventSourcing log-consistency provider backed by the host grain journaled state. The provider persists event log entries and write-vector metadata through Orleans.Journaling, performs explicit durable rereads for recovery, and clears only provider-owned state.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Correct Azure Blob WAL conflict ETag diagnostics and avoid appending new-format journal bytes before migration snapshots replace old-format storage.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keep the stored ETag unknown for Azure WAL conflicts and report the cached If-Match ETag as the current ETag passed by the caller.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@ReubenBond ReubenBond force-pushed the rebond/issue-10100-event-sourcing-journaled-state branch from 30c427c to 0069edf Compare May 17, 2026 15:30
@ReubenBond ReubenBond changed the title Add journaled state log consistency provider feat(event-sourcing): add journaled state log consistency provider May 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants