feat(streaming): add Aspire SQS integration - #10797
Conversation
There was a problem hiding this comment.
Pull request overview
Adds first-class configuration/Aspire integration for the Microsoft.Orleans.Streaming.SQS provider by introducing configuration-driven provider builders (silo + client), normalizing SQS connection string parsing, and adding Aspire app-model plus live ElasticMQ coverage in the existing AWS test project.
Changes:
- Add
SqsStreamProviderBuilderwith stable provider aliases (SQS,AmazonSQS) for both silo and client activation from configuration. - Introduce a dedicated SQS connection-string parser and update
SQSStorageto use it for more actionable startup validation. - Add Aspire app-model generation tests and a live ElasticMQ streaming test, plus update docs/snippets to include compiled Aspire examples.
Show a summary per file
| File | Description |
|---|---|
| test/Extensions/Orleans.AWS.Tests/Streaming/SQSStreamProviderBuilderTests.cs | Unit coverage for provider registration, option binding, topology determinism, adapter selection, and validation errors. |
| test/Extensions/Orleans.AWS.Tests/Streaming/SQSAspireLiveStreamTests.cs | Functional live test proving Aspire-generated config can publish/consume via ElasticMQ and cleans up queues. |
| test/Extensions/Orleans.AWS.Tests/Streaming/SQSAspireIntegrationTests.cs | App-model + activation tests validating emitted environment/config for silo and client roles. |
| test/Extensions/Orleans.AWS.Tests/Orleans.AWS.Tests.csproj | Adds Aspire AppHost/Orleans (and net10 testing) dependencies needed for the new Aspire tests. |
| src/AWS/Orleans.Streaming.SQS/Storage/SQSStorage.cs | Switches connection-string parsing to the new shared parser and improves missing-Service error handling. |
| src/AWS/Orleans.Streaming.SQS/SqsConnectionString.cs | Adds strict parsing/validation for key=value;... SQS connection strings. |
| src/AWS/Orleans.Streaming.SQS/Hosting/SqsStreamProviderBuilder.cs | New configuration-driven provider builder for silo/client, including startup validation and adapter wiring. |
| src/api/AWS/Orleans.Streaming.SQS/Orleans.Streaming.SQS.cs | Updates generated public API surface to include SqsStreamProviderBuilder. |
| docs/site/src/content/docs/streaming/sqs-streaming.md | Adds Aspire configuration guidance for SQS streaming and references compiled snippets. |
| docs/site/src/content/docs/host/snippets/aspire/Silo/SiloProgram.cs | Adds compiled silo snippet showing UseOrleans activating generated config. |
| docs/site/src/content/docs/host/snippets/aspire/Silo/Silo.csproj | Adds SQS streaming package reference for the compiled silo snippet project. |
| docs/site/src/content/docs/host/snippets/aspire/Client/ClientProgram.cs | Adds compiled client snippet showing UseOrleansClient activating generated config. |
| docs/site/src/content/docs/host/snippets/aspire/Client/Client.csproj | Adds SQS streaming package reference for the compiled client snippet project. |
| docs/site/src/content/docs/host/snippets/aspire/AppHost/AppHostExamples.cs | Adds compiled AppHost example and provider configuration snippet for SQS streaming. |
| docs/site/src/content/docs/host/snippets/aspire/AppHost/AppHost.csproj | Adds Aspire.Hosting.AWS dependency for the AppHost compiled snippet project. |
| Directory.Packages.props | Adds Aspire.Hosting.AWS package version under the net10 conditional block. |
Review details
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Suppressed comments (1)
src/AWS/Orleans.Streaming.SQS/Hosting/SqsStreamProviderBuilder.cs:236
- The DataAdapterKey/DataAdapterServiceKey conflict check is case-sensitive. Since configuration keys and most Aspire-generated values are case-insensitive, this can throw for values which resolve to the same keyed adapter (e.g., "Adapter" vs "adapter"). Consider using OrdinalIgnoreCase here, consistent with connection-string parsing using OrdinalIgnoreCase keys.
if (!string.IsNullOrWhiteSpace(dataAdapterKey)
&& !string.IsNullOrWhiteSpace(dataAdapterServiceKey)
&& !string.Equals(dataAdapterKey, dataAdapterServiceKey, StringComparison.Ordinal))
{
throw new OrleansConfigurationException(
- Files reviewed: 16/16 changed files
- Comments generated: 3
- Review effort level: Lite
There was a problem hiding this comment.
Review details
Suppressed comments (2)
Previously missed (2) — in code that hasn't changed since the last review.
src/AWS/Orleans.Streaming.SQS/Hosting/SqsStreamProviderBuilder.cs:110
- The ServiceKey/ConnectionName conflict check is case-sensitive (StringComparison.Ordinal), but connection-string keys in IConfiguration are effectively case-insensitive. This can throw an OrleansConfigurationException even when both values refer to the same connection string (e.g., "MySqs" vs "mysqs").
if (!string.IsNullOrWhiteSpace(serviceKey)
&& !string.IsNullOrWhiteSpace(connectionName)
&& !string.Equals(serviceKey, connectionName, StringComparison.Ordinal))
{
throw new OrleansConfigurationException(
docs/site/src/content/docs/host/snippets/aspire/AppHost/AppHost.csproj:16
- This snippet project pins most Aspire.Hosting.* packages to 13.4.6, but references Aspire.Hosting.AWS at 13.6.0. Mixing Aspire package versions within a single AppHost can lead to restore/downgrade warnings or runtime incompatibilities; it would be safer to align the versions (either bump the other Aspire.Hosting.* references to match AWS, or use an AWS version compatible with the rest).
<PackageReference Include="Aspire.Hosting.AppHost" Version="13.4.6" />
<PackageReference Include="Aspire.Hosting.AWS" Version="13.6.0" />
<PackageReference Include="Aspire.Hosting.Orleans" Version="13.4.6" />
<PackageReference Include="Aspire.Hosting.Redis" Version="13.4.6" />
- Files reviewed: 16/16 changed files
- Comments generated: 0 new
- Review effort level: Lite
There was a problem hiding this comment.
Review details
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
src/AWS/Orleans.Streaming.SQS/Hosting/SqsStreamProviderBuilder.cs:204
- ValidateServiceLocation only validates values which contain "://". If a user sets Service/Region (or the Service value inside ConnectionString) to a host:port value like "localhost:9324" (missing scheme), this is treated as an AWS region and will flow into SQSStorage.CreateClient, which will then use AWSUtils.GetRegionEndpoint(service) instead of ServiceURL. That yields a non-actionable misconfiguration path (likely failing at runtime with hard-to-diagnose AWS endpoint errors). Consider rejecting colon-delimited values which are not absolute http/https URIs so startup validation stays actionable.
private static void ValidateServiceLocation(string service)
{
if (service.Contains("://", StringComparison.Ordinal))
{
ValidateServiceEndpoint(service);
- Files reviewed: 16/16 changed files
- Comments generated: 0 new
- Review effort level: Lite
Code coverage
Report-only conclusion: current-main baseline stale. The newest successful coverage run tested 84352a7, not current main 325e7d6. Coverage combines every CI test matrix job, including providers, CodeGen, .NET 8/10, Linux, Windows, and macOS, using canonical physical source and branch identities. The comparison remains report-only while normal line and branch variance is calibrated. Coverage details |
|
The current head has a repository-wide restore/build blocker. Failed run: https://github.com/dotnet/orleans/actions/runs/32652145773 Please remove the unavailable dependency path (as in the updated Aspire provider integration approach), verify |
There was a problem hiding this comment.
Review details
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
src/AWS/Orleans.Streaming.SQS/Hosting/SqsStreamProviderBuilder.cs:134
- ResolveConnectionString treats "ServiceEndpoint" and its alias "Endpoint" by picking the first non-empty value, but it does not detect the case where both are configured with different values. That makes startup validation non-deterministic (one value is silently ignored) and can hide misconfiguration.
var service = configurationSection["Service"];
var region = configurationSection["Region"];
var serviceEndpoint = GetFirstNonWhiteSpace(
configurationSection["ServiceEndpoint"],
configurationSection["Endpoint"]);
- Files reviewed: 16/16 changed files
- Comments generated: 0 new
- Review effort level: Lite
There was a problem hiding this comment.
Review details
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
test/Extensions/Orleans.AWS.Tests/Orleans.AWS.Tests.csproj:23
- These Aspire integration/live tests are effectively net10-only (Aspire.Hosting.* references are conditioned on net10.0, and the test sources are wrapped in
#if NET10_0). However, the repo test TFMs arenet8.0;net10.0(test/Directory.Build.props:19) and the PR description/linked acceptance criteria call out running the Aspire-configured SQS tests on both net8.0 and net10.0. If the intent is dual-framework validation, these tests need to compile and execute for net8.0 as well (or the PR/issue criteria should be updated to match the net10-only constraint).
<ItemGroup Condition="'$(TargetFramework)' == 'net10.0'">
<PackageReference Include="Aspire.Hosting.Orleans" />
<PackageReference Include="Aspire.Hosting.Testing" />
</ItemGroup>
- Files reviewed: 16/16 changed files
- Comments generated: 0 new
- Review effort level: Lite
There was a problem hiding this comment.
Review details
Suppressed comments (2)
Previously missed (2) — in code that hasn't changed since the last review.
src/AWS/Orleans.Streaming.SQS/Hosting/SqsStreamProviderBuilder.cs:134
- ResolveConnectionString treats ServiceEndpoint and Endpoint as aliases but silently prefers ServiceEndpoint when both are set. If both keys are present with different values, the configuration is ambiguous and should throw an OrleansConfigurationException (similar to the ServiceKey/ConnectionName validation) instead of ignoring one value.
var service = configurationSection["Service"];
var region = configurationSection["Region"];
var serviceEndpoint = GetFirstNonWhiteSpace(
configurationSection["ServiceEndpoint"],
configurationSection["Endpoint"]);
docs/site/src/content/docs/host/snippets/aspire/AppHost/AppHost.csproj:16
- This snippet project pins Aspire.Hosting.AWS to 13.4.0 while all other Aspire.Hosting.* references in the same project use 13.4.6. That version skew can lead to confusing dependency resolution or restore/build warnings (which are treated as errors in this repo). Consider aligning the AWS integration package version with the rest of the Aspire stack used in this snippet, or documenting why it must differ.
<ItemGroup>
<PackageReference Include="Aspire.Hosting.AppHost" Version="13.4.6" />
<PackageReference Include="Aspire.Hosting.AWS" Version="13.4.0" />
<PackageReference Include="Aspire.Hosting.Orleans" Version="13.4.6" />
<PackageReference Include="Aspire.Hosting.Redis" Version="13.4.6" />
- Files reviewed: 16/16 changed files
- Comments generated: 0 new
- Review effort level: Lite
6f736c9 to
f950aff
Compare
This reverts commit 3bbb347.
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
It introduces new public packages/APIs and provisioning/configuration behavior across runtime, tests, restore sources, and docs which warrants final human validation.
Review tier: Lite
Findings: 1
New issues introduced by this change (1)
| Severity | Finding |
|---|---|
 |
test/Extensions/Orleans.AWS.Tests/Orleans.AWS.Tests.csproj — Microsoft.Testing.Platform.MSBuild is already centrally versioned in Directory.Packages.props… |
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Startup validation currently allows Service=host:port values which can silently resolve to an unintended AWS region instead of failing with an actionable configuration error.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review tier: Lite
Findings: 1
New issues introduced by this change (1)
| Severity | Finding |
|---|---|
 |
src/AWS/Orleans.Streaming.SQS/Hosting/SqsStreamProviderBuilder.cs — ValidateServiceLocation only treats values containing "://" as endpoints. A common misconfiguration… |
Issues resolved since last review (1)
| Severity | Finding |
|---|---|
|
test/Extensions/Orleans.AWS.Tests/Orleans.AWS.Tests.csproj — Microsoft.Testing.Platform.MSBuild is already centrally versioned in Directory.Packages.props… View resolved comment |
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
test/Extensions/Orleans.AWS.Tests/Orleans.AWS.Tests.csproj:30
- These Roslyn package references specify
VersionOverride="5.0.0", butDirectory.Packages.propsalready updatesMicrosoft.CodeAnalysis.*packages to 5.0.0 for net10.0. Keeping the redundant overrides increases maintenance cost and can drift from central package management in the future.
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
It introduces a new shipping Aspire bridge package with AWS CDK provisioning and cross-package validation logic (including private-member access), which warrants careful human review for long-term compatibility and supportability.
Review tier: Lite
Findings: None
Issues resolved since last review (1)
| Severity | Finding |
|---|---|
|
src/AWS/Orleans.Streaming.SQS/Hosting/SqsStreamProviderBuilder.cs — ValidateServiceLocation only treats values containing "://" as endpoints. A common misconfiguration… View resolved comment |
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
src/AWS/Orleans.Streaming.SQS.Aspire/OrleansSqsStreamingExtensions.cs:247
- ValidateServiceId relies on [UnsafeAccessor] to call a non-public OrleansService getter ("get_ServiceId"). This is brittle against upstream Aspire/Orleans integration changes (renames/signature changes can break at runtime) and can be incompatible with trimming/AOT scenarios, since it bypasses public surface-area contracts.
Consider avoiding private-member access by tracking the expected ServiceId via a resource/annotation owned by this package (and validating against that), or by using a public Aspire/Orleans API if one is available to read the configured ServiceId.
Problem
Orleans SQS streaming owns a deterministic partitioned queue topology, so a generic single-queue Aspire resource or a copied private provider class could not safely keep AWS infrastructure and silo/client configuration synchronized.
Solution
Microsoft.Orleans.Streaming.SQS.AspireAppHost bridge, separate from the runtime SQS packageWithSqsStreaming/AddSqsStreaming,SqsStreamingOptions, andSqsStreamingResourceAPIsOrleans__Streaming__<name>__*settingsServiceId, attach AWS SDK for .NET v4 configuration, and automatically wait for the CDK stack on referenced silos and clientsRationale
The AppHost package owns AWS CDK provisioning while the runtime package remains free of Aspire, AWS provisioning, and CDK dependencies. A shared naming invariant and one immutable configuration model keep CloudFormation resources and Orleans runtime topology aligned.
Fixes #10783
Part of #10784
Microsoft Reviewers: Open in CodeFlow