feat(networking): introduce message-oriented networking transports

[ReubenBond]

Summary

• Introduces message-oriented networking transport abstractions and socket/stream/TLS transport implementations, then rewires client, gateway, and silo connections to use them.
• Moves TLS transport pieces from Orleans.Connections.Security into Core/Runtime hosting and updates the solution/project layout accordingly.
• Updates message serialization/buffer handling for the new transport path, including buffered raw response headers and serialization failure response routing.
• Updates TestingHost in-memory/Unix socket transports, drains buffered transport messages, and scopes message handler serializer/read/write pools per MessageHandlerShared instance.
• Keeps broader callback/message/invokable pooling and SEDA code/docs out of this PR.

Validation

• git diff --check
• conflict-marker scan
• rg Seda|SEDA|ISeda src returned no matches
• dotnet build src\Orleans.Core\Orleans.Core.csproj -m
• dotnet build src\Orleans.Runtime\Orleans.Runtime.csproj -m
• dotnet build src\Orleans.TestingHost\Orleans.TestingHost.csproj -m
• security tests project build
• networking lifecycle tests
• ArcBuffer/PooledBuffer serialization tests

Notes

• No full solution/provider/external-dependency test run was performed.
• TLS tests were built but not executed.

Microsoft Reviewers: Open in CodeFlow

[Copilot]

Pull request overview

Introduces a new message-oriented transport layer (TCP/socket/stream/TLS implementations) and rewires Orleans client/gateway/silo networking plus serialization/buffering to flow through these abstractions, including updates to TestingHost transports and related tests.

Changes:

• Add MessageTransport / listener / connector abstractions and socket + TLS transport implementations.
• Rewire runtime/client connection factories & listeners to use message transports (including updated tracing/logging and lifecycle).
• Update serialization buffer handling (ArcBuffer/ArcBufferWriter) and add/adjust unit tests and benchmarks accordingly.

Show a summary per file

File	Description
test/Orleans.Serialization.UnitTests/PooledBufferTests.cs	Adjusts slice enumerator tests to focus on span enumeration.
test/Orleans.Core.Tests/Orleans.Core.Tests.csproj	Adds System.IO.Pipelines package reference for new test usage.
test/Orleans.Core.Tests/Networking/MessageTransportLifecycleTests.cs	Adds lifecycle/defaults tests for transport-related options and handler sharing.
test/Orleans.Connections.Security.Tests/TlsConnectionTests.cs	Updates TLS tests to new transport security namespace.
test/Orleans.Connections.Security.Tests/Orleans.Connections.Security.Tests.csproj	Removes reference to removed Orleans.Connections.Security project.
test/Benchmarks/Serialization/ComplexTypeBenchmarks.cs	Updates benchmark to new MessageSerializer construction path.
src/Orleans.TestingHost/UnixSocketTransport/UnixSocketConnectionOptions.cs	Removes old memory-pool factory remnants from Unix socket options.
src/Orleans.TestingHost/UnixSocketTransport/UnixSocketConnectionListenerFactory.cs	Comments out legacy Kestrel-connection listener factory (now obsolete).
src/Orleans.TestingHost/UnixSocketTransport/UnixSocketConnectionListener.cs	Removes legacy Kestrel IConnectionListener implementation.
src/Orleans.TestingHost/UnixSocketTransport/UnixSocketConnectionFactory.cs	Removes legacy Kestrel IConnectionFactory implementation.
src/Orleans.TestingHost/UnixSocketTransport/UnixSocketConnectionExtensions.cs	Removes legacy DI registration extensions for Kestrel-based Unix sockets.
src/Orleans.TestingHost/UnixSocketTransport/UnixDomainSocketMessageTransportListener.cs	Adds message-transport-based Unix domain socket listener.
src/Orleans.TestingHost/UnixSocketTransport/UnixDomainSocketMessageTransportConnector.cs	Adds message-transport-based Unix domain socket connector.
src/Orleans.TestingHost/UnixSocketTransport/SocketsLog.cs	Adds (currently commented-out) Unix socket transport logging scaffold.
src/Orleans.TestingHost/TestClusterOptions.cs	Changes default test cluster transport to TCP sockets.
src/Orleans.TestingHost/TestClusterHostFactory.cs	Adds logging namespace import for updated DI usage.
src/Orleans.TestingHost/TestCluster.cs	Rewires TestCluster transport selection to message transports (TCP/InMemory/UnixSocket).
src/Orleans.TestingHost/Orleans.TestingHost.csproj	Adds logging + pipelines package references for new transports.
src/Orleans.TestingHost/InProcTestCluster.cs	Updates in-proc cluster to use new in-memory message transport hooks.
src/Orleans.TestingHost/InMemoryTransport/InMemoryTransportHostingExtensions.cs	Adds DI extensions to configure in-memory message transport.
src/Orleans.TestingHost/InMemoryTransport/InMemoryTransportConnection.cs	Removes legacy Kestrel TransportConnection in-memory pipe implementation.
src/Orleans.Serialization/Serializers/CodecProvider.cs	Minor edits plus introduction of commented-out unfinished code.
src/Orleans.Serialization/Serializer.cs	Adds ArcBuffer deserialize overloads and formatting/style fixes.
src/Orleans.Serialization/Buffers/Writer.cs	Adds ArcBufferWriter wrapper support for Writer.Create(...).
src/Orleans.Serialization/Buffers/Adaptors/BufferSliceReaderInput.cs	Adds ArcBufferReaderInput for Reader<TInput> over ArcBuffer.
src/Orleans.Serialization.TestKit/FieldCodecTester.cs	Switches round-trip helper to ArcBufferWriter/ArcBuffer flow.
src/Orleans.Runtime/Networking/SiloConnectionListener.cs	Replaces Kestrel connection listener plumbing with message transport listeners.
src/Orleans.Runtime/Networking/SiloConnectionFactory.cs	Replaces Kestrel connection factory plumbing with message transport connector.
src/Orleans.Runtime/Networking/SiloConnection.cs	Updates connection to operate over MessageTransport and new run loop shape.
src/Orleans.Runtime/Networking/GatewayInboundConnection.cs	Updates gateway inbound connection to MessageTransport.
src/Orleans.Runtime/Networking/GatewayConnectionListener.cs	Updates gateway listener to message transport listener pipeline.
src/Orleans.Runtime/Messaging/Gateway.cs	Adjusts imports to use transport abstractions.
src/Orleans.Runtime/Hosting/SiloTlsHostingExtensions.ISiloBuilder.cs	Moves silo TLS hosting extensions onto message transport TLS middleware.
src/Orleans.Runtime/Hosting/DefaultSiloServices.cs	Registers message transports (TCP + listeners) and shared handler pools.
src/Orleans.Runtime/Configuration/SiloConnectionOptions.cs	Removes legacy Kestrel connection-builder options type.
src/Orleans.Core/Utils/SingleWaiterAutoResetEvent.cs	Adds new single-waiter auto-reset event utility.
src/Orleans.Core/Orleans.Core.csproj	Removes Kestrel connections abstractions dependency; adds InternalsVisibleTo.
src/Orleans.Core/Networking/Transport/WriteRequest.cs	Adds write request abstraction over ArcBufferReader.
src/Orleans.Core/Networking/Transport/TlsMessageTransportConnectorMiddleware.cs	Adds TLS middleware wrappers for connectors/listeners.
src/Orleans.Core/Networking/Transport/Sockets/TcpMessageTransportListener.cs	Adds TCP message transport listener implementation.
src/Orleans.Core/Networking/Transport/Sockets/TcpMessageTransportConnector.cs	Adds TCP message transport connector implementation.
src/Orleans.Core/Networking/Transport/Sockets/SocketsLog.cs	Adds socket transport logging helpers.
src/Orleans.Core/Networking/Transport/Sockets/SocketSender.cs	Adds multi-buffer socket send helper for transports.
src/Orleans.Core/Networking/Transport/Sockets/SocketReceiver.cs	Adds socket receive helper for transports.
src/Orleans.Core/Networking/Transport/Sockets/SocketOperationResult.cs	Adds result wrapper for socket operations.
src/Orleans.Core/Networking/Transport/Sockets/SocketExtensions.cs	Adds fast-path enabling extension for sockets.
src/Orleans.Core/Networking/Transport/Sockets/SocketConnectionException.cs	Adds transport-level socket connection exception type.
src/Orleans.Core/Networking/Transport/Sockets/SocketAwaitableEventArgs.cs	Adds awaitable SocketAsyncEventArgs base for send/receive.
src/Orleans.Core/Networking/Transport/Sockets/CorrelationIdGenerator.cs	Adds transport-side correlation ID generator.
src/Orleans.Core/Networking/Transport/Sockets/BufferExtensions.cs	Adds Memory<T>/ReadOnlyMemory<T> array extraction helpers for sockets.
src/Orleans.Core/Networking/Transport/Sockets/AddressInUseException.cs	Adds address-in-use exception for listener bind failures.
src/Orleans.Core/Networking/Transport/Security/TlsServerAuthenticationOptions.cs	Adds TLS server auth options wrapper.
src/Orleans.Core/Networking/Transport/Security/TlsOptions.cs	Adds new TLS options model for message transports.
src/Orleans.Core/Networking/Transport/Security/TlsMessageTransportListener.cs	Adds TLS-wrapping listener implementation.
src/Orleans.Core/Networking/Transport/Security/TlsMessageTransportConnector.cs	Adds TLS-wrapping connector implementation.
src/Orleans.Core/Networking/Transport/Security/TlsMessageTransport.cs	Adds TLS stream transport implementation.
src/Orleans.Core/Networking/Transport/Security/TlsClientAuthenticationOptions.cs	Adds TLS client auth options wrapper.
src/Orleans.Core/Networking/Transport/Security/ServerTlsMessageTransport.cs	Adds server-authenticating TLS transport.
src/Orleans.Core/Networking/Transport/Security/RemoteCertificateMode.cs	Adds remote certificate policy enum.
src/Orleans.Core/Networking/Transport/Security/ClientTlsMessageTransport.cs	Adds client-authenticating TLS transport.
src/Orleans.Core/Networking/Transport/Security/CertificateLoader.cs	Adds certificate store loading/validation helpers.
src/Orleans.Core/Networking/Transport/ReadRequest.cs	Adds read request abstraction over ArcBufferReader.
src/Orleans.Core/Networking/Transport/NetworkTransportBase.cs	Adds base class for message transport implementations.
src/Orleans.Core/Networking/Transport/MessageTransportListener.cs	Adds message transport listener abstraction.
src/Orleans.Core/Networking/Transport/MessageTransport.cs	Adds core MessageTransport/connector APIs + middleware interfaces.
src/Orleans.Core/Networking/Transport/IFeatureCollection.cs	Adds typed feature collection interface for transports.
src/Orleans.Core/Networking/Transport/FeatureCollection.cs	Adds default feature collection implementation.
src/Orleans.Core/Networking/Transport/ConnectionResetException.cs	Adds connection reset exception type.
src/Orleans.Core/Networking/Transport/ConnectionEndPointFeature.cs	Adds local/remote endpoint transport feature.
src/Orleans.Core/Networking/Transport/ConnectionAbortedException.cs	Adds aborted/closed connection exception types.
src/Orleans.Core/Networking/SocketDirection.cs	Removes legacy connection direction enum file.
src/Orleans.Core/Networking/Shared/TransportConnection.cs	Removes legacy Kestrel ConnectionContext-based transport connection.
src/Orleans.Core/Networking/Shared/SocketsTrace.cs	Removes legacy socket trace logger abstraction.
src/Orleans.Core/Networking/Shared/SocketSenderReceiverBase.cs	Removes legacy socket send/receive base.
src/Orleans.Core/Networking/Shared/SocketSender.cs	Removes legacy socket sender implementation.
src/Orleans.Core/Networking/Shared/SocketSchedulers.cs	Removes legacy IOQueue-based schedulers.
src/Orleans.Core/Networking/Shared/SocketReceiver.cs	Removes legacy socket receiver implementation.
src/Orleans.Core/Networking/Shared/SocketExtensions.cs	Removes legacy socket extensions (fast-path/keepalive).
src/Orleans.Core/Networking/Shared/SocketConnectionOptions.cs	Removes legacy socket connection options.
src/Orleans.Core/Networking/Shared/SocketConnectionListenerFactory.cs	Removes legacy socket listener factory.
src/Orleans.Core/Networking/Shared/SocketConnectionListener.cs	Removes legacy socket listener.
src/Orleans.Core/Networking/Shared/SocketConnectionFactory.cs	Removes legacy socket connection factory.
src/Orleans.Core/Networking/Shared/SocketAwaitableEventArgs.cs	Removes legacy awaitable socket args.
src/Orleans.Core/Networking/Shared/SharedMemoryPool.cs	Removes legacy shared memory pool wrapper.
src/Orleans.Core/Networking/Shared/MemoryPoolSlab.cs	Removes legacy slab memory pool internals.
src/Orleans.Core/Networking/Shared/MemoryPoolBlock.cs	Removes legacy slab memory pool internals.
src/Orleans.Core/Networking/Shared/KestrelMemoryPool.cs	Removes legacy Kestrel memory pool wrapper.
src/Orleans.Core/Networking/Shared/ISocketsTrace.cs	Removes legacy sockets trace interface.
src/Orleans.Core/Networking/Shared/IOQueue.cs	Removes legacy IOQueue scheduler.
src/Orleans.Core/Networking/Shared/DuplexPipe.cs	Removes legacy duplex pipe helpers.
src/Orleans.Core/Networking/Shared/BufferExtensions.cs	Removes legacy buffer-to-array helpers (moved to transport).
src/Orleans.Core/Networking/MessageWriteRequest.cs	Adds message framing + serialization write request.
src/Orleans.Core/Networking/MessageReadRequest.cs	Adds framed message read request with deferred body handling.
src/Orleans.Core/Networking/MessageHandlerShared.cs	Adds per-shared-instance pools for serializers/read/write handlers.
src/Orleans.Core/Networking/IUnderlyingTransportFeature.cs	Removes legacy underlying transport feature interface.
src/Orleans.Core/Networking/CorrelationIdGenerator.cs	Moves correlation ID generator to new namespace/usage.
src/Orleans.Core/Networking/ConnectionTrace.cs	Adds ConnectionTrace diagnostic/logger wrapper.
src/Orleans.Core/Networking/ConnectionShared.cs	Updates shared connection services and lazy MessageHandlerShared access.
src/Orleans.Core/Networking/ConnectionOptions.cs	Adds close connection timeout option.
src/Orleans.Core/Networking/ConnectionManager.cs	Updates run method call to RunAsync().
src/Orleans.Core/Networking/ConnectionLogScope.cs	Reduces log scope fields to connection ID only.
src/Orleans.Core/Networking/ConnectionFactory.cs	Replaces Kestrel-based connection creation with message transport connector.
src/Orleans.Core/Networking/ConnectionDirection.cs	Adds connection direction + transport protocol feature types.
src/Orleans.Core/Networking/ConnectionBuilderDelegates.cs	Removes legacy connection-builder delegate infrastructure.
src/Orleans.Core/Networking/ClientOutboundConnectionFactory.cs	Updates client outbound connection creation to message transports.
src/Orleans.Core/Networking/ClientOutboundConnection.cs	Updates client outbound connection to MessageTransport.
src/Orleans.Core/Networking/ClientConnectionOptions.cs	Removes legacy Kestrel connection options.
src/Orleans.Core/Messaging/MessageFactory.cs	Updates TTL copying logic to new helper.
src/Orleans.Core/Messaging/Message.cs	Adds deferred body deserialization via MessageReadRequest and safe formatting behavior.
src/Orleans.Core/Messaging/CachingSiloAddressCodec.cs	Adjusts timestamp capture ordering in cache lookup path.
src/Orleans.Core/Hosting/ClientTlsHostingExtensions.IClientBuilder.cs	Adds/renames public client TLS hosting extensions using transport middleware.
src/Orleans.Core/Core/DefaultClientServices.cs	Registers message transports and shared handler pools for client default services.
src/Orleans.Connections.Security/Security/TlsServerAuthenticationOptions.cs	Removes old TLS security types (moved to new transport security).
src/Orleans.Connections.Security/Security/TlsOptions.cs	Removes old TLS options (moved).
src/Orleans.Connections.Security/Security/TlsDuplexPipe.cs	Removes old TLS duplex pipe adapter.
src/Orleans.Connections.Security/Security/TlsConnectionFeature.cs	Removes old TLS features plumbing.
src/Orleans.Connections.Security/Security/TlsClientAuthenticationOptions.cs	Removes old TLS auth options wrapper.
src/Orleans.Connections.Security/Security/RemoteCertificateMode.cs	Removes old enum (moved).
src/Orleans.Connections.Security/Security/OrleansApplicationProtocol.cs	Removes old application protocol constant.
src/Orleans.Connections.Security/Security/MemoryPoolExtensions.cs	Removes old memory pool sizing helpers.
src/Orleans.Connections.Security/Security/ITlsHandshakeFeature.cs	Removes old TLS handshake feature interface.
src/Orleans.Connections.Security/Security/ITlsConnectionFeature.cs	Removes old TLS connection feature interface.
src/Orleans.Connections.Security/Security/ITlsApplicationProtocolFeature.cs	Removes old TLS ALPN feature interface.
src/Orleans.Connections.Security/Security/DuplexPipeStreamAdapter.cs	Removes old duplex pipe stream adapter.
src/Orleans.Connections.Security/Security/CertificateLoader.cs	Removes old certificate loader (moved).
src/Orleans.Connections.Security/Orleans.Connections.Security.csproj	Removes the old TLS extension package project from the repo.
src/Orleans.Connections.Security/Hosting/HostingExtensions.cs	Removes old Kestrel connection-builder TLS extensions.
Orleans.slnx	Removes the old connections security project and its test project from the solution.

Copilot's findings

Comments suppressed due to low confidence (2)

src/Orleans.Core/Hosting/ClientTlsHostingExtensions.IClientBuilder.cs:13

• This change introduces/renames public hosting extension APIs (ClientTlsHostingExtensions.UseTls). For this repo, public API changes in packable src/ projects require updating the generated API surface files under src/api. Please regenerate/update the relevant src/api entries so CI/public API checks stay consistent.
  src/Orleans.Runtime/Hosting/SiloTlsHostingExtensions.ISiloBuilder.cs:16
• This introduces/renames public hosting extension APIs (SiloTlsHostingExtensions.UseTls). Public API changes in packable src/ projects are expected to be reflected in the generated API surface files under src/api. Please regenerate/update the relevant src/api files accordingly.

• Files reviewed: 147/148 changed files
• Comments generated: 7

[Copilot]

test/Orleans.Connections.Security.Tests/Orleans.Connections.Security.Tests.csproj was removed from Orleans.slnx, so dotnet test Orleans.slnx ... (CI default) will no longer build/run the TLS test suite. If TLS tests are still intended to run, keep this project in the solution or move the tests into an included test project.