[Question] Understanding FragmentSize setting

[a-sid]

Dear colleagues, thank you for this excellent project!

Does CycloneDDS have any recommendations on choosing optimal FragmentSize for different setups? I have studied the docs, found some setup recommendations in the Internet, tried different values myself, but I'd like to know if there is a common approach.
I am transferring large messages (9MB) over 10 Gb Ethernet with large MTU, targeting for latency. As I understand, I should have MaxMessageSize close to the max UDP datagram size (65500 bytes, as often recommended). I have also enabled recommended large socket buffer size settings both in the OS and in the DDS. But what about FragmentSize? Should it be close to the network MTU, or should it be set closer to the maximum message size, if I rely on the kernel IP fragmentation? As I guess, small FragmentSize doesn't have too much sense for large data samples, because it just increases the size overhead as the small fragments are still being packed into a single datagram. But what size should I choose? The closest discussion I can find is #1938, and it mostly confirms my thoughts, but doesn't answer my questions directly.

And another question: does it make sense to increase FragmentSize for best-effort QoS transmission, as there won't be any retransmission in case of packet loss anyway?

Thank you in advance.

[eboasson]

Hi @a-sid, thanks for the kind words.

There are a few considerations:

• Retransmits are requested on fragments: it is generally good to minimize the risk of loss of a fragment, so < 1 MTU makes sense.
• Retransmits are still subject to packing into larger datagrams up to MaxRexmitMessageSize; keeping this one < 1 MTU also usually makes sense. Packing in retransmits is slightly dumber than packing during initial transmission, so there is a small efficiency benefit to keeping FragmentSize close to MaxRexmitMessageSize.
• Initial transmission can put multiple DDSI fragments into a single DATA_FRAG submessage; this mostly eliminates a difference in overhead between small and large FragmentSize.
• DDSI fragmentation is more expensive than IP fragmentation: so if IP fragmentation allows you avoid DDSI fragmentation altogether it is a big win, this pleads for making MaxMessageSize close to the maximum UDP payload size.
• If packet loss occurs, then the cost of retransmitting dwarfs the cost of fragmentation, so if retransmits occur somewhat regularly and usually concern a few fragments, reducing MaxMessageSize may help improve utilization.
• Best-effort doesn't do retransmits, so anything in the above that involves retransmits is irrelevant.

On GbE with jumbo frames, I'd go for MaxMessageSize close to 64kB and MaxRexmitMessageSize close to 9000 B, and FragmentSize a bit below the latter. Say 65000B, 8400B and 8000B. That should result the sample being divided up into 64000-byte blocks, and then you should get ethernet frames of just over 8kB. The numbers can be tweaked a bit, but I don't think it'll make a material difference.

[a-sid]

Thank you!