Skip to content

Add chunked PhoneAPI transport payloads#10928

Open
kernel-oops wants to merge 3 commits into
meshtastic:developfrom
kernel-oops:garmin-ble-chunked-phoneapi
Open

Add chunked PhoneAPI transport payloads#10928
kernel-oops wants to merge 3 commits into
meshtastic:developfrom
kernel-oops:garmin-ble-chunked-phoneapi

Conversation

@kernel-oops

@kernel-oops kernel-oops commented Jul 7, 2026

Copy link
Copy Markdown

Summary

This adds a compact chunked PhoneAPI transport wrapper so clients with a small GATT payload limit can exchange full Meshtastic client API protobufs without relying on ATT long reads or long writes.

The immediate motivation is Garmin Connect IQ BLE support. Garmin limits application BLE writes to 20 bytes and does not implement ATT long reads, while normal ToRadio and FromRadio protobufs can be much larger than that. Without explicit application-layer chunking, a Garmin client either cannot send larger requests or can misinterpret repeated fromradio reads as continuations of the same protobuf.

Depends on the corresponding protobufs schema PR: meshtastic/protobufs#980

What changed

  • Adds ChunkedPayload as a compact ordered wrapper:
    • ToRadio.chunked_payload = 8
    • FromRadio.chunked_payload = 20
    • first chunk carries payload_size
    • all chunks carry payload_chunk
  • Replaces the earlier id/count/index fields with an ordered-transfer format to keep 20-byte BLE packets efficient.
  • Adds PhoneAPI reassembly for incoming ToRadio.chunked_payload messages.
  • Adds opt-in FromRadio.chunked_payload emission for clients that have successfully completed a chunked ToRadio transfer.
  • Keeps old clients on existing unchunked behavior.
  • Regenerates mesh.pb.h for the new schema.

Transport behavior

The chunking format intentionally assumes ordered, reliable delivery on a single client connection. That matches Garmin's serialized BLE write-with-response and read flow and avoids spending bytes on payload ids and chunk indices in every 20-byte packet.

For Garmin-sized packets:

  • ToRadio first chunk carries 13 payload bytes.
  • ToRadio continuation chunks carry 16 payload bytes.
  • FromRadio first chunk carries 12 payload bytes.
  • FromRadio continuation chunks carry 15 payload bytes.

The smaller FromRadio first/continuation sizes account for the larger FromRadio.chunked_payload field tag while keeping each serialized wrapper at or below 20 bytes.

Safety and compatibility considerations

  • FromRadio chunking is opt-in only. Legacy clients continue receiving existing unchunked FromRadio messages.
  • Opt-in is recorded only after a complete chunked ToRadio transfer is reassembled and accepted into the normal PhoneAPI path.
  • Incoming chunk state is keyed per PhoneAPI* and protected by a small lock-guarded slot table.
  • The slot table is deliberately small and sized for the normal serial/BLE client footprint rather than many simultaneous chunking clients. Extra clients fail closed, and stale incoming transfers are reclaimed after 30 seconds.
  • Chunk state is cleared on PhoneAPI::close().
  • A disconnected matching PhoneAPI clears stale chunk capability state before reuse.
  • Outbound chunking has an explicit source-size guard before copying into the 512-byte chunk buffer.

Security/code-review notes

I reviewed the memory-safety paths with emphasis on untrusted BLE/client input:

  • payload_chunk cannot exceed the generated nanopb PB_BYTES_ARRAY_T(228) storage; nanopb rejects oversized bytes fields before PhoneAPI sees them.
  • First-chunk payload_size is rejected if it exceeds MAX_TO_FROM_RADIO_SIZE or is smaller than the chunk being copied.
  • Continuations are rejected and the slot is cleared if bytesUsed + chunk.size would exceed totalSize.
  • Reassembled payloads are copied out to a stack buffer only after totalSize <= MAX_TO_FROM_RADIO_SIZE has been enforced.
  • Outbound source payloads are copied into a fixed 512-byte slot only after checking fromRadioSize <= sizeof(slot->bytes).
  • Pending outbound chunks are emitted before generating the next normal FromRadio, preserving chunk continuity.

Residual risks/tradeoffs:

  • This is not a retransmission protocol; it depends on ordered reliable reads/writes for a single client connection.
  • Two simultaneous incomplete incoming chunk transfers can occupy the global ToRadio chunk slots until timeout, which is a bounded availability tradeoff rather than a memory-safety issue.
  • A complete schema rollout also requires the corresponding protobufs commit/submodule pointer to be available to reviewers.

Testing

  • Built heltec-v4 with PlatformIO:
    • $HOME/.platformio/penv/bin/pio run -e heltec-v4
  • Ran focused wrapper-size sanity checks for 22-byte, 111-byte, and 512-byte original payloads; all compact chunk wrappers serialized to <=20 bytes.
  • Ran whitespace checks:
    • git diff --check
    • git -C protobufs diff --check
  • Flashed a local heltec-v4 test node over /dev/ttyACM0 with PlatformIO upload; flashing completed with hash verification.
  • Queried the local test node over serial after reboot; the device responded with expected owner metadata.

Notes:

  • The local project .venv/bin/pio wrapper currently has a stale shebang pointing at /tmp/meshtastic-firmware/.venv/bin/python3, so the build/flash used $HOME/.platformio/penv/bin/pio.
  • A local flash helper script is untracked and intentionally excluded from this PR.

Summary by CodeRabbit

  • New Features
    • Added support for chunked transport of large radio protobuf messages to improve delivery of bigger payloads.
    • Large incoming and outgoing messages can now be split/reassembled, with support negotiated per connection.
  • Bug Fixes
    • Improved validation for chunk sizes and safer reassembly behavior to prevent corrupted/failed transfers.
    • Clear pending in-progress chunked data when connections close or reassembly times out.

@CLAassistant

CLAassistant commented Jul 7, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@coderabbitai

coderabbitai Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 07325b9f-8d36-4d8c-9b03-0f5df3a30b97

📥 Commits

Reviewing files that changed from the base of the PR and between f619b73 and 1ae9122.

📒 Files selected for processing (1)
  • src/mesh/PhoneAPI.cpp
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/mesh/PhoneAPI.cpp

📝 Walkthrough

Walkthrough

This PR bumps the protobufs submodule reference and adds chunked-payload transport in PhoneAPI.cpp for large ToRadio and FromRadio protobuf messages, including reassembly, chunk encoding, and connection-state cleanup.

Changes

Chunked Payload Support

Layer / File(s) Summary
Chunked payload infrastructure and helpers
src/mesh/PhoneAPI.cpp, protobufs
Per-connection chunk slot tables, mutexes, timeout-based stale-slot reclamation, inbound reassembly logic, outbound chunk-encoding helpers, and the protobufs submodule reference are updated together.
Inbound chunk handling and connection cleanup
src/mesh/PhoneAPI.cpp
handleToRadio() accepts meshtastic_ToRadio_chunked_payload_tag and close() clears chunked state for the connection.
Outbound chunked delivery paths
src/mesh/PhoneAPI.cpp
getFromRadio() returns pending chunk fragments first and routes both reply paths through maybeEncodeChunkedFromRadio().

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant PhoneAPI
  participant ChunkSlots

  Client->>PhoneAPI: send chunked ToRadio payload
  PhoneAPI->>ChunkSlots: allocate or locate reassembly slot
  PhoneAPI->>ChunkSlots: append chunk bytes
  alt payload complete
    PhoneAPI->>PhoneAPI: forward reassembled bytes
  else invalid or overflow
    PhoneAPI->>ChunkSlots: clear slot
  end

  Client->>PhoneAPI: getFromRadio()
  PhoneAPI->>ChunkSlots: check pending outbound chunk
  alt chunk pending
    PhoneAPI-->>Client: return chunk bytes
  else encode new FromRadio
    PhoneAPI->>PhoneAPI: maybeEncodeChunkedFromRadio(encoded bytes)
    PhoneAPI-->>Client: return chunked or original bytes
  end
Loading

Related PRs None

Suggested labels None

Suggested reviewers None

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely summarizes the main change: adding chunked PhoneAPI transport payloads.
Description check ✅ Passed The description is detailed and covers summary, behavior, safety, compatibility, and testing, so it mostly satisfies the template.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

@kernel-oops, Welcome to Meshtastic!

Thanks for opening your first pull request. We really appreciate it.

We discuss work as a team in discord, please join us in the #firmware channel.
There's a big backlog of patches at the moment. If you have time,
please help us with some code review and testing of other PRs!

Welcome to the team 😄

@github-actions github-actions Bot added first-contribution enhancement New feature or request labels Jul 7, 2026
@kernel-oops
kernel-oops marked this pull request as ready for review July 7, 2026 18:25
@kernel-oops

kernel-oops commented Jul 7, 2026

Copy link
Copy Markdown
Author

Local validation after rebasing onto current develop:

  • Resolved the protobuf submodule conflict by updating to kernel-oops/protobufs@90d0af5 from Add compact chunked client API payload protobufs#980.
  • ./.venv/bin/pio has a stale shebang in this checkout, so I ran PlatformIO via the venv Python instead.
  • Command: .venv/bin/python .venv/bin/pio run -e heltec-v4
  • Result: heltec-v4 SUCCESS (00:37:57.748, including first-run tool/dependency downloads).

The protobuf PR is now ready for review as well: meshtastic/protobufs#980.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
src/mesh/PhoneAPI.cpp (1)

179-186: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Use Throttle for the stale-slot timeout instead of raw millis() arithmetic.

This stale-transfer expiry is a "did N ms pass since X" check and should go through the Throttle helper.

♻️ Proposed change
-static ChunkedToRadioSlot *findChunkedToRadioSlot_LH(PhoneAPI *api)
-{
-    const uint32_t now = millis();
-    for (auto &slot : g_chunkedToRadioSlots) {
-        if (slot.who != nullptr && now - slot.updatedMillis > CHUNKED_TORADIO_TIMEOUT_MS) {
-            clearChunkedToRadioSlot_LH(slot);
-        }
-    }
+static ChunkedToRadioSlot *findChunkedToRadioSlot_LH(PhoneAPI *api)
+{
+    for (auto &slot : g_chunkedToRadioSlots) {
+        if (slot.who != nullptr && !Throttle::isWithinTimespanMs(slot.updatedMillis, CHUNKED_TORADIO_TIMEOUT_MS)) {
+            clearChunkedToRadioSlot_LH(slot);
+        }
+    }

As per coding guidelines: "Use Throttle for time-based rate limiting instead of raw millis() arithmetic; prefer Throttle::isWithinTimespanMs(lastMs, intervalMs) ... for 'did N ms pass since X' checks."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/mesh/PhoneAPI.cpp` around lines 179 - 186, The stale-slot expiry check in
findChunkedToRadioSlot_LH is using raw millis() subtraction to decide whether
CHUNKED_TORADIO_TIMEOUT_MS has elapsed. Replace that logic with the Throttle
helper, using the existing slot timestamp field and
Throttle::isWithinTimespanMs(...) for the “has N ms passed since X” check before
calling clearChunkedToRadioSlot_LH. Keep the behavior the same, but route the
timeout decision through Throttle to match the time-based rate-limiting
guideline.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@src/mesh/PhoneAPI.cpp`:
- Around line 179-186: The stale-slot expiry check in findChunkedToRadioSlot_LH
is using raw millis() subtraction to decide whether CHUNKED_TORADIO_TIMEOUT_MS
has elapsed. Replace that logic with the Throttle helper, using the existing
slot timestamp field and Throttle::isWithinTimespanMs(...) for the “has N ms
passed since X” check before calling clearChunkedToRadioSlot_LH. Keep the
behavior the same, but route the timeout decision through Throttle to match the
time-based rate-limiting guideline.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 8ccf108a-0720-4d93-a686-99de0a8e10ee

📥 Commits

Reviewing files that changed from the base of the PR and between d16ae2b and f619b73.

⛔ Files ignored due to path filters (1)
  • src/mesh/generated/meshtastic/mesh.pb.h is excluded by !**/generated/**, !src/mesh/generated/**
📒 Files selected for processing (2)
  • protobufs
  • src/mesh/PhoneAPI.cpp

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request first-contribution

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants