Skip to content

feat(benchmark): swappable-planner harness (latency + smoothness) for gru/flow_matching/bezier#83

Merged
m-zain-khawaja merged 1 commit into
autowarefoundation:mainfrom
gcordova10:feat/t1-planner-benchmark
Jun 23, 2026
Merged

feat(benchmark): swappable-planner harness (latency + smoothness) for gru/flow_matching/bezier#83
m-zain-khawaja merged 1 commit into
autowarefoundation:mainfrom
gcordova10:feat/t1-planner-benchmark

Conversation

@gcordova10

Copy link
Copy Markdown
Contributor

As requested in #69, moving the planner benchmark to its own PR. It runs gru, flow_matching and bezier through PLANNER_REGISTRY under identical conditions and reports inference latency (p50/p99/jitter), param count, and architectural smoothness (Var(Δaccel), Var(Δcurvature)) on the (accel, curvature) output.

planner params p50 (ms) jerk Var(Δaccel) curv Var(Δcurv)
gru 829K 50.3 1.4e-04 3.8e-04
flow_matching 988K 13.9 4.2e+00 2.0e+00
bezier 495K 0.63 5.0e-06 1.2e-06

Honest caveat: random weights → ADE/FDE, off-road, collision and closed-loop are not comparable yet; they need a trained checkpoint + dataset (KITScenes/L2D) + a simulator (NAVSIM/Bench2Drive). The harness documents those as hooks. @m-zain-khawaja — happy to extend it with the trained metrics once the training loop (#78) is wired. Relates to #56, #45.

… for gru/flow_matching/bezier

Compares the registered planners under identical conditions: inference latency
(p50/p99/jitter), param count, and architectural smoothness (Var(Δaccel),
Var(Δcurvature)) on the (accel, curvature) unicycle output. ADE/FDE, off-road,
collision and closed-loop are left as documented hooks (need trained checkpoint
+ dataset/simulator). Addresses @m-zain-khawaja's request in autowarefoundation#69 / autowarefoundation#56.

Signed-off-by: GABRIELA CORDOVA <100548769@alumnos.uc3m.es>

@m-zain-khawaja m-zain-khawaja left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved - thanks @gcordova10

@m-zain-khawaja m-zain-khawaja merged commit 2d50805 into autowarefoundation:main Jun 23, 2026
1 check passed
@gcordova10 gcordova10 deleted the feat/t1-planner-benchmark branch June 30, 2026 15:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants