feat(media): preroll move-paired sounds to remove playbin warmup skew

[tfrere]

Summary

Preroll move-paired sounds so audio starts deterministically (~1 ms) instead of a variable 15-50 ms after the motion clock. audio_lead_ms = 0 then truly means "audio and motion start together".

Problem

Every play_sound() rebuilds its audio player from scratch - detect the format, spin up the decoder, open the ALSA device. That takes 15-50 ms and is never the same twice (spikes past 100 ms under CPU load), while the motion starts instantly: the gesture always leads the sound by an amount no fixed audio_lead_ms can cancel.

Solution

Build the player before the go signal:

• prepare_sound() prerolls the playbin to PAUSED (the slow part: sink open + decode + buffer fill)
• start_prepared_sound() flips it to PLAYING at t0 (~1 ms, deterministic, load-independent)
• Backend._play_move prerolls off the event loop (executor) before the motion clock starts, then starts the prepared sound at the requested lead

Benchmarks

On the robot (wireless CM4, daemon running, wake_up.wav, 20 runs - time from the PLAYING request to the first buffer reaching the daemon's own ALSA sink):

start path	median	max	stdev
cold - today's play_sound() at t0	16.1 ms	46.5 ms	7.3 ms
primed - this PR, preroll before t0	0.9 ms	1.4 ms	0.2 ms

With 3 CPU cores saturated, cold median rises to ~48 ms with spikes >100 ms (measured with the fuller probe in scripts/measure_playbin_warmup.py, which rebuilds the daemon's complete tee sink bin); primed stays at 1-4 ms regardless of load.

How to reproduce

Standalone script, no reachy_mini import: python3 warmup_minimal.py /path/to/wake_up.wav 20

warmup_minimal.py

#!/usr/bin/env python3
"""cold = NULL->PLAYING at t0 (today). primed = preroll to PAUSED before t0,
then PAUSED->PLAYING at t0 (this PR). Latency = t0 until the first buffer
reaches the sink pad of the exact sink the daemon uses."""
import statistics, sys, threading, time
import gi
gi.require_version("Gst", "1.0")
from gi.repository import Gst

Gst.init(None)
FILE = sys.argv[1]
RUNS = int(sys.argv[2]) if len(sys.argv) > 2 else 15

def make_playbin():
    sink = Gst.ElementFactory.make("alsasink")
    sink.set_property("device", "reachymini_audio_sink")
    if sink.set_state(Gst.State.READY) == Gst.StateChangeReturn.FAILURE:
        sink.set_state(Gst.State.NULL)
        sink = Gst.ElementFactory.make("autoaudiosink")  # non-robot fallback
    else:
        sink.set_state(Gst.State.NULL)
    p = Gst.ElementFactory.make("playbin")
    p.set_property("uri", Gst.filename_to_uri(FILE))
    p.set_property("audio-sink", sink)
    return p, sink

def first_buffer_latency(p, sink, primed):
    got, stamp = threading.Event(), []
    def probe(pad, info):
        stamp.append(time.monotonic()); got.set()
        return Gst.PadProbeReturn.REMOVE
    if primed:
        p.set_state(Gst.State.PAUSED)
        p.get_state(Gst.CLOCK_TIME_NONE)      # preroll done, BEFORE t0
    pad = (sink.get_static_pad("sink")
           or sink.iterate_sinks().next()[1].get_static_pad("sink"))
    pad.add_probe(Gst.PadProbeType.BUFFER, probe)
    t0 = time.monotonic()
    p.set_state(Gst.State.PLAYING)
    got.wait(5); time.sleep(0.1); p.set_state(Gst.State.NULL)
    return (stamp[0] - t0) * 1000 if stamp else float("nan")

cold, primed = [], []
for _ in range(RUNS):
    p, s = make_playbin(); cold.append(first_buffer_latency(p, s, False))
    p, s = make_playbin(); primed.append(first_buffer_latency(p, s, True))

for name, xs in (("cold", cold), ("primed", primed)):
    print(f"{name:7s} n={len(xs)}  median={statistics.median(xs):6.1f} ms  "
          f"max={max(xs):6.1f} ms  stdev={statistics.stdev(xs):5.1f} ms")

Limitations / open questions (why this is a draft)

• End-to-end validation on a patched daemon is still to be done (motion start vs first audible sample, e.g. the antenna-collision mic test). The benchmarks above validate the GStreamer mechanism in isolation, not the full play_move path. Motor chain latency is out of scope.
• The executor treatment done for _play_move is also needed for the synchronous play_sound() callers (wake_up/goto_sleep)
• The preroll-failure fallback just re-runs a preroll; restore a true legacy path or drop it
• The prepared playbin is single-slot: a concurrent play_sound during the lead window swaps it under the move

Related

• Feature/daemon side move upload #1133 introduced audio_lead_ms as an empirical knob; this PR gives it back a single meaning (artistic offset)
• The head wobbler solved the same class of problem in April by trusting GStreamer's clock instead of latency constants (cb328c5)
• The JS SDK hardcodes audioLeadMs = -100 (ts/lib/reachy-mini.ts:1645) - that constant should not ship in this week's frozen SDK release, regardless of this PR's fate
• fix(media): tear down the play_sound playbin on EOS/error #1330 fixes the playbin's end of life (teardown on EOS/error); this PR fixes its start. Complementary; expect a media_server.py conflict for whichever lands second

Made with Cursor