Add vision subpackage with MediaPipe HeadTracker#1177
Conversation
|
Is it an issue to have reachy_mini_toolbox as a dep in the conv app? |
|
LGTM, indeed as @FabienDanieau points out. Maybe we can use toolbox as a dependency. Not strong opinion on this. Just if we do, we have to make sure that everything included runs on a RM wireless. Maybe next step is ton add start/stop/getface API entry points to trigger the head tracking in background? |
|
I made some tests. rf-detr-nano is too heavy for the rpi. Mediapipe seems to be the most suitable solution for edge devices. There is a lighter model that we can use instead of the facemesh of the tool box: see face_track.py.
I agree with @pierre-rouanet, the right call would be to enable head tracking on the daemon side (in a way similar we did for the wobbling see #1001). Basically we need to add a appsink to the video pipeline to feed the head tracker. Also the tracking could be extended with the DoA. But let's keep that for another issue :) |
# Conflicts: # uv.lock
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
…frames, simplify look-at and SDK transport
…rame, retune for responsiveness
|
let's say that mediapipe works here though it's not ideal: I'm continuing my experiment with completely another approach in this branch |
… keep daemon/SDK foundation
… the full-res IPC
…ates while closed
…ss, gate stray faces
|
next checkpoint, commit 9a5b52f
|
…nder the conv app
…t stopping the detector
Adds a vision subpackage with the MediaPipe HeadTracker, migrated from reachy_mini_toolbox. Also,
mediapipe==0.10.14is added as a new optional extra. The originalexit()on missing mediapipe is replaced with a properImportErrorto avoid killing the robot process on importCloses #1175
The conv app PR: pollen-robotics/reachy_mini_conversation_app#381