Skip to content

Commit 305662e

Browse files
srperensclaude
andauthored
feat(vision-mixer): zone borders as underlay pads, PiP layout presets, transition fixes (#625)
* feat(vision-mixer): zone borders drawn live on PGM and the multiview Mixer-state-driven graphics belong in the mixer: borders frame boxes whose geometry only the mixer knows in real time (morphs, takes and punch-ins drive the compositor pads via control bindings), so no external graphics source could stay in sync. External graphics (lower thirds, bugs) remain on DSK. - Zone gains border: Option<ZoneBorder> { color: #RRGGBB[AA], width } (serde default — additive API). The border belongs to the BOX, not the source: a capacity-1 swap zone keeps its frame whoever is pushed in. Borders draw around the actual rendered source boxes (aspect-fitted rects), hugging the picture rather than the layout rect. - Width is resolution-normalized: expressed in PGM canvas pixels and scaled by region_width / pgm_width at every render target, so 4 px on air looks proportionally identical on the PVW display and PiP tiles. - New PGM overlay (pgm_overlay.rs): cairo → appsrc → one pad on the dist compositor below the DSK stack (zorder 90), mirroring the multiview overlay pattern. Geometry is read LIVE from the compositor sink pads each timer tick (property reads, never a buffer probe) — borders track animations frame by frame and fade with their box's alpha. Dirty-hash skips redraws when nothing moved. - Multiview parity: the existing mv overlay draws the same borders on PiP tiles and the PVW big display (PGM big shows the dist output and carries them already), with a live-geometry hash in its dirty check so borders follow morphs there too. - Pad-loop audit: the classic-take reset and FTB restore leave the new PGM overlay pad alone / restore it (it sits beyond the DSK range). - Zone editor: color swatch + width field on the active-zone control row (v2.21). Operator guide updated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(vision-mixer): evaluate border geometry at composite time, not sampling time Borders trailed their boxes during morphs/takes: the overlay renderer sampled the pads' current property values, but an overlay frame pushed now is composited 1-3 output frames later — by which time the control bindings driving the animation have moved the box further. Evaluate the bindings at (compositor position + 66 ms lead) instead via GstControlBinding::get_value, so the border is drawn where the box WILL be when the frame composites. Properties without an active binding fall back to plain reads. Applied to both the PGM overlay and the multiview border pass (shared pad_geometry_at helper). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(vision-mixer): border look-ahead is one output frame period The trailing fix evaluated border geometry 2 frame periods ahead, which overshot — borders led their boxes during morphs. One period matches reality: query_position() reports the frame currently being produced and a buffer pushed now lands in the next one. Confirmed empirically (0 frames trails, 2 leads, 1 tracks). The lead derives from each output's actual framerate — PGM and multiview can differ and each renderer uses its own. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(vision-mixer): harden PGM overlay — gate on PiPs, restart race, CPU format Self-review findings on the zone-borders branch: - Build the PGM overlay machinery only when num_pips > 0: zones (and thus borders) cannot exist without PiPs, and the default mixer config has none — skipping it saves a compositor pad, an appsrc chain and a render thread on every plain mixer. The FTB/take pad-index guards remain valid either way (the pad index simply never matches). - Timer restart race (also fixed in the pre-existing multiview timer): a fast flow restart can re-register a NEW renderer under the same block id before the old timer thread observes the unregistration — the presence check alone would keep the old thread (and its strong AppSrc ref into the dead pipeline) alive forever. Both timers now verify registry identity (Arc::ptr_eq) instead. - CPU backend: match the forced output format with a capsfilter after videoconvert_pgm_overlay, mirroring the DSK and multiview overlay chains. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(vision-mixer): border lead from latency, style in dirty-hash, FTB alpha restore on take Three zone-border fixes: - Derive the border geometry look-ahead from the compositor's aggregation latency (the block's `latency` property) instead of one output frame period. The two happened to coincide at the 20 ms default / 50 fps, but a mixer configured with e.g. 500 ms latency drew borders ~480 ms behind their boxes. - Resolve border style (color/width) once per tick in border_pad_set (new MvBorder struct) and include it in the renderer dirty-hash, so a color- or width-only edit re-renders the multiview immediately instead of waiting for the next geometry change. - On take with FTB auto-cancel, strip the FTB alpha binding from the PGM graphics overlay pad and pin it back to full alpha (both the classic and PiP-aware take paths) — borders no longer stay frozen at whatever alpha a mid-flight fade last wrote. Frontend: keep an API-set #RRGGBBAA border color's alpha when re-committing border edits through the 6-digit color picker. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(vision-mixer): per-PiP layout presets — save/load named compositions Backend: new GET /api/flows/{flow_id}/blocks/{block_id}/pip/{pip_idx} returning the PiP's live composition (bg, zones, transforms) as PipState — the export half of save/restore; restore is the existing PUT on the same path. 404 when the pipeline is not running or the index is out of range. Registered in openapi.json. Frontend (v2.24): Layout preset row in the PiP editor — dropdown plus Save/Load/Delete against a browser-wide localStorage key, so a preset saved on one PiP/flow can be loaded on any other. A preset is a full composition snapshot: zones (rect, capacity, border, sources by input index), background, and every non-default crop — including retained crops for sources currently outside the PiP, mirroring strom's own swap-zone semantics. Loading is best-effort: input indices that do not exist on the target mixer are silently skipped, the bg double-booked as a zone source is stripped, and cross-zone duplicates are deduped — each of which would otherwise fail the whole PUT with 400. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(vision-mixer): zone borders as compositor underlay pads, not cairo overlays A zone border is now a solid-color compositor pad sitting in z-order directly beneath its content pad, sized to the content rect inflated outward by the region-scaled border width. This replaces the cairo-drawn borders (the PGM graphics overlay renderer and the multiview overlay's border pass) and fixes two structural problems with drawing borders on top of the mix: - Morph sync: the underlay is animated by the same control-binding timeline as its content pad, so the compositor evaluates both at the same composite time. The whole look-ahead chain (border_lead, the latency-derived lead, the per-tick geometry polling and dirty-hash, the cairo re-render, the FTB alpha special-casing in the take engine) is deleted — it existed to chase the compositor's clock from outside. - Z-order: zone slots use a doubled scheme (content at overlay_zorder + 2k + 1, underlay one below), so an overlapping higher zone covers a lower zone's border exactly like a stacked framed card, instead of all borders rendering on top of all content. Each underlay pad is fed by a 16x16 videotestsrc (pattern=solid-color, non-live so it adds no latency); the border color is written to its foreground-color. Pads exist only when PiPs are configured: dist input i's underlay at sink_{N+DSK+i} (replacing the PGM overlay pad), and on the multiview after the overlay pad (PVW big + per-tile candidates). Borders now also fade with their box (FTB, capacity-1 cross-fades) and clamp to their region, so a tile border no longer bleeds into the neighboring multiview tile. The stroke now sits fully outside the video edge (it never covers content pixels) instead of half over it. PadTarget carries an optional UnderlayTarget; plan_transition still decides actions from content geometry alone, and the controller drives each pad's underlay in lockstep with whatever action the content pad got. Verified live on the CPU backend: geometry/zorder/color on dist and multiview pads, FTB on/off, cut/fade takes input<->PiP, zone-rect morphs, and two overlapping bordered zones interleaving correctly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(transitions): correct slide/push geometry for mixed aspect ratios Slide and push transitions computed the incoming pad's destination from the OUTGOING pad's position — correct back when every pad was fullscreen, wrong since explicit per-pad aspect-fit geometry: taking between a 16:9 and a 2.39:1 source landed the incoming picture at the outgoing picture's letterbox offset (wrong y). The classic-take reset already positions every pad at its own aspect-fitted rect before the transition runs, so the incoming pad's own xpos/ypos IS the destination — read that instead. Slide also gets end-of-slide handling per the planner's rule ("what's visible needs a transition; what's hidden doesn't"): when the incoming rect fully covers the outgoing one, the old picture snaps off at the end as before (it's hidden under the new one). With mismatched aspect ratios the incoming rect can leave parts of the old picture visible — those now fade out over the slide instead of popping away at the end. Push needs neither: the outgoing picture always exits a full canvas dimension. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(vision-mixer): zone-aware source chips, crop follows zone, full transition picker Three UI improvements: - The source chips now act as checkboxes for the ACTIVE zone: checked = in the selected zone; a source sitting in another zone of the same PiP shows with a dashed outline and clicking it MOVES it to the active zone (previously the chips showed "in any zone" and clicking a taken source removed it from wherever it was). Tooltips spell out remove/move/push per chip. - Selecting a zone (zone buttons, canvas click, + Zone, preset load) points the Crop/Zoom panel at the zone's first source — selecting a box usually means "work on this box". Manually picking another crop target stays sticky until the next zone switch. - The transition picker now exposes all engine variants: MIX, DIP, PUSH and SLIDE with a direction row (left/right/up/down) for the latter two. Previously only fade, dip and push_left were reachable. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(gst): close use-after-free race in control binding handling — never remove bindings Production segfault (GP fault, confirmed via core dump): GStreamer's gst_object_sync_values() iterates an object's control_bindings GList WITHOUT any lock — the lock is deliberately disabled upstream ("FIXME: this deadlocks" in gstobject.c, present in 1.24 and current master). gst_object_remove_control_binding() mutates and frees list nodes under the object lock from whatever thread calls it. Removing a binding while a streaming thread is mid-iteration is therefore a use-after-free. We hit it through videocrop, which syncs control values on EVERY buffer: rapid PiP updates (dragging a zone rect fires many PUTs per second) had the API thread removing + re-adding crop bindings while the videocrop streaming thread walked the list — the core dump shows the iteration following a dangling node->next into freed memory. Compositor pads sync per output frame and carry the same race, just with a narrower window. Fix: persistent bindings (new gst::control_bindings module). Each (object, property) pair gets ONE DirectControlBinding with ONE InterpolationControlSource, attached on first use and never removed — first-time attachment appends, which the unlocked reader tolerates; removal is what frees memory under it. "Clearing" a binding now wipes the control source's keyframes (unset_all — guarded by the source's internal lock), which makes the binding inert: get_value returns FALSE, the property is left alone, and plain set_property writes behave exactly as before. Reprogramming reuses the same source (unset_all + set_mode + new keyframes). All former add/remove sites migrated: the transition controller, set_pad_crop (both backends), the layout appliers, the classic-take reset, and FTB (whose cleanup timeout now wipes instead of removes, keeping DSK toggles and takes writable after a fade). Verified live on the CPU backend: 300 sequential rapid PiP updates plus 3 parallel update hammers with concurrent fade/push takes — previously crashed within seconds of dragging; now survives with the API healthy. Note: volume_ramp.rs still uses add/remove on the volume element (same race class, lower frequency) — migrate separately. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(vision-mixer): update operator guide — borders, presets, transitions, zone-aware editor Catch the guide up with the branch: full transition palette (slide/push directions + dip_to_black) with mixed-aspect behavior, the broadened PiP downgrade rule, the composited zone-border semantics (outside the picture edge, stacks like framed cards, fades with its box), zone-aware source chips, crop-follows-zone, layout presets, and the PiP composition export in the controls reference + glossary. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(vision-mixer): review findings — CPU border alpha, zorder ceiling, underlay churn Three fixes from the pre-merge review pass: - CPU underlay sources always produce RGBA instead of inheriting the forced output_format: an alpha-less format (I420/NV12) silently dropped the alpha of #RRGGBBAA border colors (opaque on CPU, correct on GPU), and exotic forced formats risked failing videotestsrc negotiation. The compositor's convert pads handle per-pad conversion. - TRANSITION_FOREGROUND_ZORDER raised 50 → 60: the doubled zone z-order scheme tops out at 21 + 2·14 + 1 = 50 on a multiview tile at MAX_PIP_OVERLAYS, exactly colliding with the old lift base — a pad lifted during a concurrent morph could swap stacking order with a fully-loaded zone's top source. Lifted ceilings still clear DSK (100) on dist and the overlay (200) on the multiview. - Underlay videotestsrc framerate 30 → 5 fps (the color only changes on border edits; the mixer keeps compositing the latest buffer between pushes), and the preset localStorage parse hoisted out of the per-PiP panel loop. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 1e46ef5 commit 305662e

24 files changed

Lines changed: 1761 additions & 238 deletions

backend/src/api/flows.rs

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1704,6 +1704,63 @@ pub async fn select_preview(
17041704
}))
17051705
}
17061706

1707+
/// Get the current composition of a single PiP on a vision mixer block.
1708+
///
1709+
/// Returns the same per-PiP state as the corresponding entry in
1710+
/// `VisionMixerState::pips`. Useful for exporting one PiP's composition
1711+
/// (e.g. to save it as a reusable layout preset); restore it with a `PUT`
1712+
/// to the same path.
1713+
#[utoipa::path(
1714+
get,
1715+
path = "/api/flows/{flow_id}/blocks/{block_id}/pip/{pip_idx}",
1716+
tag = "flows",
1717+
params(
1718+
("flow_id" = String, Path, description = "Flow ID (UUID)"),
1719+
("block_id" = String, Path, description = "Vision mixer block instance ID"),
1720+
("pip_idx" = usize, Path, description = "PiP index (0-based)")
1721+
),
1722+
responses(
1723+
(status = 200, description = "Current PiP composition", body = strom_types::api::PipState),
1724+
(status = 404, description = "Block has no live state (pipeline not running) or PiP index out of range", body = ErrorResponse),
1725+
)
1726+
)]
1727+
pub async fn get_pip_config(
1728+
Path((flow_id, block_id, pip_idx)): Path<(FlowId, String, usize)>,
1729+
) -> Result<Json<strom_types::api::PipState>, (StatusCode, Json<ErrorResponse>)> {
1730+
let overlay = crate::blocks::builtin::vision_mixer::overlay::get_overlay_state(&block_id)
1731+
.ok_or_else(|| {
1732+
(
1733+
StatusCode::NOT_FOUND,
1734+
Json(ErrorResponse::with_details(
1735+
"Vision mixer state not available",
1736+
format!(
1737+
"No live overlay state for block {} in flow {} (pipeline not running)",
1738+
block_id, flow_id
1739+
),
1740+
)),
1741+
)
1742+
})?;
1743+
1744+
if pip_idx >= overlay.num_pips {
1745+
return Err((
1746+
StatusCode::NOT_FOUND,
1747+
Json(ErrorResponse::with_details(
1748+
"PiP index out of range",
1749+
format!(
1750+
"PiP index {} out of range (block {} has {} PiPs)",
1751+
pip_idx, block_id, overlay.num_pips
1752+
),
1753+
)),
1754+
));
1755+
}
1756+
1757+
Ok(Json(strom_types::api::PipState {
1758+
bg: overlay.pip_bg_input(pip_idx),
1759+
zones: overlay.pip_zones(pip_idx),
1760+
transforms: overlay.pip_transforms(pip_idx),
1761+
}))
1762+
}
1763+
17071764
/// Update a PiP composition (background + overlay inputs) on a vision mixer block.
17081765
///
17091766
/// The change is applied live to all places where the PiP is currently visible:

backend/src/blocks/builtin/vision_mixer/builder/mod.rs

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -328,6 +328,8 @@ pub(super) fn setup_overlay_renderer(
328328
initial_pvw_input,
329329
p.labels.to_vec(),
330330
mv_layout.clone(),
331+
p.pgm_w,
332+
p.pgm_h,
331333
p.show_vu_meters,
332334
pip_initial,
333335
));

backend/src/blocks/builtin/vision_mixer/builder/pad_layout.rs

Lines changed: 57 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -51,13 +51,30 @@ fn initial_pad_geom_for_input(
5151
l.w,
5252
l.h,
5353
1.0,
54-
(overlay_zorder + l.zorder_offset) as u64,
54+
vision_mixer::zone_content_zorder(overlay_zorder, l.zorder_offset) as u64,
5555
)
5656
} else {
5757
(0, 0, 1, 1, 0.0, bg_zorder as u64)
5858
}
5959
}
6060

61+
/// Initial properties for a border underlay pad: hidden, explicit geometry.
62+
/// Zones (and thus borders) are runtime-only, so every underlay starts
63+
/// invisible; the layout appliers position and reveal them when a bordered
64+
/// zone appears.
65+
fn underlay_initial_props(props: &mut HashMap<String, PropertyValue>, zorder: u32) {
66+
props.insert("xpos".to_string(), PropertyValue::Int(0));
67+
props.insert("ypos".to_string(), PropertyValue::Int(0));
68+
props.insert("width".to_string(), PropertyValue::Int(1));
69+
props.insert("height".to_string(), PropertyValue::Int(1));
70+
props.insert("alpha".to_string(), PropertyValue::Float(0.0));
71+
props.insert("zorder".to_string(), PropertyValue::UInt(zorder as u64));
72+
props.insert(
73+
"sizing-policy".to_string(),
74+
PropertyValue::String("none".to_string()),
75+
);
76+
}
77+
6178
/// Build pad_properties for compositor sink pads (applied after linking).
6279
///
6380
/// Since glvideomixerelement uses auto-created request pads (the linker uses
@@ -149,6 +166,19 @@ pub(super) fn build_pad_properties(
149166
);
150167
}
151168

169+
// --- Dist border underlay pads: sink_{N + DSK + i} ---
170+
// Only present when PiPs are configured (see the pipeline builders).
171+
// Hidden at build — zones are runtime-only.
172+
if p.num_pips > 0 {
173+
for i in 0..p.num_inputs {
174+
let pad_name = format!("sink_{}", p.num_inputs + p.num_dsk_inputs + i);
175+
underlay_initial_props(
176+
dist_pads.entry(pad_name).or_default(),
177+
vision_mixer::DIST_PIP_OVERLAY_ZORDER,
178+
);
179+
}
180+
}
181+
152182
// --- Multiview compositor pad properties ---
153183
let mv_pads = pad_props.entry(mv_comp_id).or_default();
154184

@@ -282,8 +312,9 @@ pub(super) fn build_pad_properties(
282312
}
283313

284314
// --- Overlay pad: fullscreen, highest zorder ---
315+
let overlay_pad_idx = 2 * p.num_inputs + 1 + p.num_pips * p.num_inputs;
285316
{
286-
let overlay_pad_name = format!("sink_{}", 2 * p.num_inputs + 1 + p.num_pips * p.num_inputs);
317+
let overlay_pad_name = format!("sink_{}", overlay_pad_idx);
287318
let props = mv_pads.entry(overlay_pad_name).or_default();
288319
props.insert("xpos".to_string(), PropertyValue::Int(0));
289320
props.insert("ypos".to_string(), PropertyValue::Int(0));
@@ -296,5 +327,29 @@ pub(super) fn build_pad_properties(
296327
);
297328
}
298329

330+
// --- Multiview border underlay pads (PVW big + PiP tiles), hidden ---
331+
if p.num_pips > 0 {
332+
let mv_underlay_base = overlay_pad_idx + 1;
333+
for i in 0..p.num_inputs {
334+
let pad_name = format!("sink_{}", mv_underlay_base + i);
335+
underlay_initial_props(
336+
mv_pads.entry(pad_name).or_default(),
337+
vision_mixer::MV_PVW_PIP_OVERLAY_ZORDER,
338+
);
339+
}
340+
for pip_idx in 0..p.num_pips {
341+
for i in 0..p.num_inputs {
342+
let pad_name = format!(
343+
"sink_{}",
344+
mv_underlay_base + p.num_inputs * (1 + pip_idx) + i
345+
);
346+
underlay_initial_props(
347+
mv_pads.entry(pad_name).or_default(),
348+
vision_mixer::MV_PIP_OVERLAY_ZORDER,
349+
);
350+
}
351+
}
352+
}
353+
299354
pad_props
300355
}

backend/src/blocks/builtin/vision_mixer/builder/pipeline_cpu.rs

Lines changed: 89 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -234,6 +234,56 @@ pub(super) fn build_cpu_pipeline(
234234
};
235235
// Link to mv_comp is added AFTER all other mv_comp links (pad ordering matters)
236236

237+
// --- Border underlay sources ---
238+
// Zone borders render as solid-color compositor pads directly beneath
239+
// their content pads (see `gst::underlay`). One tiny videotestsrc per
240+
// (region, input); the border color is set at runtime via
241+
// `foreground-color`. Non-live → contributes no latency. Only built when
242+
// PiPs are configured — zones (and thus borders) cannot exist without
243+
// them.
244+
if p.num_pips > 0 {
245+
let underlay_chains: Vec<String> = (0..p.num_inputs)
246+
.map(|i| format!("underlay_dist_{}", i))
247+
.chain((0..p.num_inputs).map(|i| format!("underlay_pvw_{}", i)))
248+
.chain((0..p.num_pips).flat_map(|pip| {
249+
(0..p.num_inputs)
250+
.map(move |i| format!("underlay_pip_{}_{}", pip, i))
251+
.collect::<Vec<_>>()
252+
}))
253+
.collect();
254+
// Always RGBA — the compositor's convert pads handle per-pad format
255+
// conversion, and an alpha-less forced output_format (I420/NV12)
256+
// would silently drop the alpha of #RRGGBBAA border colors. Low
257+
// framerate: the color only changes on border edits and the
258+
// compositor keeps compositing the latest buffer between pushes.
259+
let underlay_caps: gst::Caps =
260+
"video/x-raw,format=RGBA,width=16,height=16,framerate=5/1,pixel-aspect-ratio=1/1"
261+
.parse()
262+
.map_err(|e| BlockBuildError::ElementCreation(format!("underlay caps: {}", e)))?;
263+
for name in &underlay_chains {
264+
let src_id = p.id(&format!("{}_src", name));
265+
let cf_id = p.id(&format!("{}_caps", name));
266+
let src = gst::ElementFactory::make("videotestsrc")
267+
.name(&src_id)
268+
.property("is-live", false)
269+
.build()
270+
.map_err(|e| BlockBuildError::ElementCreation(format!("{}: {}", src_id, e)))?;
271+
src.set_property_from_str("pattern", "solid-color");
272+
let cf = gst::ElementFactory::make("capsfilter")
273+
.name(&cf_id)
274+
.property("caps", &underlay_caps)
275+
.build()
276+
.map_err(|e| BlockBuildError::ElementCreation(format!("{}: {}", cf_id, e)))?;
277+
elems.push((src_id.clone(), src));
278+
elems.push((cf_id.clone(), cf));
279+
links.push((
280+
ElementPadRef::pad(&src_id, "src"),
281+
ElementPadRef::pad(&cf_id, "sink"),
282+
));
283+
}
284+
}
285+
// Links to the compositor pads are added below in pad-index order.
286+
237287
// --- Per-input elements ---
238288
for i in 0..p.num_inputs {
239289
let q_id = p.id(&format!("queue_{}", i));
@@ -358,6 +408,21 @@ pub(super) fn build_cpu_pipeline(
358408
));
359409
}
360410

411+
// Dist border underlays — after the DSK links so input i's underlay
412+
// lands at sink_{num_inputs + num_dsk_inputs + i}.
413+
if p.num_pips > 0 {
414+
for i in 0..p.num_inputs {
415+
let cf_id = p.id(&format!("underlay_dist_{}_caps", i));
416+
links.push((
417+
ElementPadRef::pad(&cf_id, "src"),
418+
ElementPadRef::pad(
419+
&mixer_id,
420+
format!("sink_{}", p.num_inputs + p.num_dsk_inputs + i),
421+
),
422+
));
423+
}
424+
}
425+
361426
// Multiview compositor thumbnails: tee_i.src_1 → queue → mv_comp
362427
for i in 0..p.num_inputs {
363428
let tee_id = p.id(&format!("tee_{}", i));
@@ -416,13 +481,36 @@ pub(super) fn build_cpu_pipeline(
416481
}
417482
}
418483

419-
// Overlay pad: last overlay element → mv_comp (must be last link for correct pad index)
484+
// Overlay pad: last overlay element → mv_comp.
420485
let overlay_pad_idx = 2 * p.num_inputs + 1 + p.num_pips * p.num_inputs;
421486
links.push((
422487
ElementPadRef::pad(&overlay_last_id, "src"),
423488
ElementPadRef::pad(&mv_comp_id, format!("sink_{}", overlay_pad_idx)),
424489
));
425490

491+
// Multiview border underlays — after the overlay pad: PVW underlays at
492+
// sink_{2N+2+P+i}, then tile underlays at sink_{2N+2+P+N+pip*N+i}.
493+
if p.num_pips > 0 {
494+
let mv_underlay_base = overlay_pad_idx + 1;
495+
for i in 0..p.num_inputs {
496+
let cf_id = p.id(&format!("underlay_pvw_{}_caps", i));
497+
links.push((
498+
ElementPadRef::pad(&cf_id, "src"),
499+
ElementPadRef::pad(&mv_comp_id, format!("sink_{}", mv_underlay_base + i)),
500+
));
501+
}
502+
for pip_idx in 0..p.num_pips {
503+
for i in 0..p.num_inputs {
504+
let cf_id = p.id(&format!("underlay_pip_{}_{}_caps", pip_idx, i));
505+
let sink_idx = mv_underlay_base + p.num_inputs * (1 + pip_idx) + i;
506+
links.push((
507+
ElementPadRef::pad(&cf_id, "src"),
508+
ElementPadRef::pad(&mv_comp_id, format!("sink_{}", sink_idx)),
509+
));
510+
}
511+
}
512+
}
513+
426514
// Multiview PGM big display: tee_pgm.src_1 → queue_pgm_mv → capsfilter_pgm_mv → mv_comp.sink_N
427515
// (capsfilter breaks caps query cycle back to PGM compositor)
428516
links.push((

0 commit comments

Comments
 (0)