Headless Window + pipelined (non-blocking) pixel readback#398
Open
haixuanTao wants to merge 2 commits into
Open
Headless Window + pipelined (non-blocking) pixel readback#398haixuanTao wants to merge 2 commits into
haixuanTao wants to merge 2 commits into
Conversation
…ing buffer read_pixels indexed the mapped staging buffer byte-by-byte during BGRA->RGB conversion. Mapped readback memory is uncached (write-combined), so scalar reads run at ~10 MB/s — 99 ms for a 640x480 frame on an RTX 5080, dwarfing the 4 ms render. memcpy each row into a cached local buffer first and convert from there: 5.4 ms (18x). Also: reuse the staging buffer across calls (grown on demand) instead of allocating per call, and wait on the copy's submission index instead of polling the device indefinitely. Per-frame capture loop (sim + render + snap, 640x480): 113 ms -> 12 ms. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two additions for off-screen video capture: - Window::new_headless_with_setup(width, height, setup): a full-featured Window backed by no OS window and no swapchain (same headless canvas as OffscreenSurface, but exposing the whole Window API — custom renderers, ray tracer, snap* readbacks). Rendering is never throttled by the display's vsync and works without a display server. - snap_begin()/snap_finish() (begin_read_pixels/finish_read_pixels on the canvas): pipelined readback that enqueues the texture->buffer copy and buffer map without waiting, so the copy of frame N can be collected after frame N+1 renders instead of stalling the GPU pipeline every frame the way the blocking snap()/read_pixels() does. read_pixels() is refactored to begin+finish with unchanged behavior. Motivation: a windowed capture loop (render_frame + blocking snap) is vsync-locked — the blocking readback defeats swapchain frame pipelining, so every iteration eats ~2 vblanks (~30 fps at 60 Hz) regardless of GPU load. Headless + pipelined readback took a downstream nexus3d capture loop from 33 to 92 fps at 640x480. Builds on feat/persistent-readback-staging (dimforge#397).
haixuanTao
added a commit
to haixuanTao/nexus
that referenced
this pull request
Jul 8, 2026
…ewer, pipelined readback Makes the nexus3d Python bindings usable for off-screen video capture: - NexusViewer(width, height): configurable resolution; set_draw_ui(False) keeps the egui panel out of captured frames. - NexusViewer.render() -> (H, W, 3) uint8 numpy frame export, like mujoco.Renderer.render(). - raytrace_frame() + set_raytracer_samples_per_frame/max_bounces/denoise + raytracer_backend(): kiss3d's GPU path tracer from Python, with progressive sample accumulation. sync takes the CPU-readback path while the tracer is active (the zero-readback kernel only updates the rasterizer's instance buffers, which the tracer's BVH never reads). - NexusViewer(..., headless=True): no OS window, no swapchain — frames render into an off-screen texture, never throttled by display vsync, works without a display server. - set_vsync(enabled)/vsync(): uncap a windowed viewer instead. - render_async()/render_flush(): pipelined capture — returns the previous frame while the current frame's GPU->CPU copy runs in the background. A windowed capture loop was vsync-locked at ~30 fps (blocking readback + Fifo swapchain = ~2 vblanks per iteration); headless + pipelined readback reaches 92 gen-fps at 640x480 (GPU physics) on an RTX 5080. Requires the kiss3d headless-capture branch (dimforge/kiss3d#398). Supersedes dimforge#7 and dimforge#8 (included here).
haixuanTao
force-pushed
the
feat/headless-capture-pipeline
branch
from
4440e6b to
7003b7d
Compare
haixuanTao
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #397 — merge #397 first; this PR then reduces to its single commit (
feat: public headless Window + pipelined (non-blocking) pixel readback). Until then the diff below includes #397's read_pixels changes.Two additions for off-screen video capture:
Window::new_headless_with_setup(width, height, setup): a full-featuredWindowbacked by no OS window and no swapchain — the same headless canvas asOffscreenSurface, but exposing the wholeWindowAPI (custom renderers, ray tracer,snap*readbacks). Never throttled by display vsync; works without a display server.snap_begin()/snap_finish(): pipelined readback that enqueues the texture→buffer copy and buffer map without waiting, so frame N's pixels are collected after frame N+1 renders instead of stalling the GPU pipeline every frame.read_pixels()is refactored onto the same begin/finish path with unchanged behavior.Motivation: a windowed capture loop (
render_frame+ blockingsnap) is vsync-locked — the swapchain defaults toAutoVsync(Fifo) and the blocking readback defeats frame pipelining, so every iteration eats ~2 vblanks (~30 fps at 60 Hz) regardless of GPU load. Headless + pipelined readback took a downstream nexus3d capture loop from 33 to 92 gen-fps at 640×480.🤖 Generated with Claude Code