feat: Add encoding hints and tuning modes by DatCaptainHorse · Pull Request #20 · hgaiser/pixelforge

DatCaptainHorse · 2026-06-11T21:38:16Z

PR changes

Adds new methods and wiring to set encode usage and content hints alongside encoder tuning mode.

Comment

Starting as draft, as I've yet to add tests and methods for getting the encode latency for easy check and comparison.

Also for some drivers and devices, it may not make much difference at all. From what I remember NVIDIA might get most out of this, while AMD is hard-tuned for speed always.. though I'll have to add the tests and methods to compare for sure.

DatCaptainHorse · 2026-06-14T09:02:32Z

Update: Confirmed that atleast with current mesa-git and 9060 XT, AMD GPUs don't care about Vulkan Video encode tuning modes.

NVIDIA shows interesting behaviour though:

~/DEV/pixelforge-horse feat/encoder-tunes-hints* nestri@nestripc-1
❯ VK_DRIVER_FILES=/usr/share/vulkan/icd.d/nvidia_icd.json cargo run --example encode_bench
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.02s
     Running `target/debug/examples/encode_bench`
PixelForge Encode Bench Example

Input: 300 frames, 1920x1080 YUV420, 933120000 bytes
Config: Cbr, bitrate=8000000, GOP=120, B-frames=0, tuning=HighQuality


Encoded 300 frames, 228760 bytes, 4079.0:1 compression
GPU Encode timings: Min: 1.877728ms, Max: 2.209824ms, Avg: 2.134621ms, P99: 2.144512ms
CPU Encode timings: Min: 2.019822ms, Max: 3.241982ms, Avg: 2.303739ms, P99: 2.715553ms

~/DEV/pixelforge-horse feat/encoder-tunes-hints* nestri@nestripc-1
❯ VK_DRIVER_FILES=/usr/share/vulkan/icd.d/nvidia_icd.json cargo run --example encode_bench
   Compiling pixelforge v0.5.0 (/home/nestri/DEV/pixelforge-horse)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.18s
     Running `target/debug/examples/encode_bench`
PixelForge Encode Bench Example

Input: 300 frames, 1920x1080 YUV420, 933120000 bytes
Config: Cbr, bitrate=8000000, GOP=120, B-frames=0, tuning=UltraLowLatency


Encoded 300 frames, 225483 bytes, 4138.3:1 compression
GPU Encode timings: Min: 3.548224ms, Max: 3.713472ms, Avg: 3.667244ms, P99: 3.696704ms
CPU Encode timings: Min: 3.709208ms, Max: 4.859037ms, Avg: 3.857445ms, P99: 4.139455ms

It seems NVIDIA, instead of actually lowering the latency, instead opts to minimize jitter 🤔
Will need to test with other codecs, resolutions and all as well.

hgaiser · 2026-06-14T09:42:44Z

Thanks for the update! Interesting results too 🤨

Also, that bench looks useful if you want to submit that as a PR :)

I'm curious, are you using pixelforge for something directly, or through moonshine?

DatCaptainHorse · 2026-06-14T11:57:26Z

Thanks for the update! Interesting results too 🤨

Also, that bench looks useful if you want to submit that as a PR :)

I'm curious, are you using pixelforge for something directly, or through moonshine?

I'll make the bench bit more well-rounded and add it 👍

Using pixelforge with a piece of Nestri project called hudless (looks like it's still private, I'll have to see when Wanjohi deems it "ready" to make public and open-source). It's basically an implicit Vulkan layer that grabs frames off a Vulkan application (usually a game) directly (akin to obs-vkcapture), encodes it using pixelforge and sends over network eventually.

urwrstkn8mare · 2026-06-16T02:57:30Z

encodes it using pixelforge and sends over network eventually.

How is it different from Moonshine (other than the Moonlight protocol)?

DatCaptainHorse · 2026-06-16T10:22:16Z

How is it different from Moonshine (other than the Moonlight protocol)?

Using QUIC streams, works over internet with holepunching, also WebTransport support for browser clients.
No RTP payloading and audio-video syncing required.. coming from WebRTC mainly, switching media delivery over to QUIC is actually much nicer and simpler and gives lower latency, can recommend 🙂

But yeah if you want to ask more, just DM me over Discord, my username there is dathorse.

urwrstkn8mare · 2026-06-16T10:53:36Z

Yeah was thinking of a Moonlight alternative without the baggage of Gamestream. I was planning on raising the idea of further modularising Moonshine so you could theoretically swap out the protocol but haven't really fleshed out that idea haha. I was thinking it could be useful for other usecases as well like how Microsoft has that streamed app enterprise feature I see used alot (and works surprisingly well).

hgaiser · 2026-06-16T11:17:55Z

Yeah was thinking of a Moonlight alternative without the baggage of Gamestream. I was planning on raising the idea of further modularising Moonshine so you could theoretically swap out the protocol but haven't really fleshed out that idea haha. I was thinking it could be useful for other usecases as well like how Microsoft has that streamed app enterprise feature I see used alot (and works surprisingly well).

I've been thinking about it too, though the large support of the moonlight clients is mainly holding me back :). I also worry about maintainability of multiple protocols being supported in Moonshine. To me an important point was to have a (relatively) simple pipeline. I'd worry we lose that if it needs to be modularized or if multiple protocols need to be supported.

No RTP payloading and audio-video syncing required.. coming from WebRTC mainly, switching media delivery over to QUIC is actually much nicer and simpler and gives lower latency, can recommend 🙂

AFAIK RTP payloading was not expensive at all. Not 100% sure about the FEC part, which I assume you included in RTP.

But I do agree. There's a lot of legacy from the GameStream era. Cool project though!

urwrstkn8mare · 2026-06-16T13:04:10Z

I also worry about maintainability of multiple protocols being supported in Moonshine.

Oh yeah definitely. I guess I was envisioning it kinda like how libghostty from Ghostty allows other applications with a different feature set use the same core.

Interested in trying out Nestri to see how much of an effect a different network architecture could make.

DatCaptainHorse · 2026-06-16T14:11:29Z

Alright, rebased against #21 - AMD Radeon still cares not, but NVIDIA is showing similar behaviour for "ultra-low-latency" tune 🤔

❯ VK_DRIVER_FILES=/usr/share/vulkan/icd.d/nvidia_icd.json cargo run --example encode_bench
   Compiling pixelforge v0.6.0 (/home/nestri/DEV/pixelforge-horse)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.38s
     Running `target/debug/examples/encode_bench`
PixelForge Encode Bench Example

Input: 300 frames, 1920x1080 YUV420, 933120000 bytes

=== Testing H264 with Default tuning ===
Config: Cbr, bitrate=8000000, GOP=120, B-frames=0
Encoded 300 frames, 228760 bytes, 4079.0:1 compression
GPU Encode timings: Min: 1.865952ms, Max: 3.618816ms, Avg: 2.140005ms, P99: 2.5928ms
CPU Encode timings: Min: 2.014046ms, Max: 3.725567ms, Avg: 2.242582ms, P99: 2.693645ms

=== Testing H264 with HighQuality tuning ===
Config: Cbr, bitrate=8000000, GOP=120, B-frames=0
Encoded 300 frames, 228760 bytes, 4079.0:1 compression
GPU Encode timings: Min: 1.871232ms, Max: 3.594944ms, Avg: 2.133401ms, P99: 2.152928ms
CPU Encode timings: Min: 2.044396ms, Max: 3.699528ms, Avg: 2.246677ms, P99: 2.651694ms

=== Testing H264 with LowLatency tuning ===
Config: Cbr, bitrate=8000000, GOP=120, B-frames=0
Encoded 300 frames, 228760 bytes, 4079.0:1 compression
GPU Encode timings: Min: 1.911296ms, Max: 5.353664ms, Avg: 3.261652ms, P99: 3.92256ms
CPU Encode timings: Min: 2.026946ms, Max: 5.855355ms, Avg: 3.556617ms, P99: 4.541579ms

=== Testing H264 with UltraLowLatency tuning ===
Config: Cbr, bitrate=8000000, GOP=120, B-frames=0
Encoded 300 frames, 225483 bytes, 4138.3:1 compression
GPU Encode timings: Min: 5.892992ms, Max: 7.8136ms, Avg: 6.334878ms, P99: 6.893664ms
CPU Encode timings: Min: 6.111999ms, Max: 7.958852ms, Avg: 6.577248ms, P99: 7.300783ms

=== Testing H264 with Lossless tuning ===
Config: Cqp, bitrate=8000000, GOP=120, B-frames=0
Encoded 300 frames, 308215 bytes, 3027.5:1 compression
GPU Encode timings: Min: 2.966688ms, Max: 5.123488ms, Avg: 3.659541ms, P99: 4.2464ms
CPU Encode timings: Min: 3.408764ms, Max: 5.589132ms, Avg: 4.066712ms, P99: 4.843432ms

=== Testing H265 with Default tuning ===
Config: Cbr, bitrate=8000000, GOP=120, B-frames=0
Encoded 300 frames, 1609684 bytes, 579.7:1 compression
GPU Encode timings: Min: 2.805536ms, Max: 5.065792ms, Avg: 3.357263ms, P99: 4.636224ms
CPU Encode timings: Min: 2.934148ms, Max: 5.46848ms, Avg: 3.673968ms, P99: 4.863272ms

=== Testing H265 with HighQuality tuning ===
Config: Cbr, bitrate=8000000, GOP=120, B-frames=0
Encoded 300 frames, 1609684 bytes, 579.7:1 compression
GPU Encode timings: Min: 2.859328ms, Max: 5.581184ms, Avg: 3.351717ms, P99: 4.631072ms
CPU Encode timings: Min: 3.011088ms, Max: 6.21446ms, Avg: 3.733699ms, P99: 4.820702ms

=== Testing H265 with LowLatency tuning ===
Config: Cbr, bitrate=8000000, GOP=120, B-frames=0
Encoded 300 frames, 1609684 bytes, 579.7:1 compression
GPU Encode timings: Min: 2.981376ms, Max: 5.115904ms, Avg: 3.456038ms, P99: 5.001248ms
CPU Encode timings: Min: 3.150671ms, Max: 5.344758ms, Avg: 3.773944ms, P99: 5.252457ms

=== Testing H265 with UltraLowLatency tuning ===
Config: Cbr, bitrate=8000000, GOP=120, B-frames=0
Encoded 300 frames, 1420374 bytes, 657.0:1 compression
GPU Encode timings: Min: 5.516512ms, Max: 9.29776ms, Avg: 5.912072ms, P99: 8.795552ms
CPU Encode timings: Min: 5.772724ms, Max: 9.520362ms, Avg: 6.283903ms, P99: 9.101558ms

=== Testing H265 with Lossless tuning ===
Config: Cqp, bitrate=8000000, GOP=120, B-frames=0
Encoded 300 frames, 127653 bytes, 7309.8:1 compression
GPU Encode timings: Min: 2.9104ms, Max: 5.309408ms, Avg: 3.415358ms, P99: 4.801984ms
CPU Encode timings: Min: 3.047359ms, Max: 5.892046ms, Avg: 3.657925ms, P99: 4.981884ms

=== AV1 not supported, skipping ===

I'm starting to wonder if this is a driver bug.. using latest linux-cachyos-nvidia-open available as of today (610 something driver).

urwrstkn8mare · 2026-06-19T16:44:36Z

was able to reproduce the same results you got and yeah its really odd. The best way I can reason about this is maybe these mode changes aren't meant to change encode latency but instead network latency by reducing the resultant bitrate? A pretty far fetched theory but other than driver issue can't really explain it.

DatCaptainHorse · 2026-06-26T12:46:35Z

Well, marking as ready since the API works, but I'm stumped at the NVIDIA results still.. I added submission (wall time) stat and it's also showing that "LowLatency" and "UltraLowLatency" tunes increase the latency?

I wonder if we just aren't feeding frames fast enough so it ends up stalling the encoder, or the latency comes from elsewhere? Or both..

But yeah the options have an effect on NVIDIA atleast @hgaiser - as to if it's beneficial with current state, I am unable to say for certain due to results I get 😅

hgaiser · 2026-06-26T14:38:11Z

Well, marking as ready since the API works, but I'm stumped at the NVIDIA results still.. I added submission (wall time) stat and it's also showing that "LowLatency" and "UltraLowLatency" tunes increase the latency?

I wonder if we just aren't feeding frames fast enough so it ends up stalling the encoder, or the latency comes from elsewhere? Or both..

But yeah the options have an effect on NVIDIA atleast @hgaiser - as to if it's beneficial with current state, I am unable to say for certain due to results I get 😅

I'm wondering if the main difference between those modes is how it will setup B frames, which we already disabled anyway.

Also I think (ultra) low latency might optimize frame sizes, at the cost of encode times (which is what you observed). Look at these frame size distributions:

A more constant frame size should, in theory, make it quicker to transfer over the network (less buffer bloat / queue times). Perhaps these modes make the trade-off of having a more strict / complex encoding (and therefore wall-time) versus the size of each frame.

hgaiser

Some minor questions, but looks good to me overall!

hgaiser · 2026-06-26T22:42:14Z

+    command_buffer: vk::CommandBuffer,
+    query_pool: vk::QueryPool,
+) {
+    unsafe {


Is this unsafe not redundant? The function is already marked unsafe.

I thought so, but it gave a warning if I didn't have it.. could be 2021 edition limitation if I remember right, which was fixed in 2024 Rust edition?

I don't mind if you set edition to 2024, it's an 'artifact' from using an LLM 🤷

In general I'm not against using unsafe in unsafe-marked functions when they're like:

unsafe fn foo() { something_safe(); unsafe { something_unsafe(); } }

But in these cases it feels redundant :)

If it still gives a warning with edition set to 2024 then let's ignore it; it's not worth spending more time on it.

DatCaptainHorse · 2026-07-07T11:00:37Z

@hgaiser Was there anything you wanted to add in the comments?

hgaiser · 2026-07-14T07:28:03Z

@hgaiser Was there anything you wanted to add in the comments?

Apologies, I've been on vacation. I just got back though so I'll pick this up this week 👍

hgaiser

Just one minor remark regarding the unsafe functions, I think we're good to merge after that.

Thanks again. Sorry it took a while to work through this, I've been a bit busy. I do really appreciate the input!

hgaiser · 2026-07-17T19:22:48Z

    }
 }

+/// Resets and writes first timestamp command


When is this call not safe? Please document what the caller should ensure to make this call correct. (ie. a # Safety section)

hgaiser · 2026-07-17T19:23:02Z

+    }
+}
+
+pub(crate) unsafe fn end_timestamp(


hgaiser · 2026-07-17T19:24:12Z

+}
+
+/// Queries and possibly returns the difference between reset_start_timestamp and end_timestamp calls
+pub(crate) unsafe fn query_timestamp_diff(


And here :)

hgaiser · 2026-07-17T19:29:15Z

+    command_buffer: vk::CommandBuffer,
+    query_pool: vk::QueryPool,
+) {
+    unsafe {


I don't mind if you set edition to 2024, it's an 'artifact' from using an LLM 🤷

In general I'm not against using unsafe in unsafe-marked functions when they're like:

unsafe fn foo() { something_safe(); unsafe { something_unsafe(); } }

But in these cases it feels redundant :)

If it still gives a warning with edition set to 2024 then let's ignore it; it's not worth spending more time on it.

DatCaptainHorse force-pushed the feat/encoder-tunes-hints branch from 3bfc7f9 to bda89d6 Compare June 12, 2026 15:29

DatCaptainHorse force-pushed the feat/encoder-tunes-hints branch from 2a252eb to 58a3e11 Compare June 16, 2026 14:04

DatCaptainHorse mentioned this pull request Jun 16, 2026

Suggestion: List of possible improvements / work #19

Open

8 tasks

DatCaptainHorse force-pushed the feat/encoder-tunes-hints branch from 677d45f to 7e53ab3 Compare June 19, 2026 10:27

DatCaptainHorse added 6 commits June 26, 2026 15:00

feat: Add encode hints and tuning modes

1076212

feat: Add encode timing stats

c526dc8

feat: Add bench example and fix encode usage for h26x codecs.

2df6266

fix: Update bench for async

7c00f07

fix: add missing timestamp query pool destroy

45ac1c6

fix: Make clippy happy

aa86e36

DatCaptainHorse force-pushed the feat/encoder-tunes-hints branch from 7e53ab3 to aa86e36 Compare June 26, 2026 12:02

DatCaptainHorse added 3 commits June 26, 2026 15:13

fix: bench example and warnings

00c2514

feat: add submission wall latency timing

b8a25d5

fix: don't use flushed frames for benching

ad73994

DatCaptainHorse marked this pull request as ready for review June 26, 2026 12:43

hgaiser reviewed Jun 26, 2026

View reviewed changes

fix: Resolve comments and issues

7908395

DatCaptainHorse force-pushed the feat/encoder-tunes-hints branch from 6dac01a to 7908395 Compare June 29, 2026 15:08

fix: Update README

42755a5

hgaiser reviewed Jul 17, 2026

View reviewed changes

Conversation

DatCaptainHorse commented Jun 11, 2026

PR changes

Comment

Uh oh!

DatCaptainHorse commented Jun 14, 2026

Uh oh!

hgaiser commented Jun 14, 2026

Uh oh!

DatCaptainHorse commented Jun 14, 2026

Uh oh!

urwrstkn8mare commented Jun 16, 2026

Uh oh!

DatCaptainHorse commented Jun 16, 2026

Uh oh!

urwrstkn8mare commented Jun 16, 2026

Uh oh!

hgaiser commented Jun 16, 2026

Uh oh!

urwrstkn8mare commented Jun 16, 2026

Uh oh!

DatCaptainHorse commented Jun 16, 2026

Uh oh!

urwrstkn8mare commented Jun 19, 2026

Uh oh!

DatCaptainHorse commented Jun 26, 2026

Uh oh!

hgaiser commented Jun 26, 2026

Uh oh!

hgaiser left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

hgaiser Jun 26, 2026

Choose a reason for hiding this comment

Uh oh!

DatCaptainHorse Jun 29, 2026

Choose a reason for hiding this comment

Uh oh!

hgaiser Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

DatCaptainHorse commented Jul 7, 2026

Uh oh!

hgaiser commented Jul 14, 2026

Uh oh!

hgaiser left a comment

Choose a reason for hiding this comment

Uh oh!

hgaiser Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

hgaiser Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

hgaiser Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

hgaiser Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants