Client-side media: Add video transcoding to web-safe formats#79375
Open
adamsilverstein wants to merge 8 commits into
Open
Client-side media: Add video transcoding to web-safe formats#79375adamsilverstein wants to merge 8 commits into
adamsilverstein wants to merge 8 commits into
Conversation
Detect uploaded videos that are not web-safe (non-MP4/WebM container, non-web-safe codec, oversized, or over a bitrate budget) and transcode them to MP4/H.264 (or WebM/VP9) in the browser before they are served, mirroring the GIF-to-video flow. The original upload is preserved as the attachment (consistent with how WordPress keeps the `-scaled` and HEIC originals); the transcoded, web-safe version is sideloaded as a companion file and is what plays. A `videoKeepOriginal` setting flips the pipeline to replace-primary so only the optimized file is stored. Transcoding runs off the main thread in the existing @wordpress/video-conversion worker via mediabunny's Conversion API, with hardware acceleration where available and graceful fallback to the original when the browser cannot encode. Video encoding shares the single-concurrency WebCodecs gate with GIF conversion. Part of #76756. Implements #79363.
Add server-side support for the `optimized-video` sideload size: the transcoded, web-safe video is stored as a companion file recorded in attachment metadata under `optimized_video` (surfaced to the editor via media_details), the original stays the attachment, and the companion is cleaned up on delete_attachment. Add the `gutenberg_video_transcoding_keep_original` filter (default true). When a developer returns false, the resolved value is exposed as window.__videoTranscodingKeepOriginal and read by the block editor's media upload settings, flipping the pipeline to transcode-before-upload so only the optimized file is stored. Part of #76756. Implements #79363.
When a video with an `optimized_video` companion is uploaded or selected, the Video block now plays the web-safe transcoded version (its `src` points at the companion) while the attachment keeps pointing at the original. A new toolbar control lets the author toggle between the optimized companion and the original upload. Part of #76756. Implements #79363.
Cover the worker transcodeVideo/getVideoMetadata functions (success, codec selection, downscaling, and the Unsupported graceful-failure paths), the prepareItem decision logic (companion vs replace-primary vs skip-already-optimized), the transcodeVideoItem handler's two graceful fallbacks (silent companion cancel vs upload-original), the generateVideoCompanion sideload, and the shared video-processing concurrency selectors. Part of #76756. Implements #79363.
PHP: cover the optimized_video companion path resolution (including directory-traversal hardening), delete_attachment cleanup, and the gutenberg_video_transcoding_keep_original filter default + override. e2e: upload a browser-generated WebM into a Video block, assert the block plays the transcoded MP4 companion while the original WebM stays the attachment (with optimized_video in its metadata), and that the toolbar toggles back to the original. The fixture is generated at runtime via MediaRecorder, so no binary asset is committed. Part of #76756. Implements #79363.
github-actions
Bot
added
[Package] Block library
adamsilverstein
added
[Feature] Client Side Media
The inferred return type referenced VideoMetadata from the @wordpress/video-conversion worker package, which TypeScript flagged as non-portable (TS2883) during the project build. Annotate the wrapper with an explicit structural return type that mirrors the worker's VideoMetadata shape.
adamsilverstein
commented
Jun 20, 2026
| // See lib/media/animated-gif-to-video.php. | ||
| $metadata['animated_video_poster'] = $sub_size['file']; | ||
| } elseif ( 'optimized-video' === $image_size ) { | ||
| // Web-safe transcoded companion of an uploaded video. Stored |
Member
Author
There was a problem hiding this comment.
please use the /* */ convention for multiline comments
adamsilverstein
commented
Jun 20, 2026
| // dimension checks (the caller passes (0, 0) for this case so the | ||
| // positive-dimension assertion below would otherwise fire). | ||
| if ( 'animated-video' === $image_size ) { | ||
| // 'animated-video' and 'optimized-video' companion files are videos, |
Member
Author
There was a problem hiding this comment.
please use the /* */ convention for multiline comments
adamsilverstein
commented
Jun 20, 2026
| // upload as "corrupted or unsupported". Skip the read for this case; | ||
| // validate_image_dimensions() also short-circuits it below. | ||
| $size = 'animated-video' === $image_size ? array( 0, 0 ) : wp_getimagesize( $path ); | ||
| // 'animated-video' and 'optimized-video' companions are video files |
Member
Author
There was a problem hiding this comment.
please use the /* */ convention for multiline comments
adamsilverstein
commented
Jun 20, 2026
| // video block's poster and deleted with the video. | ||
| $sub_size_data['file'] = wp_basename( $path ); | ||
| } elseif ( 'optimized-video' === $image_size ) { | ||
| // Web-safe transcoded video companion. finalize_item() writes the |
Member
Author
There was a problem hiding this comment.
please use the /* */ convention for multiline comments
adamsilverstein
commented
Jun 20, 2026
| // render-time filtering is needed. | ||
| require_once __DIR__ . '/animated-gif-to-video.php'; | ||
|
|
||
| // Video transcoding: clean up the sideloaded web-safe companion when its |
Member
Author
There was a problem hiding this comment.
please use the /* */ convention for multiline comments
adamsilverstein
commented
Jun 20, 2026
| allowedMimeTypes: settings.allowedMimeTypes, | ||
| allImageSizes: settings.allImageSizes, | ||
| bigImageSizeThreshold: settings.bigImageSizeThreshold, | ||
| // Developer opt-out for keeping the original video upload, exposed |
Member
Author
There was a problem hiding this comment.
please use the /* */ convention for multiline comments
adamsilverstein
commented
Jun 20, 2026
| return; | ||
| } | ||
|
|
||
| // Prefer the web-safe transcoded companion when available: the |
Member
Author
There was a problem hiding this comment.
please use the /* */ convention for multiline comments
adamsilverstein
requested review from
ajitbohra,
ellatrix,
fabiankaegy and
spacedmonkey
as code owners
|
The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message. To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook. |
|
Size Change: +129 kB (+1.49%) Total Size: 8.81 MB 📦 View Changed
|
|
Flaky tests detected in 1cc51f2. 🔍 Workflow run URL: https://github.com/WordPress/gutenberg/actions/runs/27882031001
|
- Use the /* */ block convention for multi-line comments. - Delete the optimized_video companion via wp_delete_file_from_directory confined to the uploads directory, matching the source_image (HEIC) companion cleanup; trust only the basename of the recorded value. - Regenerate the @wordpress/video-conversion README API docs.
Add a small (6 KB) WebM clip under test/e2e/assets and upload it in the video transcoding e2e test, replacing the runtime MediaRecorder generation. With the default MP4 output target the WebM is a real non-web-safe input that exercises the actual transcode to an MP4 companion.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What?
Adds client-side video transcoding to web-safe formats for the client-side media feature, implementing #79363. When a video that is not already web-safe is uploaded, it is transcoded in the browser (via mediabunny / WebCodecs) to MP4/H.264 (or WebM/VP9) before it is served, mirroring what we already do for images and the GIF-to-video conversion in #78410.
Note
This PR is stacked on #78410 (
add/gif-to-video-mediabunny) and reuses its@wordpress/video-conversionpackage, worker boundary, and companion-file infrastructure. It targets that branch and should be rebased ontotrunkonce #78410 lands. Review the last 6 commits.Why?
WordPress accepts video uploads in any format (
.mov,.mkv, high-bitrate.mp4) and never transcodes them, because server-side transcoding needs FFmpeg, which most hosts lack. The result is slow page loads, high bandwidth/storage cost, and clips that won't play in some browsers. This is one of the capabilities called out for the 7.1 cycle and was only a roadmap bullet on #76756.How?
Keep the original, serve a web-safe companion. WordPress always preserves the original upload (so it can be linked to or used to regenerate later, like the
-scaledand HEIC originals). So the original video is stored as the attachment and the transcoded, web-safe version is sideloaded as a companion file (recorded in attachment metadata underoptimized_video). Thecore/videoblock points its playbacksrcat the companion; a toolbar control lets authors switch back to the original.Pipeline. Detection and transcoding reuse the
@wordpress/upload-mediaqueue:prepareItemprobes the video metadata (getVideoMetadata, a cheap header read) andneedsVideoTranscodedecides eligibility (non-web-safe container/codec, oversized past the 1920px threshold, or over an optional bitrate budget) — already-optimized small files are skipped.Upload → GenerateVideoCompanion → Finalize; the companion is transcoded off the main thread via mediabunny's high-levelConversionAPI withhardwareAcceleration: 'prefer-hardware', then sideloaded.Developer opt-out. The
gutenberg_video_transcoding_keep_originalPHP filter (defaulttrue) flips the pipeline to transcode-before-upload, storing only the optimized file. Since video files can be very large, this affords the ability to reduce storage requirements with the tradeoff that the original video is no longer available for download or regeneration.Testing Instructions
.mov, an.mkv, or a large/high-bitrate.mp4)..mp4once the upload finishes, while the Media Library still holds the original upload..mp4and confirm it is left untouched.add_filter( 'gutenberg_video_transcoding_keep_original', '__return_false' );and confirm only the optimized file is stored.Automated tests
npm run test:unit packages/upload-media packages/video-conversionvendor/bin/phpunit phpunit/media/video-transcoding-test.phpnpm run test:e2e -- test/e2e/specs/editor/various/video-transcoding.spec.jsPart of #76756.
AI use
This PR was written by Claude after careful prompting, planning and review from me. I also plan to manually review the code and test the feature directly.