feat(flame_3d): Add experimental web support using webgpu backend (#3930)

Adding experimental web support using `webgpu`, it serves as a showcase
for the backend API working as intended and as a way to bring web into
the mix.

Author: wolfenrain

.github/.cspell/gamedev_dictionary.txt

@@ -1,6 +1,7 @@
 # general development-adjacent terms and expressions
 AABB # axis aligned bounding box
 abelian # Abelian Group, also known as commutative group
+alignof # alignment of
 ARGB # alpha red green blue
 arities # plural of arity
 backgrounded # moving the app to the background
@@ -58,7 +59,12 @@ tileset # image with a collection of tiles. in games, tiles are small square spr
 tilesets # plural of tileset
 truecolor # truecolor rendering
 tweening # the process of tween
+uncapturederror # dumb webgpu javascript name
+unorm # unsigned normalized integer
 viewports # plural of viewport
 WASD # movement keys on a keyboard
 WBMP # wireless bitmap image format
+webgpu # gpu standard for web browsers 
 WebP # WebP image format
+WGSL # WGSL shader language
+wgslbundle # WGSL shader bundle format

packages/flame_3d/README.md

@@ -17,14 +17,16 @@ Adds 3D support for <a href="https://github.com/flame-engine/flame">Flame</a> us
 </p>
 
 ---
+
 <!-- markdownlint-enable MD013 -->
 
 <!-- markdownlint-disable-next-line MD002 -->
+
 # flame_3d
 
-This package provides an experimental implementation of 3D support for Flame. The main focus is to
-explore the potential capabilities of 3D for Flame while providing a familiar API to existing Flame
-developers.
+This package provides an experimental implementation of 3D support for Flame.
+The main focus is to explore the potential capabilities of 3D for Flame while
+providing a familiar API to existing Flame developers.
 
 Supported platforms:
 
@@ -35,41 +37,48 @@ Supported platforms:
 | macOS    | ✅        |
 | Windows  | ❌        |
 | Linux    | ❌        |
-| Web      | ❌        |
+| Web      | ⚠️¹       |
+
+⚠️¹ Web support is experimental, see [Web support](#web-support-experimental)
+below.
 
 
 ## Prologue
 
-**STOP**, we know you are hyped up and want to start coding some funky 3D stuff but we first have to
-set your expectations and clarify some things. So turn down your music, put away the coffee and make
-some tea instead because you have to do some reading first!
+**STOP**, we know you are hyped up and want to start coding some funky 3D stuff
+but we first have to set your expectations and clarify some things. So turn down
+your music, put away the coffee and make some tea instead because you have to do
+some reading first!
 
-This package provides 3D support for Flame but it depends on the still experimental
-[Flutter GPU](https://github.com/flutter/flutter/wiki/Flutter-GPU), which in turn depends on
-Impeller.
+This package provides 3D support for Flame but it depends on the still
+experimental [Flutter GPU](https://github.com/flutter/flutter/wiki/Flutter-GPU),
+which in turn depends on Impeller.
 
 Therefore, this package is also experimental; you can check our
 [Roadmap](https://github.com/flame-engine/flame/blob/main/packages/flame_3d/ROADMAP.md)
 for more details on our plans and what is currently supported.
 
-This package does not guarantee that it will follow correct [semver](https://semver.org/) versioning
-rules, nor does it assure that its APIs wont break. Be ready to constantly have to refactor your
-code if you are planning on using this package, and potentially to have to contribute with
+This package does not guarantee that it will follow correct
+[semver](https://semver.org/) versioning rules, nor does it assure that its APIs
+wont break. Be ready to constantly have to refactor your code if you are
+planning on using this package, and potentially to have to contribute with
 improvements and fixes. Please do not use this for production environments.
 
-Documentation and tests might be lacking for quite a while because of the potential constant changes
-of the API. Where possible, we will try to provide in-code documentation and code examples to help
-developers but our main goal for now is to enable the usage of 3D rendering within a Flame
-ecosystem.
+Documentation and tests might be lacking for quite a while because of the
+potential constant changes of the API. Where possible, we will try to provide
+in-code documentation and code examples to help developers but our main goal for
+now is to enable the usage of 3D rendering within a Flame ecosystem.
 
 
 ## Prerequisites
 
-In order to use flame_3d, you will need to ensure a few things. Firstly, the only platforms that we
-have explicitly tested so far for support were Android, iOS, and macOS.
+In order to use flame_3d, you will need to ensure a few things. Firstly, the
+only platforms that we have explicitly tested so far for support were Android,
+iOS, and macOS.
 
-Then, you need to enable Impeller, if not already enabled by default. For example, for macOS, add
-the following to the generated `macos/runner/Info.plist` directory:
+Then, you need to enable Impeller, if not already enabled by default. For
+example, for macOS, add the following to the generated `macos/runner/Info.plist`
+directory:
 
 ```xml
 <dict>
@@ -87,43 +96,71 @@ Alternatively, you can run Flutter with this flag instead:
 flutter run --enable-flutter-gpu
 ```
 
-Now everything is set up you can start doing some 3D magic! You can check out the
-[example](https://github.com/flame-engine/flame/tree/main/packages/flame_3d/example) to see how you
-can set up a simple 3D environment using Flame.
+Now everything is set up you can start doing some 3D magic! You can check out
+the
+[example](https://github.com/flame-engine/flame/tree/main/packages/flame_3d/example)
+to see how you can set up a simple 3D environment using Flame.
+
+Also check our more advanced examples,
+[Collect the Donut](https://github.com/luanpotter/collect_the_donut) and
+[Defend the Donut](https://github.com/flame-engine/defend_the_donut).
+
 
-Also check our more advanced examples, [Collect the Donut](https://github.com/luanpotter/collect_the_donut)
-and [Defend the Donut](https://github.com/flame-engine/defend_the_donut).
+## Web support (experimental)
+
+Flame 3D also runs on the web, though this is **even more experimental** than
+the rest of the package. Flutter GPU does not run in the browser (for now), so
+on web `flame_3d` renders through the browser's native
+[WebGPU](https://developer.mozilla.org/en-US/docs/Web/API/WebGPU_API) API
+instead, via a separate rendering backend. If you want to run on web add the
+following before your `runApp` call in `main.dart`:
+
+```dart
+await GpuBackend.initialize();
+```
 
 
 ## Building shaders
 
-If you are using the `SpatialMaterial` provided by `flame_3d`, you do not need to worry about shaders.
+If you are using the materials provided by `flame_3d`, you do not need to worry
+about shaders.
 
-That being said, you can write your own shaders and use them on custom materials.
-Currently, Flutter does not do the bundling of shaders for us so this package provides a simple
-Dart script. Create your fragment and vertex shader in a `shaders` directory,
-make sure the file names are identical. Like so:
+That being said, you can write your own shaders and use them on custom
+materials. Currently, Flutter does not do the bundling of shaders for us so this
+package provides a simple Dart script. Create your fragment and vertex shader in
+a `shaders` directory, make sure the file names are identical. Like so:
 
 - `my_custom_shader`.frag
 - `my_custom_shader`.vert
 
-You can then run `dart pub run flame_3d:build_shaders` to bundle the shaders. They will
-automatically be placed in `assets/shaders`.
+You can then run `dart pub run flame_3d:build_shaders` to bundle the shaders.
+They will automatically be placed in `assets/shaders`.
+
+For [web support](#web-support-experimental), also pass `--with-web-gpu` to
+build the WebGPU shader bundles. That step needs the `naga` CLI on your `PATH`
+(`cargo install naga-cli`).
+
+Shaders can also reuse shared GLSL through `#include`. An
+`#include <package_name/file.glsl>` resolves against the `shaders/` directory of
+any package in your dependency graph, so chunks can be shared across packages.
+Including ones that `flame_3d` itself ships, such as
+`#include <flame_3d/skinning.glsl>` for vertex skinning.
 
 You can check out the
-[default shaders](https://github.com/flame-engine/flame/tree/main/packages/flame_3d/shaders) if you
-want to have some examples.
+[default shaders](https://github.com/flame-engine/flame/tree/main/packages/flame_3d/shaders)
+if you want to have some examples.
 
 
 ## Contributing
 
-Have you found a bug or have a suggestion of how to enhance the 3D APIs? Open an issue and we will
-take a look at it as soon as possible.
+Have you found a bug or have a suggestion of how to enhance the 3D APIs? Open an
+issue and we will take a look at it as soon as possible.
 
-Do you want to contribute with a PR? PRs are always welcome, just make sure to create it from the
-correct branch (main) and follow the [checklist](.github/pull_request_template.md) which will
-appear when you open the PR.
+Do you want to contribute with a PR? PRs are always welcome, just make sure to
+create it from the correct branch (main) and follow the
+[checklist](.github/pull_request_template.md) which will appear when you open
+the PR.
 
-For bigger changes, or if in doubt, make sure to talk about your contribution to the team. Either
-via an issue, GitHub discussion, or reach out to the team using the
-[Discord server](https://discord.gg/pxrBmy4).
+For bigger changes, or if in doubt, make sure to talk about your contribution to
+the team. Either via an issue, GitHub discussion, or reach out to the team using
+the [Discord server](https://discord.gg/pxrBmy4).

packages/flame_3d/assets/shaders/spatial_material.shaderbundle

@@ -0,0 +1,37 @@
+{
+  "vertex": "struct JointMatrices {\n    joints: array<mat4x4<f32>, 16>,\n}\n\nstruct VertexInfo {\n    model: mat4x4<f32>,\n    view: mat4x4<f32>,\n    projection: mat4x4<f32>,\n}\n\nstruct VertexOutput {\n    @location(0) fragTexCoord: vec2<f32>,\n    @location(1) fragColor: vec4<f32>,\n    @location(2) fragPosition: vec3<f32>,\n    @location(3) fragNormal: vec3<f32>,\n    @builtin(position) gl_Position: vec4<f32>,\n}\n\nvar<private> vertexPosition_1: vec3<f32>;\nvar<private> vertexTexCoord_1: vec2<f32>;\nvar<private> vertexColor_1: vec4<f32>;\nvar<private> vertexNormal_1: vec3<f32>;\nvar<private> vertexJoints_1: vec4<f32>;\nvar<private> vertexWeights_1: vec4<f32>;\n@group(0) @binding(0) \nvar<uniform> jointMatrices: JointMatrices;\nvar<private> fragTexCoord: vec2<f32>;\nvar<private> fragColor: vec4<f32>;\nvar<private> fragPosition: vec3<f32>;\nvar<private> fragNormal: vec3<f32>;\n@group(0) @binding(1) \nvar<uniform> vertex_info: VertexInfo;\nvar<private> gl_Position: vec4<f32>;\n\nfn computeSkinMatrix() -> mat4x4<f32> {\n    let _e9 = vertexWeights_1;\n    let _e13 = vertexWeights_1;\n    let _e18 = vertexWeights_1;\n    let _e23 = vertexWeights_1;\n    if ((((_e9.x == 0f) && (_e13.y == 0f)) && (_e18.z == 0f)) && (_e23.w == 0f)) {\n        {\n            return mat4x4<f32>(vec4<f32>(1f, 0f, 0f, 0f), vec4<f32>(0f, 1f, 0f, 0f), vec4<f32>(0f, 0f, 1f, 0f), vec4<f32>(0f, 0f, 0f, 1f));\n        }\n    }\n    let _e35 = vertexWeights_1;\n    let _e37 = vertexJoints_1;\n    let _e42 = jointMatrices.joints[i32(_e37.x)];\n    let _e44 = vertexWeights_1;\n    let _e46 = vertexJoints_1;\n    let _e51 = jointMatrices.joints[i32(_e46.y)];\n    let _e54 = vertexWeights_1;\n    let _e56 = vertexJoints_1;\n    let _e61 = jointMatrices.joints[i32(_e56.z)];\n    let _e64 = vertexWeights_1;\n    let _e66 = vertexJoints_1;\n    let _e71 = jointMatrices.joints[i32(_e66.w)];\n    return ((((_e35.x * _e42) + (_e44.y * _e51)) + (_e54.z * _e61)) + (_e64.w * _e71));\n}\n\nfn main_1() {\n    var skinMatrix: mat4x4<f32>;\n    var position: vec3<f32>;\n    var normal: vec3<f32>;\n    var modelViewProjection: mat4x4<f32>;\n\n    let _e20 = computeSkinMatrix();\n    skinMatrix = _e20;\n    let _e22 = skinMatrix;\n    let _e23 = vertexPosition_1;\n    position = (_e22 * vec4<f32>(_e23.x, _e23.y, _e23.z, 1f)).xyz;\n    let _e32 = skinMatrix;\n    let _e33 = vertexNormal_1;\n    normal = normalize((_e32 * vec4<f32>(_e33.x, _e33.y, _e33.z, 0f)).xyz);\n    let _e43 = vertex_info;\n    let _e45 = vertex_info;\n    let _e48 = vertex_info;\n    modelViewProjection = ((_e43.projection * _e45.view) * _e48.model);\n    let _e53 = modelViewProjection;\n    let _e54 = position;\n    gl_Position = (_e53 * vec4<f32>(_e54.x, _e54.y, _e54.z, 1f));\n    let _e61 = vertexTexCoord_1;\n    fragTexCoord = _e61;\n    let _e62 = vertexColor_1;\n    fragColor = _e62;\n    let _e63 = vertex_info;\n    let _e65 = position;\n    fragPosition = vec3<f32>((_e63.model * vec4<f32>(_e65.x, _e65.y, _e65.z, 1f)).xyz);\n    let _e74 = vertex_info;\n    let _e77 = transpose(_naga_inverse_4x4_f32(_e74.model));\n    let _e87 = normal;\n    fragNormal = (mat3x3<f32>(_e77[0].xyz, _e77[1].xyz, _e77[2].xyz) * _e87);\n    let _e90 = gl_Position;\n    let _e92 = gl_Position;\n    gl_Position.z = ((_e90.z + _e92.w) * 0.5f);\n    return;\n}\n\n@vertex \nfn main(@location(0) vertexPosition: vec3<f32>, @location(1) vertexTexCoord: vec2<f32>, @location(2) vertexColor: vec4<f32>, @location(3) vertexNormal: vec3<f32>, @location(4) vertexJoints: vec4<f32>, @location(5) vertexWeights: vec4<f32>) -> VertexOutput {\n    vertexPosition_1 = vertexPosition;\n    vertexTexCoord_1 = vertexTexCoord;\n    vertexColor_1 = vertexColor;\n    vertexNormal_1 = vertexNormal;\n    vertexJoints_1 = vertexJoints;\n    vertexWeights_1 = vertexWeights;\n    main_1();\n    let _e43 = fragTexCoord;\n    let _e45 = fragColor;\n    let _e47 = fragPosition;\n    let _e49 = fragNormal;\n    let _e51 = gl_Position;\n    return VertexOutput(_e43, _e45, _e47, _e49, _e51);\n}\n\nfn _naga_inverse_4x4_f32(m: mat4x4<f32>) -> mat4x4<f32> {\n   let sub_factor00: f32 = m[2][2] * m[3][3] - m[3][2] * m[2][3];\n   let sub_factor01: f32 = m[2][1] * m[3][3] - m[3][1] * m[2][3];\n   let sub_factor02: f32 = m[2][1] * m[3][2] - m[3][1] * m[2][2];\n   let sub_factor03: f32 = m[2][0] * m[3][3] - m[3][0] * m[2][3];\n   let sub_factor04: f32 = m[2][0] * m[3][2] - m[3][0] * m[2][2];\n   let sub_factor05: f32 = m[2][0] * m[3][1] - m[3][0] * m[2][1];\n   let sub_factor06: f32 = m[1][2] * m[3][3] - m[3][2] * m[1][3];\n   let sub_factor07: f32 = m[1][1] * m[3][3] - m[3][1] * m[1][3];\n   let sub_factor08: f32 = m[1][1] * m[3][2] - m[3][1] * m[1][2];\n   let sub_factor09: f32 = m[1][0] * m[3][3] - m[3][0] * m[1][3];\n   let sub_factor10: f32 = m[1][0] * m[3][2] - m[3][0] * m[1][2];\n   let sub_factor11: f32 = m[1][1] * m[3][3] - m[3][1] * m[1][3];\n   let sub_factor12: f32 = m[1][0] * m[3][1] - m[3][0] * m[1][1];\n   let sub_factor13: f32 = m[1][2] * m[2][3] - m[2][2] * m[1][3];\n   let sub_factor14: f32 = m[1][1] * m[2][3] - m[2][1] * m[1][3];\n   let sub_factor15: f32 = m[1][1] * m[2][2] - m[2][1] * m[1][2];\n   let sub_factor16: f32 = m[1][0] * m[2][3] - m[2][0] * m[1][3];\n   let sub_factor17: f32 = m[1][0] * m[2][2] - m[2][0] * m[1][2];\n   let sub_factor18: f32 = m[1][0] * m[2][1] - m[2][0] * m[1][1];\n\n   var adj: mat4x4<f32>;\n   adj[0][0] =   (m[1][1] * sub_factor00 - m[1][2] * sub_factor01 + m[1][3] * sub_factor02);\n   adj[1][0] = - (m[1][0] * sub_factor00 - m[1][2] * sub_factor03 + m[1][3] * sub_factor04);\n   adj[2][0] =   (m[1][0] * sub_factor01 - m[1][1] * sub_factor03 + m[1][3] * sub_factor05);\n   adj[3][0] = - (m[1][0] * sub_factor02 - m[1][1] * sub_factor04 + m[1][2] * sub_factor05);\n   adj[0][1] = - (m[0][1] * sub_factor00 - m[0][2] * sub_factor01 + m[0][3] * sub_factor02);\n   adj[1][1] =   (m[0][0] * sub_factor00 - m[0][2] * sub_factor03 + m[0][3] * sub_factor04);\n   adj[2][1] = - (m[0][0] * sub_factor01 - m[0][1] * sub_factor03 + m[0][3] * sub_factor05);\n   adj[3][1] =   (m[0][0] * sub_factor02 - m[0][1] * sub_factor04 + m[0][2] * sub_factor05);\n   adj[0][2] =   (m[0][1] * sub_factor06 - m[0][2] * sub_factor07 + m[0][3] * sub_factor08);\n   adj[1][2] = - (m[0][0] * sub_factor06 - m[0][2] * sub_factor09 + m[0][3] * sub_factor10);\n   adj[2][2] =   (m[0][0] * sub_factor11 - m[0][1] * sub_factor09 + m[0][3] * sub_factor12);\n   adj[3][2] = - (m[0][0] * sub_factor08 - m[0][1] * sub_factor10 + m[0][2] * sub_factor12);\n   adj[0][3] = - (m[0][1] * sub_factor13 - m[0][2] * sub_factor14 + m[0][3] * sub_factor15);\n   adj[1][3] =   (m[0][0] * sub_factor13 - m[0][2] * sub_factor16 + m[0][3] * sub_factor17);\n   adj[2][3] = - (m[0][0] * sub_factor14 - m[0][1] * sub_factor16 + m[0][3] * sub_factor18);\n   adj[3][3] =   (m[0][0] * sub_factor15 - m[0][1] * sub_factor17 + m[0][2] * sub_factor18);\n\n   let det = (m[0][0] * adj[0][0] + m[0][1] * adj[1][0] + m[0][2] * adj[2][0] + m[0][3] * adj[3][0]);\n\n   return adj * (1 / det);\n}\n",
+  "fragment": "struct Material {\n    albedoColor: vec4<f32>,\n}\n\nstruct FragmentOutput {\n    @location(0) outColor: vec4<f32>,\n}\n\nvar<private> fragTexCoord_1: vec2<f32>;\nvar<private> fragColor_1: vec4<f32>;\nvar<private> fragPosition_1: vec3<f32>;\nvar<private> fragNormal_1: vec3<f32>;\nvar<private> outColor: vec4<f32>;\n@group(1) @binding(0) \nvar albedoTexture: texture_2d<f32>;\n@group(1) @binding(1) \nvar albedoTextureSampler: sampler;\n@group(1) @binding(2) \nvar<uniform> material: Material;\n\nfn main_1() {\n    var texColor: vec4<f32>;\n\n    let _e10 = fragTexCoord_1;\n    let _e11 = textureSample(albedoTexture, albedoTextureSampler, _e10);\n    texColor = _e11;\n    let _e13 = texColor;\n    let _e14 = material;\n    outColor = (_e13 * _e14.albedoColor);\n    return;\n}\n\n@fragment \nfn main(@location(0) fragTexCoord: vec2<f32>, @location(1) fragColor: vec4<f32>, @location(2) fragPosition: vec3<f32>, @location(3) fragNormal: vec3<f32>) -> FragmentOutput {\n    fragTexCoord_1 = fragTexCoord;\n    fragColor_1 = fragColor;\n    fragPosition_1 = fragPosition;\n    fragNormal_1 = fragNormal;\n    main_1();\n    let _e26 = outColor;\n    return FragmentOutput(_e26);\n}\n",
+  "slots": {
+    "JointMatrices": {
+      "group": 0,
+      "binding": 0,
+      "sizeInBytes": 1024,
+      "memberOffsets": {
+        "joints": 0
+      }
+    },
+    "VertexInfo": {
+      "group": 0,
+      "binding": 1,
+      "sizeInBytes": 192,
+      "memberOffsets": {
+        "model": 0,
+        "view": 64,
+        "projection": 128
+      }
+    },
+    "Material": {
+      "group": 1,
+      "binding": 2,
+      "sizeInBytes": 16,
+      "memberOffsets": {
+        "albedoColor": 0
+      }
+    },
+    "albedoTexture": {
+      "group": 1,
+      "binding": 0,
+      "samplerBinding": 1
+    }
+  }
+}