get wasm fast again

[louistrue]

What I mean is, it's not about the total opening time itself, but the difference in opening time. For instance, even a lightweight file might take about 20 seconds to open, but while it took 20 seconds in the old version, it takes 25 seconds in the new one. The same goes for a very large model: it opens in about 1 minute 20 seconds in the old version, but takes around 2 minutes in the new one. So we are looking at this kind of gap. It has become about 40 to 50 percent slower.

Originally posted by @Blogbotana in #1109

[Blogbotana]

I'll try to solve this issue too 👍

[louistrue]

cool, go for it @Blogbotana 👍 I'm currently looking into plane clipping (for the Tekla steel-heavy stuff) and a fast rectangular-openings path (for architectural models)...

[Blogbotana]

Profiled the regression natively — the dominant cost is mesh-count fragmentation, not the CSG kernel.

Method: ran process_geometry natively (no WASM/JS noise, same machine) on three boolean-heavy Tekla steel models, comparing the last Manifold-default commit 7c0f0d82 (parent of #1024) against current main. Per-phase numbers come from the built-in ProcessingStats.

File	metric	Manifold 7c0f0d82	main	ratio
130_KM	geometry_ms	621	2415	3.9×
	total_ms	702	2509	3.6×
	meshes	9 031	26 739	3.0×
	triangles	2.69M	2.93M	1.09×
140_KM	geometry_ms	453	1978	4.4×
	meshes	5 180	18 876	3.6×
	triangles	1.91M	2.07M	1.08×
170_KM	geometry_ms	1738	5934	3.4×
	meshes	16 622	64 314	3.9×
	triangles	7.14M	7.92M	1.11×

Parse/scan are <5% of total on every file — the cost is entirely in the geometry phase.

Findings

• Triangle counts are within ~10% and total_csg_failures == 0 on all three — the exact kernel completes every cut and never falls back to the AABB box. So this is not the booleans (and not the Regression: exact CSG kernel hangs at 95% on boolean-heavy models that loaded in ~5 s before #1109 escalation path).
• main emits 3–4× more MeshData objects for the same geometry (~2.9–3.9 meshes/element vs ~1.0 in the Manifold era).
• Time per mesh is ~equal on both kernels (~0.07–0.10 ms/mesh; on 170_KM main is even faster per mesh). So geometry_time_ms scales with mesh count, which accounts for essentially the whole ~3.5× regression. In WASM this is compounded: 3–4× more meshes ⇒ 3–4× more serialization across the WASM→JS boundary, which is the wall-clock this issue is about.

Root cause

rust/processing/src/element.rs::emit_sub_meshes emits one MeshData per representation item (the submesh path + #858 per-item palette splits) and never coalesces submeshes that resolve to the same color/material. A single-color steel member (main extrusion + several IfcBooleanResult parts + plates) fragments into 3–4 meshes; the Manifold-era pipeline merged an element's items into ~1. This came in with the pipeline unification (#874 also removed the old geometry-deduplicator; cf. the sibling #1292 "instancing dropped repeated geometry").

Proposed fix

In emit_sub_meshes, group submeshes by resolved (color, material_name, geometry_class) and merge each group into a single MeshData (via Mesh::merge, which already rebases per-mesh origins and offsets indices) before build_mesh_data. This keeps everything the submesh split was added for — #858 indexed-colour palettes, per-item window-glass transparency, per-layer wall slice colours, per-item error isolation — because parts with different colours still split; instanced submeshes stay separate. It only re-merges parts that are identically styled, collapsing the common single-colour case back to ~1 mesh/element.

Will follow up with a branch + before/after numbers.

[Blogbotana]

Correction — deeper root cause (and my proposed emit_sub_meshes fix above was wrong).

The earlier comment's direction holds — it's mesh count, not the booleans — but the specific mechanism and the colour-merge fix were off. Instancing is always on (instancing_enabled() == true), so every occurrence already carries InstanceMeta and takes the instanced path; those submeshes are not colour-merge candidates, and merging them would defeat instancing.

The real cause: the pipeline fully builds every occurrence even though the geometry is massively redundant, and collation dedups only after the heavy build. processGeometryBatchInstanced calls produce_batch (full per-occurrence tessellation / CSG / normals) and only then collate_and_encode (rust/wasm-bindings/src/api/gpu_meshes.rs). So the per-occurrence production cost is paid in full; collation just shrinks the upload.

Unique-vs-total on the same three Tekla steel files (collate_refs, min_group = 2):

File	meshes built	unique geometries	redundancy
130_KM	26 739	791 (597 templates / 26 545 occ + 194 flat)	~34×
140_KM	18 876	1 262	~15×
170_KM	64 314	2 800	~23×

~95–97% of the per-occurrence build is redundant — the same beams/plates/bolts are tessellated thousands of times. The Manifold era emitted ~1 mesh/element (5 180 / 16 622, no per-occurrence explosion), which is why it was ~3.5× faster at the same triangle total.

Fix direction: dedup geometry production by a cheap pre-build signature so each unique shape is tessellated/CSG'd once and reused with a per-instance transform — i.e. move the collation key upstream of the heavy build instead of running it after. The collation infra (rep_identity, collate_refs, templates) already exists; it just runs too late. For direct (non-mapped) swept solids the current rep_identity is a post-tessellation geometry hash, so the missing piece is a definition-level signature (profile id/params + extrusion depth/direction; mapped/representation key when present) computed before the build. These files have only ~12 IfcRepresentationMap / ~34–38 IfcMappedItem, so the redundancy is geometric (distinct entities, identical shape), not shared representations — a definition-level signature is what catches it.

[louistrue]

nice work @Blogbotana, your correction is spot on. verified it in the code: produce_batch builds every occurrence (produce_element_meshes) before collate_and_encode runs, and rep_identity for direct swept solids is a post-tessellation hash (compute_mesh_hash_full | DIRECT_SOLID_TAG), so there's no pre-build key to dedup on yet. mapped items do get cached by source_id but with ~12 RepresentationMaps in these files that catches almost nothing, so yeah, the redundancy is geometric (distinct entities, identical shape).

been profiling the CSG side in parallel and this reconciles it perfectly. per-cut the exact kernel is ~15ms on 170_KM steel, which made it look like the kernel was the bottleneck, but it's ~95% redundant: the same beams/plates/bolts cut over and over. the ~14.7s of serial CSG collapses to under 1s if you build the ~2.8k unique shapes once (same distinct count instancing already lands at). so moving the dedup upstream of the heavy build is the lever. the plane-clip / box-diff stuff i was poking at only speeds up the unique cuts, which is secondary to this.

few guardrails on the pre-build signature, learned the hard way from the last couple attempts:

1. the key has to include the boolean voids + their placements, not just profile + extrusion depth/dir. two identical beams with bolt holes in different spots have the same swept solid but different final geometry, so a profile-only key would dedup them wrong. cheap safe path: do the boolean-free direct solids first (most plates and plain members, that's the bulk, and the signature is complete there), then a richer key for the voided ones.

2. don't re-decode the subtree to build the key. that's exactly what made content-dedup feat(geometry): content-dedup identical representation items (skip re-meshing) #1130 a net loss (the hash cost ~3.5x the meshing it skipped). read it off the already-decoded entity.

3. assert the pre-build signature produces the SAME groups as the post-build rep_identity on a sample, so we don't drift from what collation already does, and native stays == wasm.

4. record the per-occurrence placement before baking it into the verts. that's the exact trap from fix(geometry): instancing dropped repeated geometry (Tekla steel showed as 'missing objects') #1292 (the "missing objects" one), and build-time dedup leans on it even harder.

happy to take the voids-inclusion + parity test if it helps, lmk 👍

[louistrue]

@Blogbotana your first comment was actually onto it imo, the mesh-count angle. dug in and the pre-build dedup the correction describes already exists and is on in prod, so heads up before building a signature from scratch:

• content_dedup_enabled() returns true (router/mod.rs:367)
• process_representation_item computes item_dedup_key = item_signature(...) BEFORE meshing (decode-free, memoized, renumber-invariant, folds inline booleans + placements), and on a hit returns the cached local mesh and skips tessellation+CSG entirely. the in-code note has 170_KM geometry 16.4s -> 2.8s (5.8x), byte-identical.

so the geometry compute is already deduped. a from-scratch definition key would mostly duplicate this and risk re-introducing the #1130 re-decode net-loss. what's actually left is your first point, per-occurrence mesh cost, in two spots:

1. on every cache HIT, tag_direct_instance still re-runs compute_mesh_hash_full over all verts to set rep_identity (processing.rs:930 -> 971), because the mesh is cached before it's tagged. O(verts) per occurrence (64k on 170_KM) instead of once per shape. cheap byte-identical fix: stash rep_identity in the cache value (Arc<Mesh> -> Arc<(Mesh, u128)>).
2. the real one is void hosts: the item cache returns the UNCUT base, then apply_void_context runs the CSG cut per occurrence at the element layer (voids.rs:2874), result never cached. so identical slabs/walls cut by identical openings get CSG'd every time, and CSG is ~85-90% of load. fix is an element-level cache of the post-cut mesh keyed by a void-inclusive sig (host sig + sorted per-opening sig + quantized host-relative placement) with a strict completeness fallback so beams with different holes never false-merge.

want to split it? happy to take the void-host element cache (#2, bigger and riskier) and leave you the rep_identity stash + widening the dedup type-gate to facesets, or carve it however. one ask either way: nothing in CI can catch a bad dedup merge today (the snapshot harness only walks process_element, not the submesh path), so whoever goes first should promote dedup_validate.rs to a real lane with a grouping-parity assert. otherwise a false-merge ships as silently-wrong geometry. 👍

[Blogbotana]

Follow-up — pinned the cost to the exact kernel itself (correcting my "mesh count" comments above).

@louistrue good call on the existing dedup. I dug further with native A/B controls on 170_KM and the earlier "mesh-count fragmentation" framing was wrong — here's the breakdown:

1. CSG is ~93% of the geometry phase. Making the kernel subtract/subtract_many return the host un-cut (everything else identical): geometry 7 603 ms → 497 ms. Tessellation alone is ~0.5 s.

2. Mesh count is not the driver. Forcing the Manifold-era "one combined mesh per element" reproduces the exact Manifold mesh counts (9 031 / 5 180 / 16 622), unchanged triangles, but geometry is still 2.7 / 2.1 / 6.9 s (~4× Manifold). Merging submeshes per element was a small net loss (extra vertex copies). So the colour-merge idea I floated is a dead end.

3. More caching won't help these files. With content-dedup on, 170_KM issues only 954 subtract calls total (subtract_many = 0 — these are IfcBooleanResult, not void-host openings). ~6 s / 954 ≈ 6.3 ms per exact subtract. So the rep_identity-stash and void-host-cache ideas would help opening-heavy Revit models, but on this Tekla corpus the booleans are already deduped to the minimum; the remaining cost is irreducible unique exact subtracts.

4. Not the #1109 escalation path. ~970k escalations across ~16.6k elements ≈ 58/element (budget calibration: p95≈150), 0 budget trips, 0 AABB fallbacks. It's the base exact-arrangement cost, not pathological escalation.

Bottom line: vs the Manifold era the gap is the exact-arithmetic kernel being ~4–5× slower per subtract than Manifold's float kernel for the same cut (same triangles, same result). The implicit-point/symbolic arrangement + exact predicate cascade is inherently heavier than float intersection coordinates. #1135 already recovered ~42% (BVH + lambda caching). Closing the rest is a kernel-perf effort — parallelise within a single boolean (currently single-threaded per op), shrink per-subtract overhead, drop the per-cut consolidate_coplanar re-triangulation — or a float fast-path, which trades against the native==wasm determinism the exact kernel exists to guarantee.

Repro: native harness around process_geometry reading the built-in per-phase ProcessingStats; Manifold baseline built from 7c0f0d82 with --features ifc-lite-geometry/manifold-csg.

[Blogbotana]

Idea: skip small detail cuts in the preview tessellation tiers.

Following the finding that the cost is the exact subtract per cut (not mesh count / dedup), on the KM steel corpus almost every cut is a small local cope/notch — the cutter's max dimension is <10% of the host's on 98% of cuts in 130_KM and 67% in 170_KM — yet each pays a full ~6 ms exact boolean for a barely-visible change.

Proposal: in the preview tessellation tiers only (Lowest/Low), skip a DIFFERENCE cut whose cutter max-dimension is < ~10% of the host max-dimension and render the host un-cut. Medium (default) and finer keep every cut, so production geometry stays byte-identical.

Measured (native, 170_KM, threshold 10%): geometry phase 7.6 s → 1.1 s (~6.9×) — i.e. Manifold-class load times — with no parallelism and full determinism. Trade-off is purely visual (tiny notches/copes vanish in preview), which is what fast viewers already do. Threshold is tunable (IFC_LITE_FAST_CUT_RATIO natively; the default applies in wasm).

[louistrue]

nice, think we can just do I256-before-I512 without loosing anything 🙌

[louistrue]

😔 doesn't move the needle