feat: tree-shakeable shader includes via lazy loading#18169
feat: tree-shakeable shader includes via lazy loading#18169RaananW wants to merge 6 commits intoBabylonJS:masterfrom
Conversation
|
Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s). |
|
WebGL2 visualization test reporter: |
|
Visualization tests for WebGPU |
1 similar comment
|
Visualization tests for WebGPU |
|
WebGL2 visualization test reporter: |
|
Visualization tests for WebGPU |
|
Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s). |
|
Snapshot stored with reference name: Test environment: To test a playground add it to the URL, for example: https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/18169/merge/index.html#WGZLGJ#4600 Links to test your changes to core in the published versions of the Babylon tools (does not contain changes you made to the tools themselves): https://playground.babylonjs.com/?snapshot=refs/pull/18169/merge To test the snapshot in the playground with a playground ID add it after the snapshot query string: https://playground.babylonjs.com/?snapshot=refs/pull/18169/merge#BCU1XR#0 If you made changes to the sandbox or playground in this PR, additional comments will be generated soon containing links to the dev versions of those tools. |
|
WebGL2 visualization test reporter: |
|
Visualization tests for WebGPU |
Add ShaderStore._PendingIncludesLoaders and LoadPendingIncludesAsync() for eagerly batch-loading shader include dependencies before processing. Update buildShaders.ts to generate _PendingIncludesLoaders registrations for main shaders and include files with nested includes. PreStripConditionalIncludes evaluates #ifdef blocks before include resolution, enabling tree-shaking of unused shader includes. Add LoadPendingIncludesAsync() call in effect._processShaderCodeAsync to prevent singleton WGSL processor state corruption from async interleaving. Update minification regex to preserve newlines before shader declaration keywords. Remove redundant IncludesShadersResolvers infrastructure. Add unit tests for LoadPendingIncludesAsync.
…ure PreStripConditionalIncludes calls - Use AsyncLock (per reviewer suggestion) instead of manual _CurrentLoadingPromise logic in LoadPendingIncludesAsync; concurrent callers now serialize through the lock rather than sharing a raw promise reference - Remove PreStripConditionalIncludes calls from Process() and PreProcess(): shader .ts files are not yet regenerated with lazy loaders, so double-evaluating #ifdef blocks was causing 27 WebGL2 visualization test regressions; the function definition is preserved for when shaders are regenerated - Update unit tests to match AsyncLock async semantics and reset the static lock in beforeEach to prevent cross-test lock leaks
LoadPendingIncludesAsync was called unconditionally in effect.ts for all effects. In the CDN bundle, all WGSL shader modules execute at load time and push their include loaders to _PendingIncludesLoaders. The first GLSL/WebGL effect created would then drain the entire WGSL loader queue before proceeding, stalling shader initialization until hundreds of dynamic imports resolved. This caused visual timing regressions in 24 WebGL2 visualization tests. Fix: gate the call on ShaderLanguage.WGSL so only WGSL effects process the include queue. GLSL effects still get LoadPendingIncludesAsync via their extraInitializationsAsync callback (in each material's shader loading code) for any WGSL includes that happen to be pending at that point.
|
WebGL2 visualization test reporter: |
|
Visualization tests for WebGPU |
|
Visualization tests for WebGPU |
|
WebGL2 visualization test reporter: |
|
Visualization tests for WebGPU |
3 participants
Summary
Make shader includes tree-shakeable by replacing eager static imports with lazy loading via
ShaderStore._PendingIncludesLoadersandLoadPendingIncludesAsync().Changes
Core Infrastructure
ShaderStore: Add_PendingIncludesLoadersarray andLoadPendingIncludesAsync()— a batch loader that eagerly loads all shader include dependencies in parallel, with support for nested includes and concurrent callers.effect.ts: CallLoadPendingIncludesAsync()in_processShaderCodeAsyncbefore shader processing begins, ensuring all includes are in the store synchronously. This prevents the singleton WGSL processor's per-effect state from being corrupted by async interleaving.shaderProcessor.ts: AddPreStripConditionalIncludes— evaluates#ifdef/#ifndef/#ifblocks before include resolution so that#includedirectives inside disabled conditional blocks are never resolved, enabling tree-shaking.Build Tooling
buildShaders.ts: Generated.tsshader files now register_PendingIncludesLoadersentries (both main shaders and include files with nested includes). Removed the previousIncludesShadersResolversapproach in favor of this simpler mechanism.varying,uniform,attribute, etc.) so WGSL line-level processors handle them correctly.Cleanup
IncludesShadersResolvers/IncludesShadersResolversWGSLstores,GetIncludesShadersResolvers(),includesShadersResolversprocessing option, and all resolver fallback paths fromProcessIncludes.Consumer Updates
await ShaderStore.LoadPendingIncludesAsync()in theirextraInitializationsAsynccallbacks.Tests
LoadPendingIncludesAsync: fast path, batch loading, nested loaders, concurrent callers, error cleanup.Verification
npm run build:devpasses clean