feat: avatar facial expressions#8746
Draft
alejandro-jimenez-dcl wants to merge 29 commits into
Draft
Conversation
Components (AvatarFaceComponent merging expression/blink/mouth-anim, AvatarMouthInputComponent, LocalPlayerFacialExpressionComponent), SO definitions (AvatarFaceExpressionConfig + Definition), shared mouth input queue, default expressions asset, and Essentials addressable.
Adds the central face animation system delegating material binding to AvatarFaceMaterialUtils and atlas/pose data to AvatarFacialExpressionConstants. Designer-tunable timings live in AvatarFaceAnimationSettings (SO). Drops AvatarMouthInputQueue: chat/voice handlers will write directly to AvatarMouthInputComponent (added alongside AvatarFaceComponent in setup).
Adds rfc4 FacialExpression message (eyebrows/eyes/mouth tile indices, ADR-317) and wires MessagePipe payload extraction for the new oneof case. Source proto change lives on protocol branch feat/facial-expressions.
Multiplayer: IFacialExpressionMessageBus + MultiplayerFacialExpressionMessageBus (subscribe to FacialExpression oneof, edge-triggered Send), RemoteFacialExpressionIntention, and PlayerFacialExpressionNetSendSystem reading LocalPlayerFacialExpressionComponent. AvatarShape: RemoteFacialExpressionSystem drains intentions and applies indices to AvatarFaceComponent of the matching remote avatar (TODO: validate non-query-driven participant lookup with architect, mirrors RemoteEmotesSystem). Face systems grouped under AvatarShape/Systems/FacialExpression and Multiplayer/FacialExpression.
Slots 1-9 bound as Y-modifier composites mirroring the emote slot shortcut style. Unblocks UpdateFaceExpressionInputSystem (Domain 2 leftover).
Y + 1-9 shortcut applies the matching expression to the local player's AvatarFaceComponent and mirrors indices into LocalPlayerFacialExpressionComponent so PlayerFacialExpressionNetSendSystem propagates the change.
AvatarPlugin loads AvatarFaceAnimationSettings + AvatarFaceExpressionConfig SOs and injects AvatarFacialExpressionSystem (null texture arrays for now — atlases come from wearables per ADR-317) and UpdateFaceExpressionInputSystem. MultiplayerMovementPlugin owns IFacialExpressionMessageBus, injects PlayerFacialExpressionNetSendSystem and RemoteFacialExpressionSystem. DynamicWorldContainer instantiates MultiplayerFacialExpressionMessageBus alongside the movement bus. RemoteFacialExpressionSystem made partial so the Arch source generator emits InjectToWorld. Added asmref so PlayerFacialExpressionNetSendSystem compiles in DCL.Plugins (which has Arch.SystemGroups). Default AvatarFaceAnimationSettings asset and Essentials addressable entry plus Global Plugins Settings.asset references for both SOs.
Brings the facial expressions wheel UI online: ring of 10 slots with icons, EYEBROWS / EYES / MOUTH cyclers, live avatar preview, and the input maps that drive it (Y+[0-9] always-on quick-apply, plain [0-9] while the wheel is open). On close, the last preview is committed via IFacialExpressionApplier if the user touched a slot or cycler; opening and closing without picking anything leaves the avatar untouched. The slot view, cycler view, controller, character preview wrapper and icon textures are introduced here. FacialExpressionWheelUtils centralises slot label / action name / wrap math so the controller and the (future) ECS input system share the same encoding. The cycler exposes a single OnCycle(int delta) instead of separate Previous/Next events.
Pulls Y-shortcut release and Y+[0-9] quick-apply out of the wheel controller so the wheel stays UI-only. FacialExpressionsWheelShortcutHandler mirrors EmoteWheelShortcutHandler: listens to Shortcuts.FaceExpression.canceled, publishes RequestToggleFacialExpressionsWheelEvent on the event bus (subscriber is the wheel plugin, follow-up). Exposes NotifyExpressionPlayed so the ECS input system can suppress the next Y release after a quick-apply. UpdateFacialExpressionInputSystem replaces the old UpdateFaceExpressionInputSystem: listens to the FaceExpressions map, writes TriggerFacialExpressionIntent on the local player, notifies the handler so the upcoming Y release is swallowed. The FaceExpressions map was previously always-on (it had no entry in ApplyInputMapsSystem) which meant Y+[0-9] fired in chat input, menus, and any state that disables player input. Adds InputMapComponent.Kind FACE_EXPRESSIONS, wires it through ApplyInputMapsSystem, and includes it in the default-enabled set in MainSceneLoader so blocking emotes also blocks face quick-apply. AvatarPlugin now owns the handler lifecycle (created when the face expression config loads, disposed with the plugin) and injects the new ECS input system. DynamicWorldContainer passes the shared emotesEventBus to AvatarPlugin so the handler and the wheel use the same event channel.
Inline FacialExpressionApplier as a static util and drop the matching interface; the wheel takes World+Entity directly and calls Apply(). Move the applier out of the FacialExpression subfolder so it lives in the AvatarShape assembly, reachable from the wheel (DCL.Social), which the prior asmref-to-Plugins indirection had silently broken. Relocate LocalPlayerFacialExpressionComponent from DCL.Multiplayer.Movement to DCL.Multiplayer.FacialExpression next to its only consumer (PlayerFacialExpressionNetSendSystem), and fix the stale xmldoc that pointed at the old PlayerMovementNetSendSystem path. In ApplyFacialExpressionIntentSystem: rename the network-flavored query to SetupLocalPlayerFacialExpression, rename the ref param network->local, and drop the redundant `in Entity` per code standards (Entity is a small value type and the query mutates). Fold FacialExpressionTriggerSource into FacialExpressionsWheelShortcutHandler (single-use enum), drop the unjustified virtual on NotifyExpressionPlayed, inline StashRemoteMessage into OnChatMessageAdded, and turn EnqueueSpeaking into a local function inside OnActiveSpeakersUpdated.
Slot prefab + HUD prefab with 10 radial slot instances (R=270). Move expression icons to Assets/Textures/FacialExpressionsWheel. Register FacialExpressionsWheelPlugin in DynamicWorldContainer. Qualify UnityEngine.Time to avoid DCL.Time namespace shadow. Cycler buttons not wired yet.
Wearable load substitutes *_expressions.png / *_expressions_mask.png when present and flags the wearable. The facial expression system stamps per-channel capability onto AvatarFaceComponent and skips MaterialPropertyBlock overrides (blink, mouth pose, eyebrow frames) on channels without an atlas, keeping the wearable's static face texture untouched.
Material-side wiring for the in-shader 4x4 atlas slicing landed in unity-shared-dependencies. AvatarFaceMaterialUtils sets _ExpressionIndex through a MaterialPropertyBlock (-1 disabled, 0..15 cell) so the renderer's existing _MainTexArr_ID slot keeps the wearable's loaded atlas while the shader picks the per-frame cell on its own. AvatarFacialExpressionSystem drops the dead eyebrows / eye / mouth global Texture2DArray params (AvatarPlugin had already been passing null); the system now only needs the animation settings. Also restores the LINQ Contains -> explicit loop fix in WearablePolymorphicBehaviour.TryGetMainFileKey: the LINQ overload picked up the wrong extension method once the ambient using shifted.
…g gaps Properties inside UnityPerMaterial CBuffer (added there for SRP batcher compatibility) can't be overridden by MaterialPropertyBlock. Each face renderer already has its own pooled material instance, so write the expression cell index directly onto sharedMaterial. Same pattern as _MainTexArr_ID. SetupFaceComponents flips IsDirty=true so the first frame's ApplyExpressionLayer actually writes the initial cell index, and bumps the per-channel index to 0 (idle) for atlas-bound channels. The -1 sentinel stays the default so non-atlas wearables keep their static face texture untouched. Also: - Register FacialExpression in MessageWrapExtensions.WRITES_MAP so the rfc4 broadcast stops throwing NotSupportedException every trigger. - Park FacialExpressionsWheelPlugin behind a commented registration in DynamicWorldContainer until wearables ship expression atlases; its Settings entry stays in the Global Plugins Settings container so re-enabling is a single uncomment.
…ressions # Conflicts: # Explorer/Assets/AddressableAssetsData/AssetGroups/Essentials.asset # Explorer/Assets/DCL/Infrastructure/Global/Dynamic/DynamicWorldContainer.cs # Explorer/Packages/packages-lock.json
Contributor
|
Windows and Mac build successful in Unity Cloud! You can find a link to the downloadable artifact below. |
Swapping wearables pool-recycles the body shape renderer with a fresh skinning material whose _ExpressionIndex starts at the shader-default sentinel. The face component still cached its prior renderer reference and Current* indices, so the equality guard short-circuited subsequent writes and the channel rendered the full atlas until the next input. - Stamp per-channel atlas capability on AvatarShapeComponent during AvatarInstantiator so AvatarFacialExpressionSystem no longer reads the (possibly replaced) WearablePromise. - Bring HasExpressionAtlas onto AvatarFaceComponent as a stable bool; resting index resolves to 0 or the channel sentinel from that flag. - Run ApplyExpressionLayer every frame and drop the Current* equality guard in AvatarFaceMaterialUtils, since the pool can swap the underlying material without changing the renderer ref.
The previous fix ran ApplyExpressionLayer every frame and dropped the equality guard in AvatarFaceMaterialUtils because the skinning material pool can swap the face material under us on a wearable swap (same renderer instance, fresh shader-default _ExpressionIndex). That bypassed the Current* optimization unconditionally. Cache the renderer.sharedMaterial alongside the renderer ref on AvatarFaceComponent and diff it in ReInitRenderersIfNeeded. A mismatch flips IsDirty and resets Current* the same way a renderer-ref or capability-bool change does, so the equality short-circuit and the IsDirty gate are reliable again.
feat(facial-expressions): rework wheel state model + enable plugin
Extract FaceChannelCyclerView to reusable DCL.UI.NumericCycler{View,Controller}.
Controller owns wrap/index, raises OnIndexChanged only on user input so external
SetIndex calls do not re-enter.
Rewire FacialExpressionsWheelController around three NumericCyclerController
instances: seed from current AvatarFaceComponent on open, auto-match a slot or
show "Custom", commit via FacialExpressionApplier only when the user touched
something.
Fix initial-seed race: preview wearables instantiate async after OnShow, so
TrySetFace was silently no-opping. SetFaceWhenReadyAsync waits for the preview
entity AvatarFaceComponent before pushing seed indices. Required exposing
CharacterPreviewController.PreviewEntity.
Shorten wearable URNs in FacialExpressionsCharacterPreviewController.Initialize
to avoid long-form lookup misses.
Make CharacterPreviewAvatarContainer FOV smooth duration configurable via
cameraSettings.fieldOfViewDuration; <= 0 snaps instantly.
Re-enable FacialExpressionsWheelPlugin in DynamicWorldContainer (was gated on
wearables shipping expression atlases).
@
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request Description
What does this PR change?
Adds facial expressions to avatars: a radial wheel HUD picks one of 9 presets or fine-tunes eyebrows/eyes/mouth independently, applies the result to the local avatar, and broadcasts changes over comms so remote players see them.
Data + animation layer (
Assets/DCL/AvatarRendering/AvatarShape)AvatarFaceComponentholds per-channel renderer/material refs, atlas-capability bools, resting indices (Eyebrows/Eyes/MouthExpressionIndex), live override indices, and blink/mouth-animation state.AvatarFacialExpressionSystem(inAvatarGroup, afterAvatarInstantiatorSystem) sets up the component, re-binds renderers/materials on wearable swap, and drives three layers each frame: eyebrows base, blink override, mouth-pose override (mouth restores toMouthExpressionIndexwhen chat/voice end, not to idle)._ExpressionIndexmaterial property +_expressionstexture; per-channel writes go throughMaterialPropertyBlockinAvatarFaceMaterialUtils. Channels whose wearable lacks an_expressionsatlas are skipped so we don't paint atlas slices onto a single-frame face texture.Rendererinstance with a freshly pooledMaterialwhose_ExpressionIndexis at the shader-default sentinel. The system diffs both the renderer ref andsharedMaterialagainst the cached pair every frame and flipsIsDirtyon mismatch, so the override is re-applied after a swap.MouthAnimationSystem/ inputs inAvatarMouthInputComponent: chat text overrides voice-chat loop; vowels and uppercase get distinct frame durations.Comms (
Assets/DCL/Multiplayer/FacialExpression)Regenerated rfc4 protobuf with a new
FacialExpressionmessage (EyebrowsIndex,EyesIndex,MouthIndex).MultiplayerFacialExpressionMessageBussubscribes on Island + Scene pipes, drops payloads withindex > 15(ADR-317), and exposes aDrainpattern keyed by wallet.PlayerFacialExpressionNetSendSystemis edge-triggered offLocalPlayerFacialExpressionComponent(only sends when indices change).RemoteFacialExpressionSystemapplies drainedRemoteFacialExpressionIntentions to remote avatars viaTriggerFacialExpressionIntent, consumed byApplyFacialExpressionIntentSystem.Input
New
Shortcuts.FaceExpression(Y) andFaceExpressions(1-9) action maps inDCLInput.inputactions; map flag added toInputMapComponent.UpdateFacialExpressionInputSystemresolves wheel-open vs. quick-press: Y down opens the wheel state, Y+digit while open applies a slot, Y+digit released without explicit open is a "shortcut" quick-apply.FacialExpressionsWheelShortcutHandlerowns Y release:ignoreNextReleaseafter aSHORTCUTapply and a 0.5sQUICK_APPLY_LOCK_TIMEafter aWHEEL_SLOTapply suppress the toggle so a quick pick doesn't immediately reopen/close the wheel. PublishesRequestToggleFacialExpressionsWheelEventotherwise.Wheel UI (
Assets/DCL/FacialExpressionsWheel)New
FacialExpressionsWheelHUDprefab +FacialExpressionWheelSlotView, controller, view, and aFacialExpressionsCharacterPreviewController.On open, the wheel seeds from the player's current
AvatarFaceComponent, auto-matches the indices to a preset slot label or shows "Custom"; commits only when the user actually cycled a stepper or clicked a slot.Three independent steppers (eyebrows / eyes / mouth) backed by a reusable
DCL.UI.NumericCycler(extracted from the wheel-specificFaceChannelCyclerView).Emote wheel adds a "face tab" entry that hands off to the facial expressions wheel.
Test Instructions
Prerequisites
_expressionsatlas (default body works).Test Steps
ignoreNextRelease).Quality Checklist
Code Review Reference
Please review our Branch & PR Standards before submitting.