Lip Sync

StoryFlowLipsync moves a character's mouth while their StoryFlow Editor dialogue audio plays. It is a separate component for the actor's face, so projects that only display text do not need it.

Supported Faces

The component supports ARKit-style facial blendshapes. This widely used facial-animation convention gives individual movements names such as jawOpen, mouthPucker and mouthFunnel. Each blendshape acts like a slider that deforms the face. The built-in mapping uses the mouth-related shapes from this set; it does not require the full set of 52. See Apple's blendshape documentation for the convention.

Your character's mesh must already contain the facial blendshapes you want to animate. The component drives those existing shapes from audio; it does not generate a facial rig. Use a Viseme Map to connect differently named blendshapes. No iPhone or face-tracking hardware is needed for this audio-driven animation.

Synty Sidekick Characters

Synty Sidekick characters with facial blendshapes are supported. The built-in poses are tuned for Sidekick faces, and the component recognizes names such as MESHBlends.jawOpen. It can drive a baked character mesh or a face assembled from separate head, teeth and tongue meshes. Set Face Root to their common parent so every matching part receives the mouth animation.

Set Up a Face

1. Add StoryFlowLipsync to the character actor.
2. Assign the StoryFlowComponent that plays dialogue to Source. If left empty, the lip sync component looks for one in the scene.
3. Set Face Root to the transform containing the face's SkinnedMeshRenderer components. If left empty, it uses its own GameObject.
4. Set Character Id to this actor's StoryFlow Editor character ID. An empty ID makes this face react to every line, which can be useful in a single-character scene.
5. Play a voiced line and adjust Strength, Sensitivity, Jaw Bias and Smoothing to suit the character.

The component analyses audio as it plays. It works without preparing a timing track for each line. Because this method follows sound energy, it does not guarantee exact consonant timing.

If your game plays dialogue audio through its own AudioSource, call StartLipsyncFor(source) with that source and call StopLipsync() when playback ends.

Mapping a Rig

Leave Viseme Map empty to use the built-in ARKit-style mouth poses. The component drives all matching face renderers under Face Root. Rigs without the additional tongueUp and tongueRaise shapes still use the available jaw and lip shapes.

For a rig with different names, create a StoryFlow > Viseme Map asset in Unity and assign it to Viseme Map. Set the name style to match the mesh and map each pose to the rig's blendshapes. A map with no custom poses keeps the built-in pose table, so it can also be used just to change the name style.

Tuning and Troubleshooting

If the mouth does not move, read the component's Level and RawPeak C# properties while dialogue audio plays. You can inspect them in a debugger or log their values. A Level near zero means the component is receiving little usable audio. Check that the dialogue AudioSource is active. If you route dialogue through an Audio Mixer Group, try clearing that route while testing. Also check Dialogue Volume Multiplier on the dialogue component.

If the audio level changes but the face stays still, check that Face Root contains renderers with the expected blendshape names. Use a Viseme Map for another naming scheme. Analysis Full Scale adjusts the level reference when quiet audio barely opens the mouth or loud audio keeps it open.