The July Sale is here! Save 50% on select products now through July 25.

Text To Lip Sync

Yuri N Kalinin - Apr 7, 2018

Subtitles-based lip sync

  • Supported Platforms
  • Supported Engine Versions
    4.19 - 4.26
  • Download Type
    Engine Plugin
    This product contains a code plugin, complete with pre-built binaries and all its source code that integrates with Unreal Engine, which can be installed to an engine version of your choice then enabled on a per-project basis.

It's a real-time subtitles-based lip sync, i.e. it requires subtitles and audio to create animation. The plugin is using provided text subtitles to generate lips sync animation for characters in real-time. Audio envelope value from the synchronously activated audio component allows to detect silent intervals and to pause animation in this intervals and adjust future animation speed. Subtitles + Sound Asset = LipSync.

Demo video

Tutorial video


>>> Please try executable demo (link below) to weigh quality of an animation before you buy the plugin! <<<


Support (forums)

Technical Details

Features:

  • easy setup & use
  • using audio envelope value to detect pauses
  • using pauses in audio for real-time animation adjustments (this allow to play quite long audio sequence using a single subtitle line)
  • subtitles can be separated to blocks with in-out time marks to get better result
  • information gathered from audio can be saved to file to improve future animations
  • can automatically load map of phrases from data table
  • sources included

Code Modules:

  •  Text2LipSync (Runtime)

Number of Blueprints: 0

Number of C++ Classes: 4 (1 actor component exposed to blueprints)

Network Replicated: No

Supported Development Platforms: Windows x64

Supported Target Build Platforms: Windows x64

Documentation: pdf + MetaHumans Setup

Example Project: [4.19] [4.26]

Executable Demo (try before you buy): zip

Additional Notes: Please contact me if you need it for educational purposes