Meshingun Studio: All products 50% off

PoseAI LiveLink

Pose AI Ltd - Code Plugins - Jun 28, 2021

Connect your phone for real-time motion capture input

  • Supported Platforms
  • Supported Engine Versions
    4.26 - 4.27, 5.0 - 5.3
  • Download Type
    Engine Plugin
    This product contains a code plugin, complete with pre-built binaries and all its source code that integrates with Unreal Engine, which can be installed to an engine version of your choice then enabled on a per-project basis.

This plugin adds a LiveLink source for connecting with Pose Camera, our full body motion capture app developed for mobile devices. This allows you to use your phone to capture and stream live, low-latency animation data into the Unreal Engine. With a single node in your animation blueprint LiveLink allows you to take an incoming stream and animate rigged characters.


Pair your phone and record animations using Unreal's tools or use motion input from players to generate exciting new gameplay.


In order to effectively use this software you need the Pose Cam app installed on your phone. The app is now available on the iOS App Store. There is a free trial period and then a subscription is required. We recommend an iPhone 8 or later (ideally an Iphone XR/XS or later), or a recent IPad (A12 chipset or better).


For developers: we have versions of our engine for iOS, Android and Windows, which can be embedded in third party applications. Try it first with our iOS showcase app and then get in contact to discuss licensing and our other versions.


No meshes, characters or content included in the plugin. Meshes used in project gallery are available from third parties on the Unreal Marketplace.


Newest Changes 

  • Face animation (blendshapes similar to ARKit) via normal camera stream and integrated with the full body pose with the 1.4 engine release (March 2023)
  • UE plugin v1.4 - see our new animation graph nodes and UE5 demo for our arm/hand IK which maps the player's pose onto an avatar with different proportions for better results. See our YouTube video for a demo and setup guide.




To upgrade to the latest version of the plugin, click under your Engine Version from the Epic Games Unreal launcher, where it says "Installed Plugins!", and then click update next to our plugin.

Technical Details

Features:

  • Works with simple phone cameras - we do not need depth sensors, and we get game ready frame rates even on older phones
  • Sandboxed AI - Our AI runs fully on the connected phone: no server queries and no competing with Unreal for desktop or console resources.
  • Multiple rigs - Select from different rigs (UE4 Mannequin, MetaHuman, Mixamo, DazUE). We include fully articulated hands.
  •  Different modes - tailor for different phone camera setups, such as Room (full body standing at a distance), Desktop (upper body seated at your pc), or Portrait (full body in tighter spaces).
  • Player Events - simplify game logic by using our motion input events similar to keyboard input
  • Convenience options - stream in third-person or mirror mode (i.e. flipped left/right sides), separate root motion etc.

Code Modules:

  •  PoseAILiveLink


Supported Development Platforms: Windows

Supported Target Build Platforms: Windows, Mac

Documentation: here

Simple Mocap Animation Project - UE4: here UE5: here

Advanced Mocap Event Gameplay Project: here