PoseAI LiveLink

Pose AI Ltd - 2021/06/28

Connect your phone for real-time motion capture input

  • 支持的平台
  • 支持的引擎版本
    4.25 - 4.27
  • 下载类型
    引擎插件
    此产品包含一款代码插件,含有预编译的二进制文件以及与虚幻引擎集成的所有源代码,能够安装到您选择的引擎版本中,并根据每个项目的需求启动。

This plugin adds a LiveLink source for connecting with Pose Camera, our full body motion capture app developed for mobile devices. This allows you to use your phone to capture and stream live, low-latency animation data into the Unreal Engine. With a single node in your animation blueprint LiveLink allows you to take an incoming stream and animate rigged characters.


Pair your phone and record animations using Unreal's tools or use motion input from players to generate exciting new gameplay.


In order to effectively use this software you need the Pose Camera app installed on your phone. The app is currently available in our open (free) beta for iOS, ONLY via Apple TestFlight. Currently the app only works on an iPhone 8 or later, or a recent IPad (A12 chipset or better). Please see the beta page on our website for participation instructions.


No meshes, characters or content included in the plugin. Meshes used in project gallery are available from third parties on the Unreal Marketplace.


Important/Additional Notes: Pose Camera is currently being developed for iOS and is undergoing open beta testing, with planned release later in 2021. We hope to release a version for Android in the future.


Beta ChangeLog

v1.01 - Some users reported UE crashes typically during a TArray<Transform> ResizeGrow operation from within LiveLink. We could not consistently replicate the issue but this fix, pre-allocating the TArray size, stopped any such crashes on our systems. Please let us know if you still see an issue post fix.

v1.02 - Plugin now also built for MacOS. Added blueprint category to PoseAILiveLinkRetargetRotations property as lack prevented some builds.

v1.10 - Plugin now provides blueprint events notifying of changes in the subject's visibility in the camera, and some helpful additional data to use the mocap as a character controller for motion. See our updated example project or website for more info on these features.

v1.11 - FIxed a couple issues which were preventing Packaged builds from running.


To upgrade to the latest version of the plugin, click under your Engine Version from the Epic Games Unreal launcher, where it says "Installed Plugins!", and then click update next to our plugin.

技术细节

Features:

  • Works with simple phone cameras - Our AI uses ordinary camera video. We do not need LIDAR or depth sensors to capture animations, and we get game ready frame rates even on older phones
  • Sandboxed AI - Our AI runs fully on the connected phone: no server queries and no competing with Unreal for desktop or console resources.
  • Multiple rigs - Select from different rig configurations (UE4 Mannequin, MetaHuman, Mixamo) in the Source setup menu. We use a full body game skeleton including fully articulated hands.
  •  Different modes - tailor recording to different phone camera setups, such as Room (full body standing at a distance), Desktop (upper body while seated at your pc), or Portrait (full body in tighter spaces). More modes in development.
  • Convenience options - choose to stream in third-person or mirror mode (i.e. flipped left/right sides), or whether to include lateral/vertical motion in root or in pelvis translation. We also include a LiveLink retargeting subclass which can automatically apply the incoming joint rotations to arbitrarily sized skeletons.

Code Modules:

  •  PoseAILiveLink


Supported Development Platforms: Windows

Supported Target Build Platforms: Windows, plus others forthcoming

Documentation: here

Example Project: here