Unreal Engine Marketplace is moving over to Fab in mid-October. Learn more.

Microsoft OpenXR

Microsoft Corp - Code Plugins - Nov 24, 2020

Microsoft OpenXR extensions providing additional functionality for mixed reality applications

  • Supported Platforms
  • Supported Engine Versions
    4.26 - 4.27, 5.0 - 5.1
  • Download Type
    Engine Plugin
    This product contains a code plugin, complete with pre-built binaries and all its source code that integrates with Unreal Engine, which can be installed to an engine version of your choice then enabled on a per-project basis.

The Microsoft OpenXR plugin contains a set of OpenXR extensions unlocking mixed reality-specific features from Microsoft. If you're developing an experience targeting OpenXR for HoloLens 2 or Windows Mixed Reality VR headsets, you'll need this plugin to access the following features:

  • Spatial mapping
  • Scene Understanding (requires 4.27.1+)
  • Spatial anchors
  • Holographic remoting
  • Voice input
  • Hand mesh
  • System keyboard
  • Camera
  • QR code tracking
  • Azure Spatial Anchors
  • Secondary View Configuration (requires 4.26.2+)
  • Azure Object Anchors (requires plugin version 1.1.11+)


Instructions for adding the Microsoft OpenXR plugin to your project can be found in the README of the plugin's GitHub repo. The GitHub repo also contains source code for the plugin, a sample HoloLens 2 project built for OpenXR, and a place to file any issues/feedback.


All plugin features are accessible through both Blueprints and C++.  For documentation on how to access these features, visit Microsoft's Mixed Reality documentation.


Note: This plugin works alongside the built-in OpenXR plugins in Unreal Engine 4.26, which provide functionality for eye tracking, hand joint tracking, and hand gesture and controller input action mapping. Enabling the Microsoft OpenXR plugin in your project will automatically enable the built-in plugins required for an OpenXR experience to run on HoloLens 2: OpenXR, OpenXREyeTracker, OpenXRHandTracking, OpenXRMsftHandInteraction, XRVisualization, and HoloLens.

Technical Details

Features:

  • Hand Mesh - Renders a real-time mesh over the user's hands.
  • Remoting - Streams an app from the editor to HoloLens.
  • Holographic Window Attachment - Guarantees only a single core window is created on HoloLens to route focus to the correct window.
  • Locatable Camera - Gets world space transforms from the PV camera and pixels in the camera frame.
  • PV (Photo/Video) Camera - Gets a live feed from the HoloLens camera.
  • QR - Gets world space transforms of QR codes scanned by the mixed reality device.
  • Secondary View Configuration (requires 4.26.2+) - Aligns holograms with the real world in Mixed Reality Capture images and videos.
  • Spatial Anchors/Azure Spatial Anchors - Integrates ARPins with HoloLens anchors to lock content to a real-world transform that persists between app launches.
  • Spatial Mapping - Adds world mesh data to the MRMesh
  • Scene Understanding (requires 1.1.7+ on UE 4.27.1+) - Adds wall, floor, ceiling, and table planes to the MRMesh
  • Speech - Reacts to speech events added in the project settings' Speech Mappings.
  • Azure Object Anchors - Detect objects in the world from a 3D model (requires 1.1.11+)


Documentation: Unreal Development Overview - Mixed Reality | Microsoft Docs

Example Project: Microsoft-OpenXR-Unreal/MsftOpenXRGame at main · microsoft/Microsoft-OpenXR-Unreal (github.com)