April 9, 2019
Pixel Streaming: delivering high-quality UE4 content to any device, anywhere
Available for Beta since Unreal Engine 4.21, and with a new sample project to get you started quickly included in Unreal Engine 4.22, Pixel Streaming enables you to stream the highest-quality content to almost any web browser on any platform—even mobile devices—with zero download and zero install requirements. It’s no more complicated for the participant than accessing a YouTube video. But, unlike YouTube, they can then interact with that content via keyboard, mouse, touch events, or custom HTML5 UI, and send those responses back to the engine.
Pixel Streaming makes it possible for you to deliver applications such as product configurators, on-site architectural design reviews, or interactive training anywhere a modern web browser can run. You can enable multiple viewers to collaborate in a single session by simply sharing a link, or you can send each connecting viewer to their own separate session.
How does Pixel Streaming work?
Until now, the device on which your end user consumed your real-time content (whether it’s a desktop PC, console, tablet, or smartphone) was typically the same device that ran the gameplay logic and rendered the results to the screen. Because of this, the performance and quality of the content were limited to the capabilities of the device.However, with Pixel Streaming, you can run your Unreal Engine application on a powerful remote computer (either in the cloud or as a local server), taking advantage of all its resources—CPU, GPU, memory, and so on—to execute the game logic and render every frame in real time. End users can then use standard web browsers on their own computer, tablet, or smartphone to connect to that running game application through a lightweight stack of web services.
What’s involved under the covers?
Despite its power, a typical Pixel Streaming setup is quite simple, consisting of just three components that connect Unreal Engine to the viewing platform—relatively easy for anyone to set up inside a local network. However, it's powerful enough for teams with experience in deploying web services to use as a basis for creating custom cloud-hosted platforms.To achieve the lowest possible latency between the viewer and the Unreal Engine application, the system uses a WebRTC peer-to-peer communication framework.
- Pixel Streaming plugin - This plugin runs inside Unreal Engine. It encodes the final results of every rendered frame using H.264 video compression, packs those video frames along with the game audio into a media stream, and sends it to the WebRTC Proxy Server.
- WebRTC Proxy Server - The WebRTC Proxy Server is responsible for relaying the media stream produced by the Pixel Streaming plugin to multiple viewers, over direct peer-to-peer connections.
- Signaling and Web Server - The Signaling and Web Server is responsible for negotiating connections between viewers and the WebRTC Proxy Server, and for providing viewers with the HTML and JavaScript environment that plays back the media stream.
You can read more about the underlying technology of Pixel Streaming in the Unreal Engine documentation.
What makes this a great idea?
The benefits of such a system are many:- Mobile devices and lightweight web browsers can display workstation-quality 3D graphics—much higher-quality graphics than otherwise possible. They can show complex scenes at high frame rates, using rendering features that rely on a powerful GPU.
- The time from loading the web page to interacting with the experience can be less than a second (try doing that with a WebGL approach). It’s well known that delays are a major reason for abandoning a web page (according to Kissmetrics, 40% gave up when delays were three seconds or more).
- The experience is predictable. You know that whatever device they’re consuming your content on, they’re seeing the same image you’re seeing. You don’t have to worry about video drivers or GPU limitations causing visual problems.
- It’s a “zero install” solution. The end user doesn't need to download plugins, large executables, or content files in advance, and doesn't need to install anything. The only thing the viewer is consuming is a video stream—just like any other video stream. And unlike WebGL, your 3D data is not sitting on someone else's hard drive, which can be a security concern for some firms.
- From a developer perspective, this is a massively easier solution to support than porting an experience to all conceivable mobile and browser configurations. What if you need to make a change? Just change it in one place and everyone instantly gets the new experience. You package your application once for Windows, and end users can choose to experience your content on any platform, and in any modern browser that supports the WebRTC connection model, which includes Google Chrome and Mozilla Firefox on desktop, iOS, and Android.
Ready to give it a try? If you haven’t already done so, download Unreal Engine 4.22 from your Epic Launcher, and enable the Pixel Streaming plugin. To get up to speed quickly, download the new Pixel Streaming Demo sample project from the Learn tab of the Launcher; you’ll find it in the Engine Feature Samples section, as shown below. You can also visit the documentation for full details on getting started with Pixel Streaming.
If you’re new to Unreal Engine altogether, you can sign up here. We hope you find Pixel Streaming useful and enjoyable to use.