November 11, 2018

Unreal Engine preserves New Zealand culture with hyperreal imagery

By Ken Pimentel

UPDATE: The Homestead is now available.

When New Zealander Simon Che de Boer started his first virtual reality project, his goal was modest—he wanted to re-experience the home he and his daughter had lost in a fire.

But this small personal project quickly turned into a much larger goal: to present large-scale environments down to the tiniest detail in virtual reality. For this reason, he founded RealityVirtual to focus on “encapsulating real-world environments for slice-of-life VR experiences.” RealityVirtual uses Unreal Engine for real-time playback of dense, hyper-real VR scenes.
For Che de Boer, the drive to recreate reality at a fine-grained level is both cultural and personal. With only 5% vision, Che de Boer is legally blind. This means that experiencing fine detail in real life, whether on a sculpture, an intricate natural rock formation, or an architectural wonder, requires him to get very close, an inch or two away, to even see it.

Initially, the thrust for Che de Boer’s VR projects was that he wanted to be able to experience the environment up close at his own pace. The results were VR worlds where you can press your virtual nose against a virtual object and still see it at the level of realism you’d experience in real life, giving a truly immersive experience. 

Putting the Reality in VR

To withstand such close scrutiny, the model and textures need to have an extremely high level of detail, including representation of tactile surfaces like rough stone at a micro level. To duplicate a true real-world experience in VR, RealityVirtual uses a combination of 3D scanning and high-resolution photos to produce environments that are “as close as possible to infinite detail,” says Che de Boer. “Everything is just overkill beyond belief.”
Blog-body-img2.jpg Simon Che de Boer captures high-resolution photos at the Sir James Wallace Homestead.

High-resolution photos capture more detail than the naked eye can see, giving RealityVirtual the resources to create a VR world with more apparent detail than the real world. For example, in creating a VR experience of an art gallery at the Sir James Wallace Homestead in Auckland, NZ, RealityVirtual’s team used high-resolution photos to accurately capture and depict the fine detail in each painting. As the VR viewer draws near, the thickness, texture, and shininess of the paint itself becomes apparent, just as it would in real life. “The artwork is down to the fibers on the canvas,” says Che de Boer. “It’s a testament to the power of Unreal Engine that this can all be rendered at 45 to 90 frames per second.”
Sir James Wallace Homestead virtual reality experience, rendered in real time.

Che de Boer’s also wants to ensure that important parts of New Zealand culture, from landscapes to architectural icons, are preserved for future generations. And in the meantime, anyone can experience them through VR at a level of fidelity that’s as good as, if not better than, the real thing.

Unreal Engine Makes Real-Time Reproduction Possible

When Che de Boer set out to create VR, Unreal Engine was a natural choice. “It was just common sense for me to go with Unreal Engine,” he says, “quite simply because it was just the easiest thing for me to pick up and use straight away.”

At the heart of RealityVirtual’s process is photogrammetry, where high-resolution photos are used to capture a level of detail not detectable by the naked eye. 3D scanning is also used to capture billions of data points, which are then distilled down to millions.

With these large datasets, speed of processing and playback is key. “Everything we’re doing with Unreal Engine is just on a massive scale,” says Che de Boer. He adds that NVIDIA’s new architecture adds a level of speed and power that’s “hugely exciting.” With this enormous amount of data, RealityVirtual’s finished scenes reach a playback speed of 90fps at 1080p, even when volumetric fog and planar reflections are used.

Between these two technologies, RealityVirtual is able to provide real-time immersive experiences of these densely textured environments at playback rates that provide a truly immersive experience. “Without Unreal Engine, without NVIDIA, we’d be lost as to how we could go ahead and make this thing a reality,” says Che de Boer.

Preserving Culture with Hyper-Real Immersive Experiences

RealityVirtual’s other VR projects include both natural and man-made New Zealand attractions, including a breathtaking Mana beach and St. Matthew-in-the-City church. Che de Boer plans to create hyper-real VR experiences of many more environments and landmarks in the years to come, not only for cultural preservation but for immersive experiences that feel like “something that you revisit as a memory.” 
Beach cove in Mana, NZ reconstructed from scans and photos, and rendered in real time as part of a VR experience.

“Personally, I actually really appreciate the detail, which in the real world is actually difficult for me,” says Che de Boer. “I literally want to scan the Mona Lisa so I can actually go up and look at the Mona Lisa.”

Che de Boer hopes that RealityVirtual’s work encourages more and better content creation and cultural preservation. “We need to be capturing and actually allow people to really experience things for what they are,” he says. With the power of NVIDIA and Unreal Engine technology, Che de Boer feels these goals are within reach. 

“When you’re striving for realism, you need Unreal,” quips Che de Boer, “which is kind of ironic. You should just call it ‘Real’.”
Building the Sir James Wallace Homestead VR experience with Unreal Engine.

Experience the Reality of Unreal

Want to preserve a cultural icon, or create your own VR experiences? Download Unreal Engine today.