Preserving European royal heritage with RealityCapture and Unreal Engine
February 28, 2023
To prepare for the future, we need to learn from the past. Ensuring the preservation of important global heritage items, from physical sites to artifacts and personal collections, is critical in helping the next generation access and understand the world’s history. Today’s technology provides the perfect solution: digitization.
Organizations like Slovakia-based Artzenal, which delivers a full range of digital art services from early 2D drafts to high-quality 3D content and character art, are embarking on cultural preservation projects to help safeguard this history.
Organizations like Slovakia-based Artzenal, which delivers a full range of digital art services from early 2D drafts to high-quality 3D content and character art, are embarking on cultural preservation projects to help safeguard this history.
Empress Sisi of Austria, Queen of Hungary
Artzenal’s latest project used RealityCapture and Unreal Engine to digitally recreate the private collection of Empress Elisabeth of Austria. Alongside the collection—which includes furniture, art, clothing, and other personal effects—the team recreated her chambers at Schönbrunn Palace and the Hofburg in Vienna as they would have looked in the late 19th century.Born as Duchess Elisabeth of Bavaria, Sisi, as she was known, became Empress of Austria in 1854 after marrying Emperor Franz Joseph. In 1867, she also became Queen of Hungary thanks to her part in the formation of the Austria-Hungary dual monarchy.
Austria’s longest reigning empress, Sisi was considered to be one of the most beautiful women in the world and was renowned for her rebellious, free-spirited nature. And more than 100 years later, she is still the subject of fascination thanks to her many accomplishments and achievements, all of which were highly unusual for a woman in the late 1800s.
Capturing history
From its initial conception, the motivation behind this project was to deliver an interactive experience with an educational element—a medium that has become known as “edutainment”—offering insights into periods of the empress’ life.The official pilot project is a partnership between Schloss Schönbrunn, which owns the collection and curates the exhibition of it; Germany-based licensing specialist Culture XR; and Artzenal, who executed and developed the digital solution using a combination of several new technologies.
To digitize the collection and recreate the imperial chambers, the team required high-quality photogrammetry tools and the ability to build lifelike 3D environments in minute detail that could be experienced in real time.
RealityCapture is a state-of-the-art photogrammetry solution that enables you to automatically create highly detailed textured 3D meshes from photographs, LiDAR scans, or a combination of the two. Artzenal had been using RealityCapture for several years prior to embarking on the Empress Sisi project, producing high-quality AAA content for game and film productions, and knew the software could handle the detail-oriented precision required of a major heritage project.
“Our long-time experience of using RealityCapture made it an easy choice for photogrammetry, and in our opinion it’s the best software available for recreating larger assets that may require thousands of photos each,” says Milan Kubinec, Business Development and Marketing Manager at Artzenal, who adds that the team perceives the results to be the highest quality on the market, as well as valuing the speed of the entire process and the possibility of automation through CLI (command line interface) scripting.
The Artzenal team used a combination of LiDAR scanning for the rooms, and photogrammetry for the assets. Starting off by scanning the entire environment with LiDAR, the team then captured every unique piece of furniture, art, and clothing down to the smallest detail using high-resolution cameras and, in some cases, macro lenses. For shiny objects, cross-polarization was used when possible and necessary. In addition, they took clean, one-shot photos to create tileable materials for use in larger areas in Unreal Engine.
The Artzenal team used a combination of LiDAR scanning for the rooms, and photogrammetry for the assets. Starting off by scanning the entire environment with LiDAR, the team then captured every unique piece of furniture, art, and clothing down to the smallest detail using high-resolution cameras and, in some cases, macro lenses. For shiny objects, cross-polarization was used when possible and necessary. In addition, they took clean, one-shot photos to create tileable materials for use in larger areas in Unreal Engine.
“The process of photogrammetry consists of shooting the entire object and its environment in a way that every square centimeter is covered, with emphasis on all of the details and specific materials (which require more images) in order to achieve the most realistic and high-quality results,” says Milan. “The scanning phase has certain steps that need to be followed. We start off with shooting the scale for reference, then the color checker, followed by the scanning itself.”
“You can immerse yourself in the late 19th-century Schönbrunn Palace in the actual setup from Empress Sisi’s lifetime. From the positioning of the furniture, environment, and building structure, to the ability to browse her personal items, the results are truly breathtaking.”
Creating ultra-realistic environments
With the 3D meshes automatically constructed from this data by RealityCapture, they could be brought into Unreal Engine to recreate rooms from Empress Sisi’s homes as they would have been when she lived there. “Unreal Engine’s unmatched capabilities in achieving realism enabled us to breathe life into our digitized content,” says Milan. “It ensures the assets look as authentic as they do in their natural habitat.“You can immerse yourself in the late 19th-century Schönbrunn Palace in the actual setup from Empress Sisi’s lifetime. From the positioning of the furniture, environment, and building structure, to the ability to browse her personal items, the results are truly breathtaking.”
As a project, it wasn’t always a straightforward process. The team encountered unexpected challenges, not from the technology, but from the environment.
“We were really surprised by the strict rules imposed,” explains Milan. “We weren’t allowed to use flashes on the vast majority of the assets we captured, we were never left unattended, and there was absolutely no food and drink allowed on set, for fear of contamination. We actually had to shoot some of the assets via a glass cage. Without RealityCapture and Unreal Engine, the quality of what we would have been able to take would have been unusable in terms of lighting, reflections, and color matching elements.”
The VR show debuted at the LEAP 2023 conference in Riyadh, where it provided thousands of visitors with their first experience of virtual tourism as they walked through the digitized rooms and interacted with objects from the collection.
Artzenal has firmly embedded these tools into its workflows, and is now using the technology in all of its projects. And with the latest features in Unreal Engine 5, such as Nanite virtualized micropolygon geometry and Lumen dynamic global illumination and reflections, the optimization process has been drastically improved, cutting down processing time from months to weeks, and achieving photorealism in real time is even easier.
“Using this potent combination of technology, the essence of reality is truly captured,” says Milan. “It enables us not only to create a high-fidelity copy of the asset in its current state, but to reimagine it in its prime glory. Using RealityCapture and Unreal Engine together creates a cinematic atmosphere that promotes storytelling and takes our scanned data to another level.”
“We were really surprised by the strict rules imposed,” explains Milan. “We weren’t allowed to use flashes on the vast majority of the assets we captured, we were never left unattended, and there was absolutely no food and drink allowed on set, for fear of contamination. We actually had to shoot some of the assets via a glass cage. Without RealityCapture and Unreal Engine, the quality of what we would have been able to take would have been unusable in terms of lighting, reflections, and color matching elements.”
Capturing the future: RealityCapture and Unreal Engine
With the combination of RealityCapture and Unreal Engine, Artzenal was able to generate what the team feels are some of the most lifelike scenes currently available in the industry, producing a rich resource of interactive, adjustable content in the process.The VR show debuted at the LEAP 2023 conference in Riyadh, where it provided thousands of visitors with their first experience of virtual tourism as they walked through the digitized rooms and interacted with objects from the collection.
Artzenal has firmly embedded these tools into its workflows, and is now using the technology in all of its projects. And with the latest features in Unreal Engine 5, such as Nanite virtualized micropolygon geometry and Lumen dynamic global illumination and reflections, the optimization process has been drastically improved, cutting down processing time from months to weeks, and achieving photorealism in real time is even easier.
“Using this potent combination of technology, the essence of reality is truly captured,” says Milan. “It enables us not only to create a high-fidelity copy of the asset in its current state, but to reimagine it in its prime glory. Using RealityCapture and Unreal Engine together creates a cinematic atmosphere that promotes storytelling and takes our scanned data to another level.”
Find out more about RealityCapture, Unreal Engine, and other tools in the Epic Games ecosystem here.
Get RealityCapture today!
Get the state-of-the-art photogrammetry software solution that’s changing the industry. With RealityCapture, you can create virtual reality scenes, textured 3D meshes, orthographic projections, geo-referenced maps, and much more from images and/or laser scans—completely automatically.