18. Oktober 2019
Cubic and UE4 create waves in the training and simulation community
If these are valid measures, Cubic Global Defense has passed with flying colors. Cubic develops innovative and realistic training solutions for the United States and allied forces in more than 35 nations. Their track record is based on decades of experience in providing simulation solutions to train warfighters and law enforcement personnel, making the world a safer place.
Having already incorporated visualization and game engines in their training production for over a decade, in 2012 Cubic began hearing whispers of something special in the works from Epic Games.“We had just made some key hires that had experience with UE3, which wasn’t widely adopted in the Department of Defense (DoD) training industry—we had hired from the film and games industry,” explains Andre Balta, VP and CTO at Cubic. “They came in and told us that Unreal 3 was great compared to what we were using, but there’s this Unreal 4 thing coming, and it’s going to be incredible, from what they were hearing.”
And so the team got a very early beta version of UE4, codenamed “Rocket”. The feedback was unanimously positive.
“Every developer that touched that Rocket beta came to me and said, ‘we have to have this now,’” says Balta. “The passion in the eyes of my developers using that Rocket beta, and seeing what it could become—it was hard for me to not do everything I could to give them the tools that they needed,” he continues. “And I mean when I say ‘passion,’ every artist, every engineer, every designer was in my office saying ‘this thing is incredible.’”
Balta and the team continued to evaluate the beta against other game engines available at the time, including UE3. Balta was asked if they were comparing apples to apples. “I said no, an apple versus an orchard, there is no comparison,” he laughs. “And this was just the Rocket beta. If you compare that to where Unreal Engine is today, it’s an apple to forty orchards.”
The team was also excited about Epic’s published roadmap for UE4, and the future directions towards an enterprise offering, along with the strength of the support and the documentation, and the decision was made—they switched their entire production to Unreal Engine 4.
Creating a virtual Littoral Combat Ship—or two!
Since adopting UE4, Cubic has used it for an extensive series of DARPA projects and large-scale exercise control/management applications—something traditionally done only in 2D. But the project that really put their decision to the test, and proved the “orchard to apple” theory, was for the U.S. Navy—to create not one, but two different 400-foot Littoral Combat Ships, each with hundreds of spaces, to train their engineering plant technicians. The team was looking at dealing with around 50,000 assets and 30,000 interactive components. “The scale was incredible in terms of the synthetic environment,” says Balta. “The engine had to be able to handle the scale.”
Not only that, but everything had to be as true-to-life as possible, even down to the lumens on the light bulbs, and the exact color of the paint on the bulkheads. To help accomplish this, the team purchased two phase-based laser scanners from Faro, enabling them to scan the entire ship, and even the uniforms of the characters. The resulting point-cloud data was accurate to within two centimeters, and also contained color information. These point clouds were then taken into 3ds Max or Maya, and converted to meshes and textures, before being brought into UE4.
Balta cites UE4’s support for common 3D formats like FBX as an important aspect of being able to create a robust asset pipeline that integrates popular 3D content creation tools. Once in UE4, the team was able to employ physically-based materials and the photorealistic real-time renderer to accurately recreate the real-life interior and exterior of the ship, together with its crew.
The simulation had to be both visual and physical; not only does the valve or firearm need to be photorealistic, but it must behave like the component does in the real world. If you turn a valve, you need to see the second- and third-order effects, such as leaks and pressure changes on a system.
The scale of the art environment and this simulation model presented challenges. In order to cope with the complexity of the ship, the team employed a level-streaming system, so when the user walks into a space, only that space and the adjacent spaces load.
It took the better part of a year for a large art team to deliver assets for the first order, which required 80% of the ship to be built, before layers of game-based learning principles were added. That first order required the generation of 1,500 hours of immersive content. When you consider that an average AAA video game campaign is 8-15 hours, the enormity of the undertaking becomes clear. The hard work paid off—the client was more than happy with the result, placing several more orders over the last five years, and even going so far as to make the video above to show off the product.
Gamification: making training more effective
Earlier in his career, Balta faced some resistance towards the use of game engines in the DoD training and simulation industry. “There was a constant word that game engines can’t do high-fidelity physics; game engines can’t do high-fidelity simulation; game engines can’t do flight dynamics; game engines can’t handle the pressure curve of a power plant; game engines don’t have the right state machines,” he says. “And to be honest, at the time, the industry wasn’t wrong.”
Back then, game engines did lack those capabilities. “But here comes our experience with Unreal and everything it came with and we realized there was quite a bit there, and if it didn’t have the high-fidelity physics that we needed it was incredibly easy to create modules and plugins that did have the high-fidelity physics,” says Balta.Another hurdle that Cubic needed to overcome was the lack of understanding at the time of the value of using gaming principles in training. So it was up to them to demonstrate how embedding knowledge in an experience is, in fact, a highly effective way to teach. Just as when you play a video game, you don’t have a tutorial to teach you how to use a weapon but instead you learn from the embedded gameplay, the same principles can be applied in training.
“One of the great principles of gamification, one of the things games do very well is they adapt to your experiences,” says Balta. “As you’re going through and you’re leveling up, things level up with you to the right degree. The experience is tailored to your actions. So if you’re really good at something, that thing will become harder for you within that scenario,” he explains. This concept is equally applicable to training; as you begin to master certain situations, more faults or procedures can be injected to further improve your skills.
“I feel that Cubic has really made gamification more digestible to our industry,” says Balta. “I think we have applied it in a way that is just ‘gamey’ enough. While staying faithful to the critical aspects of training, our goal is still driven by the training need. The way Cubic looks at it is similar to the way in which innovative animation studios approached movies, where the desire to create particular looks or effects drove technology, and that new technology made further advancements in visual quality possible. In Cubic’s case, training challenges technology, and technology inspires new forms of training.
“So we looked at commercial game engines, found the best of breed in Unreal 4, that forced us to hire technical creators; technical creators teamed up with traditional learning scientists and figured out how gamification applied to DoD training,” he continues.“It’s kind of this linear track that has happened to our industry. You can’t have a team of instructional designers and software engineers and say ‘go figure out gamification’. You need that technical designer in the middle that speaks both languages. I’d argue that training doesn’t happen without us hiring technical creatives because we had to adopt Unreal 4. So it’s all kind of connected.”
Suit and tie, meet shorts and T-shirt
Human capital is a theme that features prominently in Cubic’s story of transition. At the time of their adoption of UE4, the industry was still very much a highly conservative “suit and tie” environment. But to get the skills they needed, Cubic had to hire from the game and film industries, where team members are more likely to be wearing pajamas than a suit and tie, and certainly nothing smarter than jeans. Bringing these two cultures together was essential to the success of the projects, and one of Cubic’s biggest challenges.
“How do you bring a 3D artist who’s spent most of their career in the games industry into an area where they have to talk to a subject-matter expert who’s a retired veteran that’s spent maybe twenty years in the Navy or the Army? How do you mix those cultures where they’re really organically programmed not to talk to one another?” asks Balta.
The answer lay in compromise. “We had to change our dress requirements from business-casual to more casual; we had to have more flexible work hours; we had to create open-plan offices to support technical creatives and the free flow of communication needed between over 10 different skill sets. But we also had to explain that we do have contracts and requirements and commitments. At the same time when we had customers in, we had to make them comfortable with this environment we’d created.”
Fortunately, the industry is becoming more accepting of the gaming-related aspects of training and simulation, and the culture that comes with it.
“Four or five years ago you had to warn your staff not to say “game,’” says Balta. “We had acquired one studio and built another, we weren’t calling them studios publicly because of the industry. The industry is much more comfortable now—you’ll even see contracts with ‘gaming’ in the title.”
Blueprint: a force for change
There’s another way that the shift to Unreal Engine has impacted Cubic’s organization, and it’s something that really excites Balta, who explains that historically, the DoD industry was really made up of three major skill sets: Business Development, Program Management Office, and Engineering, which included any kind of product development.
“Unreal’s application of Blueprint, having a visual scripting tool that you don’t have to be an engineer to use, you can be a designer or subject-matter expert to use, and create content, has been huge,” he says. “And it’s been huge in two ways. One, it’s really given technical creators a seat at the table with engineers in the DoD industry. The other thing is it’s let veterans who are recently out of the military pick up high-fidelity technology and essentially become game developers, and write code without knowing it, using Blueprint. Unreal 4 has reshaped our industry.”
According to Balta, Unreal has created an engine that really serves all mediums of product development, not just engineering and not just pure art development, putting the focus on creating content and immersive experiences. And that has changed their organization. “We have an extremely large pool of designers now that continue to empathize with customers, and Unreal Engine has had a big impact on that transition we’ve made,” he says.
What does the future hold?
Cubic is not content to rest on its laurels, but continues to look for new ways to use real-time technology to improve training and simulation. Live, virtual and constructive (LVC) training is one of the areas where Balta believes using an engine like UE4 can yield significant benefits.
A great example of this is CATS Metrix, Cubic's Exercise Control software that lets instructors monitor, control, and evaluate live ground training, where soldiers and vehicles are fitted with GPS and laser simulators. Maneuvers, weapon engagements, and communications are monitored and recorded in a control center.
CATS Metrix and UE4 lets instructors visualize the recorded exercise in 3D, enabling the soldiers to evaluate their tactical performance in familiar 'gaming' view. One of the challenges is that the trained soldiers have very high expectations for realism, detail, and authenticity of the virtual display: having been in the real terrain all day exercising means that soldiers compare details from the training area with the virtual terrain. With UE4, Cubic can meet those expectations.
Another application is Cubic’s SYN-ISR product. From providing a synthetic full-motion video feed of major multinational exercises, to supporting selection and training of the nation’s most elite special mission units, Cubic’s SYN-ISR leverages UE4 to develop dynamic and highly realistic training environments. The level of realism has fooled many seasoned operators into believing they were looking at live feeds from a UAV flying live missions overseas. This realism has impressed military leadership and hardened operators, making SYN-ISR an indispensable part of the training and exercise life cycle.
The fidelity and “design first” focus of UE4 also represent a shift to the training continuum. Historically, there were three phases in training: classroom fundamentals (typically Powerpoint); hands-on simulation (such as standing up and grabbing actual hardware in front of a wraparound screen; and actual live training (such as flying a real aircraft or shooting a real weapon). Now, because it’s possible to bring so much virtual content to the classroom thanks to UE4, the trainees’ journey seamlessly transitions across the LVC phases.
Balta is also excited about how they can expand upon the virtual constructive capabilities in the live environment, for example putting an AR headset on someone and letting them practice on virtual and synthetic entities in the field with an actual rifle. He also sees where constructive entities used in live training can use artificial intelligence (AI) to start adapting to individuals’ human performance.“The industry was forced to adopt high-fidelity game engines, that forced us to hire technical creatives, those technical creatives solved the gamification problem, and now we’re extending the principle of the gamification to live training,” says Balta. “So there’s this linear chain that’s happening that’s been beautiful. I’ve been in the industry just under a decade and fortunate to be pushing quite a bit of this and it’s been exciting to see that transition.”
Interested in finding out what UE4 can do for your training and simulation requirements? Get in touch and we’d love to start that conversation.