For robots to truly be able to navigate in the real world, the software that powers them needs to be reliable enough to enter any environment and accurately navigate. Otherwise, you’re left with a robot that needs to be reprogrammed when it goes from a forest to a field of grass. This is the kind of work Duality AI is doing in its new agreement with NASA’s Jet Propulsion Laboratory, or JPL, in Pasadena, Calif.
The San Mateo, Calif.-based company today said it will continue its work on the Defense Advanced Research Projects Agency (DARPA) Robotic Autonomy in Complex Environments with Resiliency (RACER) program.
This announcement followed Phase 1 of the RACER project, which Duality completed as one of only two simulation providers selected by DARPA. The company said this latest agreement accelerates development of autonomy for NASA JPL by allowing rapid iteration and discovery of flaws through simulation and reducing risks. It will also provide the synthetic data vital to training advanced AI algorithms that push the boundaries of robotic autonomy.
“For Phase 1, we were actually part of RACER Sim,” Michael Taylor, Duality’s chief product officer, told The Robot Report. “So RACER Sim was a sister effort. We were a prime, and the other team was led by Intel. RACER Sim was envisioned to push forward the state of the art for simulation technologies for building off-road robots. DARPA recognized the gaps there.”
During the second phase of the RACER program, teams must demonstrate their ability to create advanced AI systems that allow large-scale vehicles to travel autonomously at faster speeds and over longer distances than in Phase 1.
The team from NASA JPL will use Duality’s advanced digital twin simulator, Falcon, to overcome the challenges associated with autonomous high-speed, off-road vehicles, including mapping, perception, planning, and responding to hazards and obstacles.
“As you’re dealing with more and more dynamic environments, at some point, more traditional approaches don’t work,” said Apurva Shah, CEO of Duality. “You really have to look at AI-enabled approaches, how they scale, and also remain very robust and predictable and resilient, which is very much a focus of the RACER program.”
Duality to train AI in Phase 2
As part of the agreement, Duality will provide the tools and processes required to train AI through the use of simulation and aid in the development of NASA JPL’s next generation of autonomy technologies. The contract also provides JPL with access to expansive, highly realistic environments necessary for robust software testing and training.
“As RACER moved into Phase 2, we actually just became part of the JPL team itself,” said Taylor. “RACER was about pushing technology forward and showing it could be useful, and now for Phase 2, it’s ‘Let’s go make it useful and directly help the JPL team on their needs as they’re progressing through this project.”.
The digital twins of real locations enable the NASA JPL team to generate relevant synthetic data and test the autonomy software with a higher percentage of code coverage than with conventional simulators, said Duality. The company said its AI-powered tools can build virtual environments from satellite imagery, as well as its library of configurable virtual sensors.
As a result, NASA JPL can manipulate environments with digital twins of various rocks, trees, weather, and more. The lab said this level of precise control helps its transition cutting-edge ideas from simulation into the real world.
This partnership has been valuable not only for NASA JPL, but also for the advancement of Duality’s technology. Taylor said that during the first phase, Duality focused on advancing its technology in several areas with the hope of developing large-scale, performant, and photorealistic environments. Phase 2 will likely only open more doors for the company, he said.
“I think the JPL partnership has actually been really, really valuable, both to mature Falcon, because we believe, at the end of the day, Falcon is something that brings customer value, so JPL has been incredible in that way, to work with,” said Shah. “And also, just as humans, they’re just nice, wonderful humans.”
“For the robot nerd in me, it’s been a real pleasure to be able to work with a team like this,” commented Taylor. “RACER is, I would argue, hands down, the most interesting and challenging off-road autonomy public thing that we know about happening today.”
Duality says the technology it has developed during the RACER program is already being used by some of its customers. | Source: Duality AI
What makes Falcon unique?
Shah and Taylor both bring very different backgrounds to Duality AI. Taylor led field robotics at Caterpillar for more than a decade before Duality, while Shah spent much of his career at Pixar, creating virtual worlds for movies.
Shah said Taylor approached him and said his experience with virtual worlds could be applied to robotics.
Shah explained that, even in animated movies, “there’s a physics to the world. And if you were to break that physics, it breaks your sense of reality, and it’ll pop you out of the movie.”
Although movie makers might cut corners for artistic or stylistic reasons, this isn’t the case when creating virtual worlds for robots.
“We’ve taken a lot of the tools and techniques that have been refined in filmmaking over the last three or four decades and are applying them to creating simulations, and, more importantly, simulation data that is actually relevant in an engineering context,” he said.
“From an AI perspective, we are most interested in embodied AI models,” he said. “So not AI models that are kind of running on the Internet, but AI models that are supposed to be embodied within the real world, the physical world.”
“It will be sensing [the environment] and then responding within a closed loop,” added Shah. “And that is really what makes our Falcon platform, unique. From the very beginning, we use digital twins as a building block.”
Once Duality has all these building blocks, he said, it can put them together in whatever way it likes or works for its customers.
Falcon generates high-fidelity digital twins for development of offroad autonomous vehicles. Source: Duality AI
Video games help harden system
The company also has a partnership with Epic Games. Shah and Taylor say the video game industry provides a host of research into creating virtual worlds. Duality simply needs to fill the gap between what’s used in media now and what’s needed for robotics, Shah said.
“We leverage the literal billions of dollars that the game industry has put into developing and hardening and fielding and maintaining a lot of these technologies, but we don’t require that our users leverage them,” said Taylor. “If you want to bring in a model that you have in MATLAB that’s great, co-simulate with Falcon.”
“Use Falcon for what you want to use it for, and bring in MATLAB for the physics,” he added. “So you have the flexibility to choose the components that you need and the approach that works for your team.”
While the RACER program is limited to off-road vehicles, Duality’s Falcon isn’t.
“If you look at us as a company, we have customers that fly in the air, that crawl on the ground, and that jump in the ocean,” Taylor said. “So our customer base and our software supports all that.”
