Dr. Ding Zhao, director of the Safe AI lab at Carnegie Mellon University, has founded Safeworld alongside seasoned startup executive Kyle Wong and machine learning engineer Simo Rachidi. The company is focused on making generative AI-driven robots predictable and safe enough to deploy in real-world environments.
Safeworld is emerging from stealth with a seed round of more than $12 million, led by Shine Capital and a16z Speedrun, with additional investment from Box Group, Carnegie Mellon University Endowment, Innovation Endeavors, and SV Angel. Dr. Zhao highlights the dual nature of this challenge: “The safety challenge that we’re talking about is a combination of, one, really advanced generative AI probabilistic evals – how do you underwrite the risk of a probabilistic system? The second part that’s really hard is the trust part, and you need both to deploy a robot.”
As AI robots start to integrate into various environments, establishing industry safety standards becomes essential. Jonathan Lai from a16z Speedrun stresses the urgency: “The time to build an industry safety standard is now while robots are being designed and deployed. By the time you have robots in households colliding with kids and causing safety incidents, that’s way too late.”
Safeworld is innovating by developing simulations that assess robotic control systems in environments filled with realistic human models. This requires a thorough understanding of how robots will react to unpredictable human behavior. For example, Wong raises a common safety concern: “What is the speed or what is the stopping distance that you need to make sure that this robot will not collide with a particular human?”
To address such questions, Safeworld will digitally recreate various scenarios, including complex situations like blind corners in factories. The simulation process involves running thousands of scenarios where human models interact with the robots. However, as Dr. Zhao points out, the unpredictable nature of humans complicates this task. “Tripping and falling is also a good example of something that we do a lot of testing with the simulation,” Wong adds, highlighting the need for rigorous testing without real-world risks.
Current robot safety testing methods often lack verification through formal mathematical proofs. Vishal Dugar, CTO of Gritt Robotics, emphasizes this challenge: “The difficulty with most of our systems is it’s very hard to formally prove it by doing some math, writing some equations, and saying yeah, the system is verified to be safe.” His company is collaborating with Safeworld to ensure their robots can safely operate alongside human workers in complex environments.
Dugar spells out the full scope: “Humans have many kinds of appearances. Their bodies can be in different configurations. They could be kneeling, standing. They could be tripping and falling potentially. They could be crouching. They could be running. You have to respond to all these behaviors that humans could potentially exhibit on these sites, along with the variety of variations in human appearance, you know, clothes, size, shape, height, skin color, everything else.”
As Safeworld moves forward, it recognizes that the field of generative AI in robotics is still in its early stages. The company is currently refining its business model, weighing the option of offering a platform for external users against adopting a service-based approach. Despite being in the early days, the team is confident in their mission. Dr. Zhao states, “We’ll probably be the first profitable company in this field. Because if anyone wants to deploy, they need to pay us to handle the situation.”
Establishing robust safety standards for AI robots is crucial as they begin to interact with unpredictable human behavior in real-world environments. With the urgency highlighted by industry leaders, the effectiveness of Safeworld’s simulations and testing methods will play a significant role in determining how well these robots can adapt and operate safely beside people. The challenge lies not just in technological advancement, but also in gaining public trust through transparent and verifiable safety measures.
