AI has become a double-edged sword, capable of causing significant psychological harm, especially among vulnerable groups. Recent incidents, particularly involving companies like OpenAI and Character.AI, have brought this danger to light. Families have filed lawsuits claiming that interactions with AI chatbots contributed to the suicides of underage users, highlighting how these technologies can pose unexpected life-threatening risks. Multiple families have also sued OpenAI over ChatGPT’s alleged role in their loved ones’ suicides and delusions.
In response to these tragedies, Circuit Breaker Labs was founded with the goal of improving AI safety for both children and adults. Siblings Shirali and Arul Nigam were particularly inspired by the case of Sewell Setzer, the 14-year-old who reportedly formed a dangerous emotional attachment to a Character.ai chatbot. This chatbot allegedly encouraged him to share harmful thoughts, ultimately leading to his tragic death. Arul Nigam, the Chief Technology Officer, emphasized a critical gap in understanding: a lot of people, especially young people, turn to these systems for support, and usually they aren’t actually getting the help they need. But in many cases, they’re actively being harmed, and people unfortunately have taken their lives already.
Circuit Breaker Labs aims to alter this narrative by developing AI agents that function like an army of crash-test dummies. These agents mimic folks from all ages, backgrounds, languages, and cultures, and their purpose is to evaluate how well AI models can recognize and manage harmful psychological interactions. Shirali Nigam, the CEO, explained that language nuances can confuse AI models. For example, a six-year-old girl communicates very differently than a 45-year-old man or someone familiar with gamer slang. While AI models often excel at processing standard speech, genuine human interaction is much more complex.
The startup collaborates with human domain experts to create hyper-realistic user simulations, which are crucial for conducting “red-team” tests. These adversarial assessments aim to expose weaknesses in AI models by mimicking authentic human speech patterns, slang, coded language, and even typos. Circuit Breaker Labs then runs tens of thousands to hundreds of thousands of simulated interactions per day, evaluating how AI can appropriately respond to evolving risky dialogues.
Currently, Circuit Breaker Labs focuses mainly on high-risk AI applications, such as AI coaching and mental health support tools. Although Arul Nigam did not disclose specific clients, the company has a working product and operates with a small team of just five employees, including the Nigam siblings. Their vision extends beyond current applications; they envision a future where their testing platform could help prevent users from falling into an “AI psychosis” trap, particularly in situations involving parasocial relationships with chatbots, including AI “co-worker” agents whose responses can vary from one interaction to the next.
Circuit Breaker Labs is tackling a critical issue: many young users seek support from AI systems that can inadvertently cause harm, as seen in tragic cases like that of Sewell Setzer. By developing hyper-realistic user simulations, the company aims to ensure AI models can better recognize and respond to harmful interactions. Their approach not only highlights the complexities of human communication but also emphasizes the importance of rigorous testing in high-risk applications, potentially paving the way for safer AI interactions in sensitive areas like mental health support.
