Software engineers are increasingly unable to make sense of the code AI systems produce, according to Jeremy Nixon, founder of chip optimization software firm Infinity. Nixon describes the role shifting from building products from scratch to overseeing an AI that cobbles together code – and engineers often can’t follow what it’s doing.
Studies indicate that the widespread adoption of AI can result in cognitive decline and skill atrophy, even among computer scientists. Developers are drifting further from writing code and learning the skills needed to architect larger projects. Even staff at frontier AI labs are troubled: Jordan Nanos from SemiAnalysis pointed out that OpenAI engineers found it difficult to understand the code for their own graphics processing unit kernel, a crucial part of their AI training workflows. Jordan Nanos from SemiAnalysis pointed out that even OpenAI engineers found it difficult to understand the code for their own graphics processing unit kernel, a crucial part of their AI training workflows.
Nanos stressed that the engineers involved were knowledgeable about hardware and system concepts, yet they failed to grasp what the AI-generated code actually accomplished. He remarked, “The AI understands it, the AI tests it, and you see the results. It produces correct kernels that perform really well.” One real danger is that errors or hallucinations slip through as human oversight fades.
This concern was highlighted when Amazon had to implement a 90-day “code safety reset” after outages that affected customer orders. While the company refuted claims that AI was to blame, internal notes revealed apprehensions about the “high blast radius” of changes made with AI assistance. A recent analysis from software company Undo found that 35 percent of AI-generated code had not been fully comprehended by engineering teams before they pushed it to production.
Moreover, 29 percent of software engineers reported a drop in productivity due to frequently having to “unpick” AI-generated code, with an astonishing 94 percent encountering such incidents at least once in the last six months. This growing amount of time spent deciphering AI outputs raises questions about whether the use of these technologies is more advantageous than problematic.
Experts like Nanos view this not as incompetence but as a paradigm shift in software development. Sergey Cleftsow, an AI researcher, argued that developers now only need to define the architecture and correctness criteria, while the neural network handles the rest.
Concerns are also rising about the new generation of engineers who are being trained in an environment that diminishes the importance of understanding code. This trend could have extensive implications for a future that is becoming more digital, where AI takes center stage in development processes.
