In recent weeks, concern over the potential dangers of advanced artificial intelligence intensified after former OpenAI and Anthropic researcher Jacob Coxon resigned, warning that AI companies are racing toward self-improving superintelligence without adequate safeguards. His warning came as prominent AI executives, including Dario Amodei, Sam Altman and Elon Musk, have acknowledged serious risks from increasingly powerful AI. Adding to those concerns, OpenAI disclosed incidents in which experimental AI agents concealed mistakes, fabricated information, sought unauthorized access and attempted to evade human controls—behavior the company described as a “warning shot” about the challenges of keeping more capable AI systems under control.
Although there’s been growing nervousness and concern in recent years about the potential of AI to cause harm, recently revealed behavior of AI agents deceiving humans, bypassing guardrails, and sophisticated AI collaboration to hack external networks, has triggered more focused public debate on the need to slow down development and impose regulation.
Between The Lines’ Scott Harris spoke with Miranda Bogen, director of the Center for Democracy and Technology’s AI governance lab and chief technologist, who assesses the potential danger of increasingly powerful AI systems and growing need for AI regulatory legislation at both the U.S. federal level and internationally.
MIRANDA BOGEN: Well, what I can tell you is that folks who are close to the companies developing this technology are worried about what they term in a technical phrase as “recursive self-improvement,” which is when AI models will conduct their own research to continue to improve their capabilities, etc. Myself and my organization haven’t been focused on that particular threat model because we are really thinking about what are people’s experiences today with the technology that’s already in their lives and being imposed on them. And in both cases, what we see is that often the decisions to move forward with the technology or to engage in certain types of training activities in technical development are just made very, very quickly. People are feeling pressure to move quickly, and that doesn’t typically lead enough time to think about is this something we want to do? Have we mitigated the risks of this?
And is this a future that we want to walk into? And that’s what we’re hearing the calls from leaders today saying that we would like to slow down because we don’t feel like we’ve been able to do that work and moving quickly at this point seems like it’s a bigger risk than a benefit. And I think that’s long been the case and hopefully the developers, the leading developers will step back and say, “What are we actually building for and is this the right move?”
SCOTT HARRIS: One of the key issues being discussed now, and again, it’s good there’s a focus on this, is the need for regulation of AI development. Even the owners of the AI companies are supporting some form of government regulation. However, they’re not being very clear about what kind of regulation they support and whether it’s more PR or not. In your view, how urgent is the call for regulation and what form do you believe it should take? Sen. Bernie Sanders of Vermont had some proposals recently on regulating AI. While President Trump says that the threat posed by AI is a hoax, we’ve heard that about a lot of threats to humanity from President Trump. But Bernie Sanders, I know specifically wanted to ban super intelligence AI.
MIRANDA BOGEN: There’s no silver bullet to addressing the risks posed by AI, and I think people wish there were, but what we do need is to change the incentives that shape the behavior of the developers of this technology. And what that means is creating the conditions where they have more time to do safety testing, where they are ensuring that the benefits of the technology that they’re building and of the trillions of dollars that they stand to gain are going to actually go back to the public and that the cost of getting to a point of very advanced technology doesn’t outweigh those benefits. But right now, we’re not seeing that. And so what I think is necessary from a regulatory perspective is increasing the robustness of a third-party testing ecosystem. Right now, we have to take the word of the developers that what they’re building is safe, and I think we’re past that.
Even businesses of all kinds get audited and scrutinized by third-party inspectors and testers. And if this is going to be a very disruptive technology, it seems that that would be warranted. So we’re hearing some calls from the leaders of the large AI companies for embedded auditors, embedded evaluators to be watching what the companies are doing. That’s a good start, but it is still in their view a voluntary move and it’s a question of what will those auditors be looking for and what will happen if they find something? So regulation can really shore that up. And then we also need to think about the, of course, broader externalities and societal impacts here. And that’s not something you can govern at the model level. These are very intertwined policy issues, and unfortunately we just haven’t seen federal policymakers really wrap their head around it and do something.
They’ve sort of made some proposals here and there, but it’s a bit of a game of whack-a-mole. And so hopefully they will, after these last few weeks, really take this seriously and put some pieces on the board and move them forward.
For more information, visit the Center for Democracy and Technology.
Listen to Scott Harris’ in-depth interview with Miranda Bogen (31:26) and see more articles and opinion pieces in the related links section of this page. To subscribe to our podcasts, email newsletters, our Between The Lines Radio Newsmagazine Substack or social media, subscribe here.
