How to avoid Terminators Google DeepMind releases its plan to keep AGI from running wild DeepMind says AGI could arrive in 2030, and it has some ideas to keep us safe. Ryan Whitwam Apr 3, 2025 5:43 pm | 0 Story textSizeSmallStandardLargeWidth *StandardWideLinksStandardOrange* Subscribers only Learn moreAs AI hype permeates the Internet, tech and business leaders are already looking toward the next step. AGI, or artificial general intelligence, refers to a machine with human-like intelligence and capabilities. If today's AI systems are on a path to AGI, we will need new approaches to ensure such a machine doesn't work against human interests.Unfortunately, we don't have anything as elegant as Isaac Asimov's Three Laws of Robotics. Researchers at DeepMind have been working on this problem and have released a new technical paper (PDF), which you can download at your convenience.It contains a huge amount of detail, clocking in at 108 pages before references. While some in the AI field believe AGI is a pipe dream, the authors of the DeepMind paper project that it could happen by 2030. With that in mind, they aimed to understand the risks of a human-like synthetic intelligence, which they acknowledge could lead to "severe harm."All the ways AGI could suck for humanityThis work has identified four possible types of AGI risk, along with suggestions on how we might ameliorate said risks. The DeepMind team, led by company co-founder Shane Legg, categorized the negative AGI outcomes as misuse, misalignment, mistakes, and structural risks. The four categories of AGI risk, as determined by DeepMind. Credit: Google DeepMind The four categories of AGI risk, as determined by DeepMind. Credit: Google DeepMind The first possible issue, misuse, is fundamentally similar to current AI risks. However, because AGI will be more powerful by definition, the damage it could do is much greater. A ne'er-do-well with access to AGI could misuse the system to do harm, for example, by asking the system to identify and exploit zero-day vulnerabilities or create a designer virus that could be used as a bioweapon.DeepMind says companies developing AGI will have to conduct extensive testing and create robust post-training safety protocols. Essentially, our current AI guardrails on steroids. They also suggest devising a method to suppress dangerous capabilities entirely, sometimes called "unlearning," but it's unclear if this is possible without substantially limiting models.Misalignment is largely not something we have to worry about with generative AI as it currently exists. This type of AGI harm is envisioned as a rogue machine that has shaken off the limits imposed by its designers. Terminators, anyone?To avoid that, DeepMind suggests developers use techniques like amplified oversight, in which two copies of an AI check each other's output, to create robust systems that aren't likely to go rogue. If that fails, DeepMind suggests intensive stress testing and monitoring to watch for any hint that an AI might be turning against us. Keeping AGIs in virtual sandboxes with strict security and direct human oversight could help mitigate issues arising from misalignment. Basically, make sure there's an "off" switch.If, on the other hand, an AI didn't know that its output would be harmful and the human operator didn't intend for it to be, that's a mistake. We get plenty of those with current AI systemsremember when Google said to put glue on pizza? The "glue" for AGI could be much stickier, though. DeepMind notes that militaries may deploy AGI due to "competitive pressure," but such systems could make serious mistakes as they will be tasked with much more elaborate functions than today's AI.The paper doesn't have a great solution for mitigating mistakes. It boils down to not letting AGI get too powerful in the first place. DeepMind calls for deploying slowly and limiting AGI authority. The study also suggests passing AGI commands through a "shield" system that ensures they are safe before implementation.Lastly, there are structural risks, which DeepMind defines as the unintended but real consequences of multi-agent systems contributing to our already complex human existence. For example, AGI could create false information that is so believable that we no longer know who or what to trust. The paper also raises the possibility that AGI could accumulate more and more control over economic and political systems, perhaps by devising heavy-handed tariff schemes. Then one day, we look up and realize the machines are in charge instead of us. This category of risk is also the hardest to guard against because it would depend on how people, infrastructure, and institutions operate in the future.AGI in five years?No one knows if the thinking machines are really just a few years away, but there are plenty of tech leaders who are confident enough to say so. Part of the problem in predicting the emergence of AGI is that we're still just speculating about how human-like intelligence would manifest itself in a machine. Anyone who has used generative AI systems over the past years has seen real, tangible improvements, but does that trajectory lead to true human-like capabilities?We recently talked about a range of AI topics, including AGI, with Google's Tulsee Doshi, director of product management for Gemini. "Different people have different definitions of AGI, and so depending on who you talk to, how close or far we are from AGI is a different conversation," said Doshi. "What I would say is LLMs, Gemini, and the training of smarter and smarter models is on the path to models that are going to be at extremely high intelligence. And that has a ton of value in and of itself."This paper is not the final word on AGI safetyDeepMind notes this is just a "starting point for vital conversations." If the team is right, and AGI will transform the world in five short years, those conversations need to happen soon. If not, well, a lot of people are going to look kind of silly.Ryan WhitwamSenior Technology ReporterRyan WhitwamSenior Technology Reporter Ryan Whitwam is a senior technology reporter at Ars Technica, covering the ways Google, AI, and mobile technology continue to change the world. Over his 20-year career, he's written for Android Police, ExtremeTech, Wirecutter, NY Times, and more. He has reviewed more phones than most people will ever own. You can follow him on Bluesky, where you will see photos of his dozens of mechanical keyboards. 0 Comments