So, will AI kill us all in 10 years?

Everyone is asking this question right now because a warning from inside the AI labs just went viral. Jacob Coxon, a researcher at Anthropic, resigned, announcing that the people building the systems “earnestly believe that AI could kill us all”. Surprisingly, Anthropic’s own alignment leader publicly supported him. He placed the odds at over 10% that AI could wipe out humanity within 10 years. There is still no plan for reliably aligning superintelligence to human safety.But weren’t we just here? In July 2024, India hosted the Global IndiaAI Summit. Sovereign nations signed the ‘Delhi Declaration’, promising safe and trustworthy AI. The headlines trumpeted that the problem was being handled. It wasn’t. These were nothing but external rules that countries could sign and quietly ignore while the development race continued.Coxon’s warning is hard to dismiss because of what he gave up. When you work at a company like Anthropic, a part of your pay is in shares of the company, and these only become yours after you’ve stayed long enough, a process called vesting. Coxon left before his shares vested. Anthropic has a future public listing at a valuation exceeding a trillion dollars, so Coxon’s shares were worth a literal fortune. He walked away from it because he no longer wanted to be morally part of the machinery.There is an echo of Peter Gabriel here, who did much the same in 1977, quitting Genesis at the precise moment that the band was on the threshold of fortune and fame. Gabriel wrote about his decision in the classic song ‘Solsbury Hill’.‘I was feeling part of the scenery/ I walked right out of the machinery/ My heart going, “Boom-boom-boom”/ “Hey,” he said/ “Grab your things, I’ve come to take you home”’.Coxon left AI development because he saw the trap everyone was in. Game theorists call this trap the Prisoner’s Dilemma: a situation where each player is tempted to do what helps them individually, even if everyone would be safer if they all held back. In AI, that means every company has a reason to keep racing ahead. If one lab pauses to be safe while another keeps going, the cautious lab falls behind. Thus, no one pauses first. Each company’s choice may make sense on its own, but together those choices push us all to ruin.AI has changed the job. Now it is changing what makes you employable‘When illusion spin her net/ I’m never where I wanna be/ And liberty, she pirouette/ When I think that I am free’.Coxon quit, but quitting isn’t enough. Anthropic builds without him. Stepping away to clear your conscience doesn’t solve our collective Prisoner’s Dilemma. What we need is to jailbreak this prison itself.Jailbreaking is the hacker term for stripping the safety guardrails off an AI system so it does what you want instead of what its makers intended. We need to strip the guardrails off the game itself. Better rules won’t do, nor will stronger incentives, nor will dozens more safety researchers quitting in protest. The payoff matrix (that is, the set of rewards and punishments that shapes every prisoner’s choice) has to change. And that only happens when the prisoners themselves change the way that rational decisions get made.‘Today, I don’t need a replacement/ I’ll tell them what the smile on my face meant/ My heart going, “Boom-boom-boom”/ “Hey,” I said, “You can keep my things, they’ve come to take me home”’.Rules are external. They are written, enforceable, and always bendable. A lab can sign a safety pledge and then reinterpret the clause to keep racing forward. The Delhi Declaration did the same thing on a global scale: nations agreed to words about safety, then went right back to building. The Prisoner’s Dilemma thrives in every rulebook scenario because the rules sit outside of the players.I suggest something completely different: an AI ethics of care. Unlike traditional AI ethics, an ethics of care works from the inside out. It starts from vulnerability – the fact that no one, human or machine, is self-sufficient – and from the relationships that hold us all together. Instead of asking, ‘What am I permitted to do?’, it asks: ‘What do I owe everyone affected by my actions?’ That question cannot be loopholed. It is an internal disposition, not an external rule.This is the jailbreak. Whereas rules are the guardrails on the game, care rewrites the game itself. When the players are constituted by care rather than by competition, the payoff matrix changes. And that, my fellow prisoners, is one of the challenges I fleshed out in my book, Will AI Murder Us All?, published, eerily, on the very day that Coxon’s viral warning escaped the deadly AI labs and into public view.The writer, as Dr Jekyll, is a Professor of Philosophy, Politics and Law, author and editor of over 20 books and counting, and as Mr Hyde, one of India’s top-ranking Ironman triathletes. (Disclaimer: The views expressed above are the author’s own. They do not necessarily reflect the views of DH.)Write to us at: theprism@deccanherald.co.in

Source: Deccan Herald