ye olde skynet 161 years ago, a New Zealand sheep farmer predicted AI doom Butler's "Darwin among the machines" warned of a future mechanical race that could subjugate humanity. Benj Edwards Jan 11, 2025 7:15 am | 0 Credit: Aurich Lawson | Getty Images Credit: Aurich Lawson | Getty Images Story textSizeSmallStandardLargeWidth *StandardWideLinksStandardOrange* Subscribers only Learn moreWhile worrying about AI takeover might seem like a modern idea that sprung from War Games or The Terminator, it turns out that a similar concern about machine dominance dates back to the time of the American Civil War, albeit from an English sheep farmer living in New Zealand. Theoretically, Abraham Lincoln could have read about AI takeover during his lifetime.On June 13, 1863, a letter published in The Press newspaper of Christchurch warned about the potential dangers of mechanical evolution and called for the destruction of machines, foreshadowing the development of what we now call artificial intelligenceand the backlash against it from people who fear it may threaten humanity with extinction. It presented what may be the first published argument for stopping technological progress to prevent machines from dominating humanity.Titled "Darwin among the Machines," the letter recently popped up again on social media thanks to Peter Wildeford of the Institute for AI Policy and Strategy. The author of the letter, Samuel Butler, submitted it under the pseudonym Cellarius, but later came to publicly embrace his position. The letter drew direct parallels between Charles Darwin's theory of evolution and the rapid development of machinery, suggesting that machines could evolve consciousness and eventually supplant humans as Earth's dominant species."We are ourselves creating our own successors," he wrote. "We are daily adding to the beauty and delicacy of their physical organisation; we are daily giving them greater power and supplying by all sorts of ingenious contrivances that self-regulating, self-acting power which will be to them what intellect has been to the human race. In the course of ages we shall find ourselves the inferior race."In the letter, he also portrayed humans becoming subservient to machines, but first serving as caretakers who would maintain and help reproduce mechanical lifea relationship Butler compared to that between humans and their domestic animals, before it later inverts and machines take over."We take it that when the state of things shall have arrived which we have been above attempting to describe, man will have become to the machine what the horse and the dog are to man... we give them whatever experience teaches us to be best for them... in like manner it is reasonable to suppose that the machines will treat us kindly, for their existence is as dependent upon ours as ours is upon the lower animals," he wrote.The text anticipated several modern AI safety concerns, including the possibility of machine consciousness, self-replication, and humans losing control of their technological creations. These themes later appeared in works like Isaac Asimov's The Evitable Conflict and the Matrix films. A model of Charles Babbage's Analytical Engine, a calculating machine invented in 1837 but never built during Babbage's lifetime. Credit: DE AGOSTINI PICTURE LIBRARY via Getty Images Butler's letter dug deep into the taxonomy of machine evolution, discussing mechanical "genera and sub-genera" and pointing to examples like how watches had evolved from "cumbrous clocks of the thirteenth century"suggesting that, like some early vertebrates, mechanical species might get smaller as they became more sophisticated. He expanded these ideas in his 1872 novel Erewhon, which depicted a society that had banned most mechanical inventions. In his fictional society, citizens destroyed all machines invented within the previous 300 years.Butler's concerns about machine evolution received mixed reactions, according to Butler in the preface to the second edition of Erewhon. Some reviewers, he said, interpreted his work as an attempt to satirize Darwin's evolutionary theory, though Butler denied this. In a letter to Darwin in 1865, Butler expressed his deep appreciation for The Origin of Species, writing that it "thoroughly fascinated" him and explained that he had defended Darwin's theory against critics in New Zealand's press.What makes Butler's vision particularly remarkable is that he was writing in a vastly different technological context when computing devices barely existed. While Charles Babbage had proposed his theoretical Analytical Engine in 1837a mechanical computer using gears and levers that was never built in his lifetimethe most advanced calculating devices of 1863 were little more than mechanical calculators and slide rules.Butler extrapolated from the simple machines of the Industrial Revolution, where mechanical automation was transforming manufacturing, but nothing resembling modern computers existed. The first working program-controlled computer wouldn't appear for another 70 years, making his predictions of machine intelligence strikingly prescient.Some things never changeThe debate Butler started continues today. Two years ago, the world grappled with what one might call the "great AI takeover scare of 2023." OpenAI's GPT-4 had just been released, and researchers evaluated its "power-seeking behavior," echoing concerns about potential self-replication and autonomous decision-making.GPT-4's release inspired several open letters signed by AI researchers and tech executives warning of potential extinction-level risks posed by advanced artificial intelligence. One of the letters, reminiscent of fears about nuclear weapons or pandemics, called for a global pause on AI development. Around the same time, OpenAI CEO Sam Altman testified of AI dangers in front of the US Senate.A year later, California legislator Scott Wiener proposed a bill to regulate AI, backed by prominent figures that critics labeled as "AI doomers"those who feared the uncontrolled progression of machine intelligence. Opponents of the bill argued such measures were overblown and could stifle innovation, much as Butlers fictional society had done. Yet his 19th century call for pausing mechanical progress bears a striking resemblance to recent open letters and policy proposals about AI safety.Perhaps the great AI takeover scare will one day be viewed as another chapter in humanitys long struggle to reconcile progress with appropriate human oversighta struggle Butler foreshadowed over 160 years ago. But in some ways, even if machines never become truly intelligent, he was still eerily accurate about our dependence on the ways they algorithmically regulate our lives."Day by day, however, the machines are gaining ground upon us; day by day we are becoming more subservient to them," Butler wrote in 1863. "The upshot is simply a question of time, but that the time will come when the machines will hold the real supremacy over the world and its inhabitants is what no person of a truly philosophic mind can for a moment question."Butler didn't end his letter with passive acceptance of this fate. Somewhat like Eliezer Yudkowsky's 2023 proposal of bombing data centers to prevent AI takeover, Butler's letter concluded with a dramatic call to arms: "War to the death should be instantly proclaimed against them. Every machine of every sort should be destroyed by the well-wisher of his species. Let there be no exceptions made, no quarter shown; let us at once go back to the primeval condition of the race."Even then, he feared it might already be too late, writing that if such destruction proved impossible because of our growing dependency on them: "This at once proves that the mischief is already done, that our servitude has commenced in good earnest, that we have raised a race of beings whom it is beyond our power to destroy."Benj EdwardsSenior AI ReporterBenj EdwardsSenior AI Reporter Benj Edwards is Ars Technica's Senior AI Reporter and founder of the site's dedicated AI beat in 2022. He's also a tech historian with almost two decades of experience. In his free time, he writes and records music, collects vintage computers, and enjoys nature. He lives in Raleigh, NC. 0 Comments