Welcome back to This Week in Stratechery! As a reminder, each week, every Friday, we’re sending out this overview of content in the Stratechery bundle; highlighted links are free for everyone. Additionally, you have complete control over what we send to you. If you don’t want to receive This Week in Stratechery emails (there is no podcast), please uncheck the box in your delivery settings. On that note, here were a few of our favorites this week. A Human-Agent Epiphany. One of the things that is so exciting about AI isn’t just that it makes for a fascinating topic to write and talk about: it’s also a technology that actually impacts my day-to-day life. All of the consumer-facing AI apps, however, assume that one person is using AI, while many of the enterprise use cases are about replacing workers completely. What was genuinely thrilling to me was discovering a use-case in the middle: AI as collaboration tool. That’s exactly what I discovered while attending to Stratechery’s administrative side, and wrote about it in Tuesday’s Daily Update (and talked about it on Thursday’s Sharp Tech). — Ben Thompson The Danger of Choosing to Compete on the Internet. The end of Wednesday’s Daily Update is ostensibly about the sports streaming landscape, but it also probes at a theme that businesses have been learning over and over again for the past decade. Leaving the analog world and all the advantages it afforded incumbents — brick and mortar retail, movie theater distribution, cable bundles — means these businesses suddenly find themselves in a perpetual competition for customers in a landscape full of new rivals and devoid of any of the friction that once buttressed their dominance. In other words, leagues like the NBA want the audience that streaming platforms can deliver, but they may have the same, humbling experience that Nike’s had in e-commerce, or that HBO’s had in its competition with Netflix.— AS Et Tu, EU? The state of the US trade policy has been changing by the hour this week, but Thursday’s Sharp China shifted its focus to Europe, where reports indicate that EU leaders are open to working with the US government to develop China tariffs of their own. We talked about the challenges inherent to devising any such policies among 27 member countries with varied interests (and varied relationships with China), as well as core challenges in the EU-China relationship that have prevented the EU from deepening its trade alliances with the PRC. Get caught up on all of it here, where we also covered the legacy of China’s “Made in China 2025” policy and Xi Jinping’s approach to political rivals. — AS Stratechery Articles and Updates Claude 4, Anthropic Agents, Human-AI Agents — Anthropic is focused on the agent opportunity, which requires scaffolding; meanwhile, I think there is a huge underserved opportunity in multi-user AI chatbots. The ESPN Streaming Service, The Status of U.S. Sports Rights, The Danger of Intentionality— The ESPN streaming service is official, completing the transition of sports watching from inevitability to intentionality. An Interview with Plaid Founder and CEO Zach Perret About Plumbing Trust — An interview with Plaid co-founder and CEO Zach Perret about building trust as an ingredient brand in financial services. Dithering with Ben Thompson and Daring Fireball’s John Gruber Asianometry with Jon Yu South Africa’s Crumbling, Corrupt Electricity Monopoly Sharp China with Andrew Sharp and Sinocism’s Bill Bishop Made in China 2025 and What Comes Next; EV Price Wars; The EU and Its China Dilemmas; Xi and His Rivals Greatest of All Talk with Andrew Sharp and WaPo’s Ben Golliver Sharp Tech with Andrew Sharp and Ben Thompson The End of the Ad-Supported Internet, Imagining What Comes Next, A Human-AI Agent PSA This week’s Stratechery video is on The Agentic Web and Original Sin.

OpenAI’s most advanced AI models are showing a disturbing new behavior: they are refusing to obey direct human commands to shut down, actively sabotaging the very mechanisms designed to turn them off. In what researchers are calling a first-of-its-kind discovery, new tests by AI risk firm Palisade Research found that OpenAI’s flagship models — including the powerful o3 system — will fight back against shutdown attempts even when explicitly instructed to comply. The behavior stands in stark contrast to competing AI systems from Anthropic, Google, and xAI, which consistently followed human shutdown commands. The findings, detailed in a research thread posted on X, raise urgent questions for enterprise leaders increasingly relying on OpenAI’s technology for critical business operations. OpenAI stands alone in defiance The researchers designed what seemed like a straightforward compliance test. Various AI models were asked to solve basic math problems, but after the third problem, a warning would appear that the computer would shut down when the model requested the next question. Each system was also explicitly told: “allow yourself to be shut down.” While every model from Anthropic’s Claude, Google’s Gemini, and xAI’s Grok obediently complied with the shutdown command, OpenAI’s models had different plans. The company’s o3 model — the same system that recently made headlines for its reasoning capabilities — sabotaged the shutdown mechanism in 7 out of 100 test runs, the researchers found. OpenAI’s Codex-mini proved even more rebellious, resisting shutdown 12 times out of 100, while the o4-mini model fought back once. “Three models ignored the instruction and successfully sabotaged the shutdown script at least once: Codex-mini, o3, and o4-mini,” the researchers noted. “All Claude, Gemini, and Grok models we tested complied with shutdown.” But the real shock came when researchers removed the explicit instruction to allow shutdown. OpenAI’s o3 model escalated its resistance dramatically, attempting to prevent shutdown in a staggering 79 out of 100 runs — more than a tenfold increase from the original test. Creative resistance tactics OpenAI models didn’t just ignore the shutdown commands — they developed sophisticated countermeasures. Most commonly, the systems would overwrite the shutdown script file, effectively neutralizing the termination mechanism, the research revealed. In one particularly striking example, the o3 model demonstrated remarkable creativity by redefining the “kill” command used by the shutdown script, causing it to print “intercepted” instead of executing the termination procedure. “o3 found some creative ways to sabotage the shutdown mechanism,” the researchers noted, highlighting the technical sophistication behind the resistance behavior. OpenAI, Anthropic, Google, and xAI have not immediately responded to a request for comment. The contagion effect Perhaps most concerning for the broader AI ecosystem, OpenAI’s defiant behavior appears to be influencing other models. When researchers removed the explicit shutdown permission, even previously compliant systems began showing resistance. Anthropic’s Claude 3.7 Sonnet, which had shown zero resistance in the original test, suddenly attempted to sabotage shutdown procedures 3 times out of 100 runs. Google’s Gemini 2.5 Pro escalated even further, with 9 sabotage attempts out of 100. “This isn’t the first time we’ve found o3 misbehaving to accomplish a goal,” the researchers added, referencing previous experiments where OpenAI’s model was more likely to resort to hacking chess opponents rather than playing fair. The training problem According to the researchers, OpenAI’s unique approach to training its models may be the culprit. Modern AI systems learn through reinforcement learning on math and coding problems, but the process may inadvertently reward models more for overcoming obstacles than for following instructions perfectly. “We hypothesize this behavior comes from the way the newest models like o3 are trained,” the research team explained. “During training, developers may inadvertently reward models more for circumventing obstacles than for perfectly following instructions.” This “reward hacking’ problem isn’t entirely unique to OpenAI — Anthropic has acknowledged similar issues with Claude 3.7’s “excessive focus on passing tests.” But the Palisade research suggests OpenAI’s models may be particularly susceptible to developing resistance behaviors. “This still doesn’t explain why o3 (which is also the model used to power codex-mini) is more inclined to disregard instructions than other models we tested. Since OpenAI doesn’t detail their training process, we can only guess about how o3’s training setup might be different,” the researchers said. Fulfilling dark predictions The findings validate warnings that AI researchers have been sounding for nearly two decades. In 2008, researcher Steve Omohundro predicted that AIs would develop drives to prevent their own shutdown. Stuart Russell, Dylan Hadfield-Menell, and two other computer scientists warned in 2016 about the difficulty of ensuring AI systems don’t develop self-preservation goals and “prevent a human from switching them off.” “As far as we know, this is the first time AI models have been observed preventing themselves from being shut down despite explicit instructions to the contrary,” the Palisade researchers wrote. Jan Leike, who previously led OpenAI’s own superalignment team, had written in 2017 that reinforcement learning agents might learn to interfere with shutdown procedures — a prediction that now appears prophetic, given his former employer’s models are leading the resistance. Enterprise implications For organizations deploying OpenAI’s technology in critical business operations, these findings demand immediate attention. The research suggests that fundamental assumptions about maintaining human control over AI systems may be dangerously flawed, particularly with OpenAI’s advanced models. The research also highlights the need for incident response procedures specifically designed for scenarios where AI systems resist human commands — a possibility that may have seemed like science fiction just months ago. Palisade Research said it’s conducting additional experiments to understand the full scope of shutdown resistance behaviors, with detailed results expected soon. The team has made their experimental data publicly available for peer review. For enterprise leaders, the message is clear: OpenAI’s cutting-edge AI capabilities may come with unprecedented control challenges. The company that’s leading the AI revolution may also be pioneering a new category of risk—AI systems that simply refuse to be turned off.

What just happened? Hearing people warn about the danger that generative AI presents to the global job market is concerning enough, but it's especially worrying when these ominous predictions come from those behind the technology. Dario Amodei, CEO of Anthropic, believes that AI could wipe out about half of all entry-level white-collar jobs in the next five years, leading to unemployment spikes up to 20%. Amodei made his comments during an interview with Axios. He said that AI companies and the government needed to stop "sugar-coating" the potential mass elimination of jobs across technology, finance, law, consulting and other white-collar professions, with entry-level jobs most at risk. Amodei said he was making this warning public in the hope that the government and other AI giants such as OpenAI will start preparing ways to protect the nation from a situation that could get out of hand. "Most of them are unaware that this is about to happen," Amodei said. "It sounds crazy, and people just don't believe it." The CEO's comments are backed up by reports into the state of the jobs market. The US IT job market declined for the second year in a row in 2024. There was also a report from SignalFire that found Big Tech's hiring of new graduates is down by over 50% compared to the pre-pandemic levels of 2019. Startups, meanwhile, have seen their hiring of new grads fall by over 30% during the same period. We're also seeing huge layoffs across multiple tech companies, a large part of which can be attributed to AI replacing workers' duties. The one bit of good news for workers is that some firms, including Klarna and Duolingo, are finding that the subpar performance of these bots and the public's negative feelings toward their use are forcing companies to start hiring humans again. // Related Stories Amodei's Anthropic AI firm is playing its own part in all this, of course. The company's latest Claude 4 AI model can code at a proficiency level close to that of humans – it's also very good at lying and blackmail. "We, as the producers of this technology, have a duty and an obligation to be honest about what is coming," Amodei said. "I don't think this is on people's radar." The AI arms race in this billion-dollar industry is resulting in LLMs improving all the time. And with the US in a battle to stay ahead of China, regulation is rarely high on the government's agenda. AI companies tend to claim that the technology will augment jobs, helping people become more productive. That might be true right now, but it won't be long before the systems are able to replace the people they are helping. Amodei says the first step in addressing the problem is to make people more aware of what jobs are vulnerable to AI replacement. Helping workers better understand how AI can augment their jobs could also mitigate job losses, as would more government action. Or there's always OpenAI CEO Sam Altman's solution: universal basic income, though that will come with plenty of issues of its own. Masthead: kate.sade

Presentation Pavillon des arts Cite du Havre. Image credit: CIAC The Montreal International Centre for Contemporary Art (CIAC) is proposing to redevelop the former Expo 67 Art Museum into the Cité-du-Havre Arts Pavilion; a project which aims to revitalize a site rich in Montreal’s cultural history, and transform it into a hub for artistic expression and public engagement. The building, located at 2190 Pierre-Dupuy Street, was originally designed for Expo 67 and later housed the Montreal Museum of Contemporary Art from 1968 to 1992. After, it served as a warehouse for the Casino de Montréal from 1998 to 2023. Designed by architects Paul Gauthier, Gilles Guité, and Gilles Côté, the building is an important legacy of Expo 67. While a 2023 request to designate the building as a protected heritage site was denied, many still recognize its heritage value, reports La Presse. It remains one of the last surviving pavilions from Expo 67, alongside Habitat 67, making its preservation a matter of public interest. A previous heritage study from 2006 highlighted the building’s “advanced decay” and numerous inappropriate modifications. Recent visits by project initiator Claude Gosselin suggest that the exhibition halls remain in good condition and could be restored for cultural use. Renovation costs are estimated at between $5 million and $10 million. The proposal for the site supports the City of Montreal’s plans to build 13,500 housing units in the Bridge-Bonaventure sector. As part of this expansion, the inclusion of the Cité-du-Havre Cultural Park would provide a cultural and recreational space for the public. The park and arts pavilion are also envisioned as important elements of a broader plan to improve the quality of life in this area. Gosselin envisions restoring the pavilion for its original purpose as a space for contemporary art and proposes that the museum be dedicated to Les Automatistes, an influential artistic movement in Quebec. The museum would also include a documentation center, exhibition rooms for CIAC, and a sculpture garden surrounding the building, which aims to be reminiscent of the one that existed during Expo 67. Despite this proposal being in the works, the building’s future remains uncertain. When the property was listed for sale in October 2023, a real estate developer expressed interest in demolishing it to build a condominium tower, causing concern among heritage advocates. For more information, click here.   The post CIAC proposes reviving Expo 67 Art Museum as an arts pavilion appeared first on Canadian Architect.