WAR OF THE WEIGHTS Cutting-edge Chinese reasoning model rivals OpenAI o1and its free to download DeepSeek R1 is free to run locally and modify, and it matches OpenAI's o1 in several benchmarks. Benj Edwards Jan 21, 2025 1:28 pm | 56 Credit: Wong Yu Liang Credit: Wong Yu Liang Story textSizeSmallStandardLargeWidth *StandardWideLinksStandardOrange* Subscribers only Learn moreOn Monday, Chinese AI lab DeepSeek released its new R1 model family under an open MIT license, with its largest version containing 671 billion parameters. The company claims the model performs at levels comparable to OpenAI's o1 simulated reasoning (SR) model on several math and coding benchmarks.Alongside the release of the main DeepSeek-R1-Zero and DeepSeek-R1 models, DeepSeek published six smaller "DeepSeek-R1-Distill" versions ranging from 1.5 billion to 70 billion parameters. These distilled models are based on existing open source architectures like Qwen and Llama, trained using data generated from the full R1 model. The smallest version can run on a laptop, while the full model requires far more substantial computing resources.The releases immediately caught the attention of the AI community because most existing open-weights modelswhich can often be run and fine-tuned on local hardwarehave lagged behind proprietary models like OpenAI's o1 in so-called reasoning benchmarks. Having these capabilities available in an MIT-licensed model that anyone can study, modify, or use commercially potentially marks a shift in what's possible with publicly available AI models."They are SO much fun to run, watching them think is hilarious," independent AI researcher Simon Willison told Ars in a text message. Willison tested one of the smaller models and described his experience in a post on his blog: "Each response starts with a <think>...</think> pseudo-XML tag containing the chain of thought used to help generate the response," noting that even for simple prompts, the model produces extensive internal reasoning before output.Simulated reasoning in actionThe R1 model works differently from typical large language models (LLMs) by incorporating what people in the industry call an inference-time reasoning approach. They attempt to simulate a human-like chain of thought as the model works through a solution to the query. This class of what one might call "simulated reasoning" models, or SR models for short, emerged when OpenAI debuted its o1 model family in September 2024. OpenAI teased a major upgrade called "o3" in December.Unlike conventional LLMs, these SR models take extra time to produce responses, and this extra time often increases performance on tasks involving math, physics, and science. And this latest open model is turning heads for apparently quickly catching up to OpenAI.For example, DeepSeek reports that R1 outperformed OpenAI's o1 on several benchmarks and tests, including AIME (a mathematical reasoning test), MATH-500 (a collection of word problems), and SWE-bench Verified (a programming assessment tool). As we usually mention, AI benchmarks need to be taken with a grain of salt, and these results have yet to be independently verified. A chart of DeepSeek R1 benchmark results, created by DeepSeek. Credit: DeepSeek TechCrunch reports that three Chinese labsDeepSeek, Alibaba, and Moonshot AI's Kimihave now released models they say match o1's capabilities, with DeepSeek first previewing R1 in November.But the new DeepSeek model comes with a catch if run in the cloud-hosted versionbeing Chinese in origin, R1 will not generate responses about certain topics like Tiananmen Square or Taiwan's autonomy, as it must "embody core socialist values," according to Chinese Internet regulations. This filtering comes from an additional moderation layer that isn't an issue if the model is run locally outside of China.Even with the potential censorship, Dean Ball, an AI researcher at George Mason University, wrote on X, "The impressive performance of DeepSeek's distilled models (smaller versions of r1) means that very capable reasoners will continue to proliferate widely and be runnable on local hardware, far from the eyes of any top-down control regime."Benj EdwardsSenior AI ReporterBenj EdwardsSenior AI Reporter Benj Edwards is Ars Technica's Senior AI Reporter and founder of the site's dedicated AI beat in 2022. He's also a tech historian with almost two decades of experience. In his free time, he writes and records music, collects vintage computers, and enjoys nature. He lives in Raleigh, NC. 56 Comments