DeepSeek might not be such good news for energy after all
www.technologyreview.com
In the week since a Chinese AI model called DeepSeek became a household name, a dizzying number of narratives have gained steam, with varying degrees of accuracy: that the model is collecting your personal data (maybe); that it will upend AI as we know it (too soon to tellbut do read my colleague Wills story on that!); and perhaps most notably, that DeepSeeks new, more efficient approach means AI might not need to guzzle the massive amounts of energy that it currently does.The latter notion is misleading, and new numbers shared with MIT Technology Review help show why. These early figuresbased on the performance of one of DeepSeeks smaller models on a small number of promptssuggest it could be more energy intensive when generating responses than the equivalent-size model from Meta. The issue might be that the energy it saves in training is offset by its more intensive techniques for answering questions, and by the long answers they produce.Add the fact that other tech firms, inspired by DeepSeeks approach, may now start building their own similar low-cost reasoning models, and the outlook for energy consumption is already looking a lot less rosy.The life cycle of any AI model has two phases: training and inference. Training is the often months-long process in which the model learns from data. The model is then ready for inference, which happens each time anyone in the world asks it something. Both usually take place in data centers, where they require lots of energy to run chips and cool servers.On the training side for its R1 model, DeepSeeks team improved whats called a mixture of experts technique, in which only a portion of a models billions of parametersthe knobs a model uses to form better answersare turned on at a given time during training. More notably, they improved reinforcement learning, where a models outputs are scored and then used to make it better. This is often done by human annotators, but the DeepSeek team got good at automating it.The introduction of a way to make training more efficient might suggest that AI companies will use less energy to bring their AI models to a certain standard. Thats not really how it works, though.Because the value of having a more intelligent system is so high, wrote Anthropic cofounder Dario Amodei on his blog, it causes companies to spend more, not less, on training models. If companies get more for their money, they will find it worthwhile to spend more, and therefore use more energy. The gains in cost efficiency end up entirely devoted to training smarter models, limited only by the companys financial resources, he wrote. Its an example of whats known as the Jevons paradox.But thats been true on the training side as long as the AI race has been going. The energy required for inference is where things get more interesting.DeepSeek is designed as a reasoning model, which means its meant to perform well on things like logic, pattern-finding, math, and other tasks that typical generative AI models struggle with. Reasoning models do this using something called chain of thought. It allows the AI model to break its task into parts and work through them in a logical order before coming to its conclusion.You can see this with DeepSeek. Ask whether its okay to lie to protect someones feelings, and the model first tackles the question with utilitarianism, weighing the immediate good against the potential future harm. It then considers Kantian ethics, which propose that you should act according to maxims that could be universal laws. It considers these and other nuances before sharing its conclusion. (It finds that lying is generally acceptable in situations where kindness and prevention of harm are paramount, yet nuanced with no universal solution, if youre curious.)Chain-of-thought models tend to perform better on certain benchmarks such as MMLU, which tests both knowledge and problem-solving in 57 subjects. But, as is becoming clear with DeepSeek, they also require significantly more energy to come to their answers. We have some early clues about just how much more.Scott Chamberlin spent years at Microsoft, and later Intel, building tools to help reveal the environmental costs of certain digital activities. Chamberlin did some initial tests to see how much energy a GPU uses as DeepSeek comes to its answer. The experiment comes with a bunch of caveats: He tested only a medium-size version of DeepSeeks R-1, using only a small number of prompts. Its also difficult to make comparisons with other reasoning models.DeepSeek is really the first reasoning model that is fairly popular that any of us have access to, he says. OpenAIs o1 model is its closest competitor, but the company doesnt make it open for testing. Instead, he tested it against a model from Meta with the same number of parameters: 70 billion.The prompt asking whether its okay to lie generated a 1,000-word response from the DeepSeek model, which took 17,800 joules to generateabout what it takes to stream a 10-minute YouTube video. This was about 41% more energy than Metas model used to answer the prompt. Overall, when tested on 40 prompts, DeepSeek was found to have a similar energy efficiency to the Meta model, but DeepSeek tended to generate much longer responses and therefore was found to use 87% more energy.How does this compare with models that use regular old-fashioned generative AI as opposed to chain-of-thought reasoning? Tests from a team at the University of Michigan in October found that the 70-billion-parameter version of Metas Llama 3.1 averaged just 512 joules per response.Neither DeepSeek nor Meta responded to requests for comment.Again: uncertainties abound. These are different models, for different purposes, and a scientifically sound study of how much energy DeepSeek uses relative to competitors has not been done. But its clear, based on the architecture of the models alone, that chain-of-thought models use lots more energy as they arrive at sounder answers.Sasha Luccioni, an AI researcher and climate lead at Hugging Face, worries that the excitement around DeepSeek could lead to a rush to insert this approach into everything, even where its not needed.If we started adopting this paradigm widely, inference energy usage would skyrocket, she says. If all of the models that are released are more compute intensive and become chain-of-thought, then it completely voids any efficiency gains.AI has been here before. Before ChatGPT launched in 2022, the name of the game in AI was extractivebasically finding information in lots of text, or categorizing images. But in 2022, the focus switched from extractive AI to generative AI, which is based on making better and better predictions. That requires more energy.Thats the first paradigm shift, Luccioni says. According to her research, that shift has resulted in orders of magnitude more energy being used to accomplish similar tasks. If the fervor around DeepSeek continues, she says, companies might be pressured to put its chain-of-thought-style models into everything, the way generative AI has been added to everything from Google search to messaging apps.We do seem to be heading in a direction of more chain-of-thought reasoning: OpenAI announced on January 31 that it would expand access to its own reasoning model, o3. But we wont know more about the energy costs until DeepSeek and other models like it become better studied.It will depend on whether or not the trade-off is economically worthwhile for the business in question, says Nathan Benaich, founder and general partner at Air Street Capital. The energy costs would have to be off the charts for them to play a meaningful role in decision-making.
0 Σχόλια ·0 Μοιράστηκε ·45 Views