TECHCRUNCH.COM
Nvidia CEO says his AI chips are improving faster than Moores Law
Nvidia CEO Jensen Huang says the performance of his companys AI chips is advancing faster than historical rates set by Moores Law, the rubric that drove computing progress for decades.Our systems are progressing way faster than Moores Law, said Huang in an interview with TechCrunch on Tuesday, the morning after he delivered a keynote to a 10,000-person crowd at CES in Las Vegas. Coined by Intel co-founder Gordon Moore in 1965, Moores Law predicted that the number of transistors on computer chips would roughly double every year, essentially doubling the performance of those chips. This prediction mostly panned out, and created rapid advances in capability and plummeting costs for decades.In recent years, Moores Law has slowed down. However, Huang claims that Nvidias AI chips are moving at an accelerated pace of their own; the company says its latest datacenter superchip is more than 30x faster for running AI inference workloads than its previous generation.We can build the architecture, the chip, the system, the libraries, and the algorithms all at the same time, said Huang. If you do that, then you can move faster than Moores Law, because you can innovate across the entire stack.The bold claim from Nvidias CEO comes at a time when many are questioning whether AIs progress has stalled. Leading AI labs such as Google, OpenAI, and Anthropic use Nvidias AI chips to train and run their AI models, and advancements to these chips would likely translate to further progress in AI model capabilities.This isnt the first time Huang has suggested Nvidia is surpassing Moores law. On a podcast in November, Huang suggested the AI world is on pace for hyper Moores Law.Huang rejects the idea that AI progress is slowing. Instead he claims there are now three active AI scaling laws: pre-training, the initial training phase where AI models learn patterns from large amounts of data; post-training, which fine tunes an AI models answers using methods such as human feedback; and test-time compute, which occurs during the inference phase and gives an AI model more time to think after each question.Moores Law was so important in the history of computing because it drove down computing costs, Huang told TechCrunch. The same thing is going to happen with inference where we drive up the performance, and as a result, the cost of inference is going to be less.(Of course, Nvidia has grown to be the most valuable company on Earth by riding the AI boom, so it benefits Huang to say so.)Nvidia CEO Jensen Huang using a gb200 nvl72 like a shield (image credits: Nvidia)Nvidias H100s were the chip of choice for tech companies looking to train AI models, but now that tech companies are focusing more on inference, some have questioned whether Nvidias expensive chips will still stay on top.AI models that use test-time compute are expensive to run today. Theres concern that OpenAIs o3 model, which uses a scaled up version of test-time compute, would be too expensive for most people to use. For example, OpenAI spent nearly $20 per task using o3 to achieve human-level scores on a test of general intelligence. A ChatGPT Plus subscription costs $20 for an entire month of usage.Huang held up Nvidias latest datacenter superchip, the GB200 NVL72, onstage like a shield during Mondays keynote. This chip is 30 to 40x faster at running AI inference workloads than Nvidias previous best selling chips, the H100. Huang says this performance jump means that AI reasoning models like OpenAIs o3, which uses a significant amount of compute during the inference phase, will become cheaper over time.Huang says hes overall focused on creating more performant chips, and that more performant chips create lower prices in the long run.The direct and immediate solution for test-time compute, both in performance and cost affordability, is to increase our computing capability, Huang told TechCrunch. He noted that in the long term, AI reasoning models could be used to create better data for the pre-training and post-training of AI models.Weve certainly seen the price of AI models plummet in the last year, in part due to computing breakthroughs from hardware companies like Nvidia. Huang says thats a trend he expects to continue with AI reasoning models, even though the first versions weve seen from OpenAI have been rather expensive.More broadly, Huang claimed his AI chips today are 1,000x better than what it made 10 years ago. Thats a much faster pace than the standard set by Moores law, one Huang says he sees no sign of stopping soon.
0 Comments 0 Shares 49 Views