PCs have been through a number of evolutions in terms of form factor, transforming from big, heavy, and quite ugly beige boxes that took up a lot of desk space to something no bigger than a hardback book that packs enough power to handle all but the most demanding of workloads.Say hello to the mini PC: A PC that's just as comfortable tucked behind a TV, popped on a shelf, or hidden in a cupboard as it is sitting on a desk. These mini PCs pack a punch and have everything that you could want from a modern PC -- and take up the least amount of space possible.Also:The best power banks: Expert testedWhat is the best mini PC right now?I've tested and researched the current mini PC offerings on the market, examining specs and paying close attention to user reviews, and my top pick for the best mini PC is the Intel NUC 13 Pro Arena Canyon. Whether you want a mini PC for the home, office, or meeting room, this is a solid choice. But I understand that this might not be for everyone, so I've picked a handful more mini PCs, each aimed at a specific use case. Sort by All The best mini PCs of 2024 Show less View now at Amazon When it comes to mini PCs, it's hard to beat a mini PC built to the Intel NUC specs. I've owned and tested a number of them myself, and have been very happy with them.With this model, we have a system running a 16-core Core i7 chip that can power up to 3.7 GHz with a built-in Iris Xe GPU. Backing that is 32GB of RAM and 1TB M.2 SSD. That's quite a spec for a system that you can hold in the palm of your hand. As for ports, this system has a multitude of USB ports, along with Thunderbolt 4, HDMI, and Ethernet.As mini PCs go, this one is overkill for most, but if you are looking for a really powerful system that runs whisper-quiet and can be mounted behind a TV or put on a shelf, this is the system for you.Given the price of this system, there aren't a lot of reviews for it, but the ones that have been written by owners are overwhelmingly positive.Intel NUC 13 Pro Arena Canyon tech specs: 12th Generation Intel Core i7-1360P | Intel Iris Xe Graphics 96EU | 32GB DDR4 RAM | 2TB M.2 NVMe SSD | 3x USB 3.2 type A, 1x USB-A, 2x Thunderbolt 4, 2x HDMI, Ethernet, audio jack | Windows 11 Pro Pros Powerful Quiet Excellent reviews Cons Expensive When it comes to mini PCs, it's hard to beat a mini PC built to the Intel NUC specs. I've owned and tested a number of them myself, and have been very happy with them.With this model, we have a system running a 16-core Core i7 chip that can power up to 3.7 GHz with a built-in Iris Xe GPU. Backing that is 32GB of RAM and 1TB M.2 SSD. That's quite a spec for a system that you can hold in the palm of your hand. As for ports, this system has a multitude of USB ports, along with Thunderbolt 4, HDMI, and Ethernet.As mini PCs go, this one is overkill for most, but if you are looking for a really powerful system that runs whisper-quiet and can be mounted behind a TV or put on a shelf, this is the system for you.Given the price of this system, there aren't a lot of reviews for it, but the ones that have been written by owners are overwhelmingly positive.Intel NUC 13 Pro Arena Canyon tech specs: 12th Generation Intel Core i7-1360P | Intel Iris Xe Graphics 96EU | 32GB DDR4 RAM | 2TB M.2 NVMe SSD | 3x USB 3.2 type A, 1x USB-A, 2x Thunderbolt 4, 2x HDMI, Ethernet, audio jack | Windows 11 Pro Read More Show Expert Take Show less Show less Just when you thought this list would only include Windows systems, we've got the Mac Mini. This was one of the original mini PCs, and with the switch to Apple Silicon processors, it has seen a new lease of life.Considered at one point the gateway Mac that got people over to MacOS from Windows (because they could replace their Windows PC with the Mac Mini and keep using their existing keyboard, mouse, and monitor with their new Mac), it's now a staple of the Mac lineup, and the perfect choice for those who want a compact system.I've owned a few different Mac Mini systems over the years, and I've been more than satisfied by them. They're reliable, well-built, and run cool and quiet. The biggest downside to this system is upgrading -- bottom line, this Mac will live and die with the same processor, RAM, and storage that it left the factory with. If you want to give it a storage boost, Satechi makes a hub/stand for it that adds a bunch of ports and a port for an M.2 SSD.As I'd expect from an Apple product, the reviews are strongly positive, with users reporting that they are happy with the performance and overall reliability of the system.Apple 2023 Mac mini tech specs: Apple Silicon M4 processor with 10 CPU cores, 10 GPU cores | 16GB RAM | 256GB storage | 2x USB-C, 3x Thunderbolt 4, HDMI, Ethernet, audio jack | MacOS Pros Powerful Close to silent Excellent Apple build quality Cons Runs MacOS (although you can run Windows on it using virtualization) Limited upgrade options Just when you thought this list would only include Windows systems, we've got the Mac Mini. This was one of the original mini PCs, and with the switch to Apple Silicon processors, it has seen a new lease of life.Considered at one point the gateway Mac that got people over to MacOS from Windows (because they could replace their Windows PC with the Mac Mini and keep using their existing keyboard, mouse, and monitor with their new Mac), it's now a staple of the Mac lineup, and the perfect choice for those who want a compact system.I've owned a few different Mac Mini systems over the years, and I've been more than satisfied by them. They're reliable, well-built, and run cool and quiet. The biggest downside to this system is upgrading -- bottom line, this Mac will live and die with the same processor, RAM, and storage that it left the factory with. If you want to give it a storage boost, Satechi makes a hub/stand for it that adds a bunch of ports and a port for an M.2 SSD.As I'd expect from an Apple product, the reviews are strongly positive, with users reporting that they are happy with the performance and overall reliability of the system.Apple 2023 Mac mini tech specs: Apple Silicon M4 processor with 10 CPU cores, 10 GPU cores | 16GB RAM | 256GB storage | 2x USB-C, 3x Thunderbolt 4, HDMI, Ethernet, audio jack | MacOS Read More Show Expert Take Show less Show less View now at Amazon This is a tiny mini PC that is a cube measuring 72 x 72 x 44.5mm, weighing only 206g. Inside the cube is a 4-core Intel N5105 processor, 8GB of RAM, and 128HB M.2 SSD running Windows 10 Pro.Despite the tiny size, this mini PC features a handful of USB ports, two HDMI ports, an audio jack, and an Ethernet port. This system has no problems running two 4K@60FPS displays for maximum productivity.And yet given the power, this system is quiet and very energy efficient (using only 10W), making it a great choice for someone who wants a PC for the home, office, or meeting room, or even to drive digital signage.Reviewers of this system comment on the performance and overall value of the system, and it's an outstanding system for the price.GMKtec Mini PC tech specs: 11th Generation Intel N5105 | Intel UHD Graphics | 8GB DDR4 RAM | 128GB M.2 NVMe SSD | 3x USB 3.2 type A, USB-C, 2x HDMI, Ethernet, TF/microSD slot, audio jack | Windows 11 Pro Pros Tiny Powerful enough to drive two 4K displays Cons Not fanless, so may become a little noisy when pushed with heavy workloads This is a tiny mini PC that is a cube measuring 72 x 72 x 44.5mm, weighing only 206g. Inside the cube is a 4-core Intel N5105 processor, 8GB of RAM, and 128HB M.2 SSD running Windows 10 Pro.Despite the tiny size, this mini PC features a handful of USB ports, two HDMI ports, an audio jack, and an Ethernet port. This system has no problems running two 4K@60FPS displays for maximum productivity.And yet given the power, this system is quiet and very energy efficient (using only 10W), making it a great choice for someone who wants a PC for the home, office, or meeting room, or even to drive digital signage.Reviewers of this system comment on the performance and overall value of the system, and it's an outstanding system for the price.GMKtec Mini PC tech specs: 11th Generation Intel N5105 | Intel UHD Graphics | 8GB DDR4 RAM | 128GB M.2 NVMe SSD | 3x USB 3.2 type A, USB-C, 2x HDMI, Ethernet, TF/microSD slot, audio jack | Windows 11 Pro Read More Show Expert Take Show less Show less View now at Amazon This mini PC has a few features that set it apart from the competition. While dual HDMI isn't all that rare, finding a system with dual Ethernet and dual M.2 slots is, and finding one with a built-in filter to keep dust out of the innards is even rarer.The price of this system makes it a perfect choice for those who want a cheap yet powerful mini PC -- as long as you're not running AAA-title games or other heavy workloads, this system has you covered.The only con to this system is that it runs the older 12th-gen Intel silicon, but for $300, this is still a system that both holds its own and won't break the bank.Beelink EQ13 Mini PC tech specs: 12th Generation Intel N200 | Intel UHD Graphics 32EUS | 16GB DDR4 RAM | 500GB M.2 NVMe SSD | 3x USB 3.2 type A, 2x HDMI, 2x Ethernet, USB-C, audio jack | Windows 11 ProReviews of this system are strongly positive, with owners praising the power and performance of the system, and how quiet it is in day-to-day use. Pros Dual HDMI Dual Ethernet Dual M.2 slots Dust filter to keep the fan and insides clean Cons Older, 12th-generation processors This mini PC has a few features that set it apart from the competition. While dual HDMI isn't all that rare, finding a system with dual Ethernet and dual M.2 slots is, and finding one with a built-in filter to keep dust out of the innards is even rarer.The price of this system makes it a perfect choice for those who want a cheap yet powerful mini PC -- as long as you're not running AAA-title games or other heavy workloads, this system has you covered.The only con to this system is that it runs the older 12th-gen Intel silicon, but for $300, this is still a system that both holds its own and won't break the bank.Beelink EQ13 Mini PC tech specs: 12th Generation Intel N200 | Intel UHD Graphics 32EUS | 16GB DDR4 RAM | 500GB M.2 NVMe SSD | 3x USB 3.2 type A, 2x HDMI, 2x Ethernet, USB-C, audio jack | Windows 11 ProReviews of this system are strongly positive, with owners praising the power and performance of the system, and how quiet it is in day-to-day use. Read More Show Expert Take Show less Show less View now at Amazon This mini PC describes itself as a gaming system, and with good reason. The combination of AMD Ryzen 9 6900HX and AMD Radeon 680M GPU gives this system a lot of horsepower to tackle a variety of workloads, from work to games.It also throws away the design rulebook. While most mini PCs are cube-like, this one is an upright tent shape. While that might seem like purely a design choice, it does in fact make opening up the system to carry out upgrades a snap as the side panel is held in place with magnets. This system features a power dial on the top to switch it between silent mode, auto, and performance, depending on whether you want power or silence. It also includes a multitude of LED lights, which you might love or hate. If you love them, great, if not, there's an app that turns them off for you.But the lights aside, this is the perfect mini PC for those looking for a gaming system but who don't want to have a huge box on their desk or beside their TV. There's no harder set of people to please than gamers, which is why it's quite reassuring to find strong reviews for this gaming mini PC.AceMagician AMD Ryzen 9 Mini PC tech specs:AMD Ryzen 9 6900HX | AMD Radeon 680M GPU | 32GB DDR4 RAM | 1TB M.2 NVMe SSD | 3x USB 3.2 type A, 2x HDMI, Ethernet, USB-C, audio jack | Windows 11 ProThis is a mini PC that breaks a lot of the rules. Pros A mini PC that legitimately describes itself as a gaming system Powerful AMD processor and GPU Convenient dial for switching between power modes Cons Different shape LED lights might be annoying This mini PC describes itself as a gaming system, and with good reason. The combination of AMD Ryzen 9 6900HX and AMD Radeon 680M GPU gives this system a lot of horsepower to tackle a variety of workloads, from work to games.It also throws away the design rulebook. While most mini PCs are cube-like, this one is an upright tent shape. While that might seem like purely a design choice, it does in fact make opening up the system to carry out upgrades a snap as the side panel is held in place with magnets. This system features a power dial on the top to switch it between silent mode, auto, and performance, depending on whether you want power or silence. It also includes a multitude of LED lights, which you might love or hate. If you love them, great, if not, there's an app that turns them off for you.But the lights aside, this is the perfect mini PC for those looking for a gaming system but who don't want to have a huge box on their desk or beside their TV. There's no harder set of people to please than gamers, which is why it's quite reassuring to find strong reviews for this gaming mini PC.AceMagician AMD Ryzen 9 Mini PC tech specs:AMD Ryzen 9 6900HX | AMD Radeon 680M GPU | 32GB DDR4 RAM | 1TB M.2 NVMe SSD | 3x USB 3.2 type A, 2x HDMI, Ethernet, USB-C, audio jack | Windows 11 ProThis is a mini PC that breaks a lot of the rules. Read More Show Expert Take Show less What is the best mini PC? Mini PCPriceProcessorGPURAMStorageOperating systemIntel NUC 13 Pro Arena Canyon$949Intel Core i7-1360PIntel Iris Xe Graphics 96EU32GB DDR42TB M.2 NVMe SSD Windows 11 ProApple 2024 Mac Mini$599Apple Silicon M4Apple Silicon M416GB DDR4256GB SSDMacOS 15GMKtec Mini PC$140Intel N5105Intel UHD Graphics8GB DDR4128GB M.2 NVMe SSDWindows 11 ProBeelink EQ13 Mini PC$299Intel N200Intel UHD Graphics 32EUS16GB DDR4500GB M.2 NVMe SSDWindows 11 ProAceMagician AMD Ryzen 9 Mini PC$570AMD Ryzen 9 6900HXAMD Radeon 680M32GB DDR41TB M.2 NVMe SSDWindows 11 Pro Show more Which is the right mini PC for you? Choose this mini PCIf you wantIntel NUC 13 Pro Arena Canyona general-purpose mini PC. The Intel NUC PCs continue to be the best all-around mini PCs on the market. Yes, there are better, faster, cheaper systems out there, but these systems have continually impressed me over the years.Apple 2024 Mac minito go with MacOS. The Mac Mini used to be the way that Windows PC users would start to make the transition over to Mac. Nowadays, they are all-purpose desktop computers that are perfect for anyone who doesn't want a portable system, from students to developers.GMKtec Mini PCa truly tiny PC. This is one of the smallest systems I've seen, but it packs quite a punch, hitting well above its size and weight.Beelink EQ13 Mini PCa mini PC with a difference. This mini PC breaks the mold: not only does it have a special dust filter to keep out debris, it also has 2 Ethernet ports, and two M.2 slots to compliment the two HDMI ports.AceMagician AMD Ryzen 9 Mini PCa proper mini gaming PC. Here you have a full gaming PC packed into a tiny footprint. Sure, there are going to be desktop systems that are way more powerful than this system, but for the size, you'll be hard-pressed to find anything faster. Show more Factors to consider when choosing a mini PC In many ways, choosing a mini PC is much like a regular PC. There are all the usual performance, price, and support for all your peripherals to keep in mind. But if you are after a mini PC, then the mini part does come into play, so there is a size issue to bear in mind. The good news is that mini PCs come in a range of sizes, from systems that can fit in the palm of your hand to systems so small you could fit several of them in the palm of your hand.Do you need a mini PC? This is the top question when choosing a mini PC. Do you actually need one, or do you just need a regular PC?Size: This really is the differentiator when it comes to mini PCs. Do you want a PC that's the size of a biscuit tin, or something significantly smaller?Performance: Pretty standard PC consideration here. What do you want your system to do? Do you want to carry out general PC tasks like word processing and spreadsheets and browsing the web, or do you have more specialist needs in mind, such as gaming?Budget: Small doesn't always mean cheap, but you can save money by choosing a mini PC that suits your needs.Ports: Most mini PCs come with a decent array of ports, but if you need something like two Ethernet ports or a bunch of USB ports, you need to ensure you're getting the right mini PC for you. Show more How did we choose these mini PCs? There are a lot of mini PCs on the market from a variety of different vendors, and I've tested and used mini PCs from all the vendors listed here, I've based my selections on my experience using these brands, as well as current model user reviews.Product specs and price also come into play, but being able to draw on dozens, and sometimes hundreds, of other people's views allows me to get a broad sense of how well-received a selected mini PC is. I've paid particular attention to reports of performance, stability, how quiet or noisy the PC is, and weeded out many that seemed to have a poor lifespan.What we're left with is the best of the best, in a broad selection of categories that are applicable to mini PCs. Show more What is a mini PC? A mini PC is a compact yet fully functional computer that offers many, if not all of, the capabilities of a desktop PC, but is packaged into a smaller form factor. Mini PCs are designed to save space and can be used for a variety of tasks, from basic computing to more demanding applications such as programming or even gaming. Show more What are the advantages of using a mini PC? People often choose to invest in a mini PC for these reasons:Space-saving:The top benefit of going for a mini PC is that it takes up significantly less space than traditional desktop PCs.Portable:Mini PCs are lightweight and easy to move, making them ideal for users who need computing power on the go.Energy efficiency:Most mini PCs consume significantly less power than a similarly-specced desktop system, which can lead to cost savings on electricity.Quiet:Some models are fanless, resulting in quieter operation, and even those that still have a fan are far quieter than a standard desktop system. Show more Are mini PCs upgradeable? It depends on the model. Some mini PCs allow for upgrades -- sometimes very easy upgrades -- to components like RAM and storage, while others have fixed configurations.On the whole however, like laptops, mini PCs are better aimed at those situations where you buy the PC with the spec that you need rather than relying on an upgrade down the line. Show more Are there alternative mini PCs worth considering? Here are a few mini PCs that didn't make the cut to the main list, but are still worthy of an honorable mention. Show more Further ZDNET Tech Coverage Smartphones Smartwatches Tablets Laptops TVs Other Tech Resources ZDNET Recommends

A year of tumultBrad Templeton / Tesla / Waymo / Zoox2024 was a year of tumult (again) for the self-driving industry, with growth and success for Waymo and Chinese players, middling results for a few others, and doom for some of the biggest. Lets count em down:Honorable Mention: The othersOur top 10 focus on the biggest players, but the next tier had a fair bit of action. Aurora kept improving their trucks and promises real, unmanned operation last year, but the world is now less interested in promises of next year and wants to know what you can really do.We did see improvements though from them and other trucking players like Kodiak, Gatik and Plus. In China, WeRide announced it will also operate in the UAE and AutoX has been quiet with announcements. Pony, on the other hand, completed an IPO in the USA and said it would quadruple to 1,000 vehicles in 2025. On the other hand, new US laws effectively forbid Chinese cars from operating in the USA.Israels MobilEye both canceled their internal LIDAR work and continued a shift away from planning its own robotaxi offering; it will instead work with partners. One notable partner was Rimacs Verne dedicated robotaxi service which showed a nice custom robotaxi, but only as a concept.MORE FOR YOUAt #10 was trouble at Motional, the MIT/Singapore based robocar startup that was funded by Hyundai and Aptiv after Aptiv bought it. Its all Hyundai now, and the CEO/founder left. With the universal trouble at Automaker-based robocar projects, things look dim for Motional, but hope remains that they can pull something together.At #9 was one of the years biggest shutdowns of a major project, as Apple killed its secretive Project Titan after many years. The worlds top tech company will stay out of self-driving, which is a shame as it has resources to match Google and Amazon (which remain in the game) and this takes that level of resources. For Apple, though, they will do just fine either wayin fact a large fraction of the worlds ride hails will still require Apple products to happen.Nuro is the best funded of the self-driving pure startups, and it had a focus on delivery with its own custom small road-based robots. Theyve dropped that path and now plan to license their self-driving tech (which is one of the very few projects to put vehicles out on the road with nobody on board) to the various OEMs and others who will need it. Meanwhile, trucking companies, including small-truck operator Gatik, continue to focus on delivery, and sidewalk robots are going strong. Starship, a company I helped build, announced reaching 7 million autonomous deliveries, which is more autonomous trips than anybody else, including Waymothough on sidewalks, not roads.#7: The rise of new AI techniques has been the tech story of the decade, particularly the application of Googles Transformers to Large Language Models. LLMs dont just do language, though, they do anything that can be turned into a string of tokens, including the stream of events perceived by a car driving down the road and its future plans. Now every team is using both the deep learning networks of the 2010s as the key tool for perception and LLMs for prediction and planning, if theyre not going all the way to trying an end to end network which takes pixels/point clouds in and outputs driving commands. LLM planners are showing powerful abilities at producing natural driving and understanding situations, though theyve also resulted in some surprising behaviors, including some crashes (though the teams dont ever admit that.) Some believe in pure neural networks, others have found that they plateau and more complex approaches combining several methods are necessary. No pure machine learning vehicle has yet reached the bar of driving on its own without a safety driver.#6: The Chinese leader is Baidu, which now claims to be doing 75,000 autonomous robotaxi rides per week, with 70-80% having no safety driver. Thats a clear #2 behind Waymo. They also announced their new custom Robotaxi design which they claim they can build for around $28,000 in China, including LIDARs and other extra gear. Shame the USA wants to slap a 100% tariff on imported Chinese EVs and ban Chinese software in them.The timelineBrad TempletonAt #5 is Waymos growth, which includes doing 4 million autonomous rides in 2024, rising to do 150,000 per week, and expanding to more territory in Los Angeles (with access to the general public) and more in the SF Bay Area, with approval for the whole peninsula. They also announced they will open in 2024 in Atlanta and Austin with a new test relationship with Uber, which will handle ride-hail and depots and fleet management. (Uber is a ride-hail option in Phoenix.) They will also expand to Miami in 2026 with Moove doing depots and the Waymo One app doing exclusive ride-hail. This is the beginning of the long awaited scaling for Waymo. Theyve also published impressive safety data and cemented a clear leadership. This timeline infographic paints their progress and that of their competitors.At #4, different news for Waymo. They had developed a 6th generation platform to replace the discontinued Jaguar, based on a minivan-like custom robotaxi design from Chinas Geely/Zeekr. But US plans for a 100% tariff on Chinese EVs put a big barrier in the way, so Waymo also announced a partnership with Hyundais new advanced vehicle group to produce an alternate version of the 6th generation vehicle on top of an Ioniq 5. Tariffs wont make the Chinese tiger go away for long, though, particularly globally.#3: Elon Musk, armed with data from Twitter and an estimated quarter billion dollars bought deep influence in the U.S. government by backing the victory of Donald Trump, and being named to a new Department of Government Efficiency. That makes his companies, including Tesla, seemingly immune from regulation, which in turn suggests a massive change (and reduction) in regulation of self-driving cars, particularly Tesla FSD. Musk has always said the one thing he could not predict about when his technology would deploy is regulation, but now he practically controls it. Unless Musk gets deregulation only of Tesla, this should be good for all players, except the Chinese.Musk is already the most distracted CEO in the world, though, and his governmental role wont make that better.#2: Tesla always makes news. Their FSD stack has shown significant improvement this year, though its still a very long way from being able to operate without supervision, though as he has for the last 7 years, Musk predicts that it will in just one year. Ironically, the rapid progress is a sign the product is still quite immaturea mature product wont see much visible improvement in its last few years before deployment. If a new version appears much better, it means it was seriously flawed before and surely still is.The hard reality is that Tesla FSD can complete only a few drives in a row without a problem, while Waymo is doing tens of thousands in a row. Waymo only drives a few cities because thats the right plan, they could easily drive all cities at a level better than Tesla if that made sense, but it doesnt.That made Teslas big We Robot launch for the Tesla Robotaxi/Cybercab a bit of a non-event, because while the concept cars and van looked nice, the software still has far to go. It showed Teslas dedication to being a robotaxi company, and most of the event was Musk evangelizing the reasonswith a speech very similar to the ones Ive been doing for over a decade.The top story is a sad one: GM has shut down Cruises Robotaxi efforts, and will redirect whats left of the company to work on ADAS tools for GMs consumer cars. This wasnt too shocking. Cruise had gone into semi-hibernation after being kicked off the California streets a year ago, and it put their custom Origin vehicle on the back-burner too. Signs of life were rare, but they kept spending money on the leaner company, which suggested they really wanted to bring it back to lifeuntil they didnt.While its far from assured, theres a distinct chance that this shutdown will be marked later in history as the moment GM died, afraid of the future. Yes, it will save money, as will Aptiv, Ford and others. Mercedes just got permission for their highway-only car to go 95 km/h in Germany, so its still in the game in a more limited way. Broadly though, the automaker-backed ventures have all faced trouble, though non-traditional automakers like Tesla and the Chinese EV makers arent flagging in their enthusiasm.Of course, GM might be able to rejoin the game by licensing from Nuro or MobilEye or another provider. This was always the likely path for automakers, but leaves them not in control of what will become the most important component of their vehicles, not a place they wanted to be.It will also be marked that Cruise and Uber, the two projects which have hit pedestrians to serious results, both received the corporate death penalty and were shut down as a result. In this case, the DMV played a large role, surely hastening Cruises end by removing them from the streets, in part for that incident. While some question whether robocar projects need more regulation, theres also a compelling case for less, or at least better regulation, less likely to kill multi-billion dollar projects unless their failings are clear.We enter 2025 with only Waymo operating a robotaxi service in the west. Zoox planned to open one but has delayed it until next year. May mobility has a limited shuttle service with no staffer on board, but full-time remote supervision. Chinese teams continue to advance. Tesla remains a wild card, many years behind Waymo and the others in safety performance, but trying different and brash approaches with the most unpredictable CEO behind the rudder, able to literally control governments to get his way. The trucking players are eager to go into production and many startups continue their quest.The shakeout may not even be over, but 2025 will certainly offer interesting times.

In this weeks edition of The Prototype, we look at a new lunar mission for startup company Firefly, how to teach old robots new tricks, our new robotic ant overlords and more. You can sign up to get The Prototype in your inbox here.FireflyThis week, NASA announced that it has awarded a new $179 million contract to Texas-based Firefly Aerospace as part of the Artemis lunar missions, the fourth it has awarded to the company to deliver projects to the Moons surface.The mission, which aims to launch in 2028, will utilize the space startups Blue Ghost cargo lander to deliver six scientific experiments to an area on the Moon called Gruithuisen Domes, which were formed by lava millions of years ago in order to better understand Lunar geology.Fireflys first Moon mission for NASA is currently slated to launch in mid-January 2025 on top of a SpaceX Falcon 9 rocket. That mission is scheduled to land at Mare Crisium in order to study a variety of surface conditions to help prepare for future crewed missions to the Moon.Stay tuned.P.S. This is the last edition of The Prototype for 2024. Ill be back on January 3rd. Happy holidays!This Startup Is Teaching Old Robots New TricksNo Coding RequiredT-RoboticsFactories across the country rely on large, robotic arms to carry out a variety of tasks. These are large, versatile and already embedded within the facilitys infrastructure. Theyre also difficult to program. Asad Tirmizi, CEO and cofounder of Bay Area startup T-Robotics, wants to change thatby making them smarter by letting them be trained for new tasks by people without programming knowledge.The company announced this week that it has raised a $5.4 million investment seed round, and that it will launch its first commercial product with robotics company ABB next year.Our thesis is, we can make your language the new programming language of robots, Tirmizi told me.To that end, his company has developed a software layer, called ActGPT, for these robotsregardless of the brand or model. Users can tell the robot what they want to have accomplished by having a conversation with ittelling it what to do while getting feedback the robot.T-Robotics software uses whats called a Visual Language Haptic Action model, including not only a language component to interact with users, but also sensors and feedback to learn to correctly do a task. There is a planning component that uses all this information to see if it has the correct information it needs, Tirmizi explained. If the robot needs more information, it prompts the user. And this back-and-forth enables someone to program the robot without knowing a bit of code.You can be a complete rookie in robot programming, but end up with a competent industrial-grade application, Tirmizi said.DISCOVERY OF THE WEEK: A SWARM OF ANT-LIKE ROBOTSResearchers in South Korea have developed tiny magnetic robots that are capable of working together in swarms, much like ants, to perform tasks like throwing things, lifting much heavier objects, and pushing down obstacles. A group of a thousand of the cube-shaped robots were able to shape themselves into a raft and move along water. They are incredibly small (about 1/50th of an inch) and made from an epoxy containing particles of neodymium-iron-boron. The movements are powered by a magnetic field that enables them to self-assemble and are programmed by varying the angle by which they are magnetized. The research teams findings were published in the journal Device this week.FINAL FRONTIER: NEXT CREWED NASA LAUNCH DELAYEDNASA astronauts Suni Williams and Butch Wilmore were only supposed to be on the International Space Station on a weeklong trip. But much like the passengers and crew of the S.S. Minnow, their journey is taking more time than expected. The pair have been on board more than six monthsand it looks like theyll be there even longer now.Previously, NASA planned to return them to Earth, along with Nick Hague and cosmonaut Aleksandr Gorbunov, in February. But that dates been pushed back to no earlier than late March as the space agency and SpaceX complete work on the new Dragon spacecraft that will be used for the mission. That capsule will carry NASA astronauts Anne McClain and Nichole Ayers, JAXA astronaut Takuya Onishi and cosmonaut Kirill Peskov to the station.FORBES CALLED IT: NEXT GENERATION IVFTwo years ago, we recognized Dina Radenkovic for our annual 30 Under 30 list in Healthcare. That was based on the potential of the next generation IVF technology her startup, Gameto, had developed. It enables extracted eggs to mature in synthetic ovaries, shortening the length of time the IVF procedure requires while lowering the overall cost. This week, that potential proved out as the company announced the first birth of a baby facilitated by its technology was born to a couple in Brazil.WHAT ELSE I WROTE THIS WEEKThe Federal Reserve warned this week that it is forecasting fewer rate cuts in 2025 as inflation persists. I wrote about why that could be bad news for biotech startups.In my other newsletter, InnovationRx, I wrote about the persistence of avian flu, which has prompted a state of emergency in California, due to the number of dairy farms affected by the disease, on the same day the CDC announced the first severe case in a patient in Louisiana.SCIENCE AND TECH TIDBITS Iran has a thriving black market for Starlink terminals, with an underground worldwide network of smuggling and advocacy aiming to bring an uncensored internet into the country.Anthropologists argued in the journal Nature Astronomy this week that space agencies should include the tracking and preservation of human spacecraft, landers, etc. in their planetary protection plans for future Mars missions. (Though hopefully they'd forgive Mark Watney for disturbing Pathfinder.)Commonwealth Fusion Systems says it plans to build its first grid-scale fusion power plant in Virginia, with an aim of being operational in the early 2030s.An Alabama woman is in good health after becoming the third living person to receive a pig kidney transplant, NYU Langone Health announced earlier this week.Chinese astronauts Cai Xuzhe and Wang Haoze completed a 9 hour, 6 minute spacewalk while installing devices to protect Chinas Tiangong space station from space debris, reported SpaceNews. This breaks the previous record for a spacewalk, which was 8 hours and 56 minutes.A research team created a new type of sunscreen that both protects skin from harmful UV rays and also cools the skin at the same time.PRO SCIENCE TIP: AVOID INJURIES WITH FLATTER RUNNING SHOESScientists at the University of Florida discovered that runners who wear shoes with thick heels are more likely to suffer an injury than those who wear thinner, flatter shoes. The culprit appears to be sensationthicker heels make it harder to feel how your foot lands, making an injury more likely. But that doesnt mean you should ditch your shoes right awaythe researchers add that a transition should be gradual so you can work on strengthening your feet and learn to land in a more controlled way. The research findings were published in Frontiers in Sports and Active Living. WHATS ENTERTAINING ME THIS WEEKThis week I watched the series finale of What We Do In The Shadows, a mockumentary TV series about a group of vampires living together on Staten Island. Over the course of six seasons, this was consistently one of the funniest shows on TVparticularly thanks to Matt Berrys incredible line readings. Series finales are hard to pull off well, but the show stuck the landing. All episodes are currently streaming on Hulu.MORE FROM FORBES

In brief: Recent leaks have revealed nearly all the important details regarding Nvidia's upcoming desktop RTX 5000 graphics cards, but information on their mobile counterparts has been slightly more elusive. New retail listings for upcoming Asus laptops confirm the memory, wattages, and corresponding CPUs for the next-generation mobile GPUs. As Nvidia and AMD prepare to launch new chips in the coming weeks, retailers and manufacturing partners have repeatedly spoiled hardware surprises by listing products too early. The latest false start has divulged critical details for Nvidia's entire next-generation laptop GPU lineup.According to 2 Cent and other retailers, Asus is preparing to launch multiple new ROG laptops in early 2025 with most featuring RTX 5000 discrete graphics. Prior reports revealed scant details regarding each GPU's memory and power draw, but a new chart summarizing the retail leaks offers a mostly complete picture.Like the mobile RTX 4090, the 5090 features 16GB of VRAM. However, the 5080 will also include 16GB an upgrade from its predecessor's 12GB. This amount was easy to guess from a ransomware leak targeting Taiwanese laptop manufacturer Clevo in June, but that leak also listed unnamed 12GB and 8GB cards, which can now be identified.The RTX 5070 Ti is the sole 12GB GPU in Nvidia's new mobile lineup. Everything below it is 8GB, including the entry-level RTX 5050, which receives a bump from the 4050's 6GB. All RTX 5000 graphics cards, including the mobile 5050 and desktop 5060, feature GDDR7 VRAM. // Related StoriesAlso Read: Nvidia GeForce RTX 4090 Laptop vs. Desktop GPUMoreover, the top two laptop GPUs draw a maximum of 175W while the 5070 Ti uses 140W. Wattages for the mid-range and low-end cards remain unavailable, but the CEO of Chinese OEM Hasee claimed in August that the 5060 draws only 115W.Additionally, Asus' upcoming laptops will pair RTX 5000 cards with the latest Intel and AMD CPUs, refuting prior reports claiming they would stick with older Intel processors. Variants will include the company's Core Ultra 9 285H, 285HX, and 275HX, along with AMD's Ryzen 9 7945HX and 7940HX. Flagship units include 2K 240Hz monitors while most others settle for 1080p at the same refresh rate.Meanwhile, AMD's upcoming enthusiast APU, the Ryzen AI Max+ 395, will debut in Asus' 2025 ROG Flow Z13. The integrated GPU, which features 40 RDNA 3.5 CUs, is paired with 32GB of RAM and a 13-inch 2K 180Hz screen.Nvidia is expected to reveal the desktop and laptop RTX 5000 graphics cards at CES in January.

MGMTable of ContentsTable of Contents10. Quantum of Solace (2008)9. Road to Perdition (2002)8. Logan Lucky (2017)7. The Adventures of Tintin (2011)6. Skyfall (2012)5. Layer Cake (2004)4. Munich (2005)3. Knives Out (2019)2. Casino Royale (2006)1. The Girl with the Dragon Tattoo (2011)This month, Daniel Craig earned the best reviews of his career for Luca Guadagninos acclaimed drama Queer. But the veteran English actor has always had an interesting career, from his early roles in The Power of One and Tales from the Crypt to his more recent films like the Knives Out mysteries. A reconsideration of his filmography is, therefore, expected and warranted not dismissing his revolutionary 15 years as James Bond but putting them in the context of the variegated career out of which they sprang. So prep those martinis (shaken or stirred) and sharpen those knives as here are the 10 best Daniel Craig films.Recommended VideosMGMRegarded as a disappointment at the time of its release, Quantum of Solace, Craigs second outing as James Bond, is easily the third-strongest of his Bond pictures.RelatedTheres a distinct, hazy visual style thanks to director Marc Forster (the youngest helmer in the franchises history), a killer spycrafty set piece at an ultra-modern opera house, and the always-excellent Gemma Arterton as a secondary Bond girl named, in the classic tradition of the franchise, Strawberry Fields. Her murder by being covered in crude oil (sure, why not?) brings to mind a parallel sequence from Goldfinger (1964), another of the films nods to Bonds past.DreamWorksCraig and Tom Hanks both play against type Hanks as an as amoral Chicago mobster, Craig as his bosss neer-do-well son in Sam Mendess adaptation of the 1998 graphic novel Road to Perdition. With its stark blacks and whites and eerie static shots, it anticipates Zack Snyders more self-consciously comic-book-adjacent films of the middle 2000s.This one is of interest as a necessary precursor to Mendes and Craigs later collaborations on Skyfall and Spectre, as Paul Newmans last on-screen role, and as a showcase for Craig in his early days as a character actor/heavy. Craig, as Connor, a mobsters son no one has any particular interest in keeping alive, makes use of the highly questionable American accent hed later supplant with his Benoit Blanc Foghorn Leghorn drawl.Bleeker StreetIn the sixth of Stephen Soderberghs (thus far) seven heist movies (the guy just loves heists!), the thickheaded robbers are yokel brothers Jimmy (Channing Tatum) and Clyde Logan (Adam Driver), the target is the Charlotte Motor Speedway, and Craig is the demolitions expert who blows a hole in its vault.The film, Soderberghs first since his short-lived retirement in 2012, may ultimately have a bit too much on its mind. But its still a candy-colored carousel of outrageous cameos and Bible Belt-isms, a working-class Oceans Eleven (or, as its winkingly referred to in the film, Oceans Seven-Eleven).Paramount PicturesAn utterly bizarre relic of early 2010s culture, The Adventures of Tintin was meant to be the first in a long-running animated franchise co-spearheaded by Steven Spielberg and Peter Jackson. That no sequel has since appeared is perhaps thanks to its conspicuous use of the now-archaic motion-capture technology pioneered by Robert Zemeckis in the early 2000s, or Spielbergs schedule, or Jacksons (utterly superfluous Hobbit sequels wont make themselves).But its not for lack of charm on the part of this light-stepping lark of an adventure pic, nor for lack of a memorable villain, as Craig plays the malevolent treasure hunter Ivan Ivanovitch Sakharine, surely the only man in history thus named with a British accent.Is it, as has been suggested, the best Bond movie ever? Absolutely not its not even Craigs best Bond film ever (see below). But Skyfall is an action extravaganza that also manages to be a reckoning with mortality and its own franchises mythos, a subtle retelling of The Odyssey, and the mantelpiece for what is unequivocally the best Bond theme ever. (Adeles Skyfall tune is so good that it managed through pure residual staying power to win Oscars not just for itself but also for Sam Smiths and Billie Eilishs textureless and odorless themes for the next two Bond films.)Sony Pictures ClassicsThe directorial debut of future Kingsman doyenne Matthew Vaughn, Layer Cake is a conscious throwback to an all-too-brief era of British crime films that was already over by the time of its release. (Im referring to dialectically incomprehensible bits of scenery-chewing and gunplay like Lock, Stock and Two Smoking Barrels, Snatch, and Sexy Beast, the former two also produced by Vaughn.)Craigs a gangster and drug dealer, name never revealed, who makes his way through an ensemble of British heavy hitters including but not limited to Michael Gambon, Ben Whishaw, Tom Hardy (eons before Venom: The Last Dance), Sally Hawkins, and Sienna Miller Miller at her best in the femme fatale/damsel in distress blend shed come to perfect. Often credited as the deciding factor in Craigs Bond casting, this fleet crime drama is a delectable mishmash of high and low, the alleyway and the country club.Universal PicturesCraig came to Steven Spielbergs Tintin through this earlier collaboration, the story of a Mossad-sponsored team of freelancers hunting down those responsible for the massacre of Israeli athletes at the 1972 Olympics. Munich ranks high among Spielbergs historical dramas thanks to its star-studded (if dubiously Jewish) team of Craig, Eric Bana, Ciarn Hinds, and Geoffrey Rush.As a South African Jewish driver for the team of agents, Craig embodies the surging moral motivation for the crusade but also the ambiguity in an officially unaffiliated team of killers taking out radicals around the world.Lionsgate FilmsThe new film characters strong enough to carry a franchise that have been created in the past decade can be counted on the fingers of one hand actually, now that I think about it, on one finger.That honor goes to Benoit Blanc, the consulting detective with the high-camp southern accent and the proclivity for Stephen Sondheim, who debuted in Rian Johnsons excellent 2019 whodunit Knives Out and was played by whats this! James Bond himself, seeking an escape from a straitjacketing identification with a single character. As Blanc jump-started an ongoing Netflix franchise that looks likely to go on for years, perhaps it was a mixed blessing.MGMBond was, simply, never better, not in the 60s and not since, than in Craigs muscular debut in Casino Royale, the latest and perhaps last Bond film to be explicitly adapted from an Ian Fleming novel (of the same name).Yes, this Bond is grittier, yes, hes a street brawler with a face that looks as if its taken one too many punches, and yes, hes blond, but Bond himself matters less in these movies than the sumptuous plots and circumstances with which the filmmakers manage to surround him. Casino Royale, set in the world of high-stakes gambling and featuring both an all-time Bond girl (Eva Green) and an all-time baddie (Mads Mikkelsen), gives the character stakes and heart and humanity but never ceases to be pure red-meat fun.SonyMiles better than its Swedish-language adaptation from 2009, director David Finchers take on the 2005 Swedish mystery novel by Stieg Larsson is as maneuverable and sinuous as its protagonist, hacker Lisbeth Salander (Rooney Mara). Craigs crusading journalist is manifestly in over his head in this story of serial murders linked by eerie familial connections.But Craig himself is magisterially brooding and plays off Mara with aplomb. That the film never spawned the franchise it deserved itself can likely be blamed on Bond. But then, so can everything.Editors Recommendations

html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd" Hisense has a little pre-CES 2025 teaser for us, and it looks like the company is taking a page from Sonys home theater playbook. The new Hisense HT Saturn is a 4.1.2 Dolby Atmos wireless home theater system with four speakers and a single subwoofer, which uses a small HDMI ARC/eARC breakout box as a transmitter a similar configuration to Sonys Bravia Theater Quad.Hisense hasnt released pricing or availability for the HT Saturn yet.HisenseHisense has equipped the Dolby Atmos soundbar alternative with two features that should make it especially appealing to some Hisense TV owners: an automatic room calibration system that gives you flexibility of placement for the speakers, and in another move that mimics the Bravia Theater Quads S-center channel tech, Hisenses Hi-Concerto feature lets a Hisense TVs speakers share the audio duties with the wireless speakers.Recommended VideosThese features only work with some Hisense TVs, but we havent received an official list of the compatible models.Please enable Javascript to view this contentThe system itself is compatible, however, with any TV that has an HDMI ARC/eARC or optical output (it also works with Bluetooth), but as with Hisenses other soundbar offerings, youll be able to control and configure the HT Saturn using a Hisense TV remote, once again making the system feel much more integrated for Hisense TV owners.RelatedThe system also works with DTS:X and has a total of 13 speakers. Hisense hasnt given us a breakdown of the kinds and sizes of drivers, other than the 6.5-inch subwoofer. It can be tuned using one of five available EQ modes, including presets for movies, music, sports, and more.Hisense hasnt said if, like the Bravia Theater Quad, the transmitter box includes an HDMI input(s) to offset the HDMI port that the system will occupy on your TV.Our first chance to see and hear the HT Saturn will be at CES 2025, where we also expect to see Hisenses latest TVs, laser TVs, and other audio and video products.Editors Recommendations

A documentary revisits the case of Xaviar Babudar, a Kansas City football obsessive who was well known at games and onlineand had another, decidedly more criminal pastime.

Scaling problems Why AI language models choke on too much text Compute costs scale with the square of the input size. That's not great. Timothy B. Lee Dec 20, 2024 8:00 am | 9 Credit: Aurich Lawson | Getty Images Credit: Aurich Lawson | Getty Images Story textSizeSmallStandardLargeWidth *StandardWideLinksStandardOrange* Subscribers only Learn moreLarge language models represent text using tokens, each of which is a few characters. Short words are represented by a single token (like "the" or "it"), whereas larger words may be represented by several tokens (GPT-4o represents "indivisible" with "ind," "iv," and "isible").When OpenAI released ChatGPT two years ago, it had a memoryknown as a context windowof just 8,192 tokens. That works out to roughly 6,000 words of text. This meant that if you fed it more than about 15 pages of text, it would forget information from the beginning of its context. This limited the size and complexity of tasks ChatGPT could handle.Todays LLMs are far more capable:OpenAIsGPT-4o can handle 128,000 tokens (about 200 pages of text).AnthropicsClaude 3.5 Sonnet can accept 200,000 tokens (about 300 pages of text).GooglesGemini 1.5 Pro allows 2 million tokens (about 2,000 pages of text).Still, its going to take a lot more progress if we want AI systems with human-level cognitive abilities.Many people envision a future where AI systems are able to do manyperhaps mostof the jobs performed by humans. Yet many human workers read and hear hundreds of millions of words during our working yearsand we absorb even more information from sights, sounds, and smells in the world around us. To achieve human-level intelligence, AI systems will need the capacity to absorb similar quantities of information.Right now the most popular way to build an LLM-based system to handle large amounts of information is called retrieval-augmented generation (RAG). These systems try to find documents relevant to a users query and then insert the most relevant documents into an LLMs context window.This sometimes works better than a conventional search engine, but todays RAG systems leave a lot to be desired. They only produce good results if the system puts the most relevant documents into the LLMs context. But the mechanism used to find those documentsoften, searching in avector databaseis not very sophisticated. If the user asks a complicated or confusing question, theres a good chance the RAG system will retrieve the wrong documents and the chatbot will return the wrong answer.And RAG doesnt enable an LLM to reason in more sophisticated ways over large numbers of documents:A lawyer might want an AI system to review and summarize hundreds of thousands of emails.An engineer might want an AI system to analyze thousands of hours of camera footage from a factory floor.A medical researcher might want an AI system to identify trends in tens of thousands of patient records.Each of these tasks could easily require more than 2 million tokens of context. Moreover, were not going to want our AI systems to start with a clean slate after doing one of these jobs. We will want them to gain experience over time, just like human workers do.Superhuman memory and stamina have long been key selling points for computers. Were not going to want to give them up in the AI age. Yet todays LLMs are distinctly subhuman in their ability to absorb and understand large quantities of information.Its true, of course, that LLMs absorb superhuman quantities of information at training time. The latest AI models have been trained on trillions of tokensfar more than any human will read or hear. But a lot of valuable information is proprietary, time-sensitive, or otherwise not available for training.So were going to want AI models to read and remember far more than 2 million tokens at inference time. And that wont be easy.The key innovation behind transformer-based LLMs is attention, a mathematical operation that allows a model to think about previous tokens. (Check out our LLM explainerif you want a detailed explanation of how this works.) Before an LLM generates a new token, it performs an attention operation that compares the latest token to every previous token. This means that conventional LLMs get less and less efficient as the context grows.Lots of people are working on ways to solve this problemIll discuss some of them later in this article. But first I should explain how we ended up with such an unwieldy architecture.GPUs made deep learning possibleThe brains of personal computers are central processing units (CPUs). Traditionally, chipmakers made CPUs faster by increasing the frequency of the clock that acts as its heartbeat. But in the early 2000s, overheating forced chipmakers to mostly abandon this technique.Chipmakers started making CPUs that could execute more than one instruction at a time. But they were held back by a programming paradigm that requires instructions to mostly be executed in order.A new architecture was needed to take full advantage of Moores Law. Enter Nvidia.In 1999, Nvidia started selling graphics processing units (GPUs) to speed up the rendering of three-dimensional games like Quake III Arena. The job of these PC add-on cards was to rapidly draw thousands of triangles that made up walls, weapons, monsters, and other objects in a game.This isnota sequential programming task: triangles in different areas of the screen can be drawn in any order. So rather than having a single processor that executed instructions one at a time, Nvidiasfirst GPUhad a dozen specialized coreseffectively tiny CPUsthat worked in parallel to paint a scene.Over time, Moores Law enabled Nvidia to make GPUs with tens, hundreds, and eventually thousands of computing cores. People started to realize that the massive parallel computing power of GPUs could be used for applications unrelated to video games.In 2012, three University of Toronto computer scientistsAlex Krizhevsky, Ilya Sutskever, and Geoffrey Hintonused a pair ofNvidia GTX 580 GPUsto train a neural network for recognizing images. The massive computing power of those GPUs, which had 512 cores each, allowed them to train a network with a then-impressive 60 million parameters. Theyentered ImageNet, an academic competition to classify images into one of 1,000 categories, andset a new record for accuracyin image recognition.Before long, researchers were applying similar techniques to a wide variety of domains, including natural language.Transformers removed a bottleneck for natural languageIn the early 2010s, recurrent neural networks (RNNs) were a popular architecture for understanding natural language. RNNs process language one word at a time. After each word, the network updates its hidden state, a list of numbers that reflects its understanding of the sentence so far.RNNs worked fairly well on short sentences, but they struggled with longer onesto say nothing of paragraphs or longer passages. When reasoning about a long sentence, an RNN would sometimes forget about an important word early in the sentence. In 2014, computer scientists Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengiodiscoveredthey could improve the performance of a recurrent neural network by adding an attention mechanism that allowed the network to look back at earlier words in a sentence.In 2017, Google publishedAttention Is All You Need,one of the most important papers in the history of machine learning. Building on the work of Bahdanau and his colleagues, Google researchers dispensed with the RNN and its hidden states. Instead, Googles model used an attention mechanism to scan previous words for relevant context.This new architecture, which Google called the transformer, proved hugely consequential because it eliminated a serious bottleneck to scaling language models.Heres an animation illustrating why RNNs didnt scale well:This hypotheticalRNN tries to predict the next word in a sentence, with the prediction shown in the top row of the diagram. This network has three layers, each represented by a rectangle. It is inherently linear: it has to complete its analysis of the first word, How, before passing the hidden state back to the bottom layer so the network can start to analyze the second word, are.This constraint wasnt a big deal when machine learning algorithms ran on CPUs. But when people started leveraging the parallel computing power of GPUs, the linear architecture of RNNs became a serious obstacle.The transformer removed this bottleneck by allowing the network to think about all the words in its input at the same time:The transformer-based model shown here does roughly as many computations as the RNN in the previous diagram. So it might not run any faster on a (single-core) CPU. But because the model doesnt need to finish with How before starting on are, you, or doing, it can work on all of these words simultaneously. So it can run alotfaster on a GPU with many parallel execution units.How much faster? The potential speed-up is proportional to the number of input words. My animations depict a four-word input that makes the transformer model about four times faster than the RNN. Real LLMs can have inputs thousands of words long. So, with a sufficiently beefy GPU, transformer-based models can be orders of magnitude faster than otherwise similar RNNs.In short, the transformer unlocked the full processing power of GPUs and catalyzed rapid increases in the scale of language models. Leading LLMs grew fromhundreds of millions of parametersin 2018 tohundreds of billions of parametersby 2020. Classic RNN-based models could not have grown that large because their linear architecture prevented them from being trained efficiently on a GPU.Transformers have a scaling problemEarlier I said that the recurrent neural network in my animations did roughly the same amount of work as the transformer-based network. But they dont do exactlythe same amount of work. Lets look again at the diagram for the transformer-based model:See all those diagonal arrows between the layers? They represent the operation of the attention mechanism. Before a transformer-based language model generates a new token, it thinks about every previous token to find the ones that are most relevant.Each of these comparisons is cheap, computationally speaking. For small contexts10, 100, or even 1,000 tokensthey are not a big deal. But the computational cost of attention grows relentlessly with the number of preceding tokens. The longer the context gets, the more attention operations (and therefore computing power) are needed to generate the next token.This means that the total computing power required for attention grows quadratically with the total number of tokens. Suppose a 10-token prompt requires 414,720 attention operations. Then:Processing a 100-token prompt will require 45.6 million attention operations.Processing a 1,000-token prompt will require 4.6 billion attention operations.Processing a 10,000-token prompt will require460 billionattention operations.This is probably why Google charges twice as much, per token, for Gemini 1.5 Pro once the context gets longer than 128,000 tokens. Generating token number 128,001 requires comparisons with all 128,000 previous tokens, making it significantly more expensive than producing the first or 10th or 100th token.Making attention more efficient and scalableA lot of effort has been put into optimizing attention. One line of research has tried to squeeze maximum efficiency out of individual GPUs.As we saw earlier, a modern GPU contains thousands of execution units. Before a GPU can start doing math, it must move data from slow shared memory (called high-bandwidth memory) to much faster memory inside a particular execution unit (called SRAM). Sometimes GPUs spend more time moving data around than performing calculations.In aseriesofpapers, Princeton computer scientist Tri Dao and several collaborators have developed FlashAttention, which calculates attention in a way that minimizes the number of these slow memory operations. Work like Daos has dramatically improved the performance of transformers on modern GPUs.Another line of research has focused on efficiently scaling attention across multiple GPUs. One widely cited paper describesring attention, which divides input tokens into blocks and assigns each block to a different GPU. Its called ring attention because GPUs are organized into a conceptual ring, with each GPU passing data to its neighbor.I once attended a ballroom dancing class where couples stood in a ring around the edge of the room. After each dance, women would stay where they were while men would rotate to the next woman. Over time, every man got a chance to dance with every woman. Ring attention works on the same principle. The women are query vectors (describing what each token is looking for) and the men are key vectors (describing the characteristics each token has). As the key vectors rotate through a sequence of GPUs, they get multiplied by every query vector in turn.In short, ring attention distributes attention calculations across multiple GPUs, making it possible for LLMs to have larger context windows. But it doesnt make individual attention calculations any cheaper.Could RNNs make a comeback?The fixed-size hidden state of an RNN means that it doesnt have the same scaling problems as a transformer. An RNN requires about the same amount of computing power to produce its first, hundredth and millionth token. Thats a big advantage over attention-based models.Although RNNs have fallen out of favor since the invention of the transformer, people have continued trying to develop RNNs suitable for training on modern GPUs.In April, Googleannounced a new modelcalled Infini-attention. Its kind of a hybrid between a transformer and an RNN. Infini-attention handles recent tokens like a normal transformer, remembering them and recalling them using an attention mechanism.However, Infini-attention doesnt try to remember every token in a models context. Instead, it stores older tokens in a compressive memory that works something like the hidden state of an RNN. This data structure can perfectly store and recall a few tokens, but as the number of tokens grows, its recall becomes lossier.Machine learning YouTuber Yannic Kilcherwasnt too impressedby Googles approach.Im super open to believing that this actually does work and this is the way to go for infinite attention, but Im very skeptical, Kilcher said. It uses this compressive memory approach where you just store as you go along, you dont really learn how to store, you just store in a deterministic fashion, which also means you have very little control over what you store and how you store it.Could Mamba be the future?Perhaps the most notable effort to resurrect RNNs is Mamba, an architecture that was announced in aDecember 2023 paper. It was developed by computer scientists Dao (who also did the FlashAttention work I mentioned earlier) and Albert Gu.Mamba does not use attention. Like other RNNs, it has a hidden state that acts as the models memory. Because the hidden state has a fixed size, longer prompts do not increase Mambas per-token cost.When I started writing this article in March, my goal was to explain Mambas architecture in some detail. But then in May, the researchers released Mamba-2, which significantly changed the architecture from the original Mamba paper. Ill be frank: I struggled to understand the original Mamba and have not figured out how Mamba-2 works.But the key thing to understand is that Mamba has the potential to combine transformer-like performance with the efficiency of conventional RNNs.In June, Dao and Guco-authored a paperwith Nvidia researchers that evaluated a Mamba model with 8 billion parameters. They found that models like Mamba were competitive with comparably sized transformers in a number of tasks, but they lag behind Transformer models when it comes to in-context learning and recalling information from the context.Transformers are good at information recall because they remember every token of their contextthis is also why they become less efficient as the context grows. In contrast, Mamba tries to compress the context into a fixed-size state, which necessarily means discarding some information from long contexts.The Nvidia team found they got the best performance from a hybrid architecture that interleaved 24 Mamba layers with four attention layers. This worked better than either a pure transformer model ora pure Mamba model.A model needssomeattention layers so it can remember important details from early in its context. But a few attention layers seem to be sufficient; the rest of the attention layers can be replaced by cheaper Mamba layers with little impact on the models overall performance.In August, an Israeli startup called AI21 announced itsJamba 1.5 familyof models. The largest version had 398 billion parameters, making it comparable in size to Metas Llama 405B model.Jamba 1.5 Large has seven times more Mamba layers than attention layers. As a result, Jamba 1.5 Large requires far less memory than comparable models from Meta and others. For example, AI21 estimates that Llama 3.1 70B needs 80GB of memory to keep track of 256,000 tokens of context. Jamba 1.5 Large only needs 9GB, allowing the model to run on much less powerful hardware.The Jamba 1.5 Large model gets an MMLU score of 80, significantly below the Llama 3.1 70Bs score of 86. So by this measure, Mamba doesnt blow transformers out of the water. However, this may not be an apples-to-apples comparison. Frontier labs like Meta have invested heavily in training data and post-training infrastructure to squeeze a few more percentage points of performance out of benchmarks like MMLU. Its possible that the same kind of intense optimization could close the gap between Jamba and frontier models.So while the benefits of longer context windows is obvious, the best strategy to get there is not. In the short term, AI companies may continue using clever efficiency and scaling hacks (like FlashAttention and Ring Attention) to scale up vanilla LLMs. Longer term, we may see growing interest in Mamba and perhaps other attention-free architectures. Or maybe someone will come up with a totally new architecture that renders transformers obsolete.But I am pretty confident that scaling up transformer-based frontier models isnt going to be a solution on its own. If we want models that can handle billions of tokensand many people dowere going to need to think outside the box.Tim Lee was on staff at Ars from 2017 to 2021. Last year, he launched a newsletter,Understanding AI,that explores how AI works and how it's changing our world. You can subscribehere.Timothy B. LeeSenior tech policy reporterTimothy B. LeeSenior tech policy reporter Timothy is a senior reporter covering tech policy and the future of transportation. He lives in Washington DC. 9 Comments

Lisa Morgan, Freelance WriterDecember 20, 20249 Min ReadRancz Andrei via Alamy StockData governance isnt where it needs to be in many organizations, despite the widespread use of AI and analytics. This is risky on several levels such as cybersecurity and compliance, not to mention the potential impacts to various stakeholders. In short, data governance is becoming more necessary as organizations rely more heavily on data, not less.Steve Willis, principal research director, data, analytics, enterprise architecture and AI at Info-Tech Research Group offers a sobering statistic: Some 50% to 75% of data governance initiatives fail.Even in highly regulated industries where the acceptance and understanding of the concept and value of governance more broadly are ingrained into the corporate culture, most data governance programs have progressed very little past an expensive [check] boxing exercise, one that has kept regulatory queries to a minimum but returned very little additional business value on the investment, says Willis in an email interview.Most data professionals cite things like lack of business understanding and/or executive engagement, limited funding, the complexity of the data landscape or general organizational change resistance as the root-cause or causes as barriers to data governance implementation and the reason(s) why most data governance initiatives fail, though Willis disagrees.Related:A lack of a deep connection between the tangible outcomes business stakeholders care about and the activities and initiatives undertaken in the name of data governance is the primary cause of failure, says Willis. The few who have successfully implemented data governance can easily point to the value that data governance initiatives have delivered. [They are] able to provide a direct line of sight not only to tactical wins but to deep contributions to an organization achieving its strategic goals and objectives.Where the Problems LieMany data teams, particularly data governance teams, lack the proper relationships with business stakeholders, so the business has no visibility into how data governance works.Data governance teams should be rigorously focused on understanding how improvements in [data use] will tangibly make life easier for those managing and using data, be it removing critical pain points or creating new opportunities to add value, says Info-Techs Willis. By not focusing on their customers needs, many data governance professionals are over-focused on adding workload to those they are purporting to help in return for providing little measurable value.Related:Steve Willis, Info-Tech Research GroupWhy the disconnect? Data teams dont feel they can spend time understanding stakeholders or even challenging business stakeholder needs. Though executive support is critical, data governance professionals are not making the most out of that support. One often unacknowledged problem is culture.Unfortunately, in many organizations, the predominant attitude towards governance and risk management is that [they are] a burden of bureaucracy that slows innovation, says Willis. Data governance teams too frequently perpetuate that mindset, over-rotating on data controls and processes where the effort to execute is misaligned to the value they release.One way to begin improving the effectiveness of data governance is to reassess the organizations objectives and approach.Embed data governance activities, small step by small step into your current business operations, make managing data part of a business process owners day to day responsibilities rather than making the governance and management of data a separate thing, saysWillis. This abstraction of data governance and management away from business operations is a key reason why nominated data stewards, who are typically business process owners, dont understand what they are being asked to do. As a data governance team, you need to contextualize data management activities into the language the business understands and make it a part of what they do.Related:Common Mistakes and How to Avoid ThemBusinesses are struggling to make data accessible for users and protect it from misuse or breaches. This often results in either too much bureaucracy or insufficient control, leaving organizations vulnerable to inefficiencies and regulatory fines.The solution is to start small, focus on delivering results, and build from there. Begin with high-priority areas, like fixing compliance gaps or cleaning up critical datasets, to show quick wins, says Arunkumar Thirunagalingam, senior manager, data and technical operations at healthcare company McKesson, in an email interview. These early successes help build momentum and demonstrate the value of governance across the organization.He says the biggest mistakes companies make include trying to fix everything at once, relying too much on technology without setting up proper processes and ignoring the needs of end users.Overly restrictive governance often leads to workarounds that create even more problems, while waiting until a crisis forces action leaves companies in a reactive and vulnerable position, says Thirunagalingam. [W]hen done right, data governance is much more than a defense mechanism -- its an enabler of innovation and efficiency.Stephen Christiansen, principal security consultant at cybersecurity consulting firm Stratascale,says the shortage of data professionals, exploding data growth, and ever-increasing requirements for AI and data security are causing organizations to take a more conservative approach.Companies need to be continually investing in data technologies that help them manage, secure, and integrate data across their enterprise systems, says Christiansen in an email interview. Internally, companies need to [build] a data-driven culture, so employees better understand the importance of data governance and how it benefits them.David Curtis, chief technology officer at global fintech RobobAI, says the average amount of data is growing 63% monthly. The speed and velocity of this growth is overwhelming, and companies are struggling to manage the storage, protection, quality, and consistency of this data.Data is often collected in multiple different ERPs across an organization. This often means that data is disparate in format and incomplete. Eighty percent of companies estimate that 50% to 90% of their data is unstructured, says Curtis in an email interview. Unstructured data creates challenges for large organizations due to its lack of standardization, making it difficult to store, analyze, and extract actionable insights, while increasing costs, compliance risks and inefficiencies.Companies need to start with a data governance strategy. As part of that, they need to review relevant business goals, define data ownership, identify reference data sources, and align the data governance strategy KPIs. For ongoing success, they need to establish an iterative process of continuous improvement by developing data processes and committing to a master data governance framework.For every dollar you invest in AI you should invest five dollars in data quality. In my experience, the most common data challenges are due to a lack of clear objectives and measurable success metrics around master data management initiatives, says Curtis. Often insufficient or poor-quality data, often at scale, and limited integration with existing systems and workflows, prevents scalability and real-world application. Evolving regulations are also adding fuel to the fire.Organizations are continually challenged with complying with the constant stream of regulations from various jurisdictions, such as GDPR, HIPAA, and CCPA. These regulations keep evolving, and just when IT leaders think theyve addressed one set of compliance requirements, a new one emerges with slight nuances, necessitating continuous adjustments to data governance programs, says Kurt Manske, information assurance andcybersecurity leader at professional services firm Cherry Bekaert. The reality is that companies cant simply pause their operations to align with these ever-changing regulations. Consequently, developing, deploying and managing a data governance program and system is a lot like changing tires on the car as it goes down the highway. [Its] an undeniably daunting task.This underscores the need to establish a resilient culture versus a reactive one.Leading companies see regulatory compliance as a differentiator for their brand and products, says Manske in an email interview. [One] key reason data governance programs and system deployment projects fail is that organizations try to take on too much at once. Big bang deployment strategies sound impressive but they often encounter numerous technical and cultural problems when put into practice. Instead, a metered or scaled deployment approach across the enterprise allows the team, vendor and governance leadership to continuously evaluate, correct and improve.The Sobering TruthOrganizations that lack strong governance are drowning in data, unable to harness its value, and leaving themselves vulnerable to growing cyber threats. According to Klaus Jck, partner at management consulting firm Horvth USA, incidents like the recent CrowdStrike breach are stark reminders of whats at stake. Data quality issues, silos, unclear ownership and a lack of standardization are just the tip of the iceberg.Klaus Jck, Horvth USAThe root cause of these struggles is simple: Data is everywhere. Thanks to new sensor technologies, process mining and advanced supervisory systems, data is produced at every step of every business process, says Jck in an email interview. The drive to monetize this data has only accelerated its growth. Unfortunately, many organizations are simply not equipped to manage this deluge.A truly effective strategy must go beyond policies and frameworks; it must include clear metrics to measure how data is used and how much value it creates. Assigning ownership is also key -- data stewards can help create a control environment sensitive to the nuances of modern data sources, including unstructured data.Failing to connect governance to business goals or neglecting executive sponsorship are major mistakes, says Jck. Poor communication and training also derail efforts. If employees dont understand governance policies or dont see their value, progress will stall. Similarly, treating governance as a one-time project rather than an ongoing process ensures failure.Dimitri Sirota, CEO and co-founder at security, privacy, compliance, and AI data management company BigID, says the root cause of data governance challenges often stem from poor data quality and insufficient governance frameworks.Inconsistent data collection practices, lack of standardized formats for key data elements such as dates and numeric values, and failure to monitor data quality over time exacerbate the problem, says Sirota in an email interview. Additionally, organizational silos and outdated systems can perpetuate inconsistencies, as different teams may define or manage data differently. Without a rigorous framework to identify, fix and monitor data issues, organizations face an uphill battle in maintaining reliable, high-quality data.Ultimately, the absence of a centralized governance strategy makes it difficult to enforce standards, creating noise and clutter in data environments.Marc Rubbinaccio, head of compliance at security compliance provider Secureframe, points to a related issue, which is understanding where sensitive data resides and how it flows within organizations.[T]he rush to adopt and implement AI within organizations and software products has introduced new risks, says Rubbinaccio in an email interview. While the efficiency gains from AI are widely recognized, the vulnerabilities it may introduce often go unaddressed due to a lack of thorough risk evaluation. Many organizations are bypassing detailed AI risk assessments in their eagerness to stay ahead, potentially exposing themselves to long-term consequences.About the AuthorLisa MorganFreelance WriterLisa Morgan is a freelance writer who covers business and IT strategy and emergingtechnology for InformationWeek. She has contributed articles, reports, and other types of content to many technology, business, and mainstream publications and sites including tech pubs, The Washington Post and The Economist Intelligence Unit. Frequent areas of coverage include AI, analytics, cloud, cybersecurity, mobility, software development, and emerging cultural issues affecting the C-suite.See more from Lisa MorganNever Miss a Beat: Get a snapshot of the issues affecting the IT industry straight to your inbox.SIGN-UPYou May Also LikeWebinarsMore WebinarsReportsMore Reports

Ganesh Seetharaman, Managing Director, Deloitte ConsultingDecember 20, 20244 Min ReadValentin Valkov via Alamy StockDuring peak times like holiday periods, retailers, consumer goods companies, insurance firms, and others involved in seasonal crunch-time sectors face a delicate balance between opportunity and risk. Seasonal spikes can be a stringent test for executives, revealing the strength of their business and operational resilience. To understand why, just think back to recent incidents with organizations that may have experienced mass website outages due to holiday spikes or that suffered prolonged log-in issues.Indeed, downtime during peak periods can result in financial impacts measured in millions of dollars per hour, so its clear that the user experience is paramount. Even minor issues can lead to significant consequences, including customer churn, wasted ad spending, and long-term brand damage. The takeaway? Failure when the world is watching can have cascading effects, and a track-record of 99.99% uptime is insufficient if the 0.01% downtime occurs at critical moments. With that in mind, lets explore a strategic approach to building game-ready resilience.Game-ready resilience means that your systems can manage adversity -- from ecosystem impacts, including third-party services -- to unprecedented traffic peaks. Most importantly, it also means creating a culture of reliability with constant learning and cross-functional teams that understand the business impacts of downtime and can respond effectively to outages.Related:To enhance business and operational resilience during the holidays, tech leaders should focus on four key areas.1. Forecast and define measurable requirements.Start enhancing resilience by developing a reliable forecast of expected transaction volumes and user behavior. Seek to understand normal traffic patterns as well as how spikes in traffic might affect your systems during peak periods. Prioritize critical services; for example, with an e-commerce platform, the checkout process should take precedence over less-essential features like recommendation engines.Use service level objectives (SLOs) to define availability expectations and measure them. For instance, aim for 99.99% shopping-cart availability -- which you can foster by forecasting transaction volumes across all channels. Then, translate those forecasts into performance requirements like the ability to accommodate a specific number of concurrent users while meeting reliability expectations. It's also crucial to identify potential architectural bottlenecks and failure points.2. Map dependencies and mitigate risks.Related:Modern retail ecosystems are complex webs of internal systems and third-party services. To identify vulnerabilities and mitigate risk, create a comprehensive map of all dependencies. Then, assess the services scalability and reliability, and develop failure contingency plans that include circuit breakers and fallback options.In addition to infrastructure, focus on key business and foundational services, especially in hybrid and multi-cloud environments. Next, to build agility and minimize recovery time, develop a clear view of all dependency layers and build fault tolerance. An example of dependency management could look like an e-commerce organization simplifying its shipping infrastructure to achieve more efficient package delivery.3. Implement robust reliability checks.Establish clear, measurable reliability objectives aligned with business outcomes. For example, you might set granular targets, such as sub-2-millisecond log-in times. Such metrics create a common language across development, operations, and business teams, fostering a unified approach to reliability. Also, to ensure build stability, avoid last minute changes, and implement rigorous process controls for continuous validation.Related:Integrate SLOs and synthetic monitoring into your operational framework. Develop real-time observability solutions that provide actionable insights and rapid response capabilities. Implement observability to balance innovation and stability during peak loads and align technical metrics with indicators like net promoter scores. Also, adopt site reliability engineering to translate technical metrics directly into customer experience.4. Develop and refine incident-response procedures.Swift and effective responses to system challenges can prevent minor issues from becoming major crises. So, its essential to develop incident response procedures that include comprehensive system dependency maps that create communication channels, action plans, and escalation pathways that help minimize confusion. Automatic failure notifications are a must as well, as are self-healing approaches to incidents and solutions driven by error budgets and burn rates.Next, ensure organizational readiness through training, communication protocols, and regular response drills. Implement proactive monitoring systems to detect and address issues early. Also, learning from high-profile incidents underscores the importance of transparent, timely communication during disruptions.The Path ForwardBuilding resilience requires both a cultural and technical shift to align critical services with customer journeys, refine resilience policies, and adapt to changing demands. Practices like game day drills enhance readiness, reinforcing that resilience is an ongoing effort that requires continuous refinement, not a one-time project. True resilience requires a holistic approach that ensures people, processes, and technology work in sync to handle both surges and scale-downs effectively. By adopting the strategies weve discussed here, you can prepare your systems for peak times while building stronger, more resilient year-round operations.About the AuthorGanesh SeetharamanManaging Director, Deloitte ConsultingGanesh Seetharaman is a managing director at Deloitte Consulting LLP. He leads Deloittes Technology Resiliency market offering and is recognized for delivering innovative solutions for his clients, as well as for helping organizations navigate technology challenges and capitalize on market opportunities.See more from Ganesh SeetharamanNever Miss a Beat: Get a snapshot of the issues affecting the IT industry straight to your inbox.SIGN-UPYou May Also LikeWebinarsMore WebinarsReportsMore Reports