We test a wide variety of laptops each year here at The Verge, and many of them are very good. But you cant buy them all. So weve done the very difficult (very difficult, trust us) task of selecting some of the best laptops you can get. Whether youre looking for a Chromebook, a gaming laptop, a 2-in-1 convertible, an ultralight notebook, or something thats a little bit of each, weve got options for you.Our current pick for the best laptop is the MacBook Air M3. Its a device that does just about everything right. It has no major flaws and it even got better recently thanks to a free RAM upgrade from Apple. Its a great pick for productivity, browsing, and even light gaming.RelatedIts getting harder to buy a bad laptop, but what separates the best laptops from the merely good laptops is how they balance power, efficiency, portability, and comfort. A great laptop should have a fantastic keyboard and trackpad; Its display should be bright and sharp; its processor should be powerful enough for most anything short of intensive video editing and advanced gaming. And the whole package should be easy to carry around and able to last all day without needing to be plugged in.What were looking forPrice is a very important factor in evaluating devices for this page. Were seeking laptops that offer great value for their asking price and that are the best of the best compared to other laptops in their price category. The more expensive a device, the more exceptional it has to be to make it on here.We want laptops with enough power to accommodate their target audience. An ultraportable laptop should be able to handle a standard office workload without excessive heat or fan noise. A gaming laptop should run todays popular games.These are the two biggest reasons youd get a laptop over a smartphone or tablet. Theyd better be good or youll hate every minute.We like displays that are color-accurate, bright, and sharp enough that you arent distracted by jagged edges or visible pixels.Were looking for laptops with a solid, durable build that can handle a standard amount of briefcase wear-and-tear while (in most categories) easy to carry around from place to place.Battery life is much more important in some categories than it is in others. That said, in general, we assume that most shoppers are choosing a laptop over a desktop because they value some ability to work away from a plug. Battery life gives a user more freedom and makes a device more reliable for travel, so we consider it here.The more ports, the better.Our other picks for the best laptops, Chromebooks, and 2-in-1s include the MacBook Pro 16 and the 13.8-inch Microsoft Surface Laptop 7th Edition with a Snapdragon X Elite chip. Check out the full list of best laptop picks below, if youre looking for a powerful machine that doesnt sacrifice portability.In this articleThe best laptop13-inch Apple MacBook Air M3 (2024)$1004$10999% off$1004The MacBook Air M3 is a jack-of-all-trades, with a balanced combination of performance and power efficiency. It also now supports dual displays with the lid closed, storage speed that is noticeably faster, and a base 16GB of RAM after a mid-cycle update.CPU: M3 (8-core) / GPU: M3 (8- or 10-core) / RAM: 16GB, 24GB / Storage: 256GB, 512GB, 1TB, 2TB / Display: 13.6-inch or 15.3-inch Liquid Retina display, 2560 x 1664 or 2880 x 1864 , 60Hz, no touch option / Dimensions: 11.97 x 8.46 x 0.44 inches or 0.45 x 13.40 x 9.35 inches / Weight: 2.7 pounds (13-inch) or 3.3 pounds (15-inch)Apples MacBook Air M3 is the best laptop for most people Mac users, of course, but also the platform-agnostic or anyone who wants a no-fuss, straightforward machine that doesnt bombard them with advertisements or bloatware. Its a productivity laptop that can do a bit of everything, and it now starts with 16GB of RAM and comes in a 15-inch version for those who like their laptops a little larger. Its hard to find another laptop that offers this kind of combination of performance and battery life in a thin and light chassis.Despite losing its way around the mid-2010s, Apple has a long history of sending quality MacBooks to market, and the Air M3 is no different. A smooth, almost ethereal trackpad, check. A chiclet-style keyboard that makes typing feel like a dance, check. Fast Wi-Fi adapter, color-rich display, 1080p webcam, and MagSafe charging, check. The Air M3 wasnt made for heavy gaming, video editing, or for those who want more than two USB-C ports, but for nearly everyone else, its a great option.However, if you do need some of that, the 14-inch MacBook Pro M4 is a great bang-for-your-buck laptop in its own right at $1,599. But opting for a Pro over an Air is best for students in creative fields and content creators needing more headroom and features like a third USB-C port and an SD card slot.The Apple MacBook Air M3 is the best laptop of 2024. Photo by Chris Welch / The VergeIts absolutely worth spending an extra $100 to get the Air M3 over the MacBook Air M2, which Apple still sells. Its about 16 to 18 percent faster than Apples M2 chip, and the storage on the 256GB configurations is up to twice as fast. The new Airs also support two external displays with the lid closed, which is another improvement from the previous generation. If you opt for the larger model, youll find that the speakers on 15-inch Air M3 are vastly superior, especially on the low end. You can actually hear the bass, whereas on the 13-inch, its nearly nonexistent. And if you get the midnight color in either model, keep a microfiber cloth handy. The amount of fingerprints it collects turns the chassis into a CSI playground.We previously recommended upgrading the MacBook Air M3 to at least 16GB of memory, but Apple now does that for you, as all current Air models now start at 16GB of RAM. You can still find 8GB configurations at some retailers, but its not worthwhile unless you get it on some kind of fire sale. The increased RAM will improve the laptops ability to multitask now and will give you some extra years of service down the line. (Important since youd have to buy a whole new laptop to upgrade the memory.) 256GB of storage is easier to get away with, given the plethora of cloud and external storage options available.If you dont need to buy right away, you might consider waiting for the M4 version of the MacBook Air, which is expected in early 2025. But the M3 Air is a great option if you cant wait.Apple MacBook Air M3 (15-inch)$1189$12998% off$1189The 15-inch MacBook Air is also equipped with Apples M3 chip. It features a larger display and better speaker array than the 13-inch MacBook Air M3. As of October 2024, it starts with 16GB of RAM instead of the original 8GB when it launched.Read our review of the Apple MacBook Air M3.Best Chromebook$699Acers latest Spin Chromebook is configured with Intels first generation of processor with an NPU. It also has Google Gemini baked right into the OS, and if you sign up for a new Google One AI premium plan, your first year of service is free.$699 at Best BuyCPU: Intel Core Ultra 5 115U / GPU: Intel UHD / RAM: 8GB, 16GB LPDDR5 / Storage: 256GB, 512GB / Display: Touch 14-inch FHD 1920 x 1200, 60Hz IPS with stylus support, 340 nits / Dimensions: 12.35 x 8.84 x 0.71 inches / Weight: 3.21 pounds Like last year, Acers newest Chromebook Spin is the best Chromebook weve seen so far in 2024 and its still part of the Plus line of Chromebooks, which now have Google Gemini features integrated with ChromeOS. It also comes with a 12-month subscription to the Google One AI premium plan, which includes 2TB of cloud storage space.The Core Ultra 5 115U is slower than last gens Intel Core i5-1335U (the Ultra has eight cores with clock speeds up to 4.20GHz, while the Core i5 has 10 cores and goes up to 4.60GHz), but the Ultra Core has an NPU, which means its better suited for running AI-related tasks on-device. The new Spin Chromebook also supports the current LPDDR5 memory standard, which is faster than the previous generation in the 2023 Spin 714.Acer upgraded some of the connectivity port options, too. The Spin 714 now has two Thunderbolt 4 USB-C ports, instead of two USB-C 3.2 ports, for much faster data transfer and power delivery.Best Snapdragon X laptopMicrosoft Surface Laptop 7th Edition (13.8-inch)$799$100020% off$799The 13.8-inch Surface Laptop is Microsofts closest rival to the MacBook Air, using Qualcomms Snapdragon X chipsets for exceptional battery life.CPU: Qualcomm Snapdragon X Plus X1P-64-100, Snapdragon X Elite X1E-80-100 / GPU: Qualcomm Adreno / RAM: 16GB, 32GB, 64GB LPDDR5X (soldered) / Storage: 256GB, 512GB, 1TB / Display: 13.8-inch touchscreen LCD, 2304 x 1536 120Hz, 600 nits / Dimensions: 11.85 x 8.67 x 0.69 inches / Weight: 2.96 poundsOut of all the Snapdragon Copilot Plus PCs weve tested so far, the 13- and 15-inch Microsoft Surface Laptop 7th edition stood out for its balance of performance, exceptional battery life and standby time, and quality components (120Hz screen, keyboard, webcam, trackpad, etc.). Microsoft was obviously gunning for Apples MacBook Air with this laptop, and it mostly delivers that level of hardware experience for a Windows machine. Theyre the full package if you want a thin-and-light productivity machine running Windows that easily lasts all day and into the night.The greater concern with Snapdragon X-equipped laptops is if Windows on Arm has all the app compatibility you need to get your work done. Most everyday apps work fine, via native Arm versions or emulation, but there remain outliers. Also, if you like the idea of your productivity machine being able to pull light gaming duties (one of Windows advantages over Mac) then youre likely better off with a laptop using Intels Lunar Lake or AMDs Ryzen AI chips. Snapdragon X laptops can only run a fraction of the games you find on Steam.Microsoft recently announced Intel Lunar Lake- based versions of the Surface Laptop 7, due out in mid-February, but theyre priced a ridiculous $500 higher and aimed solely at businesses. Its a shame, because having this hardware with Lunar Lakes compatibility might have been an appealing prospect.The best 16-inch Windows laptopAsus Zenbook S 16$1700$1700The newest Asus Zenbook S 16 features AMDs latest Ryzen AI mobile processors. Its one of the thinnest and lightest 16-inch laptops available and doesnt cut corners on performance. Its a mighty machine that can handle all sorts of heavy workloads with ease and grace. Read our review.$1700 at Best Buy$1700 at AsusCPU: AMD Ryzen AI 9 HX 370, AMD Ryzen AI 9 365 / GPU: Radeon 890M, Radeon 880M / RAM: 32GB, 24GB / Storage: 1TB / Display: touch 16-inch 2880 x 1800, 120Hz OLED with stylus support, 500 nits peak HDR / Dimensions: 13.92 x 9.57 x 0.47 ~ 0.51 inches / Weight: 3.21 pounds The 16-inch Asus Zenbook S 16 is the best-looking, best-performing Windows laptop we tested over the summer; its also so thin and lightweight you wouldnt know it just by holding it! It can handle a little bit of everything, from emails to graphic design work, and it tackles gaming surprisingly well for a laptop without a separate graphics card. Its miles ahead of what any Copilot Plus PC can do, too, including the Intel Core Ultra laptops weve recently tested. Its a lovely, catch-all device.There are only two Zenbook S 16 models as of now. The $1,700 one comes in gray with AMDs flagship Ryzen AI 9 HX 370 chip and 32GB of memory, and the $1,400 model comes in white with the lower-tier AMD Ryzen AI 9 365 and 24GB of memory. Otherwise, they are identical. Both feature OLED touch displays with a native 2880 x 1800 (3K) resolution and 120Hz refresh rate, stylus support, 1TB of storage, and the same ports / Wi-Fi adapter. Everything but the Windows version. (The more expensive model gets Home; the cheaper one gets Pro.)At just 11 hours, its battery life doesnt last as long as similar laptops weve tested, such as the Samsung Galaxy Book4 Edge and Dell XPS 14, but its a small tradeoff. Theres currently no other Windows laptop that offers as much performance and versatility in a 16-inch chassis for the price.Read our review of the Asus Zenbook S 16.The best laptop for high-end gaming$3700The ROG Strix Scar 17 X3D is one of the fastest gaming laptops you can get, thanks to its AMD Ryzen 9 7945HX3D processor and Nvidia RTX 4090 GPU. With 32GB of RAM, 1TB storage, and a 240Hz 1440p screen, its the high-end laptop to beat.$3700 at AsusCPU: AMD Ryzen 9 7945HX3D / GPU: Nvidia Geforce RTX 4090 / RAM: 32GB / Storage: 1TB / Display: 17-inch IPS QHD, 240Hz display, 3ms, 300 nits, 100 percent DCI-P3 / Dimensions: 15.55 x 11.1 x 1.11 inches / Weight: 6.51 poundsThe ROG Strix Scar 17 X3D is big. Its loud, its garish, and its flat-out the fastest gaming laptop weve tested. Thanks to its AMD Ryzen 9 79045HX3D processor, it leaves models with the same top-tier RTX 4090 graphics card and Intels fastest CPUs in the dust. It can run many of todays AAA titles at 1440p with triple-digit frame rates.The Scar 17 X3D has a 17-inch 2560 x 1440 240Hz screen with G-Sync, oodles of ports, a pleasant keyboard, and RGB galore. At over six and a half pounds and 17 inches on the diagonal, its your classic high-performance, barely portable gaming laptop.Its webcam is potato, battery life is exactly as bad as youd expect from everything we just listed, and its expensive, but for now this is the high-water mark for gaming laptops.For more down-to-earth performance and price, you can also get the ROG Strix Scar 17 with an RTX 4070 for about $2,200 or with an RTX 4080 for about $2,900.Read our Asus ROG Strix Scar 17 X3D review.The best MacBook for photo and video editingApple MacBook Pro (M4 Pro / Max, 16-inch)$2239$249910% off$2239The 16-inch MacBook Pro with M4 Pro starts with 24GB of RAM, 512GB SSD, and 14-core CPU / 20-core GPU. The M4 Max version offers 32GB of RAM, 1TB SSD, 14-core CPU / 32-core GPU, and can be configured to ridiculously high prices. Both models have Thunderbolt 5 ports, 12-megapixel Center Stage webcam, and options for anti-glare displays.CPU: M4 Pro, M4 Max / GPU: M4 Pro, M4 Max / RAM: 24GB or 48GB (M4 Pro), 36GB - 128 GB (M4 Max) / Storage: 512GB, 1TB, 2TB, 4TB, 8TB / Display: 14.2 / 16.2-inch Liquid Retina XDR, 3024 x 1964 / 3456 x 2234, adaptive refresh up to 120Hz, no touch option / Dimensions: 12.31 x 8.71 x 0.61 (14-inch) / 14.01 x 9.77 x 0.66 inches (16-inch) / Weight: 3.5 / 4.7 pounds (M4 Pro), 3.6 / 4.8 pounds (M4 Max) If you need more power for intensive creative work like 3D rendering and working with ultra-high-resolution photos and video the MacBook Pro is your best bet. Both the 14- and 16-inch models are available with powerful M4 Pro or M4 Max processors. Theres also a 14-inch Pro with a standard M4 processor, and while its a formidable step up from the MacBook Air as an everymans pro laptop, the M4 Pro and Max processors still outclass it in performance.Processors aside, the MacBook Pro has remained largely unchanged since 2021. It has a bright, beautiful, color-accurate, high-res screen with HDR and an adaptive refresh rate of up to 120Hz; amazing speakers, a comfortable keyboard and trackpad; and a good port loadout: three Thunderbolt 5 / USB-C ports, plus HDMI 2.1 and an SD card slot. The 14-inch MacBook Pro is a few hundred dollars cheaper than the 16-inch, but aside from the very lowest processor options, most configurations are available in either size, so pick whichever works for you.In our benchmarks, which test a variety of creative tasks including encoding, playback, and export time, the MacBook Pro 16 did better than any laptop weve ever used the only other machines that have come close to matching this thing in some of our benchmarks are high-end desktop PCs. The battery life is also record-shattering. The top-of-the-line 16-inch M4 Max model easily lasted all day in our most recent testing, with no battery-saving features enabled and even keeping the screen on full-time.Most people who need much more power than a MacBook Air including all but the most demanding pro photographers will be fine with an M4 Pro model, which starts at $1,999 for the 14-inch with a 12-core CPU, 16-core GPU, 24GB of memory, and a 512GB SSD. The 16-inch starts at $2,499 for a 14-core CPU, 20-core GPU, 24GB of memory, and 512GB SSD. The M4 Pro chip can be configured with up to 48GB of memory and a 4TB SSD, at the usual absurd Apple markups, and it supports up to two external 6K displays.If you absolutely need more GPU power or more than two external monitors you can step up to the M4 Max. The base M4 Max with 14 CPU cores and 32 GPU cores, plus 36GB of RAM and 1TB SSD, starts at $3,199 in the 14-inch and $3,499 in the 16-inch. Theres also an M4 Max with 16 CPU and 40 GPU cores, which starts at $3,699 and $3,999, respectively. The M4 Max models are configurable with up to 128GB of RAM and 8TB of storage and can support up to four external monitors. The vast majority of people dont have workloads heavy enough to notice a significant difference between the M4 Pro and M4 Max; if you do, you probably know it.14-inch MacBook Pro with M4$1459$15999% off$1459The entry-level MacBook Pro with M4 starts with 16GB of RAM double that of its predecessor and a 512GB SSD for the same starting price of $1,599. It also gets a third USB-C / Thunderbolt 4 port and comes in a new space black option.Read our reviews of the MacBook Pro 16 M4 Pro / Max and MacBook Pro 14 M4.The best 14-inch gaming laptop$1600This 14-inch ROG Zephyrus is an astonishingly balanced gaming laptop for its performance and price. Its AMD Ryzen 9 CPU and Nvidia RTX 4070 GPU can push high frame rates and smooth graphics in games with its OLED display.$1600 at Best Buy (RTX 4060)$1750 at Best Buy (RTX 4070)CPU: AMD Ryzen 9 8945 HS / GPU: Nvidia GeForce RTX 4060, RTX 4070 / RAM: 16GB, 32GB / Storage: 1TB / Display: 14-inch OLED, 2880 x 1800, 60Hz and 120Hz, 400 nits / Dimensions: 12.24 x 8.66 x 0.63 inches / Weight: 3.31 poundsThe Asus ROG Zephyrus G14 with an RTX 4070 is the most well-balanced 14-inch gaming laptop. Its powerful enough to reach 60 frames per second on ultra graphics at its native resolution (with or without DLSS) and thin and lightweight enough without trapping too much heat in its chassis. Also, its battery can last up to 6.5 hours on a single charge, which is good for a gaming laptop.Its display has been upgraded from an IPS to an OLED, it has 1TB of SSD storage and 32GB of memory, and Asus put lighting back on the lid not a dot matrix, but a strip of LEDs spanning diagonally across.All that for $2,000 to $2,200 makes it hard to justify getting anything else, though you can save up to $500 if you get the RTX 4060 model. Though it has only half the RAM, it has the same great screen and build, and its frame rates are only about ten percent lower than the 4070 Zephyrus at 1080p/ultra.If you spend up to $700 more on the Razer Blade 14, youll get higher frame rates, a 240Hz screen, and upgradeable RAM but its battery life barely hits four hours in general use, and its display doesnt support G-Sync or HDR.The best dual-screen laptopLenovo Yoga Book 9i (2024)$1600$200020% off$1600The Lenovo Yoga Book 9i is a dual-screen laptop that can fold and flip into several different modes. A Bluetooth keyboard, mouse, and stylus come with it, along with a keyboard folio that can transform into a laptop stand. Its the Swiss Army knife of laptops. Read our head-to-head review.$1600 at Lenovo$2000 at Best BuyCPU: Intel Core Ultra 7 155U / GPU: Intel UHD (integrated) / RAM: 16GB LPDDR5X / Storage: 1TB M.2 NVMe PCIe 4.0 SSD / Display: 13.3-inch (2880 x 1800) 60Hz OLED touchscreen w/ stylus support / Dimensions: 11.78 x 8.03 x 0.63 inches / Weight: 2.95 poundsA dual-screen laptop is exactly what it sounds like: a laptop with a second screen where the keyboard normally goes.The Lenovo Yoga Book 9i has a 360-degree hinge, which lets you use it as a regular laptop, a (large) tablet, and more. You can put it in clamshell mode and write or sketch on the bottom screen with a stylus or fold the keyboard folio into a stand to prop it up and take advantage of both screens. Thats how I usually use it at home: propped up and plugged into an external monitor as a three-screen desktop replacement. But when Im away, I use the included Bluetooth keyboard and mouse. Its touchscreen gestures are super responsive but too responsive for the virtual keyboard and trackpad. Its easy to mistype and accidentally minimize windows.The Yoga Books only real competitor is Asus Zenbook Duo. The Duos physical keyboard includes a trackpad and makes it look a lot more like a traditional laptop. Its 14-inch OLED screens are a little bigger and brighter, it has a ton of port options, its more powerful, and its a little easier to fold up and put away. But its also heavier (at 3.64 pounds), its top lid doesnt fold back far enough to use it as a tablet, and its touchscreen gestures arent as responsive.Read our head-to-head review of the Lenovo Yoga Book 9i and the Asus Zenbook Duo.The best repairable laptop$899The Framework Laptop 13 is the best laptop for those who want full control of their experience. Everything from the processor and RAM to the keyboard and bezels is user-upgradeable. It comes in Intel and AMD variants; most people should get the AMD. Read our review.$899 at FrameworkCPU: Intel Core i5-1340P / i7-1360P / 17-1370P, AMD Ryzen 7 7840U / GPU: Intel UHD / Iris Xe, Radeon 700M / RAM: 8GB, 16GB, 32GB, 64GB / Storage: 250GB, 500GB, 1TB, 2TB / Display: 13.5 inch IPS, 2256 x 1540, 60Hz, no touch option / Dimensions: 11.68 x 9.01 x 0.62 inches / Weight: 2.87 poundsIf you want a 13-inch laptop that you can configure and assemble yourself, from the ground up, the Framework Laptop 13 is pretty much your only option. The Framework is a modular laptop that users can repair and upgrade over the length of their ownership. Everything from the RAM, to the storage, to even the processor can be upgraded down the line. The company has even come out with upgrades for parts like speakers and hinges that you can install yourself. Not only can you buy it as a prebuilt system, but you can also order it as a DIY kit, allowing you to assemble the entire thing yourself and swap out parts as you please. Repairability aside, the Framework has a number of laudable features as a laptop itself. The display is bright and high resolution, the speakers are great, and the chassis is quite portable, coming in at under three pounds. The 3:2 aspect ratio provides a lot of room to work and is still a somewhat rare find on todays market.The Framework Laptop 13 is one of very few 13-inch laptops with this degree of repairability. Photo by Monica Chin / The VergeThat said, I wont pretend that this is the best 13-inch laptop you can buy. On its own merits, it is an unremarkable system with a somewhat generic look and plasticky build, particularly compared to others in its price category. Nevertheless, the Frameworks standout feature is the unprecedented access it allows its users to replace and repair its parts. Wed love to see more companies make that kind of commitment to sustainable design.Read our Framework Laptop 13 review.The best laptop under $400$440The Asus Chromebook Plus CX34 is the most affordable Chromebook with the new Plus label. It has a 12th Gen Intel Core i3 processor, 8GB of RAM, 128GB of storage, and a 1080p screen for just under $400. Read our review.$440 at Amazon (256GB)CPU: Intel Core i3-1215U / GPU: Intel UHD / RAM: 8GB / Storage: 128GB, 256GB UFS / Display: 14-inch IPS, 1920 x 1080, 60Hz, non-touch / Dimensions: 12.9 x 8.4 x 0.74 inches / Weight: 3.17 poundsThe Asus Chromebook Plus CX34 is the least expensive laptop with Googles new Plus certification. At around $400 for a Core i3 processor, 8GB of RAM, and 128GB of storage, its the baseline Chromebook you should consider if you cant spend any more.Its so easy to get a bad Chromebook at this price, so its a relief that the Chromebook Plus CX34 is so good. Not only does it have respectable internals, but the 1080p screen and 1080p webcam are good for the price, the keyboard is great, and the trackpad is fine, if a bit stiff. Battery life is decent, too, and like all Plus Chromebooks, it comes with 10 years of software updates. You can spend more on a laptop, and you probably should, but dont buy a Chromebook less powerful than this one.Read our review of the Asus Chromebook Plus CX34.Update, February 4th: Updated pricing and replaced the Dell XPS 13 with the Microsoft Surface Laptop 7th Edition as the pick for Best Snapdragon X laptop.See More:

Reinforcement learning (RL) for large language models (LLMs) has traditionally relied on outcome-based rewards, which provide feedback only on the final output. This sparsity of reward makes it challenging to train models that need multi-step reasoning, like those employed in mathematical problem-solving and programming. Additionally, credit assignment becomes ambiguous, as the model does not get fine-grained feedback for intermediate steps. Process reward models (PRMs) try to address this by offering dense step-wise rewards, but they need costly human-annotated process labels, making them infeasible for large-scale RL. In addition, static reward functions are plagued by overoptimization and reward hacking, where the model takes advantage of the reward system in unforeseen ways, eventually compromising generalization performance. These limitations restrict RLs efficiency, scalability, and applicability for LLMs, calling for a new solution that effectively combines dense rewards without high computational expense or human annotations.Existing RL methods for LLMs mostly employ outcome reward models (ORMs), which offer scores only for the final output. This results in low sample efficiency as models must generate and test whole sequences before getting feedback. Some methods employ value models that estimate future rewards from past actions to counter this. However, these models have high variance and do not properly handle reward sparsity. PRMs offer more fine-grained feedback but need costly manual annotations for intermediate steps and are prone to reward hacking because of static reward functions. Additionally, most existing methods need an extra training phase for the reward model, adding to the computational expense and making them infeasible for scalable online RL.A group of researchers from Tsinghua University, Shanghai AI Lab, University of Illinois Urbana-Champaign, Peking University, Shanghai Jiaotong University, and CUHK has proposed a reinforcement learning framework that eliminates the need for explicit step-wise annotations using efficient utilization of dense feedback. The main contribution proposed is the introduction of an Implicit Process Reward Model (Implicit PRM), which produces token-level rewards independently of outcome labels, thus eliminating the need for human-annotated step-level guidance. The approach allows for continuous online improvement of the reward model, eliminating the problem of overoptimization without allowing dynamic policy rollout adjustments. The framework can successfully integrate implicit process rewards with outcome rewards during advantage estimation, offering computational efficiency and eliminating reward hacking. Unlike previous methods, which require a separate training phase for process rewards, the new approach initializes the PRM directly from the policy model itself, thus greatly eliminating developmental overhead. It is also made compatible with a range of RL algorithms, including REINFORCE, PPO, and GRPO, thus making it generalizable and scalable for training large language models (LLMs).This reinforcement learning system provides token-level implicit process rewards, calculated through a log-ratio formulation between a learned reward model and a reference model. Rather than manual annotation, the reward function is learned from raw outcome labels, which are already obtained for policy training. The system also includes online learning of the reward function to avoid overoptimization and reward hacking. It uses a hybrid advantage estimation approach that combines implicit process and outcome rewards through a leave-one-out Monte Carlo estimator. Policy optimization is achieved through Proximal Policy Optimisation (PPO) using a clipped surrogate loss function for stability. The model was trained using Qwen2.5-Math-7B-Base, an optimized model for mathematical reasoning. The system is based on 150K queries with four samples per query, compared to Qwen2.5-Math-7B-Instruct using 618K in-house annotations, which demonstrates the effectiveness of the training process.The reinforcement learning system demonstrates significant gains in sample efficiency and reasoning performance across several benchmarks. It provides a 2.5 gain in sample efficiency and a 6.9% gain in mathematical problem-solving compared to standard outcome-based RL. The model outperforms Qwen2.5-Math-7B-Instruct on benchmarking mathematical benchmarks, with better accuracy on competition-level tasks like AIME and AMC. Models trained from this process outperform larger models, including GPT-4o, by pass@1 accuracy for challenging reasoning tasks, even when using only 10% of the training data used by Qwen2.5-Math-7B-Instruct. The results affirm that online updates to the reward model avoid over-optimization, enhance training stability, and enhance credit assignment, making it an extremely powerful method for reinforcement learning in LLMs.This reinforcement learning approach provides an efficient and scalable LLM training process with dense implicit process rewards. This eliminates step-level explicit annotations and minimizes training costs while enhancing sample efficiency, stability, and performance. The process combines online reward modeling and token-level feedback harmoniously, solving long-standing problems of reward sparsity and credit assignment in RL for LLMs. These improvements optimize reasoning capability in AI models and make them suitable for problem-solving applications in mathematics and programming. This research is a substantial contribution to RL-based LLM training, paving the way for more efficient, scalable, and high-performing AI training approaches.Check outthePaper and GitHub Page.All credit for this research goes to the researchers of this project. Also,dont forget to follow us onTwitterand join ourTelegram ChannelandLinkedIn Group. Dont Forget to join our75k+ ML SubReddit. Aswin AkAswin AK is a consulting intern at MarkTechPost. He is pursuing his Dual Degree at the Indian Institute of Technology, Kharagpur. He is passionate about data science and machine learning, bringing a strong academic background and hands-on experience in solving real-life cross-domain challenges.Aswin Akhttps://www.marktechpost.com/author/aswinak/Meta AI Introduces MILS: A Training-Free Multimodal AI Framework for Zero-Shot Image, Video, and Audio UnderstandingAswin Akhttps://www.marktechpost.com/author/aswinak/4 Open-Source Alternatives to OpenAIs $200/Month Deep Research AI AgentAswin Akhttps://www.marktechpost.com/author/aswinak/Meet Satori: A New AI Framework for Advancing LLM Reasoning through Deep Thinking without a Strong Teacher ModelAswin Akhttps://www.marktechpost.com/author/aswinak/Meta AI Introduces VideoJAM: A Novel AI Framework that Enhances Motion Coherence in AI-Generated Videos [Recommended] Join Our Telegram Channel

Best Buy is offering a great deal on a well equipped Alienware laptop. For Friday and Saturday only, you can get the Alienware m16 R2 RTX 4070 gaming laptop for just $1,374.99 shipped. That's $500 less than buying it directly from Dell. The m16 R2 is Alienware's most popular gaming laptop, which isn't surprising considering its excellent build quality, top-of-the-line gaming prowess, and reasonable price point. The "R2" is the second generation model that was released in 2024; it's 14% lighter and more compact than its predecessor without compromising on performance.Alienware m16 R2 16" Gaming LaptopAlienware m16 R2 16" QHD+ Intel Core Ultra 7 155H RTX 4070 Gaming Laptop with 16GB RAM, 1TB SSDThis laptop features a 16" 2560x1600 QHD+ display, Intel Core Ultra 7 155H processor, GeForce RTX 4070 GPU, 16GB of DDR5-5600MHz RAM, and a 1TB NVMe SSD. The Intel Core Ultra 7 155H is a power efficient, low voltage Meteor Lake CPU with a max turbo frequency of 4.8GHz and 16 total cores. Although Best Buy does have a Core Ultra 9 model for $300 more, gamers will find little reason to upgrade to it since most games are limited by the GPU, not the CPU.You absolutely will, however, need a decent amount of GPU power to be able to run games smoothly on the display's bumped-up 2560x1600 resolution. Alienware never throttles the GPU on its m16 laptops, and that includes the RTX 4070 found in this laptop, which boasts a 115W (140W with Dynamic Boost) TGP. The 4070 is powerful enough to comfortabley run most games at 60fps or higher on the 1600p display, moreso if the game supports DLSS 3.0.Best Buy is an authorized Dell retailer, so you'll be getting the same 1 year warranty as buying from Dell direct. This laptop is also currently in stock and will be delivered to you within a week.Looking for more deals? Check out the best Alienware gaming laptop and PC deals today.Why Should You Trust IGN's Deals Team?IGN's deals team has a combined 30+ years of experience finding the best discounts in gaming, tech, and just about every other category. We don't try to trick our readers into buying things they don't need at prices that aren't worth buying something at. Our ultimate goal is to surface the best possible deals from brands we trust and our editorial team has personal experience with. You can check out our deals standards here for more information on our process, or keep up with the latest deals we find on IGN's Deals account on Twitter.Eric Song is the IGN commerce manager in charge of finding the best gaming and tech deals every day. When Eric isn't hunting for deals for other people at work, he's hunting for deals for himself during his free time.

"We would like to continue releasing new titles".Nintendo has already confirmed it will be supporting the Switch in 2025 with new software, but what is the plan for the system beyond this year?During the Q&A of Nintendo's latest financial results briefing, Nintendo President Shuntaro Furukawa acknowledged the balancing act shifting from one generation to another - mentioning how the Japanese firm would like to continue releasing new titles on the original system, but also noting how "crucial" exclusives are when it comes to new hardware.Read the full article on nintendolife.com

Looking for help with today's NYT Mini Crossword puzzle? Here are some hints and answers for the ... [+] puzzle.Credit: NYTIn case you missed Fridays NYT Mini Crossword puzzle, you can find the answers here:Well its Saturday once again and I have returned, dearest readers, to help you solve another NYT Mini Crossword puzzle. Today Ill hand out a few clues before I give out all the answers.The NYT Mini is a smaller, quicker, more digestible, bite-sized version of the larger and more challenging NYT Crossword, and unlike its larger sibling, its free-to-play without a subscription to The New York Times. You can play it on the web or the app, though youll need the app to tackle the archive.CLUES1 Across is also a word for a body part.1 Down contains a stringed musical instrument thats very large that you pluck with both hands.5 Across rhymes with a tropical fruit.2 Down is also something youll find in the hood of your car.This should get you started. If you need specific words, answers to todays Mini Crossword below . . . .MORE FOR YOUSpoilers ahead!ACROSS1- Beer foam HEAD5- Dance that it takes "two" to do TANGO6- Annoying thing to forget on a trip CHARGER8- Color associated with Barbie HOTPINK9- "For all ___ and purposes " INTENTS10- Maryland governor Wes MOORE11- Cons' counterparts PROSDOWN1- Oldest monthly magazine in the U.S., launched in 1850 HARPERS2- Front car of a train ENGINE3- Athlete's representative AGENT4- Nerds DORKS5- Body image? TATTOO6- Bonobo relative CHIMP7- Pay tribute to HONORToday's MiniScreenshot: Erik KainThis was actually a very easy Mini Crossword despite having so many words. Saturday is usually a bit more challenging. Or I just lucked out knowing all of these and guessing correctly. All except for one rather massive brainfart that messed me up and made my time not so great. I entered MANGO instead of TANGO. In my brain, I thought TANGO but a funny thing happened on the way to the keyboard and somehow I plugged that in and went on my merry way. When I filled in MOORE and PROS at the end, I was confused when I got the So Close message. Was it not MOORE? I thought. What other name could Wes Mo_re be? It took me reading over the puzzle several times before it registered. MANGO! Mango? Thats not a dance, thats a fruit you idiot! 2:16 but it should have been 1:30 for me today. Cest la vie.How did you do? Let me know on Twitter, Instagram or Facebook.If you also play Wordle, I write guides about that as well. You can find those and all my TV guides, reviews and much more here on my blog. Thanks for reading!

How to solve today's Wordle.SOPA Images/LightRocket via Getty ImagesLooking for Fridays Wordle hints, clues and answer? You can find them here:Saturday is upon us and not a moment too soon. I have taken a bit of a writing break this past weeknot entirely, just slowed down my output to focus on other thingsbut I still feel like this week was long and Id like a break. Then again, weekdays and weekends blur together when you work from home, work every day, and your eldest child is in online school and so shes always home also. What is a weekday? What is a weekend? The lines are too blurry.Oh well. At least I have Wordle! Lets solve todays.How To Solve Todays WordleThe Hint: How is a mountain like a teabag?The Clue: This Wordle has a double letter in it.Okay, spoilers below!...The Answer:Today's WordleCredit: Erik KainWordle AnalysisEvery day I check Wordle Bot to help analyze my guessing game. You can check your Wordles with Wordle Bot right here. DRAIN was not a very good opener today, leaving me with 438 possible solutions. I tried to zoom in a bit with SCOPE and cut 430 of those away. My third guess, SPELT, finished off the narrowing down process, leaving me with just one word to pick: STEEP for the win!Competitive Wordle ScoreToday's Wordle BotCredit: Erik KainPlay Puzzles & Games on ForbesA total wash today. 0 points for guessing in four and 0 for tying the Bot. So it goes.How To Play Competitive WordleGuessing in 1 is worth 3 points; guessing in 2 is worth 2 points; guessing in 3 is worth 1 point; guessing in 4 is worth 0 points; guessing in 5 is -1 points; guessing in 6 is -2 points and missing the Wordle is -3 points.If you beat your opponent you get 1 point. If you tie, you get 0 points. And if you lose to your opponent, you get -1 point. Add it up to get your score. Keep a daily running score or just play for a new score each day.Fridays are 2XP, meaning you double your pointspositive or negative.You can keep a running tally or just play day-by-day. Enjoy!Todays Wordle EtymologyThe word steep comes from Old English stap, meaning "high, lofty, or deep." It is related to Proto-Germanic staupaz, meaning "high, steep, or precipitous," and is cognate with Old Norse staupr (steep place) and Dutch stoop (pitcher or steep descent). Over time, steep evolved to describe something sharply inclined or excessive, both in physical and figurative senses.Let me know how you fared with your Wordle today on Twitter, Instagram or Facebook. Also be sure to subscribe to my YouTube channel and follow me here on this blog where I write about games, TV shows and movies when Im not writing puzzle guides. Sign up for my newsletter for more reviews and commentary on entertainment and culture.

Elon Musk is a close associate of US president Donald TrumpK C Alfred/San Diego U-T/ZUMA Press Wire/ShutterstockIs a team led by Elon Musk going to crash US government computer systems? In recent weeks, there have been numerous reports of Musk and his associates gaining access to servers at departments ranging from the US National Oceanic and Atmospheric Administration (NOAA) and the US Treasury. It isnt clear if these complex, sensitive and ageing computer systems are being updated, or if private data is being accessed, but there are fears of serious harm being caused.Musks staff are operating under a task force called the Department

Nature, Published online: 05 February 2025; doi:10.1038/s41586-024-08419-4This Review delves into the fungal kingdom, exploring the relationships among fungi, animals, plants and the environment, and investigating both the threats posed by fungi and their potential benefits.

Nature, Published online: 05 February 2025; doi:10.1038/d41586-025-00057-8A superconducting state can be induced in a twisted carbon-based material, but it cant be described with conventional theory. A technique using high-frequency circuits could reveal the mechanism behind this fascinating state.

Have a look at this cute trombone rig that Noah Li set up in Maya. Inspired by Chris Crenshaw, he gave the instrument more character in its cartoonish movements.The process was not as simple as it might seem, however.The problem I encountered is the slide can be variable length but deformation must stay continuous without mesh poking through the slide. Deformers on one shared SplineIK curve seem the most accurate solution, but restrictive posing-wise. Control for each curve vertex is untenable, and the curve needs a high number of vertices to match the deformers. Various stretchy setups dont accommodate one section sliding over the other, he shared on LinkedIn.Noah LiDespite that, Li managed to make something unique and funny. If youd like to explore his other works, visit his LinkedIn and Vimeo.Also, join our80 Level Talent platformand ournew Discord server, follow us onInstagram,Twitter,LinkedIn,Telegram,TikTok, andThreads,where we share breakdowns, the latest news, awesome artworks, and more.Source link The post This Funny Trombone Rig Looks Adorably Alive appeared first on CG SHARES.