Sparse autoencoders are central tools in analyzing how large language models function internally. Translating complex internal states into interpretable components allows researchers to break down neural activations into parts that make sense to humans. These methods support tracing logic paths and identifying how particular tokens or phrases influence model behavior. Sparse autoencoders are especially valuable for interpretability applications, including circuit analysis, where understanding what each neuron contributes is crucial to ensuring trustworthy model behavior.A pressing issue with sparse autoencoder training lies in aligning training objectives with how performance is measured during model inference. Traditionally, training uses mean squared error (MSE) on precomputed model activations. However, this doesnt optimize for cross-entropy loss, which is used to judge performance when reconstructed activations replace the originals. This mismatch results in reconstructions that perform poorly in real inference settings. More direct methods that train on both MSE and KL divergence solve this issue, but they demand considerable computation, which limits their adoption in practice.Several approaches have attempted to improve sparse autoencoder training. Full end-to-end training combining KL divergence and MSE losses offers better reconstruction quality. Still, it comes with a high computational cost of up to 48 higher due to multiple forward passes and lack of activation amortization. An alternative involves using LoRA adapters to fine-tune the base language model around a fixed autoencoder. While efficient, this method modifies the model itself, which isnt ideal for applications that require analyzing the unaltered architecture.An independent researcher from Deepmind has introduced a new solution that applies a brief KL+MSE fine-tuning step at the tail end of the training, specifically for the final 25 million tokensjust 0.510% of the usual training data volume. The models come from the Gemma team and Pythia project. It avoids altering the model architecture and minimizes complexity while achieving performance similar to full end-to-end training. It also allows training time savings of up to 90% in scenarios with large models or amortized activation collection without requiring additional infrastructure or algorithmic changes.To implement this, the training begins with standard MSE on shuffled activations, followed by a short KL+MSE fine-tuning phase. This phase uses a dynamic balancing mechanism to adjust the weight of KL divergence relative to MSE loss. Instead of manually tuning a fixed parameter, the system recalculates the KL scaling factor per training batch. The formula ensures the total combined loss maintains the same scale as the original MSE loss. This dynamic control prevents the need for additional hyperparameters and simplifies transfer across model types. Fine-tuning is executed with a linear decay of the learning rate from 5e-5 to 0 over the 25M token window, aligning the process with practical compute budgets and preserving sparsity settings from earlier training.Performance evaluations show that this approach reduced the cross-entropy loss gap by 20% to 50%, depending on the sparsity setting. For example, on Pythia-160M with K=80, the KL+MSE fine-tuned model performed slightly better than a full end-to-end model, requiring 50% less wall-clock time. At higher sparsity (K=160), the fine-tuned MSE-only model achieved similar or marginally better outcomes than KL+MSE, possibly due to the simplicity of the objective. Tests with LoRA and linear adapters revealed that their benefits do not stack, as each method corrects a shared error source in MSE-trained autoencoders. Even very low-rank LoRA adapters (rank 2) captured over half the performance gains of full fine-tuning.Although cross-entropy results consistently favored the fine-tuned method, interpretability metrics showed mixed trends. On SAEBench, ReLU-based sparse autoencoders saw improvements in sparse probing and RAVEL metrics, while performance on spurious correlation and targeted probe tasks dropped. TopK-based models showed smaller, more inconsistent changes. These results suggest that fine-tuning may yield reconstructions better aligned with model predictions but may not always enhance interpretability, depending on the specific evaluation task or architecture type.This research underscores a meaningful advancement in sparse autoencoder training: a computationally light, technically simple method that improves reconstruction accuracy without modifying base models. It addresses key alignment issues in training objectives and delivers practical results across models and sparsity levels. While not uniformly superior in all interpretability metrics, it offers a favorable trade-off between performance and simplicity for tasks like circuit-level analysis.Check outthe Paper.All credit for this research goes to the researchers of this project. Also,feel free to follow us onTwitterand dont forget to join our85k+ ML SubReddit. NikhilNikhil is an intern consultant at Marktechpost. He is pursuing an integrated dual degree in Materials at the Indian Institute of Technology, Kharagpur. Nikhil is an AI/ML enthusiast who is always researching applications in fields like biomaterials and biomedical science. With a strong background in Material Science, he is exploring new advancements and creating opportunities to contribute.Nikhilhttps://www.marktechpost.com/author/nikhil0980/This AI Paper Introduces FASTCURL: A Curriculum Reinforcement Learning Framework with Context Extension for Efficient Training of R1-like Reasoning ModelsNikhilhttps://www.marktechpost.com/author/nikhil0980/This AI Paper Unveils a Reverse-Engineered Simulator Model for Modern NVIDIA GPUs: Enhancing Microarchitecture Accuracy and Performance PredictionNikhilhttps://www.marktechpost.com/author/nikhil0980/The Complete Beginners Guide to Terminal/Command PromptNikhilhttps://www.marktechpost.com/author/nikhil0980/How to Use Git and Git Bash Locally: A Comprehensive Guide

Emma Frost is joining the Rivals roster in season 2 as a vanguard with a set of abilities that change depending on her form.

What we've been playing - office nightmares, games with kids, and Tetris building gamesA few of the things that have us hooked this week.Image credit: Eurogamer / Galactic Cafe Feature by Robert Purchese Associate Editor Additional contributions byTom OrryPublished on April 5, 2025 5th AprilHello and welcome back to our regular feature where we write a little bit about some of the games we've been playing. This week, Bertie gets his starchiest white shirt on and descends into the corporate purgatory of The Stanley Parable, cleansing himself in the nearest river in between, while Tom O both dips back into Avowed and tries Split Fiction with his son.What have you been playing?Catch up with the older editions of this column in our What We've Been Playing archive.The Stanley Parable: Ultra Deluxe, PCEven the trailers for The Stanley Parable are genius.Watch on YouTubeI have a remarkable ability to allow things to completely pass me by - just whoosh! they're gone. Perhaps I'll do a masterclass on it one day. "One simply turns their head in the opposite direction and hey presto, the world passes them by." Oh now I've given it away.It's how I found myself only this week playing seminal 2013 indie banger The Stanley Parable, partially because Lottie loves it and appears as if summoned whenever someone even slightly references it, and also because I was told it was a lot like Severance, the TV show. And it is by the way - it's exactly like Severance, but I ended up writing a thing about that which I don't want to echo here.What I wanted to say here was that The Stanley Parable absolutely holds up. It feels as refreshing and magical and intelligent to me in 2025 as I expect it did to people in 2013 - as I know it did to people in 2013, because they wouldn't stop going on about it. And I don't think that's a given. A game like that has to work harder in 2025, partially because of the legacy that precedes it, partly because it's aging, and partly because there are that many more intelligent indie games around it. That The Stanley Parable should still shine so bright is borderline remarkable.-BertieAvowed, Xbox Series X / Split Fiction, PS5 ProSplit Fiction is great. The end.Watch on YouTubeIf there's an annoyance I have with Avowed (other than the ghosting visuals when moving the camera), it's the difficulty I have working out where to go, to get to where I want to be. I've started taking on some side quests having been focusing mainly on the central storyline, but it's thrown some map markers way out beyond where I've been before. I often head over in the direction of a marker, only to find that the door I go through takes me to an area that is locked off from where I need to go, so I end up going down blind alleys and retracing my steps back again.My solution has been to make a quick exit to the open world outside of whichever built up area I'm in, then go what I presume is the long way round to the marker. I know GPS-style navigation isn't everyone's cup of tea, but I feel Avowed would benefit from having it.I've also put a couple of hours into Split Fiction alongside my son. I wasn't really sold on the art direction in the pre-release footage, it coming across as a little bit generic, but in the heat of the action it's perfectly solid and changes things up frequently. We managed to complete the pig section just before we called it a night, and it was a lot of fun, if rather traumatic. I'm not sure what was more disturbing, though, the twerking pig or the meat grinder. But we're looking forward to seeing what happens next.-Tom ORiver Towns, PCRiver Towns is much more like Tetris than I realised.Watch on YouTubeI've had this one earmarked for a while, but I was completely wrong - I now realise - about what kind of game it would be. What I thought it would be was a cosy building game about making settlements alongside rivers, and it sort of is that, but much more immediately it's a kind of Tetris game. And I like that because I know that.The premise is really simple: lay down pieces of a town shaped as Tetronimoes are, with the hope of fitting them in perfectly to a confined space by the side of a river. Place them perfectly and you'll get a big score. It really is that simple. A town literally pops into being, as in a pop-up picture book, as you lay the pieces down.You follow a dried out river across an overland map, gradually restoring life and activity as you go, and the complexity gently increases. I now have two types of building - two factions, sort of - that I need to place away from each other so as not cross-contaminate, and I get bonus points at the end for whichever faction is larger. It's like playing against myself, which is odd.I've only had a brief blast at it so how it develops from here, I don't know, but the introduction is strong - immediate and tactile and pleasant. I like it.-Bertie

submitted by /u/indig0sixalpha [link] [comments]

Image: NintendoFollowing the announcement GameCube titles would be coming to Switch Online, Nintendo's Treehouse event provided an update about what exactly is on offer across both tiers.Some content is also noticeably exclusive to Switch 2 (and yes, this includes the GameCube library and upgrade packs).Here's the full breakdown, along with a graphic:Nintendo Switch Online- Access Nintendo Classics (150+ titles):Super NESGame BoyNES- Online Play- Nintendo Music (mobile app)- Save Data Cloud- Access to F-Zero 99, Tetris 99- Exclusively on Nintendo Switch 2:GameChatExpansion Pack- Includes all Nintendo Switch Online features- Access more Nintendo ClassicsGame Boy AdvanceNintendo 64Access SEGA Genesis / Mega Drive ClassicsAccess Select DLC- Exclusively on Nintendo Switch 2:GameChatNintendo GameCubeSelect Nintendo Switch 2 Edition upgrade packsHere's the official graphic of this information:Image: NintendoIt's worth noting how Nintendo will also be allowing Switch 2 users to try out GameChat for free until 31st March 2026.Switch Online's base tier costs $19.99 USD per year and including the Expansion Pack tier it will set you back $49.99 USD (or your regional equivalent). Including Zelda: Wind Waker!It's not just GameCubeActive Expansion Pack subscription requiredPlayable nostalgiaHow are you finding the NSO + Expansion Pack offerings? Will you be checking out GameCube games and GameChat when it arrives? Let us know in the comments.See AlsoShare:121 Liam is a news writer and reviewer for Nintendo Life and Pure Xbox. He's been writing about games for more than 15 years and is a lifelong fan of Mario and Master Chief. Hold on there, you need to login to post a comment...Related ArticlesSwitch 2 Games Cost A Bit More Than You're Probably ExpectingYou'd better sit down for this...Hollow Knight: Silksong Will Launch On Switch 2 In 2025Still no specifics, mind youPSA: Switch Online Includes Switch 2 Edition Upgrade Pack Access At "No Additional Cost"Active Expansion Pack subscription requiredGameCube Games Confirmed For Nintendo Switch Online On Switch 2Including Zelda: Wind Waker!

One of the odd aspects of the Switch 2 announcement was the quiet news that a special Multi-Language Switch 2 would be sold online in Japan.

OpenAI is reconfiguring its rollout plan for upcoming AI models. The companys CEO, Sam Altman shared on social media on Friday that it will delay the launch of its GPT-5 large language model (LLM) in favor of some lighter reasoning models to release first.The brand will now launch new o3 and o4-mini reasoning models in the coming weeks as an alternative to the GPT-5 launch fans were expecting. In this time, OpenAIwill be smoothing out some issues in developing the LLM before a final rollout. The company hasnt detailed a specific timeline, just indicating that GPT-5 should be available in the coming months.Recommended VideosThere are a bunch of reasons for this, but the most exciting one is that we are going to be able to make GPT-5 much better than we originally thought, Altman said in an X post. We also found it harder than we thought it was going to be to smoothly integrate everything, and we want to make sure we have enough capacity to support what we expect to be unprecedented demand, he added.Prior reports suggested that GPT-5 might have been prepared for release in the May timeframe; however, several unforeseen developments have popped up since then. TechRadar noted that OpenAI is likely having to tackle the tons of new users its ChatGPT service has recently acquired. Its user base recently jumped from 400 million to 500 million in about an hour, after a design trend prompted by its latest GPT-4o image generation update went viral.While the GPT-5 update has been long anticipated, the incremental updates are expected to help set up the introduction of the major rollout. The publication noted that once the o3 and o4-mini models are available, OpenAI will have products called o4 and 4o within the ChatGPT ecosystem. This might be confusing for users, but by the time the GPT-5 model rolls out, it will have the capability to select the best model for your task.Users on the free tier of ChatGPT are also set to have limited access to the GPT-5 model. However, those with Plus and Pro subscriptions will really be able to take advantage of the coming developments.Editors Recommendations

Mickey (Robert Pattinson) can be reprinted when he dies if he can take itWarner Bros. PicturesMickey 17Bong Joon HoOn general releaseIn Mickey 17, Robert Pattinson stars as the expendable Mickey. Put him in harms way and if he dies, you can just print another. And for human colonists on the ice planet of Niflheim, there is plenty of harm to get into. Theres the cold. And the general lack of everything, so the settlers must count every calorie and weigh every metal shaving. Most troublesome are the weevil-like creatures that chomp through the planets ice and rock. What they will do to the humans tin-can settlement is anyones guess.

Airbus unveiled early sketches of a future aircraft with open-fan engines. Courtesy of Airbus 2025-04-05T08:41:01Z SaveSaved Read in app This story is available exclusively to Business Insider subscribers. Become an Insider and start reading now.Have an account? Airbus engineers shared plans for its next aircraft, planned for the second half of next decade.They're aiming to increase fuel efficiency reducing emissions and making operations cheaper.Innovations include open-fan engines, folding wings, and automatic taxiing.While Airbus has pushed back plans for a hydrogen-powered plane, it plans to build a high-tech plane that is cheaper and more environmentally friendly. Business Insider got a look at the tech that might drive it. At its recent summit in at its headquarters Toulouse, France, CEO Guillaume Faury told reporters it was developing a new single-aisle aircraft to succeed its A320 family and enter service in the second half of the 2030s.He said the hydrogen-powered plane was canceled as it risked being "a Concorde of hydrogen" and not commercially viable at scale.He said a next-generation plane would not come with "incremental optimization" but "clean sheet designs." He also predicted a "gloves-off" competition to build a next-generation airplane with Airbus' US rival Boeing.This is the new tech it's developing.Open-fan engines Airbus and CFM are planning open-fan engine flight tests on the A380 by 2030. Pete Syme/BI Higher fuel efficiency is a major discussion point in aviation, but making it happen will require "a fundamental change in the shape of the engine," said Mohamed Ali, GE Aerospace's chief technology and operations officer.Turbofan jet engines work by taking in air. Some of this air enters the engine core where it is mixed with fuel and combusts to drive the turbines. The rest of the air is accelerated by the fan and bypasses the core.Engines which have a higher proportion of bypassed air are more fuel efficient. However, this ratio is limited by the size of the intake duct."Here is the solution, and that's the beauty of physics. You can remove that duct and go to open fan," Ali said.While today's engines have a bypass ratio of up to 12:1, Ali said current designs for an open-fan engine would improve that to 60:1.Airbus and CFM a joint venture between GE and Safran plan to test open-fan engines on an Airbus A380 by the end of the decade.They are also planning for the new engines to be able to operate entirely with sustainable aviation fuel. This can be produced from plants and cooking oil, but its takeup has so far been hindered by high costs and a lack of availability.Folding wings A Wing of Tomorrow model on display at the Airbus Summit. Pete Syme/BI Airbus engineers are looking to longer wings to create more lift and reduce drag, which in turn means using less fuel. However, the size of a plane's wings is limited by the size of airport gates.Yet if the wingtips can fold, planes can still have longer wings in flight and fit into airport gates on the ground.If this sounds familiar, you might've seen something similar on the coming Boeing 777X, although Airbus is planning to first use it on narrow-body planes."If you think about the next generation of single-aisle aircraft, it will fly many times a day, 24 hours a day, 7 days a week," said Sue Partridge, head of the Wing of Tomorrow program. "That folding wing system needs to work reliably."New materials and some automationAnother step forward in fuel efficiency could be found in new composite materials.The latest generation of airliners, like the Airbus A350 and Boeing 787, use carbon fiber-reinforced polymers in the wings and fuselage. These are stronger and lighter than aluminum, but more advances could be made."We are working on the next generation of composite materials, with our material suppliers working as partners with us," Partridge told the summit.Just before the summit began, Reuters reported that both planemakers are looking at thermoplastics for future aircraft.Single-aisle planes are more popular than widebodies, with Airbus eyeing a target of producing 75 a month."We need to be able to make our aircraft and our wings at high rate and at a cost that's actually sustainable for our business," Partridge said.She added that the Wing of Tomorrow program was looking at robotic and automation technologies to see how they could speed up manufacturing.Automation won't just take place in the factories, however.At the summit, BI rode in a self-driving van testing Airbus' new autonomous technology, Optimate.It's been testing cameras, radar, and lidar to help follow airport lines for autonomous taxiing and to avoid potential collisions. Near misses have been growing more common in recent years, and Airbus also predicts there will be twice the number of aircraft in 20 years.AI could also be used to predict traffic and help pilots reroute around turbulence or bad weather. Airbus' Optimate van uses lidar and cameras. Pete Syme/BI There are still more than 10 years before Airbus sees its next generation of planes taking to the skies.Faury, the CEO, compared the process to "a caterpillar in a cocoon becoming a butterfly.""In this transition, there's a bit of paralysis," he said. "There's a lot of forces at stake, it's painful, and then the butterfly takes off. We don't know what it looks like until we're there.""We remain committed to our purpose, which is to pioneer sustainable aerospace for a safe and united world," he added. "We think it's a beautiful purpose, but it's not an easy one."Recommended video

Nintendo played it safe with Switch 2 and Im already disappointed Readers FeatureGameCentralPublished April 5, 2025 9:00am Did you feel the Direct was a let-down? (Nintendo)A reader is unimpressed with the Switch Nintendo Direct and complains that only a minimum effort is being made with the games and hardware.I had a bad feeling about the Nintendo Switch 2 from the moment it was unveiled. Even before that we had lots of people demanding it be just a souped-up Switch, with the same basic look and design, and I always thought that was a boring and very un-Nintendo idea. But they got their wish and here it is: the Switch 2. Or rather Switch 1.2, amirite?Im not going to pretend that the Switch 2 Direct was a disaster or that Im not going to buy one I probably will eventually but everything Ive seen of it so far is disappointingly predictable and predictable is the last thing I usually associate with Nintendo.The mystery C button has turned out to be a chat feature nobodys ever going to use (and you need a webcam for anyway), there were no other major secrets about the design and mouse controls really were just mouse controls. It all looks perfectly fine but nothing about it excites me yet, and that worries me.In terms of games we got Mario Kart World (aka Mario Kart 9), Metroid Prime 4 (which looked to be doing nothing different to the previous 3) and an upgrade for Super Mario Party Jamboree that clearly nobody is ever going to play more than once, while shrugging their shoulders and going Is that it?The Zelda remasters are such minor improvements youll need a tedious Digital Foundry video to explain what the differences are. Meanwhile, that wheelchair basketball game looked like something a junior programmer knocked up in an afternoon. Welcome Tour, on the other hand, looks like something they made while on the toilet.There was other stuff in the trailer that doesnt seem to have been playable, so we dont know how it is, but it all seems like smoke and mirrors. A third Hyrule Warriors game, for the three people that like that, and a sequel to Kirby Air Ride for the even smaller crowd that like that (seriously, why that of all things when we already have Mario Kart?). Oh, a new FromSoftware exclusive that turned out to be a very unexciting sounding multiplayer game. Urgh.Where was Splatoon, Animal Crossing, Zelda, Super Smash Bros., Fire Emblem, Pikmin, and all the rest? Where was the new IP that showed off how the mouse works, considering Mario Kart and Donkey Kong dont use it at all? That was clearly the point of the basketball game but was that really the best Nintendo could do? And why was it so grey?In the end, all we got that was new was the Donkey Kong game, which looked okay? I cant say it blew me away in any way. Given theyve had eight years to come up with a groundbreaking line-up I cannot say that the Direct exceeded my expectations or even really met them.Everything seems so safe and unsurprising. It wasnt a terrible reveal, but it does feel like the minimum effort necessary not to be one.You can tell it was a disappointment because people immediately started inventing conspiracy theories about there being a secret big name game being held back for Christmas. But I dont think so. Theres actually a ton of games with 2025 dates, that we dont know the exact time for yet, but its all the boring junk I already mentioned, like Hyrule Warrior and Kirby.More TrendingYou could argue the SNES was just a souped-up NES, so its not like just making the same thing but more powerful cant work, but the SNES had tons of great games right from the start and I dont see that for the Switch 2. Maybe Nintendo is holding back but I really dont see why they would. This is it and its not that great.By reader Olliephant Donkey Kong Bananza not a sequel (Nintendo)The readers features do not necessarily represent the views of GameCentral or Metro.You can submit your own 500 to 600-word reader feature at any time, which if used will be published in the next appropriate weekend slot. Just contact us at gamecentral@metro.co.uk or use our Submit Stuff page and you wont need to send an email.GameCentralSign up for exclusive analysis, latest releases, and bonus community content.This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply. Your information will be used in line with our Privacy Policy