USC Games announced its 9th annual USC Games Expowill return to USCs campus on May 13 with more than 60 student games.Read More

Generative AI art made without human "creativity" cannot be protected by copyright lawsRepeatedly revising prompts does not change the US Copyright's Office's position, eitherImage credit: US Copyright Office News by Vikki Blake Contributor Published on Jan. 30, 2025 The US Copyright Office has confirmed that unless an author or artist contributes to the "creative process" of generative AI, it "cannot be protected by copyright" laws.The report by the US Copyright Office affirms that copyright protection in the United States "requires human authorship", giving the example that photographs taken by a monkey, for instance, "could not be protected".However, authors could "make the grade [to qualify for copyright] quite easily" as "the requisite level of creativity is extremely low"."In most cases, however, humans will be involved in the creation process, and the work will be copyrightable to the extent that their contributions qualify as authorship," the report explains."It is axiomatic that ideas or facts themselves are not protectible by copyright law, and the Supreme Court has made clear that originality is required, not just time and effort. In Feist Publications, Inc. v. Rural Telephone Service Co., the Court rejected the theory that 'sweat of the brow' alone could be sufficient for copyright protection."The vast majority of works make the grade quite easily, as they possess some creative spark, 'no matter how crude, humble or obvious' it might be."Repeatedly revising generative art prompts does not change the office's position, either."No matter how many times a prompt is revised and resubmitted, the final output reflects the users acceptance of the AI systems interpretation, rather than authorship of the expression it contains," the report stated.The team behind Catly, a pet sim announced at last month's The Game Awards, was forced to deny its debut trailer was created using generative AI after rumours began to swirl that the game may use blockchain tech or NFTs after some connected the studio's co-founder, Kevin Yeung, to other blockchain projects he is working on with a different team at TenthPlanet.

Jan 29Andrew WebsterDino Crisis launches on PC as GOG adds new tool to bring back more classicsImage: CapcomA survival horror classic is getting a new life on PC and it might be the first of many. PC games shop GOG announced the release of Dino Crisis and its sequel, which are both available starting today. Alongside the launch, the service also announced a new tool called dreamlist, with the goal of letting players vote on which old games that want to see hit the service next.For the uninitiated, Dino Crisis is essentially Resident Evil with dinosaurs. Thats not an insult: the first game was directed by original Resident Evil director Shinji Mikami, so the two series have a lot in common. Dino Crisis features the same methodical, survival horror gameplay but with faster (and occasionally much bigger) enemies to contend with. GOG says the new port supports Windows 10 / 11 and includes updates like support for ~4K resolution and modern gamepads. The two games can be purchased individually on GOG or as a bundle.Read Article >Nov 13, 2024Ash ParrishGOGs new preservation program intends to keep classic games playable foreverPC game platform GOG has launched a new preservation program dedicated to keeping beloved older games playable, now and in the future.If a game is part of the Preservation Program, it means that we commit our own resources to maintaining its compatibility with modern and future systems, the announcement blog reads.Read Article >Oct 25, 2024Sean HollisterVideo game preservationists have lost a legal fight to study games remotelyCollage by Cath Virginia / The Verge | Photos from Getty ImagesWhen video game scholars want to study games that are no longer on sale, they sometimes have to drive many hours to do it legally and that wont be changing anytime soon. The US Copyright Office has just denied a request from video game preservationists to let libraries, archives and museums temporarily lend individuals some virtual, remotely accessible copies of those works.Kendra Albert, who made the argument on behalf of the Software Preservation Network and the Library Copyright Alliance, says preservationists werent asking for a lot: It was the thing that basically exists for all kinds of special collections in libraries: the library reviews the request, makes sure its not harmful, and allows access to the work.Read Article >Sep 17, 2024Jay PetersSpellbreak lives.The game fantasy battle royale game was shut down, but a community version was released on Itch.io at the end of 2022 that has a thriving community around it, as reported by Aftermath. I wish more games would get second lives like this.Spellbreak, Shut Down When Its Developer Was Acquired By Blizzard, Is Still Alive - Aftermath[aftermath.site]Jun 26, 2024Andrew WebsterSurvival horror classics, sans DRM.Classic gaming site GOG.com is bringing the original Resident Evil series to its service, which means you can play the PlayStation-era horror trilogy on PC without fussing with DRM. The first game is available now, while parts two and three are on their way.Image: CapcomJun 25, 2024Andrew WebsterAn excellent retro collection gets even better.The playable documentary Atari 50 is getting an expansion that will add 39 classic games, split across two new interactive timelines, with even more behind-the-scenes details. Its not clear yet when the update will drop (or for how much), but later this year the collection is also getting a physical release.Mar 8, 2024Ash ParrishOne fan spent three years saving a Final Fantasy game before it shut downImage: Square EnixFebruary 29th was the last day of service for Dissidia Final Fantasy Opera Omnia, a mobile game based on Square Enixs Final Fantasy fighting game series. Since launching in the US in 2018, the game has amassed over 170 characters from across 30 years of Final Fantasy history, scattered throughout four acts composed of multiple chapters and side stories representing hundreds of hours of a game that can no longer be played. Though Opera Omnia is officially gone, one person has shouldered the task to ensure the game hasnt completely disappeared.On June 6th 2021 I began to work towards my goal of recording and rendering every Dissidia Final Fantasy Opera Omnia cutscene and upload[ing] them to YouTube, wrote Hatok, a video editor and self-described video game enthusiast. And now, 7 hours from the end of service, Ive reached the end of this massive project.Read Article >Jan 18, 2024Jay PetersAnother cool piece of Limited Run Games history.The companys physical version of Shantae and the Pirates Curse was was literally the last game produced on the US 3DS production line, with our cartridge order managing to get to Nintendo at the absolute last possible second, CEO Josh Fairhurst says on X. The game is Limited Runs final 3DS release.You can see the post on X. Screenshot by Jay Peters / The VergeJan 11, 2024Jay PetersFelix the Cat and Sparkster are back.Konami and Limited Run Games are partnering on physical rereleases of two Felix the Cat games and three Rocket Knight games for PS4, PS5, and Nintendo Switch. I was obsessed with Rocket Knight Adventures as a kid, so Im happy to see it available in physical formats on modern consoles.Image: Konami and Limited Run GamesDec 6, 2023Andrew WebsterJeff Minter is getting a playable documentary.Digital Eclipse is continuing its interactive documentary series with an upcoming collection about game designer and noted llama enthusiast Jeff Minter. The studio previously made Atari 50 and The Making of Karateka, and the new release will feature 42 different and very trippy classic games when it launches in 2024.Dec 5, 2023Jay PetersPlayStation keeps reminding us why digital ownership sucksPhoto by Vjeran Pavic / The VergeIn less than a week, Sony has given us two timely reminders of the tenuousness of digital ownership and both reminders involve things on PlayStation.Last week, Sony said that, because of content licensing arrangements, users wouldnt be able to watch Discovery content theyve purchased and that the content would be removed from their libraries as of December 31st, 2023. The resulting list of shows that will suddenly disappear because of corporate agreements is very long. Shows disappearing from streaming services is commonplace, but in this case, people are losing access to shows they bought to watch on demand whenever they wanted.Read Article >Dec 1, 2023Andrew WebsterAtari 50 is getting a big update.Now that developer Digital Eclipse is under the Atari umbrella, the studio is about to drop the first major update for its brilliant retro collection / playable documentary Atari 50. It launches on December 5th, and will add 12 new (old) Atari 2600 games. Unfortunately, its not clear yet what games will be coming, but it sounds like this is just the start of many updates to come.Nov 13, 2023David PierceThe fight to save old video gamesIllustration by Samar Haddad / The VergeWhen I ask Frank Cifaldi, the founder and director of the Video Game History Foundation, to explain the importance of preserving and maintaining old video games, he answers with a movie analogy. Imagine, he said, if movies were only released on, like, VHS, ever. You want to watch Back to the Future? All right, you have to go on eBay, and you have to find an antique VHS copy thats degraded a bit from use. You have to find a VCR that works, a TV that it plugs into or the external scalers that make it look correct on your modern TV and you might need a time-base corrector because the magnetic flux signal is out of sync.For too many games, this is the state of the industry. For the most part, decades worth of games now exist only in their original form: on a disk or cartridge that goes into a console nobody has anymore. Many of those games are going to be hard for players to ever find again and if we dont do anything to save them, they might disappear altogether.Read Article >Oct 31, 2023Jon PorterAtari is buying the developer behind its excellent 50th anniversary retro compilationImage: AtariAtari is acquiring Digital Eclipse, a studio that specializes in preserving and rereleasing retro games on modern hardware. The releases are often rich in additional materials and historical content. Think the Criterion Collection but for video games. In a press release, Atari says its paying up to $20 million for the studio, including an initial $6.5 million paid in a combination of cash and shares and a further $13.5 million, which is due to be paid in cash over the next decade subject to Digital Eclipses performance. It expects to complete the deal in the coming days.The two companies previously worked together on last years excellent Atari 50: The Anniversary Celebration, which included ports of over 90 classic games as well as unreleased prototypes and neat extras like short documentaries and old photos and magazine articles. But Digital Eclipse has also produced a number of other well-received rereleases of classic games including Teenage Mutant Ninja Turtles: The Cowabunga Collection and the interactive documentary The Making of Karateka.Read Article >Oct 16, 2023Jay PetersThe Portland Retro Gaming Expo helps keep the classics aliveSega had their own console?That was a devastating comment I heard as I sat playing Sonic the Hedgehog 2 on a Sega Genesis Mini at the Portland Retro Gaming Expo (PRGE) this weekend. The Sonic games on Genesis were my favorites as a kid; one time, I played so much Sonic that I peed my pants. How could this person not know about Sega consoles?Read Article >Oct 10, 2023Jay PetersAnother Stadia exclusive just ate a Power Pellet.Pac-Man Mega Tunnel Battle: Chomp Champs, a battle royale take on Pac-Man that was previously only available on Google Stadia before Stadia shut down, will get another chance on Nintendo Switch, PS5, PS4, Xbox Series X / X, Xbox One, and Steam, Bandai Namco announced on Tuesday. Its set to release in early 2024.Also, I regret to inform you that Nintendo just shut down Pac-Man 99. We knew the shutdown was coming, but Im still sad to see it go.Sep 28, 2023Andrew WebsterA fun way to support video game preservation.The Video Game History Foundation does great work when it comes to preserving the history of the medium, and now the institute is offering a unique way for people to show support. Right now on GOG you can purchase a supporter pack filled with all kinds of rare art and promotional materials for franchises like Yakuza and The Elder Scrolls, with proceeds going towards the VGHF.Check out this interview for more on the work theyre doing.GOG Interview: learn more about games preservation with Video Games History Foundation and support their cause![GOG.com]Sep 15, 2023Andrew WebsterThe original Wizardry has been remastered and you can play it right nowFresh off the launch of The Making of Karateka, retro game studio Digital Eclipse has announced a remaster of the original action RPG title Wizardry. Even better its out now in early access on both Steam and GOG.First released in 1981 and developed by Sir-Tech Software, the first Wizardry or Wizardry: Proving Grounds of the Mad Overlord, to give it its very long full title was a first-person, party-based, D&D-inspired RPG that proved incredibly influential and kicked off a long-running franchise. Heres the official description for the uninitiated:Read Article >Aug 29, 2023Andrew WebsterDigital Eclipse is preserving classic games in the most entertaining way possibleThe Making of Karateka. Image: Digital EclipseThe first game you play in The Making of Karateka, a release about the making of the cinematic action game Karateka, is an Asteroids clone. It may seem strange, but the attempt at recreating Asteroids was a seminal moment for Karateka creator Jordan Mechner, a humble beginning for a designer who would go on to create some of the most influential action games ever made, including Prince of Persia. The clone appears early on during a long, winding, and interactive timeline, which takes you through Mechners early days of experimentation before getting into how Karateka was developed and released.For studio Digital Eclipse, the team behind the interactive documentary, its another attempt at telling the story of classic games in a way thats completely native to the medium. As classic games become increasingly endangered, this approach is an important tool for keeping them alive. I think its the best way to tell these stories, says Digital Eclipse president Mike Mika. If youre going to tell the story of an interactive experience, it better be interactive.Read Article >Jul 14, 2023Ash ParrishAlmost 90 percent of classic games are critically endangered, say archivistsPhoto by Vjeran Pavic / The VergeWith advancing console generations and the slow demise of both backward compatibility and digital game storefronts, the ability to play older games has always been tough. Through a new study from the Video Game History Foundation (VGHF) and the Software Preservation Network, we now have a better understanding of just how difficult accessing older games really is.87% of classic video games released in the United States are critically endangered, wrote Kelsey Lewin, co-director of the Video Game History Foundation.Read Article >Dec 25, 2022Ash ParrishThe quest to save a Stadia exclusiveImage: Q-GamesEarlier this year, Google announced that it was shutting down its game streaming service Stadia, a short three years since its launch in 2018. While its mostly fans of the service feeling the impact of the closure, there are a handful of developers with Stadia exclusives that will unfortunately lose their games when the service shuts down for good in January. One of those is Q-Games, makers of PixelJunk Raiders. The Verge spoke with Q-Games founder and CEO, Dylan Cuthbert, who explained the unique situation Q-Games is in, trying to get their exclusive off Stadias foundering ship and somewhere safe where people can play it.PixelJunk Raiders is a space exploration roguelike that takes advantage of Stadias unique state share feature that allows people to share instances of their game in which other players can jump into and experience for themselves. Read Article >Nov 11, 2022Andrew WebsterAtari 50 is an incredible playable tour through video game historyAtari Karts on the Jaguar. Image: AtariOne of the biggest challenges in video game preservation is figuring out how to actually present old games. In 2022, there are more ways than ever to play the classics, whether its mini consoles, updated hardware, subscription services, retro collections, or modern rereleases. While these can make old games playable to new audiences, they arent always able to put them in a proper context which is especially important for really old games like, say, Adventure on the Atari 2600.But an expansive new release, made by Digital Eclipse to celebrate Ataris 50th anniversary, is the best attempt at a retro collection Ive ever experienced. Its available on just about every console out there right now as well as the PC, letting me put my PS5 to use for its intended purpose: playing Asteroids. The collection is huge, detailed, and does an amazing job of explaining why these games are so important.Read Article >Oct 12, 2022Ash ParrishTwo never-published NES games are up for auction on eBayPhoto by TENGKU BAHAR / AFP via Getty ImagesWhen Frank Cifaldi, founder and co-director at the Video Game History Foundation, finds an unreleased original NES game on eBay, its cause for celebration. Unearthing never-before-published games is his research kink, and according to him, such discoveries only happen once every five years. Yesterday, he found two.One game, Battlefields of Napoleon, got so far along in its development process that it has finalized box art, but the game never saw the light of a retail store. The other game is even rarer. Its a demo for the infamous Power Glove developed by Rare (yes, apparently that Rare) and is one of only five known games for the peripheral in existence (if you count the two games Nintendo announced but never released). Cifaldi is now working to raise funds to win both auctions so these never-before-seen pieces of video game history can be preserved and their contents shared with the public.Read Article >Sep 1, 2022Alicia HaddickThe ticking time bomb of modern free-to-play gamesNintendos Dragalia Lost is shutting down at the end of 2022. Photo by Amelia Holowaty Krales / The VergeDragalia Lost launched in 2018 as a statement of intent from Nintendo in partnership with Japanese developer Cygames. Nintendo may have first jumped into the field of mobile games in 2016 with the launch of games like Super Mario Run and Miitomo, but this was the first original property the company had produced exclusively for mobile devices. This free-to-play gacha game (a game whose content is generally free to access while charging microtransactions for loot boxes and randomized lotteries for rare and limited-time characters) had a flashy multi-region launch campaign collaborating with major Japanese musician DAOKO, banking on the games success at home and abroad.And it was a hit. Less than a year after launch, the game had already earned over $100 million, with a steady stream of merchandise following soon after. Yet, as of last month, Nintendo and Cygames published the games final update, and this week, it was revealed that the game would shut down on November 29th after just three years of operation.Read Article >Apr 19, 2022Ash ParrishHabitat for humanity: how a classic MMO got a second lifeIllustration by Melissa Mathieson / The VergeHabitat, the worlds first MMO developed for the Commodore 64 personal computer, went offline in 1992. It came back online in 2017 through the efforts of MADE, the Museum of Art and Digital Entertainment.Founded by Alex Handy in 2011, MADE seeks to legitimize the preservation of video games as both a historic and artistic medium within the context of our time. To that end, MADE amassed a collection of working video game consoles and a library of old games for patrons to play. Read Article >

DeepSeek has secured a completely open database that exposed user chat histories, API authentication keys, system logs, and other sensitive information, according to cloud security firm Wiz. The security researchers said they found the Chinese AI startups publicly accessible database in minutes, with no authentication required.The exposed information was housed within an open-source data management system called ClickHouse and consisted of more than 1 million log lines. As noted by Wiz, the exposure allowed for full database control and potentialprivilege escalation within the DeepSeek environment, which couldve given bad actors access to the startups internal systems. These findings were first reported by Wired.DeepSeek promptly secured the database after Wiz notified the startup about the issue.Its still not clear whether anyone else was able to access the exposed data, but the researchers told Wired, it wouldnt be surprising, given how simple it was to discover. Wizs researchers also told the outlet that DeepSeeks systems are designed similarly to those used by OpenAI, down to details like the format of the API keys. OpenAI accused DeepSeek of using its data to train its AI models earlier this week.

GameDev Archives GameDev ArchivesSelect Month January 2025 (17) December 2024 (29) November 2024 (25) October 2024 (26) September 2024 (22) August 2024 (14) July 2024 (20) June 2024 (20) May 2024 (21) April 2024 (23) March 2024 (30) February 2024 (23) January 2024 (17) December 2023 (18) November 2023 (17) October 2023 (30) September 2023 (30) August 2023 (13) July 2023 (17) June 2023 (20) May 2023 (24) April 2023 (20) March 2023 (25) February 2023 (22) January 2023 (19) December 2022 (15) November 2022 (25) October 2022 (18) September 2022 (19) August 2022 (18) July 2022 (13) June 2022 (22) May 2022 (23) April 2022 (28) March 2022 (29) February 2022 (21) January 2022 (21) December 2021 (18) November 2021 (26) October 2021 (22) September 2021 (21) August 2021 (23) July 2021 (24) June 2021 (23) May 2021 (24) April 2021 (28) March 2021 (35) February 2021 (26) January 2021 (28) December 2020 (33) November 2020 (38) October 2020 (34) September 2020 (38) August 2020 (30) July 2020 (33) June 2020 (34) May 2020 (27) April 2020 (34) March 2020 (31) February 2020 (26) January 2020 (22) December 2019 (20) November 2019 (23) October 2019 (34) September 2019 (30) August 2019 (17) July 2019 (25) June 2019 (22) May 2019 (26) April 2019 (21) March 2019 (28) February 2019 (24) January 2019 (27) December 2018 (14) November 2018 (19) October 2018 (22) September 2018 (23) August 2018 (23) July 2018 (30) June 2018 (32) May 2018 (33) April 2018 (21) March 2018 (29) February 2018 (16) January 2018 (18) December 2017 (15) November 2017 (15) October 2017 (15) September 2017 (20) August 2017 (12) July 2017 (15) June 2017 (25) May 2017 (20) April 2017 (16) March 2017 (30) February 2017 (31) January 2017 (24) December 2016 (32) November 2016 (42) October 2016 (27) September 2016 (34) August 2016 (32) July 2016 (35) June 2016 (37) May 2016 (48) April 2016 (83) March 2016 (56) February 2016 (43) January 2016 (54) December 2015 (44) November 2015 (49) October 2015 (17) September 2015 (13) August 2015 (13) July 2015 (17) June 2015 (27) May 2015 (20) April 2015 (15) March 2015 (18) February 2015 (9) January 2015 (11) December 2014 (17) November 2014 (22) October 2014 (47) September 2014 (9) August 2014 (5) July 2014 (14) June 2014 (17) May 2014 (12) April 2014 (13) March 2014 (18) February 2014 (13) January 2014 (15) December 2013 (6) November 2013 (8) October 2013 (8) September 2013 (21) August 2013 (13) July 2013 (16) June 2013 (17) May 2013 (13) April 2013 (21) March 2013 (18) February 2013 (13) January 2013 (13) December 2012 (15) November 2012 (22) October 2012 (24) September 2012 (24) August 2012 (41) July 2012 (16) June 2012 (32) May 2012 (28) April 2012 (19) March 2012 (12) February 2012 (18) January 2012 (11) December 2011 (12) November 2011 (13) October 2011 (22) September 2011 (7) August 2011 (11) July 2011 (5) June 2011 (7) May 2011 (16) April 2011 (10) January 1970 (1)

Tokenization plays a fundamental role in the performance and scalability of Large Language Models (LLMs). Despite being a critical component, its influence on model training and efficiency remains underexplored. While larger vocabularies can compress sequences and reduce computational costs, existing approaches tie input and output vocabularies together, creating trade-offs where scaling benefits larger models but harms smaller ones. This paper introduces a framework called Over-Tokenized Transformers that reimagines vocabulary design by decoupling input and output tokenization, unlocking new pathways for model efficiency and performance.Reference: https://arxiv.org/pdf/2501.16975Traditional tokenization methods use identical vocabularies for input processing and output prediction. While larger vocabularies allow models to process longer n-gram tokens (e.g., multi-character sequences), they force smaller models to handle overly granular output predictions, increasing underfitting risks. For instance, a 3-gram tokenizer reduces sequence length by 66% but requires predicting three characters jointlya task manageable for large models but overwhelming for smaller ones. Previous work like multi-token prediction (MTP) attempted to address this by predicting future tokens in parallel, but these methods still entangled input/output granularity and struggled with smaller architectures.The research team identified a critical insight through synthetic experiments with context-free grammars: input and output vocabularies influence models differently. Larger input vocabularies consistently improved all model sizes by enriching context representations through multi-gram embeddings. Conversely, larger output vocabularies introduced fine-grained prediction tasks that only benefited sufficiently large models. This dichotomy motivated their Over-Tokenized framework, which separates input encoding (Over-Encoding) and output decoding (Over-Decoding) vocabularies.Over-Encoding (OE) scales input vocabularies exponentially using hierarchical n-gram embeddings. Instead of a single token ID, each input token is represented as the sum of 1-, 2-, and 3-gram embeddings. For example, the word cat might decompose into embeddings for c, ca, and cat, allowing the model to capture multi-scale contextual cues. To avoid impractical memory costs from large n-gram tables (e.g., 100k entries), the team used parameter-efficient techniques:Modulo-based token hashing: Maps n-gram tokens to a fixed-size embedding table using modular arithmetic, enabling dynamic vocabulary expansion without storing all possible combinations.Embedding decomposition:Over-Decoding (OD) approximates larger output vocabularies by predicting multiple future tokens sequentially, a refinement of earlier MTP methods. For instance, instead of predicting one token at a time, OD trains the model to predict the next two tokens conditioned on the first prediction. Crucially, OD is selectively appliedonly larger models benefit from this granular supervision, while smaller ones retain single-token decoding to avoid underfitting.The researchers performed experiments on OLMo and OLMoE architectures and demonstrated three key findings:Log-Linear Scaling: Training loss decreased linearly as input vocabulary size grew exponentially (Figure 1). A 400M parameter model with a 12.8M-entry input vocabulary matched the performance of a 1B-parameter baseline, achieving 2.5 effective scaling at equal computational cost.Convergence Acceleration: Over-Encoding reduced training steps needed for convergence by 35 across tasks like MMLU and PIQA, suggesting richer input representations accelerate learning.Sparse Parameter Efficiency: Despite using 128 larger input vocabularies, memory and computation overheads increased by <5% due to sparse embedding access and optimized sharding strategies.On evaluations, the framework demonstrated consistent performance improvements across various model types. For dense models, a 151M Over-Encoded (OE) model achieved a 14% reduction in perplexity compared to its baseline. Similarly, in sparse Mixture-of-Experts (MoE) models, the OLMoE-1.3B with OE reduced validation loss by 0.12 points, although the gains were less pronounced as the benefits of sparse experts diluted the impact of embedding enhancements. Beyond synthetic experiments, real-world evaluations on large-scale datasets further validated these findings. Over-Encoded models consistently improved performance across multiple benchmarks, including MMLU-Var, Hellaswag, ARC-Challenge, ARC-Easy, and PIQA. Notably, the framework accelerated convergence, achieving a 5.7 speedup in training loss reduction. Additionally, downstream evaluations showed significant acceleration, with OE delivering speedups of 3.2 on MMLU-Var, 3.0 on Hellaswag, 2.6 on ARC-Challenge, 3.1 on ARC-Easy, and 3.9 on PIQA, highlighting its efficiency and effectiveness across diverse tasks.In conclusion, this work redefines tokenization as a scalable dimension in language model design. By decoupling input and output vocabularies, Over-Tokenized Transformers break traditional trade-offs, enabling smaller models to benefit from compressed input sequences without grappling with overly complex prediction tasks. The log-linear relationship between input vocabulary size and performance suggests embedding parameters represent a new axis for scaling laws, complementing existing work on model depth and width. Practically, the framework offers a low-cost upgrade path for existing architecturesintegrating Over-Encoding requires minimal code changes but yields immediate efficiency gains. Future research could explore hybrid tokenization strategies or dynamic vocabulary adaptation, further solidifying tokenizations role in the next generation of efficient, high-performing LLMs.Check out the Paper. All credit for this research goes to the researchers of this project. Also,dont forget to follow us onTwitter and join ourTelegram Channel andLinkedIn Group. Dont Forget to join our70k+ ML SubReddit.(Promoted) Vineet KumarVineet Kumar is a consulting intern at MarktechPost. He is currently pursuing his BS from the Indian Institute of Technology(IIT), Kanpur. He is a Machine Learning enthusiast. He is passionate about research and the latest advancements in Deep Learning, Computer Vision, and related fields.Vineet Kumarhttps://www.marktechpost.com/author/vineet1897/Creating An AI Agent-Based System with LangGraph: A Beginners GuideVineet Kumarhttps://www.marktechpost.com/author/vineet1897/Unlocking Autonomous Planning in LLMs: How AoT+ Overcomes Hallucinations and Cognitive LoadVineet Kumarhttps://www.marktechpost.com/author/vineet1897/Plurai Introduces IntellAgent: An Open-Source Multi-Agent Framework to Evaluate Complex Conversational AI SystemVineet Kumarhttps://www.marktechpost.com/author/vineet1897/Create Portrait Mode Effect with Segment Anything Model 2 (SAM2) [Recommended] Join Our Telegram Channel

#60: DeepSeek, CAG, and the Future of AI Reasoning 0 like January 30, 2025Share this postAuthor(s): Towards AI Editorial Team Originally published on Towards AI. Good morning, AI enthusiasts! The last two weeks in AI have been all about Deepseek-R1. So this weeks issue includes resources and discussions on that, along with emerging techniques such as CAG, AI agent frameworks like AutoGen, AG2, and Semantic Kernel, and more. Enjoy the read!Whats AI WeeklyThis week in Whats AI, I explore Cache-Augmented Generation (CAG), which has emerged as a true alternative to RAG. RAG is great for accuracy but needs time to search through and compare documents. And it gets worse the more data you have. This is where CAG comes in and says, What if we just preloaded all that knowledge directly into the models memory? So, lets understand what CAG is, how it differs from RAG, and when to use both. Read the article here or watch the video on YouTube. Louis-Franois Bouchard, Towards AI Co-founder & Head of CommunityLearn AI Together Community section!Featured Community post from the DiscordFuwafuwari. has built a resource website that can serve as a roadmap for anyone starting in the field of AI. It includes curated roadmaps, videos, articles, and other learning materials. You can start learning through the open-sourced GitHub repository. You can also contribute material and support a community member. If you have any questions or suggestions, reach out in the thread!AI poll of the week!Only a handful of people knew about DeepSeek a few days ago. Yet, thanks to the release of DeepSeek-R1, its been arguably the most discussed company, and the polls also show a clear inclination towards using DeepSeek. Is price guiding your decision? Tell us in the thread!Meme of the week!Meme shared by richardlhkTAI Curated sectionArticle of the weekAdvancing Time Series Forecasting: A Comparative Study of Mamba, GRU, KAN, GNN, and ARMA Models By Shenggang LiThis article evaluates five models for multivariate time series forecasting: Mamba, GRU, KAN, GNN, and ARMA. It highlights the strengths and limitations of each, with KAN excelling due to its dynamic weighting for nonlinear relationships, achieving the lowest error rates. GRU performed well in capturing sequential dependencies, while Mamba balanced accuracy and interpretability. GNN showed moderate results, requiring further tuning, and ARMA struggled with multivariate complexity. It also proposes integrating Mamba and KAN for a unified framework, combining temporal modeling with nonlinear adaptability. This comparative study provides insights into selecting and enhancing models for complex forecasting tasks.Our must-read articles1. AutoGen, AG2, and Semantic Kernel: Complete Guide By Naveen KrishnanThis article provides a comprehensive guide to three AI agent frameworks: AutoGen, AG2, and Semantic Kernel. It explores their architectures, features, and use cases, offering practical examples for implementation. AutoGen introduces asynchronous messaging, modularity, debugging tools, and applications like AutoGen Studio for rapid prototyping. AG2, a community-driven evolution of AutoGen, focuses on agent orchestration and collaboration. Semantic Kernel, a lightweight framework, supports enterprise-grade AI integration. It concludes by highlighting the strengths of each framework, helping developers select the most suitable option for building intelligent, autonomous systems tailored to their needs.2. DeepSeek-R1: The Open-Source AI That Thinks Like OpenAIs Best By Yash ThubeThis article introduces DeepSeek-R1, an open-source language model designed to rival OpenAIs advanced models in reasoning tasks at a fraction of the cost. Using innovative techniques like Group Relative Policy Optimization (GRPO) and reinforcement learning, the model self-learns reasoning strategies without extensive human feedback. DeepSeek-R1 excels in benchmarks like AIME 2024 and coding tasks, offering clear, structured outputs. By releasing its weights and distillation recipes, DeepSeek democratizes AI, enabling developers to create specialized, cost-effective models.3. Why Phi-4 14B Is So Much Better Than GPT-4o And o1 Here The Results By Gao Dalie ()This article compares Microsofts Phi-4 model with GPT-4o and O1, highlighting Phi-4s strengths in mathematical reasoning and efficiency. Phi-4 features 14 billion parameters and excels in tasks requiring logical thinking, such as solving equations and financial modeling. It achieves high benchmark scores, surpassing larger models like Googles Gemini Pro. It also demonstrates Phi-4s usability on standard hardware through quantization techniques, making it accessible for local deployment. While O1 is faster, Phi-4s open-source nature and performance make it a practical choice for developers with limited resources.4. Bayesian State-Space Neural Networks (BSSNN): A Novel Framework for Interpretable and Probabilistic Neural Models By Shenggang LiThis article introduces the Bayesian State-Space Neural Network (BSSNN), a framework combining Bayesian principles, state-space modeling, and neural networks to enhance interpretability and probabilistic forecasting. BSSNN explicitly models joint and marginal probabilities, enabling it to predict outcomes (Y|X) and reverse inferences (X|Y). The article details its architecture, training process, and performance evaluation, comparing it to logistic regression. While BSSNN demonstrates improved accuracy and flexibility, challenges like computational demands and potential underfitting are noted.If you are interested in publishing with Towards AI, check our guidelines and sign up. We will publish your work to our network if it meets our editorial policies and standards.Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming asponsor. Published via Towards AITowards AI - Medium Share this post

There has never been a better time to read comics than right now.This would probably pass as a controversial opinion if you were to belt it out at your local comic shop, with a guy buying a stack of Amazing Spider-Man books immediately telling you how terrible Amazing Spider-Man is and so on, but speaking as someone who waited a full decade to read the first four issues of Legion Lost (but not the last eight), I can tell you that every single issue of every comic ever printed is more accessible now than it was a decade ago. And this is true in print as well as digital.But that makes for some confusing purchases. You could get the new Spencer Ackerman/Julius Ohta Iron Man in singles, or wait for the trade? Why get the Starman compendiums when an annual subscription to the DC Universe Infinite on-demand comics app that gives you access to most of DCs massive archive of books costs about the same? Believe it or not, Ive thought a lot about this, and I have some advice.First, if you like it, there is no wrong way to read it. Comics should be read to your hearts content. Dont worry too much about how to read it, as long as you can read it.Second, if it looks interesting but youre not quite ready to commit, ask your library to get it. Most libraries have digital options, and many are part of consortiums that have bigger purchasing power than you might think. And Ive never met a group of people more excited to get requests than librarians. Now onto format specific recommendations.You should check out a streaming service if youre planning to race through a huge backlog of books. Im doing a Claremont-era X-Book run now, and its so much easier being able to bring 35 issues on a plane on an ipad than it is to bring a trade or an omnibus. I have LOTS of recommendations for what to stream at Marvel Unlimited and DC Infinite if thats whats holding you back.You should pre-order single issues if you like a creative team and want to keep seeing their take on a character, or if its creator-owned or from a small press. There will always be a comic called X-Men or Green Lantern on shelves, but your favorite artists run might get cut short if the book doesnt sell well upfrontsee Omega Men in 2016.You should chase down back issues if you like the thrill of the hunt, you can spot a key issue at 60 paces, or you cant find a copy anywhere else. It took 10 years before I fully read Abnett & Lannings Legion Lost, and it took another few years before I had gathered most of their full run in singles, so that when DC finally started collecting their work in trade, inertia made me complete the set. But that was actually a lot of fun! And I find a certain collectors wanderlust pushing me to start thinking about other runs, too. If you get that feeling, hit those back issue bins at your local comic shop!You should get trade paperback collections if you like a story, want a more compact way to collect physical copies, and want it now. Trades look really nice on bookshelves, and they provide immediate reading material that is (usually) certain to be printed through to the end, which you cant always say for larger formats.You should buy compendiums if you dont trust streaming services entirely and you want to own something youre going to read to death. These booksDC calls them compendiums, Marvel has Epic Collections, Dark Horse does library editions, IDW calls them omnisusually have slightly worse paper quality and less fancy bindings in exchange for a TON of comics. This isnt universally the casethe Black Hammer Library Editions are as good or better than Marvels (slightly more upscale than this) omnibuses, and theyre way cheaper. These are, to me, the best bang for your buck.Fancy editions are a wonderful way to collect the stories you love the most. Marvel omnis, DCs Absolute Editions, IDW Artists Editions, and the like are all top of the line collections. They do different thingsMarvel omnis are a mostly straightforward collection of comics, printed with a fancy book jacket and usually an amazing hardcover underneath. Absolutes come in slipcases with a ton of secondary material. Artists Editions are laser focused on showing off a particular artists process and work. The unifying factor here is that they are all made with incredible care by people who adore the work and the medium. Theyre pricier, but if you have a story that you lovefor me, its Walt Simonsons Thor, Kirbys Fourth World, and Darwyn Cookes Parker storiesthese are wonderful to read and a joy to own and share.Join our mailing listGet the best of Den of Geek delivered right to your inbox!

The second season of Netflixs The Recruit is full of action, espionage, and intrigue as Owen Hendricks (Noah Centineo) tries to get off the CIAs shit list by looking into a graymail case in South Korea. No matter how much he tries to avoid the mistakes of his last mission, he cant help but fall face first into another deadly thread of conspiracies and secrets that could threaten his job, and his life, if he doesnt succeed.Heres how The Recruit season 2 ending goes down. How Many Episodes Are in The Recruit Season 2?This season of The Recruit is shorter than the last, with only six episodes compared to eight in season 1. Which is unfortunate, because this season could have really used a little breathing room to wrap things up.While no definitive reason has been given for why this season of The Recruit is shorter than the last, it seems to be part of a larger trend of shorter sophomore seasons across streaming services and cable networks. Some like Squid Game and The Diplomat have been shortened at the behest of the writers behind them, but others are likely do to a combination of factors like shorter budgets and production timelines affected by the WGA and SAG work stoppages of 2023.Is Owens Mission Successful?This section contains spoilers for the season 2 finale of The Recruit.Owen, Jang Kyun (Teo Yoo), Lester (Colton Dunn), Dawn (Angel Parker), and her crew of mercenaries make it to Russian waters aboard the fishing boat Owen bought/rented from his childhood friend,Yoo Jin (Shin Do-hyun). Once they pass inspection with a hefty bribe, all thats left to do is find a way to get Nan-hee (Lee Sang-hee)back from the Yakuza.After doing some recon, Nichka (Maddie Hasson), reveals that the Yakuza are basically holding Nan-hee in a fortress. Infiltrating that seems virtually impossible with their small crew and lack of resources, so Owen proposes luring the Yakuza out onto the water for an ambush by once again posing as a K&R agent looking to get Nan-hee released.The operation seems like it has every potential to succeed. Owen and Jang Kyun get the Yakuza right where they want them. Dawn and her mercenaries are poised to infiltrate. But then a Russian boat starts patrolling the area a bit too close, leading Dawn to call off the attack and leaving Owen and Jang Kyun to fend for themselves.Jang Kyun is shot and captured and Owen barely escapes the boat alive. He manages to swim to shore, taking refuge in a nearby houseboat. Owen is presumed dead by Dawn, Lester, and the CIA, who task Amelia (Kaylah Zander) with repapering Owen and pinning everything they can on him.Meanwhile, the houseboats residents, Russian officers, come home and Owen is forced to take them out. Once theyre at least temporarily taken care of, Owen uses one of their phones to call Lester, who tries to talk him out of trying to rescue Jang Kyun and Nan-hee on his own. When that fails, he sends Nichka to help.Does Nichka Betray Owen and the CIA?But Nichka has no interest in actually helping. Surprising no one, the Russian agent who killed her own mother without hesitation sold Owen and the others out to the FSB and paid the Russian Coast Guard to patrol the water. Shes the reason that the original plan failed. Owen realizes this when she comes to help him infiltrate the airfield that Jang Kyun and Nan-hee are being held at. He fights her, knocks her out, and decides to proceed alone.Join our mailing listGet the best of Den of Geek delivered right to your inbox!Somehow against all odds, Owen manages to sneak in and get Jang Kyun and Nan-hee out of Russian custody. They make a break for the water where Yoo Jin, Lester and the fishing boat are waiting for them. They sail away as fast as they can, with Nichka and the Russians in hot pursuit until, miraculously, a U.S. submarine comes out of the water and chases them off.Whats Next for Owen and the Others in Season 3?Owen and Jang Kyun have a brief talk about Jang Kyun and Nan-hees future Owen highly encourages him to let the CIA fake their deaths and give them new identities in America in exchange for intel. Jang Kyun agrees, not really having much of a choice, and they shake on it as the submarine sails off into the sunset.While Owen is technically successful in his mission, we dont really get to see any of the aftermath. Is the CIA still going to try and take him to task for all of his mistakes? Is he going to officially be more than just a lawyer now that hes recruited Jang Kyun? Was this submarine really just in the right place at the right time?This ending has left us with so many questions, and we really wish that there had been another episode or two to give this intense and action-packed finale a little more room to explore the aftermath. Owen has been ready to quit his job since season 1, but with his own flipped agent to take care of now, the path toward becoming a true field agent is looking pretty open, should he choose to go down it. Fingers crossed another season is on the way to answer these questions, and more.

Typically, if Im holding my breath while playing a game, its because Im focused on the rhythm of say, an intense combat sequence or platforming section. The tension stems from the precise, real-time input needed to triumph in those situations, a perfectly-aligned leap or parry. Citizen Sleeper 2: Starward Vector, an RPG in which the main mechanic is placing dice into a slot at your leisure, does not require the lightning fast touch of a button, and that in itself is a testament to how exhilarating the game can be.Citizen Sleeper 2 is the sequel to 2022s Citizen Sleeper, and a compelling evolution to the first instalment that introduces new characters, stories and mechanics. You dont necessarily need to have played the first game, but it does help in understanding the nuance and complexities of the unforgiving, dystopian world that Citizen Sleeper is set in. While the first game often felt like an isolated journey where helping yourself was often a zero-sum game, Citizen Sleeper 2 introduces a crew system, alongside some other clever new features, allowing your Sleeper to become the quirky, unappointed protagonist of your very own episodic sci-fi adventure.Mechanically, the sequel feels fresh but familiar. Like the first game, Citizen Sleeper 2 is driven by Cycles, and at the start of each, youll roll a set of six-sided dice that will determine your output during any given Cycle. These dice will then slot into numerous activities, and the higher the roll, the higher the chance of success. Where Citizen Sleeper 2 builds on this is in a brand-new Glitch system, a quirk that can debuff one or multiple of your dice rolls substantially, and the Push system, which allows you to decrease the risk of using a specific dice to accrue Stress, another new feature.Stress is an exchange, letting you exchange short term success for further complications later on like breaking a die, rendering it useless for a Cycle. This system forces you to gamble, to analyse whether success in the moment is worth whatever damage youll do to yourself in the long run. This is where those tense, exhilarating moments come into play, Citizen Sleeper 2 is at its best when a critical success or punishing failure hinges on an unpredictable dice roll.Its a much more engaging system, turning the act of simply rolling dice into a wider risk/reward system that enables you to essentially push through a difficult situation and pay for it later. Citizen Sleeper 2 is effectively a game about being alive in a body that doesnt quite work as youd like it to, and its mechanics are masterfully built to force you to wrestle with that at all times, yet it never feels a burden to play.Another new addition that impacts how you play is the class system, which introduces three distinct Sleeper archetypes with their own strengths and weaknesses. Each class starts with strength in one skill, with the ability to unlock points and build out other skills as you play. However, each of these has one skill completely blocked off, youll not be able to build on it and will always be at a disadvantage on activities that require it. For example, I went with the Machinist role, a pragmatic archetype that excels with the Engineer skill and falls short with the Engage skill. In short, great at fixing things, not so great at socialising. Its a balanced system, but one that drives a natural instinct to play a role, which will also impact how you build your party, another excellent addition to Citizen Sleeper 2.As you progress, youll meet new characters that can be recruited as part of your crew, and each of these new additions comes with their own skills. This changes the dynamic of Citizen Sleeper 2 dramatically youre no longer only dependent on your own dice rolls, your crew members can contribute additional die in certain moments.However, these characters are more than just tactical additions to your tabletop arsenal. Citizen Sleeper 2s colorful cast is compelling to follow, because theyre not merely compelled to follow you. Theyre individual personalities, each with their own drives and dreams. Sometimes, crew members will be indisposed on errands of their own making, and sometimes their actions will put you in interesting positions, and sometimes those situations may conflict with how you have chosen to roleplay your Sleeper. These characters are not written to be necessarily good or bad personalities, but everyone you meet has been molded by the dystopian landscape they all share in some way. Theres moral ambiguity in characters that have been shaped by loneliness, conflict and desperation, and theres a muted cordiality in everyone just trying to survive.One thing to really heed is that there is no right or wrong way to play Citizen Sleeper 2. Progression is not confined by the binaries of success and failure here, there is no game over if you find yourself stumbling through a run of bad luck and poor dice rolls. The decisions that you or your crew decide or deign are the ones that truly matter, and your priorities may change as you uncover new threads to pull on.The main story of Citizen Sleeper 2 is propelled forward by Major Drives, main missions thatll advance the plot, but youll also pick up Minor Drives, side quests that are optional but will uncover integral additions such as new crew members and valuable items, some of which will pertain to the personal goals of your crew members. However, the new Contract system is where Citizen Sleeper 2 really shines in its homage to the episodic television format the game draws influence from. Contracts are little missions that your crew can embark on with their own engaging narratives and rewards, and theyre also where camaraderie with your crew is key to survival. Opportunities are often shrouded behind inconspicuous interactions that require good use of every skill, and not just your own.Contracts are accessed via a new map layout, which allows your crew to travel freely between several locations. Citizen Sleeper 2 is far less linear, some Minor Drives will take you to new locations, or you can just choose to spend a few Cycles exploring a place with no specific goal. While its predecessor kept you isolated and tied to one place, Citizen Sleeper 2 extends an open invitation to explore a sprawling galaxy, as long as you have the supplies to get where youre going.There is, however, a constant clock running in the background as you partake in all of this wild-eyed wandering. The next narrative beat will often be forced upon you after a certain number of Cycles, and the way to keep failure at bay is to always be moving around. Again, this isnt a pending failure, but a persistent reminder that there are consequences to your actions in this unrelenting universe.Citizen Sleeper 2 constantly dances on the line between a calm, controlled stroll and a high-octane sprint. While you may be engaging in some non-important churn, taking your time with its many engaging dialogue sequences, the everpresent thrum of danger is always creeping up to you, making sure you can never truly relax wherever you are. Its a masterful feat of thoughtful worldbuilding, where your success is often either clutched through chaos or entirely incalculable, which is also a salient question that Citizen Sleepers world poses about existence: What if everything just went wrong anyway, and that was okay?Citizen Sleeper 2: Starward Vector is out January 31 for Xbox Series X|S and Xbox One, and is available day one with Xbox Game Pass.