0 Комментарии
0 Поделились
164 Просмотры
Каталог
Каталог
-
Войдите, чтобы отмечать, делиться и комментировать!
-
WWW.MARKTECHPOST.COMYuLan-Mini: A 2.42B Parameter Open Data-efficient Language Model with Long-Context Capabilities and Advanced Training TechniquesLarge language models (LLMs) built using transformer architectures heavily depend on pre-training with large-scale data to predict sequential tokens. This complex and resource-intensive process requires enormous computational infrastructure and well-constructed data pipelines. The growing demand for efficient and accessible LLMs has led researchers to explore techniques that balance resource use and performance, emphasizing achieving competitive results without relying on industry-scale resources.Developing LLMs is filled with challenges, especially regarding computation and data efficiency. Pre-training models with billions of parameters demand advanced techniques and substantial infrastructure. High-quality data and robust training methods are crucial, as models face gradient instability and performance degradation during training. Open-source LLMs often struggle to match proprietary counterparts because of limited access to computational power and high-caliber datasets. Therefore, the challenge lies in creating efficient and high-performing models, enabling smaller research groups to participate actively in advancing AI technology. Solving this problem necessitates innovation in data handling, training stabilization, and architectural design.Existing research in LLM training emphasizes structured data pipelines, using techniques like data cleaning, dynamic scheduling, and curriculum learning to improve learning outcomes. However, stability remains a persistent issue. Large-scale training is susceptible to gradient explosions, loss spikes, and other technical difficulties, requiring careful optimization. Training long-context models introduce additional complexity as attention mechanisms computational demands grow quadratically with sequence length. Existing approaches like advanced optimizers, initialization strategies, and synthetic data generation help alleviate these issues but often fall short when scaled to full-sized models. The need for scalable, stable, and efficient methods in LLM training is more urgent than ever.Researchers at the Gaoling School of Artificial Intelligence, Renmin University of China, developed YuLan-Mini. With 2.42 billion parameters, this language model improves computational efficiency and performance with data-efficient methods. By leveraging publicly available data and focusing on data-efficient training techniques, YuLan-Mini achieves remarkable performance comparable to larger industry models.YuLan-Minis architecture incorporates several innovative elements to enhance training efficiency. Its decoder-only transformer design employs embedding tying to reduce parameter size and improve training stability. The model uses Rotary Positional Embedding (ROPE) to handle long contexts effectively, extending its context length to 28,672 tokens, an advancement over typical models. Other key features include SwiGLU activation functions for better data representation and a carefully designed annealing strategy that stabilizes training while maximizing learning efficiency. Synthetic data was critical, supplementing the 1.08 trillion tokens of training data sourced from open web pages, code repositories, and mathematical datasets. These features enable YuLan-Mini to deliver robust performance with a limited computing budget.YuLan-Minis performance achieved scores of 64.00 on HumanEval in zero-shot scenarios, 37.80 on MATH-500 in four-shot settings, and 49.10 on MMLU in five-shot tasks. These results underscore its competitive edge, as the models performance is comparable to much larger and resource-intensive counterparts. The innovative context length extension to 28K tokens allowed YuLan-Mini to excel in long-text scenarios while still maintaining high accuracy in short-text tasks. This dual capability sets it apart from many existing models, which often sacrifice one for the other.Key takeaways from the research include:Using a meticulously designed data pipeline, YuLan-Mini reduces reliance on massive datasets while ensuring high-quality learning.Techniques like systematic optimization and annealing prevent common issues like loss spikes and gradient explosions.Extending the context length to 28,672 tokens enhances the models applicability to complex, long-text tasks.Despite its modest computational requirements, YuLan-Mini achieves results comparable to those of much larger models, demonstrating the effectiveness of its design.The integration of synthetic data improves training outcomes and reduces the need for proprietary datasets.In conclusion, YuLan-Mini is a great new addition to evolving efficient LLMs. Its ability to deliver high performance with limited resources addresses critical barriers to AI accessibility. The research teams focus on innovative techniques, from data efficiency to training stability, highlights the potential for smaller-scale research to contribute to the field significantly. With just 1.08T tokens, YuLan-Mini sets a benchmark for resource-efficient LLMs.Check out the Paper and GitHub Page. All credit for this research goes to the researchers of this project. Also,dont forget to follow us onTwitter and join ourTelegram Channel andLinkedIn Group. Dont Forget to join our60k+ ML SubReddit. Asif RazzaqAsif Razzaq is the CEO of Marktechpost Media Inc.. As a visionary entrepreneur and engineer, Asif is committed to harnessing the potential of Artificial Intelligence for social good. His most recent endeavor is the launch of an Artificial Intelligence Media Platform, Marktechpost, which stands out for its in-depth coverage of machine learning and deep learning news that is both technically sound and easily understandable by a wide audience. The platform boasts of over 2 million monthly views, illustrating its popularity among audiences. [Download] Evaluation of Large Language Model Vulnerabilities Report (Promoted)0 Комментарии 0 Поделились 161 Просмотры
-
WWW.IGN.COMThe Beyonc Bowl, Squid Game, and Everything Else You Should Watch This WeekendWelcome to Streaming Rewind, a weekly breakdown of the new and noteworthy as we work to help readers wade through the absolute deluge of television series and movies in the streaming space.Welcome to Christmas and New Years limbo, where the time is made up and the date doesnt matter. Theres a limited number of releases this week, because Hollywood typically just shuts down for the last month of the year, but that doesnt mean there arent a few surprises. The Beyonc Bowl (Netflix) If youre not a football fan, you may not have known that Beyonc did a halftime special for the Ravens vs. Texans game on Christmas (thats right, Netflix does football now too). In said special, she performed some of the songs from her Cowboy Carter album live for the very first time. If youre a member of the Beyhive who wasnt willing to sit through a football game to watch Queen Bey perform, Netflix released a stand-alone special of the performance today. And, for those wondering, Netflix did, indeed, manage to host two whole live events on Christmas day without its typical buffering issues.Squid Game is Back for Round 2 (Netflix) Its been a Netflix week! The highly anticipated second season of Squid Game finally dropped, ironically resulting in many across the industry working during Christmas and continuing the trend of Netflix completely missing the point of the series. Reviews have been mixed due to it being painfully obvious that Seasons 2 and 3 were meant to be one complete story rather than split in half, but our critic Shannon Miller liked the season. If youve already finished your binge, check out how Squid Games Season 2 ending sets up Season 3. PlayThe Order (On Demand)Nicholas Hoult has yet another film out on demand, this time alongside Jude Law and Jurnee Smollett. The films been met with early acclaim, and is based on a true story (and adapted from Kevin Flynns novel The Silent Brotherhood) centered on a string of bank robberies in the Pacific Northwest. You may have seen it during its brief theatrical run when it released on December 6 but, if you missed it, its available for purchase now.New and Notable: Gladiator II December 24 (On Demand)Y2K December 24 (On Demand)Doctor Who Christmas Special December 25 (Disney+)Thats, well thats pretty much it for this week. You can check out the few titles that are available on PVOD up above in the New and Notable section and, if you havent caught up just yet, there are a few options from last week that are still noteworthy. The Simpsons Christmas Special, O Cmon All Ye Faithful springs to mind! You can check out our review of the special if youre not sure if you want to spend your precious lounge time on it. Dont forget that What If? Is dropping new episodes weekly, and that Star Wars: Skeleton Crew and Creature Commandos are in the middle of thier runs as well!0 Комментарии 0 Поделились 159 Просмотры
-
WEWORKREMOTELY.COMSyncWith: Senior Full Stack EngineerAbout SyncWithSyncWith is a small, passionate, engineering-led company on a mission to simplify data access for marketers, product managers and business owners everywhere. We help teams connect their data with tools they already know, like Google Sheets and Looker Studio, giving them the power to see all their key metrics in one place. Since our start in 2020, over 1,500 marketing teams have chosen SyncWith to keep their data accessible and actionable. We're growing fast, profitable and looking for talented people who love building software that makes an impact.The RoleWere looking for a Senior Full Stack Engineer who thrives on crafting intuitive web experiences and wants to take on a lead role in building out our user-facing products. Youll be hands-on, working across our stack (Typescript, Node, Remix, React, Tailwind) to bring features from idea to launch. If youre someone who enjoys the challenge of creating great software without red tape, values clear communication and wants to work directly with a small, tight-knit team, this might be the perfect fit.What Youll DoLead Development: Take charge of our web applications, driving new features and improvements that make a real difference to our users.Full Stack Ownership: Youll work across front-end and back-end, taking responsibility for delivering features that meet user needs from start to finish.Impactful Engineering: Ship code thats maintainable, well-tested, and loved by users, adapting based on feedback from analytics and real-world use.Develop Features to Grow User Base: Engage users by developing and iterating on new features, running experiments to drive success.Data Processing Optimization: Improve and scale our data processing infrastructure to enhance speed, cost-efficiency, and robustness.Collaborative Culture: Youll work closely with the founders and the team, contributing to the companys growth with your ideas and skills.About YouExperienced and Efficient: Youre a senior engineer whos shipped robust, maintainable software in fast-paced environments.UI/UX Enthusiast: You appreciate good design and know how to build intuitive, user-friendly interfaces that look great and perform well.Problem Solver: Youre a skilled debugger with a knack for diving deep to identify and resolve root causes of issues.Clear Communicator: You can articulate complex ideas clearly, debate solutions constructively, and collaborate effectively with teammates.Self-Starter: You take ownership of your work, enjoy working autonomously and get excited about seeing your code in the hands of users.Passionate Builder: You care deeply about building software that makes an impact.Relevant Bachelors Degree: You hold a degree in computer science, computer engineering or related field.Our Tech StackFrontend: Remix, React, Typescript, Tailwind, FigmaBackend: Node/Express, TypescriptData: Postgres for primary storage, SQLite for aggregationHosting: Render.com, AWS, and GCPTools: Amplitude Analytics, Sentry, Linear, Slack, GitHubAPIs: Integrations with platforms like Facebook Ads, Google Analytics, and ShopifyWhy SyncWith?Competitive Compensation: $135,000 - $155,000 USD per year, plus options, a health spending account and 4 weeks of vacation.Flexible Work Environment: Work fully remotely within PST/EST time zones, enjoying flexible hours, minimal meetings and plenty of autonomy to focus on meaningful work.Impact-Driven Culture: Be part of a small, dynamic team where your work directly impacts our users and the company's success.Growth Opportunities: Collaborate closely with founders, contribute to the company's direction and shape a product that helps teams harness the power of their data.Autonomy and Efficiency: We value end-to-end ownership, efficiency and meaningful contributions without the red tape.What's Next?Excited to build impactful solutions with a passionate team? Wed love to hear from you! Apply using the link below.Our Streamlined Interview ProcessHeres what you can expect as we get to know you better:Application Submission: Start by completing the application form to share an overview of your experience and skills.Video Submission: Record a brief 1-2 minute video introducing yourself, your background and why youre excited about joining SyncWith.Discovery Call: Join us for a quick 15-minute chat to explore your fit for the role and learn more about your goals.Interview: Engage in a deeper conversation with our founder to discuss your experience and technical skills.Were excited to meet you and explore how you can make an impact at SyncWith!0 Комментарии 0 Поделились 162 Просмотры
-
WWW.CNET.COMBest Internet Providers in Sarasota, FloridaResidents of Sarasota have access to a decent selection of fast and budget-friendly internet plans. We've done the work to find the best internet service providers for Sarasota households.0 Комментарии 0 Поделились 153 Просмотры
-
WWW.CNET.COMBest Internet Providers in St. Petersburg, FloridaSt. Petersburg has a few good internet options, including a light presence of fiber internet. Here are CNETs top picks for the best broadband in the area.0 Комментарии 0 Поделились 150 Просмотры
-
WWW.CNET.COMBest Internet Providers in St. George, UtahIf you're a St. George resident, TDS Telecom is the default internet service pick, but it's got competition. Here are the best providers in the city, according to CNET's experts.0 Комментарии 0 Поделились 154 Просмотры
-
WWW.THEVERGE.COMTrump asks the Supreme Court to let him rescue TikTokPresident-elect Donald Trump has asked the Supreme Court to let him negotiate a deal to save TikTok from an imminent US ban.In an amicus brief filed to the court, Trump says he seeks the ability to resolve the issues at hand through political means once he takes office, and that he alone possesses the consummate dealmaking expertise, the electoral mandate, and the political will to negotiate a resolution to save the platform.Last week, the Supreme Court agreed to hear arguments that a bill passed by Congress banning TikTok on national security grounds violates the First Amendment. While Trump pushed a TikTok ban during his first term, he has changed his tune after his campaign successfully used the platform during the 2024 election. He recently met with TikTok CEO Shou Chew at Mar-a-Lago and told a crowd that maybe we gotta keep this sucker around for a little while.The bill that would see TikTok banned in January gives wide latitude to the president to delay its enforcement if theres progress being made towards a deal ensuring TikTok isnt fully controlled by its Chinese parent company, ByteDance. But the deadline for that determination is January 19th, which is one day before Trump is set to assume the presidency. In his Supreme Court filing, Trump asks for the January 19th deadline to be stayed, arguing that the deal hed negotiate would obviate the need for this Court to decide the historically challenging First Amendment question presented here on the current, highly expedited basis.Developing...0 Комментарии 0 Поделились 167 Просмотры
-
ARSTECHNICA.COMYouTuber won DMCA fight with fake Nintendo lawyer by detecting spoofed email | Gamer urges YouTube to change DMCA takedown process to end copyright abuse.Losing game? YouTuber won DMCA fight with fake Nintendo lawyer by detecting spoofed email Gamer urges YouTube to change DMCA takedown process to end copyright abuse. Ashley Belanger Dec 27, 2024 2:16 pm | 1 Credit: PHILIP FONG / Contributor | AFP Credit: PHILIP FONG / Contributor | AFP Story textSizeSmallStandardLargeWidth *StandardWideLinksStandardOrange* Subscribers only Learn moreA brave YouTuber has managed to defeat a fake Nintendo lawyer improperly targeting his channel with copyright takedowns that could have seen his entire channel removed if YouTube issued one more strike.Sharing his story with The Verge, Dominik "Domtendo" Neumayera German YouTuber who has broadcasted play-throughs of popular games for 17 yearssaid that it all started when YouTube removed some videos from his channel that were centered on The Legend of Zelda: Echoes of Wisdom. Those removals came after a pair of complaints were filed under the Digital Millennium Copyright Act (DMCA) and generated two strikes. Everyone on YouTube knows that three strikes mean you're out and off the platform permanently.Suddenly at risk of losing the entire channel he had built on YouTube, Neumayer was stunned, The Verge noted, partly because most game companies consider "Let's Play" videos like his to be free marketing, not a threat to their business. And while Nintendo has been known to target YouTubers with DMCA takedowns, it generally historically took no issues with accounts like his.For many YouTubers, a DMCA takedown request is considered too risky to challenge, even if it's obviously fake. The risk of losing their channels outweighs the risk of losing income from removing specific videos at issue, so users often choose to delete content voluntarily, rather than defend their content. Copyright trolls try to benefit from this, getting content removed that otherwise would remain on the platform and sometimes attempting to push users to submit unnecessary payments.No one knows how much copyright abuse occurs on YouTube. According to YouTube, about 6 percent of removals from July to December 2023 were abusive, along with 10 times more attempted abusive removals. But if a significant number of users never flag abuseout of fear they could be sued for contributing to copyright infringementthen the true figure could be higher.Neumayer clearly took a long hard look at the DMCA takedown requests before making any rash decisions about submitting to the claims. That's when he noticed something strange. The requests were signed by "Tatsumi Masaaki, Nintendo Legal Department, Nintendo of America," but the second one curiously "came from a personal account at an encrypted email service: 'tatsumi-masaaki@protonmail.com,'" The Verge reported.Defending his livelihood, Neumayer started asking questions. At first, that led to his videos being reinstated. But that victory was short-lived, as the supposed Nintendo lawyer only escalated his demands, spooking the YouTuber into voluntarily removing some videos, The Verge reported, while continuing to investigate the potential troll.Reaching out directly to Nintendo helped, but questions remainThe Verge has all the receipts, sharing emails from the fake lawyer and detailing Neumayer's fight blow-for-blow. Neumayer ultimately found that there was a patent lawyer with a similar name working for Nintendo in Japan, although he could not tell if that was the person sending the demands and Nintendo would not confirm to The Verge if Tatsumi Masaaki exists.Only after contacting Nintendo directly did Neumayer finally get some information he could work with to challenge the takedowns. Reportedly, Nintendo replied, telling Neumayer that the fake lawyer's proton email address "is not a legitimate Nintendo email address and the details contained within the communication do not align with Nintendo of America Inc.s enforcement practices."Nintendo promised to investigate further, as Neumayer continued to receive demands from the fake lawyer. It took about a week after Nintendo's response for "Tatsumi" to start to stand down, writing in a stunted email to Neumayer, "I hereby retract all of my preceding claims." But even then, the troll went down fighting, The Verge reported.The final messages from "Tatsumi" claimed that he'd only been suspended from filing claims and threatened that other Nintendo lawyers would be re-filing them. He then sent what The Verge described as "in some ways the most legit-looking email yet," using a publicly available web tool to spoof an official Nintendo email address while continuing to menace Neumayer.It was that spoofed email that finally ended the faade, though, The Verge reported. Neumayer detected the spoof by checking the headers and IDing the tool used.Although this case of copyright trolling is seemingly over, Neumayeralong with a couple other gamers trolled by "Tatsumi"remain frustrated with YouTube, The Verge reported. After his fight with the fake Nintendo lawyer, Neumayer wants the streaming platform to update its policies and make it easier for YouTubers to defend against copyright abuse.Back in May, when Ars reported on a YouTuber dismayed by a DMCA takedown over a washing machine chime heard on his video, a YouTube researcher and director of policy and advocacy for the Electronic Frontier Foundation, Katharine Trendacosta told Ars that YouTube's current process discourages YouTubers from disputing copyright strikes.Every idiot can strike every YouTuber and there is nearly no problem to do so. Its insane, Neumayer said. It has to change NOW.Ashley BelangerSenior Policy ReporterAshley BelangerSenior Policy Reporter Ashley is a senior policy reporter for Ars Technica, dedicated to tracking social impacts of emerging policies and new technologies. She is a Chicago-based journalist with 20 years of experience. 1 Comments0 Комментарии 0 Поделились 159 Просмотры
-
WWW.NINTENDOLIFE.COMGuilty Gear Strives Next DLC Character Will Be A Free Update On SwitchJoining Season 4.Back in September, Arc System Works shared a first in-game look at Guilty Gear Strives next DLC fighter Queen Dizzy.This fighter will be added to the Switch version of the game as part of a free content update. Arc System Works confirmed this on social media alongside the news this update is scheduled to arrive on February 2025.Read the full article on nintendolife.com0 Комментарии 0 Поделились 165 Просмотры