Generative AI (genAI) companies are adopting a larger assortment of open-source and smaller language models that excel at automating specific tasks. A number of genAI players, including HubSpot, Microsoft, and ServiceNow, are hot on the trail of open-source models that can be easily customized to create AI agents that better meet customer needs. “Companies are going to have hundreds of models…, they’re going to be domain-specific models that are getting built rapidly now,” said Craig LeClair, vice president and principal analyst at Forrester Research. Service providers are adopting open-source AI models for several reasons, including flexibility, customization, smaller footprints, and lower computing costs. Microsoft is building an ensemble of small-footprint AI tools that will include open-source AI models for Microsoft 365 that will work offline, said Aparna Chennapragada, Microsoft’s chief product officer of experiences and devices. These smaller models are aimed at offline users who want to use M365 on Windows AI PCs with Copilot and neural processors. “Our own internal teams are also looking at post-training these models for specific use cases, for example, for writing versus analysis versus image creation, etc.,” Chennapragada said. Open-source models that are smaller in size are an important part of the pipeline, Chennapragada said. Microsoft’s own open-source small language model, called Phi Silica, is available for Windows developers to write offline applications that take advantage of NPUs. HubSpot last week expanded its availability of agents in its Breeze AI platform, which already includes genAI tools to improve employee productivity. Then company’s offerings include a back-end of open-source models to automate tasks. Customers can add Mistral AI SAS’s open-source Mistral AI LLM to their AI agent pipeline for sales, marketing and support processes, and Stability Diffusion 3 Large for text-to-image generation. The open-source models are in addition to OpenAI’s proprietary GPT and Anthropic’s Claude large language models (LLMs). HubSpot’s AI model card lists the models available to customers. “Our AI strategy is not trying to develop any sort of deep proprietary models or do something that no one else is doing,” said Nicholas Holland, head of AI and senior vice president of product at HubSpot. “Our job is to apply the best AI to our customers’ problems.” The back-end models for agents depend on the task, speed, quality and accuracy, needed he said. Some agents require extensive reasoning, while  smaller models built on open-source principles might excel at text or image generation, he said. “We work with all the vendors; we have open-source models that we use. We look at the customer problem and are able to apply the best solution at the time,” Holland said. Customers don’t have to worry about models or tokens, as orchestrators do the job in the background. Humans are still in the loop — and remain an important part of the process, he said. “To use a reasoning model requires a certain degree of depth, thinking complexity, and cost. Once you have the reasoning done, if you want to generate text or imagery, that’s a different model that might be less expensive,” he said. HubSpot opted for the Stability Diffusion 3 model for image generation after trying out DALL-E 3 from OpenAI. The latest Stability Diffusion model 3.5 is considered open source under specific terms. ServiceNow collaborated with Nvidia to develop an open-source genAI model called Apriel to create learning AI agents that decide on processes such as IT support, human resources, and customer-service functions. With only 15 billion parameters, Apriel is lean and specializes in reasoning, said Dorit Zilbershot, ServiceNow’s group vice president of AI experiences and innovation. “If you look at the foundation models, they’re very big, very slow…. This is only a 15-billion-parameters model, and it’s highly trained on reasoning,” Zilbershot said. The Apriel model should provide faster inferencing while also saving on computational costs, Zilbershot said. A fully governed AI platform is going to have a list of approved models that fit within a company’s AI policies, Forrester’s LeClair said. “I think there’s a drift back to on-premise. Companies are going to want models that have their proprietary information training them. They’re going to want really good control because they’re worried about IT leakage. So open-source models are going to be run in controlled on-premise environments,” said LeClair, who has authored a book about automation and AI in the workplace. Source: https://www.computerworld.com/article/3984310/genai-companies-go-granular-with-open-source-models-for-agents.html #genai #companies #granular #with #opensource #models #for #agents