Lisa Morgan, Freelance WriterMarch 14, 20258 Min ReadDragos Condrea via Alamy StockMany organizations have learned that AI models need to be monitored, fine-tuned, and eventually retired. This is as true of large language models (LLM) as it is of other AI models, but the pace of generative AI innovation has been so fast, some organizations are not managing their models as they should be, yet.Senthil Padmanabhan, VP, platform and infrastructure at global commerce company eBay, says enterprises are wise to establish a centralized gateway and a unified portal for all model management tasks as his company has done. EBay essentially created an internal version of Hugging Face that eBay has implemented as a centralized system.Our AI platform serves as a common gateway for all AI-related API calls, encompassing inference, fine-tuning, and post-training tasks. It supports a blend of closed models (acting as a proxy), open models (hosted in-house), and foundational models built entirely from the ground up, says Padmanabhan in an email interview. Enterprises should keep in mind four essential functionalities when approaching model management: Dataset preparation, model training, model deployment and inferencing, and continuous evaluation pipeline. By consolidating these functionalities, weve achieved consistency and efficiency in our model management processes.Related:Previously, the lack of a unified system led to fragmented efforts and operational chaos.Rather than building the platform first during its initial exploration of GenAI, the company focused on identifying impactful use cases.As the technology matured and generative AI applications expanded across various domains, the need for a centralized system became apparent, says Padmanabhan. Today, the AI platform is instrumental in managing the complexity of AI model development and deployment at scale.Senthil Padmanabhan, eBaySenthil Padmanabhan, eBayPhoenix Childrens Hospital has been managing machine learning models for some time because predictive can models drift.Weve had a model that predicts malnutrition in patients [and] a no-show model predicting when people are not going to show up [for appointments], says David Higginson, executive vice president and chief innovation officer at Phoenix Children's Hospital. Especially the no-show model changes over time so you have to be very, very conscious about, is this model still any good? Is it still predicting correctly? Weve had to build a little bit of a governance process around that over the years before large language models, but I will tell you, like with large language models, it is a learning [experience], because different models are used for different use cases.Related:Meanwhile, LLM providers, including OpenAI and Google, are rapidly adding new models turning off old ones, which means that something Phoenix Childrens Hospital built a year ago might suddenly disappear from Azure.Its not only that the technical part of it is just keeping up with whats being added and what's being removed. Theres also the bigger question of the large language models. If youre using it for ambient listening and youve been through a vetting process, and everybodys been using a certain model, and then tomorrow, theres a better model, people will want to use it, says Higginson. Were finding there are a lot of questions, [such as], is this actually a better model for my use case? What's the expense of this model? Have we tested it?How to Approach Model ManagementEBays Padmanabhan says any approach to model management will intrinsically establish a lifecycle, as with any other complex system. EBay already follows a structured lifecycle, encompassing stages from dataset preparation to evaluation.To complete the cycle, we also include model depreciation, where newer models replace existing ones, and older models are systematically phased out, says Padmanabhan. This process follows semantic versioning to maintain clarity and consistency during transitions. Without such a lifecycle approach, managing models effectively becomes increasingly challenging as systems grow in complexity.Related:EBays approach is iterative, shaped by constant feedback from developers, product use cases and the rapidly evolving AI landscape. This iterative process allowed eBay to make steady progress.With each iteration of the AI platform, we locked in a step of value, which gave us momentum for the next step. By repeating this process relentlessly, weve been able to adapt to surprise -- whether they were new constraints or emerging opportunities -- while continuing to make progress, says eBays Padmanabhan. While this approach may not be the most efficient or optimized path to building an AI platform, it has proven highly effective for us. We accepted that some effort might be wasted, but well do it in a safe way that continuously unlocks more value.To start, he recommends setting up a common gateway for all model API calls.This gateway helps you keep track of all the different use cases for AI models and gives you insights into traffic patterns, which are super useful for operations and SRE teams to ensure everything runs smoothly, says Padmanabhan. Its also a big win for your InfoSec and compliance teams. With a centralized gateway, you can apply policies in one place and easily block any bad patterns, making security and compliance much simpler. After that, one can use the traffic data from the gateway to build a unified portal. This portal will let you manage a models entire lifecycle, from deployment to phasing it out, making the whole process more organized and efficient as you scale.Phoenix Childrens Hospitals Higginson says its wise to keep an eye on the industry because its changing so fast.David Higginson, Phoenix Children's HospitalDavid Higginson, Phoenix Children's HospitalWhen a new model comes out, we try to think about it in terms of solving a problem, but we've stopped chasing the [latest] model as GPT-4 does most of what we need. I think what weve learned over time is dont chase the new model because were not quite sure what it is or youre limited on how much you can use it in a day, says Higginson. Now, were focusing more on models that have been deprecated or removed, because we get no notice of that.Its also important for stakeholders to have a baseline knowledge of AI so there are fewer obstacles to progress. Phoenix Childrens Hospital began its governance processes with AI 101 training for stakeholders, including information about how the models work. This training was done during the groups first three meetings.Otherwise, you can leave people behind, says Higginson. People have important things to say, [but] they just don't know how to say them in an AI world. So, I think thats the best way to get started. You also tend to find out that some people have an aptitude or an interest, and you can keep them on the team, and people who dont want to be part of it can exit.Jacob Anderson, owner of Beyond Ordinary Software Solutions, says a model is no different than a software product thats released to the masses.If you have lifecycle management on your product rollouts, then you should also implement the same in your model stewardship, says Anderson. You will need to have a defined retirement plan for models and have a policy in place to destroy the models. These models are just amalgamations of the data that went into training them. You need to treat models with the same care as you would the training data.Sage AdviceEBays Padmanabhan recommends that organizations still in the early stages of exploring GenAI refrain from building a complex platform to start, which is exactly what eBay did.At eBay, we initially focused on identifying impactful use cases rather than investing in a platform. Once the technology matured and applications expanded across different domains, we saw the need for a centralized system, says Padmanabhan. Today, our AI platform helps us manage the complexity of AI development and deployment at scale -- but we built it when the timing was right.He also thinks it wise not to become overwhelmed by the rapid changes in this field.Its easy to get caught up in trying to create a system that supports every type of model out there. Instead, take a step back and focus on what will truly make a difference for your organization. Tailor your model management system to meet your specific needs, not just what the industry is buzzing about, says Padmanabhan. Lastly, from our experience we see that quality of the dataset is what really matters. Quality trumps quantity. It is better to have 10,000 highly curated high-quality rows than 100,000 average rows.Phoenix Childrens Hospitals Higginson recommends experimenting with guardrails so people can learn.Have a warning that says, Don't put PII in there and use the output carefully, but absolutely use it, says Higginson. Don't believe everything it says, but other than that, don't be scared. The use cases coming from our staff, employees and physicians are way more creative than I would have ever thought of, or any committee would have thought of.Beyond Ordinarys Anderson recommends understanding the legal obligations of jurisdictions in which the models are operating because they vary.Take care to understand those differences and how your obligations bleed into those regulatory theatres. Then you need to have a well-defined operational plan for model stewardship, says Anderson. This is very much akin to your data stewardship plan, so if you don't have one of those, then it's time to slow the bus and fix that flat tire.He also recommends against putting hobbyist AI practitioners in charge of models.Find qualified professionals to help you with the policy frameworks and setting up a stewardship plan, says Anderson. Cybersecurity credentials play into the stewardship of AI models because the models are just data. Your cyber people don't need to know how to train or evaluate an AI model. They just need to know what data went into training and how the model is going to be used in a real-world scenario.About the AuthorLisa MorganFreelance WriterLisa Morgan is a freelance writer who covers business and IT strategy and emergingtechnology for InformationWeek. She has contributed articles, reports, and other types of content to many technology, business, and mainstream publications and sites including tech pubs, The Washington Post and The Economist Intelligence Unit. Frequent areas of coverage include AI, analytics, cloud, cybersecurity, mobility, software development, and emerging cultural issues affecting the C-suite.See more from Lisa MorganWebinarsMore WebinarsReportsMore ReportsNever Miss a Beat: Get a snapshot of the issues affecting the IT industry straight to your inbox.SIGN-UPYou May Also Like