WWW.MICROSOFT.COM
Real-world healthcare AI development and deploymentat scale
[THEME MUSIC FADES]The passage I read at the top there is from Chapter 7 of the book, The Ultimate Paperwork Shredder.Paperwork plays a particularly important role in healthcare. It helps convey treatment information that supports patient care, and its also used to help demonstrate that providers are meeting regulatory responsibilities, among other things. But if were being honest, its taxingfor everyoneand its a big contributor to the burnout our clinicians are experiencing today. Carey, Zak, and I identified this specific pain point as one of the best early avenues to pursue as far as putting generative AI to good work in the healthcare space.In this episode, Im excited to welcome Dr. Matt Lungren and Seth Hain to talk about matching technological advancements in AI to clinical challenges, such as the paperwork crisis, to deliver solutions in the clinic and in the health system back office.Matt is the chief scientific officer for Microsoft Health and Life Sciences, where he focuses on translating cutting-edge technology, including generative AI and cloud services, into innovative healthcare applications. Hes a clinical interventional radiologist and a clinical machine learning researcher doing collaborative research and teaching as an adjunct professor at Stanford University. His scientific work has led to more than 200 publications, including work on new computer vision and natural language processing approaches for healthcare.Seth is senior vice president of research and development at Epic, a leading healthcare software company specializing in electronic health record systems, also known as EHR, as well as other solutions for connecting clinicians and patients. During his 19 years at Epic, Seth has worked on enhancing the core analytics and other technologies in Epics platforms as well as their applications across medicine,Ive had the pleasure of working closely with both Matt and Seth. Matt, as a colleague here at Microsoft, really focused on our health and life sciences business. And Seth, as a collaborator at Epic, as we embark on the questions of how to integrate and deploy generative AI into clinical applications at scale. [TRANSITION MUSIC]Heres my conversation with Dr. Matt Lungren:LEE: Matt, welcome. Its just great to have you here.MATTHEW LUNGREN: Thanks so much, Peter. Appreciate being here.LEE: So, Id like to just start just talking about you. You know, I had mentioned your role as the chief scientific officer for Microsoft Health and Life Sciences. Of course, thats just a title. So, what the heck is that? What is your job exactly? And, you know, what does a typical day at work look like for you?LUNGREN: So, really what you could boil my work down to is essentially cross collaboration, right. We have a very large company, lots of innovation happening all over the place, lots of partners that we work with and then obviously this sort of healthcare mission.And so, what innovations, what kind of advancements are happening that can actually solve clinical problems, right, and sort of kind of direct that. And we can go into some examples, you know, later. But then the other direction, too, is important, right. So, identifying problems that may benefit from a technologic application or solution and kind of translating that over into the, you know, pockets of innovation saying, Hey, if you kind of tweaked it this way, this is something that would really help, you know, the clinical world.And so, its really a bidirectional role. So, my day to day is every day is a little different, to be honest with you. Some days its very much in the science and learning about new techniques. On the other side, though, it can be very much in the clinic, right. So, what are the pain points that were seeing? Where are the gaps in the solutions that weve already rolled out? And, you know, again, what can we do to make healthcare better broadly?LEE: So, you know, I think of you as a technologist, and, Matt, you and I actually are colleagues working together here at Microsoft. But you also do spend time in the clinic still, as well, is that right?LUNGREN: You know, initially it was kind of a very much a non-negotiable for me in sort of taking an industry role. I think like a lot of, you know, physicians, you know, were torn with the idea of like, hey, I spent 20 years training. I love what I do, you know, with a lot of caveats there in terms of some of the administrative burden and some of the hassle sometimes. But for the most part, I love what I do, and theres no greater feeling than using something that you trained years to do and actually see the impact on a human life. Its unbelievable, right.So, I think part of me was just, like, I didnt want to let that part of my identity go. And frankly, as I often say, to this day, I walk by a fax machine in our office today, like in 2025.So just to be extra clear, it really grounds me in, like, yes, I love the possibilities. I love thinking about what we can do. But also, I have a very stark understanding of the reality on the ground, both in terms of the technology but also the burnout, right. The challenges that were facing in taking care of patients has gotten, you know, much, much more difficult in the last few years, and, you know, I like to think it keeps my perspective, yeah.LEE: You know, I think some listeners to this podcast might be surprised that we have doctors on staff in technical roles at Microsoft. How do you explain that to people?LUNGREN: [LAUGHS] Yeah, no, yeah, it is interesting. I would say that, you know, from, you know, the legacy Nuance world, it wasnt so far-fetched that you have physicians that were power users and eventually sort of, you know, became, Hey, listen, I think this is a strategic direction; you should take it or whatever. And certainly maybe in the last, I want to say, five years or so, Ive seen more and more physicians who have, you know, taken the time, sometimes on their own, to learn some of the AI capabilities, learn some of the principles and concepts; and frankly, some are, you know, even coding solutions and leading companies.So, I do think that that has shifted a bit in terms of like, Hey, doctor, this is your lane, and over here, you know, heres a technical person. And I think thats fused quite a bit more.But yeah, it is an unusual thing, I think, in sort of how weve constructed what at least my group does. But again, I cant see any other way around some of the challenges.I think, you know, an anecdote Id like to tell you, when I was running the AIMI [Artificial Intelligence in Medicine and Imaging] Center, you know, we were bringing the medical school together with the computer science department, right, at Stanford. And I remember one day a student, you know, very smart, came into my office, you know, a clinical day or something, and hes like, is there just, like, a book or something where I can just learn medicine? Because, like, I feel like theres a lot of, like, translation you have to do for me.It really raised an important insight, which is that you can learn the, you know, medicine, so to speak. You know, go to med school; you know, take the test and all that. But it really you dont really understand the practice of medicine until you are doing that.And in fact, I even push it a step further to say after training those first two or three years of you are the responsible person; you can turn around, and theres no one there. Like, you are making a decision. Getting used to that and then having a healthy respect for that actually I think provides the most educational value of anything in healthcare.LEE: You know, I think what youre saying is so important because as I reflect on my own journey. Of course, Im a computer scientist. I dont have medical training, although at this point, I feel confident that I could pass a Step 1 medical exam.LUNGREN: I have no doubt. [LAUGHS]LEE: But I think that the tech industry, because of people like you, have progressed tremendously in having a more sophisticated and nuanced understanding of what actually goes on in clinic and also what goes on in the boardrooms of healthcare delivery organizations. And of course, at the end of the day, I think thats really been your role.So roughly speaking, your job as an executive at a big tech company has been to understand what the technology platforms need to be, particularly with respect to machine learning, AI, and cloud computing, to best support healthcare. And so maybe lets start pre-GPT-4, pre-ChatGPT, and tell us a little bit, you know, about maybe some of your proudest moments in getting advanced technologies like AI into the clinic.LUNGREN: You know, when I first started, so remember, like you go all the way back to about 2013, right, my first faculty job, and, you know, were building a clinical program and I, you know, I had a lot of interest in public health and building large datasets for pop [population] health, etc. But I was doing a lot of that, you know, sort of labeling to get those insights manually, right. So, like, I was the person that youd probably look at now and say, What are you doing? Right?So but I had a complete random encounter with Andrew Ng, who I didnt know at the time, at Stanford. And I, you know, went to one of the seminars that he was holding at the Gates building, and, you know, they were talking about their performance on ImageNet. You know, cat and dog and, you know, tree, bush, whatever. And I remember sitting in kind of the back, and I think I maybe had my scrubs on at the time and just kind of like, what? Like, why like, this we could use this in healthcare, you know. [LAUGHS]But for me, it was a big moment. And I was like, this is huge, right. And as you remember, the deep learning really kind of started to show its stuff with, you know, Fei-Fei Lis ImageNet stuff.So anyway, we started the collaboration that actually became a NIDUS. And one of the first things we worked on, we just said, Listen, one of the most common medical imaging examinations in the world is the chest x-ray. Right? Two, three billion are done every year in the world, and so is that not a great place to start?And of course, we had a very democratizing kind of mission. As you know, Andrew has done a lot of work in that space, and I had similar ambitions. And so, we really started to focus on bringing the, you know, the sort of the clinical and the CS together and see what could be done.So, we did CheXNet. And this is, remember this is around the time when, like, Geoffrey Hinton was saying things like we should stop training radiologists, and all this stuff was going on. [LAUGHTER] So theres a lot of hype, and this is the narrow AI days just to remind the audience.LEE: How did you feel about that since you are a radiologist?LUNGREN: Well, it was so funny. So, Andrew is obviously very prolific on social media, and I was, who am I, right? So, I remember he tagged me. Well, first he said, Matt, you need to get a Twitter account. And I said OK. And he tagged me on the very first post of our, what we call, CheXNet that was kind of like the Hello, World! for this work.And I remember it was a clinical day. I had set my phone, as you do, outside the OR. I go in. Do my procedure. You know, hour or so, come back, my phones dead. Im like, oh, thats weird. Like I had a decent charge. So, you know, I plug it in. I turn it on. I had like hundreds of thousands of notifications because Andrew had tweeted out to his millions or whatever about CheXNet.And so, then of course, as you point out, I go to RSNA that year, which is our large radiology conference, and that Geoffrey Hinton quote had come out. And everyones looking at me like, What are you doing, Matt? You know, like, are you coming after our specialty? Im like, No, no, thats, [LAUGHS] you know, its a way to interpret it, but you have to take a much longer horizon view, right.LEE: Well, you know, were going to, just as an enticement for listeners to this podcast to listen to the very end, Im going to pin you down toward the end on your assessment of whether Geoffrey Hinton will eventually be proven right or not. [LAUGHTER] But lets take our time to get there.Now lets go ahead and enter the generative AI era. When we were first exposed to what we now know of as GPT-4this was before it was disclosed to the worlda small number of people at Microsoft and Microsoft Research were given access in order to do some technical assessment.And, Matt, you and I were involved very early on in trying to assess what might this technology mean for medicine.LUNGREN: It was the weirdest thing, Peter. Like I joined that summer, so the summer before, you know, the actual GPT came out. I had literally no idea what I was getting into.So, I started asking it questions, you know, kind of general stuff, right. Just, you know, I was like, oh, all right, its pretty good. And so, then I would sort of go a little deeper. And eventually I got to the point where Im asking questions that, you know, maybe theres three papers on it in my community, and remember Im a sub-sub specialist, right, pediatric interventional radiology. And the things that we do in vascular malformations and, you know, rare cancers are really, really strange and not very commonly known.And I kind of walked away from thatfirst I said, can I have this thing, right? [LAUGHS]But then I, you know, I dont want to sound dramatic, but I didnt sleep that well, if Im being honest, for the first few nights. Partially because I couldnt tell anybody, except for the few that I knew were involved, and partially because I just couldnt wrap my head around how we went from what I was doing in LSTMs [long short-term memory networks], right, which was state of the artish at the time for NLP [natural language processing].And all of a sudden, I have this thing that is broadly, you know, domain experts, you know, representations of knowledge that theres no way you could think of it would be in distribution for a normal approach to this.And so, I really struggled with it, honestly. Interpersonally, like, I would be like, uh, well, lets not work on that. Theyre like, why not? You were just excited about it last week. Im like, I dont know. I think that we could think of another approach later.And so yeah, when we were finally able to really look at some of the capabilities and really think clearly, it was really clear that we had a massive opportunity on our hands to impact healthcare in a way that was never possible before.LEE: Yeah, and at that time you were still a part of Nuance. Nuance, I think, was in the process of being acquired by Microsoft. Is that right?LUNGREN: Thats right.LEE: And so, of course, this was also a technology that would have profound and very direct implications for Nuance. How did you think about that?LUNGREN: Nuance, for those in the audience who dont know, for 25 years was, sort of, the medical speech-to-text thing that all, you know, physicians used. But really the brass ring had always been and I want to say going back to like 2013, 2014, Nuance had tried to figure out, OK, we see this pain point. Doctors are typing on their computers while theyre trying to talk to their patients, right.We should be able to figure out a way to get that ambient conversation turned into text that then, you know, accelerates the doctor takes all the important information. Thats a really hard problem, right. Youre having a conversation with a patient about their knee pain, but youre also talking about, you know, their cousins wedding and their next vacation and their dog is sick or whatever and all that gets recorded, right.And so, then you have to have the intelligence/context to be able to tease out whats important for a note. And then it has to be at the performance level that a physician who, again, 20 years of training and education plus a huge, huge amount of, you know, need to get through his cases efficiently, thats a really difficult problem.And so, for a long time, there was a human-in-the-loop aspect to doing this because you needed a human to say, This transcripts great, but heres actually what needs to go on the note. And that cant scale, as you know.When the GPT-4, you know, model kind of, you know, showed what it was capable of, I think it was an immediate light bulb because there was no you can ask any physician in your life, anyone in the audience, you know, what are your what is the biggest pain point when you go to see your doctor? Like, Oh, they dont talk to me. They dont look me in the eye. Theyre rushing around trying to finish a note.If we could get that off their plate, thats a huge unlock, Peter. And I think that, again, as you know, its now led to so much more. But that was kind of the initial, I think, reaction.LEE: And so, maybe that gets us into our next set of questions, our next topic, which is about the book and all the predictions we made in the book. Because Carey, Zak, and Iactually we did make a prediction that this technology would have a huge impact on this problem of clinical note-taking.And so, youre just right in the middle of that. Youre directly hands-on creating, I think, what is probably the most popular early product for doing exactly that. So, were we right? Were we wrong? What else do we need to understand about this?LUNGREN: No, you were right on. I think in the book, I think you called it like a paper shredder or something. I think you used a term like that. Thats exactly where the activity is right now and the opportunity.Ive even taken that so far as to say that when folks are asking about what the technology is capable of doing, we say, well, listen, its going to save time before it saves lives. Itll do both. But right now, its about saving time.Its about peeling back the layers of the onion that if you, you know, put me in where I started medicine in 2003, and then fast-forward and showed me a day in the life of 2025, I would be shocked at what I was doing that wasnt related to patient care, right. So, all of those layers that have been stacked up over the years, we can start finding ways to peel that back. And I think thats exactly what were seeing.And to your point, I think you mentioned this, too, which is, well, sure, we can do this transcript, and we can turn a note, but then we can do other things, right. We can summarize that in the patients language or education level of choice. We can pend orders. We can eventually get to a place of decision support. So, Hey, did you think about this diagnosis, doctor? Like those kinds of things.And all those things, I think you highlighted beautifully, and again, it sounds like with, you know, a lot of, right, just kind of guesswork and prediction, but those things are actually happening every single day right now.LEE: Well, so now, you know, in this episode, were really trying to understand, you know, where the technology industry is in delivering these kinds of things. And so from your perspective, you know, in the business that youre helping to run here at Microsoft, you know, what are the things that are actually shipping as product versus things that clinicians are doing, lets say, off label, just by using, say, ChatGPT on their personal mobile devices, and then what things arent happening?LUNGREN: Yeah. Ill start with the shipping part because I think you, again, you know my background, right. Academic clinician, did a lot of research, hadnt had a ton of product experience.In other words, like, you know, again Im happy to show you what benchmarks we beat or a new technique or, you know, get a grant to do all this, or even frankly, you know, talk about startups. But to actually have an audience that is accustomed to a certain level of performance for the solutions that they use, to be able to deliver something new at that same level of expectation, wow, thats a big deal.And again, this is part of the learning by, you know, kind of being around this environment that we have, which is we have this, you know, incredibly focused, very experienced clinical product team, right.And then I think on the other side, to your point about the general-purpose aspect of this, its no secret now, right, that, you know, this is a useful technology in a lot of different medical applications. And lets just say that theres a lot of knowledge that can be used, particularly by the physician community. And I think the most recent survey I saw was from the British Medical Journal, which said, hey, you know, which doctors are using are you willing to tell us, you know, what youre doing? And it turns out that folks are, what, 30% or so said that they were using it regularly in clinic [1]. And again, this is the general, this is the API or whatever off the shelf.And then frankly, when they ask what theyre using it for, tends to be things like, Hey, differential, like, help me fill in my differential or suggest and to me, I think what that created, at leastand youre starting to see this trend really accelerate in the US especiallyis, well, listen, we cant have everybody pulling out their laptops and potentially exposing, you know, patient information by accident or something to a public API.We have to figure this out, and so brilliantly, I think NYU [New York University] was one of the first. Now I think theres 30 plus institutions that said, listen, OK, we know this is useful to the entire community in the healthcare space. Right?We cant allow this sort of to be a very loosey-goosey approach to this, right, given this sort of environment. So, what well do is well set up a HIPAA-compliant instance to allow anyone in the communityyou know, in the health systemto use the models, and then whatever the newest model comes, it gets hosted, as well.And whats cool about thatand thats happened now a lot of placesis that at the high level first of all, people get to use it and experiment and learn. But at the high level, theyre actually seeing what are the common use cases. Because you could ask 15 people and you might get super long lists, and it may not help you decide what to operationalize in your health system.LEE: But let me ask you about that. When you observe that, are there times when you think, Oh, some specific use cases that were observing in that sort of organic way need to be taken into specialized applications and made into products? Or is it best to keep these things sort of, you know, open-chat-interface types of general-purpose platform?LUNGREN: Honestly, its both, and thats exactly what were seeing. Im most familiar with Stanford, kind of, the work that Nigam Shah leads on this. But he, he basically, you know, theres a really great paper that is coming out in JAMA, but basically saying, Heres what our workforce is using it for. Here are the things in the literature that would suggest what would be popular.And some of those line up, like helping with a clinical diagnosis or documentation, but some of them dont. But for the most part, the stuff that flies to the top, those are opportunities to operationalize and productize, etc. And I think thats exactly what were seeing.LEE: So, lets get into some of the specific predictions. Weve, I think, beaten note-taking to death here. But theres other kinds of paperwork, like filling out prior authorization request forms or referral letters, an after-visit note or summary to give instructions to patients, and so on. And these were all things that we were making guesses in our book might be happening. Whats the reality there?LUNGREN: Ive seen every single one of those. In fact, Ive probably seen a dozen startups too, right, doing exactly those things. And, you know, we touched a little bit on translation into the actual clinic. And thats actually another thing that I used to kind of underappreciate, which is that, listen, you can have a computer scientist and a physician or nurse or whatever, like, give the domain expertise, and you think youre ready to build something.The health IT [LAUGHS] is another part of that Venn diagram thats so incredibly critical, and then exactly how are you going to bring that into the system. Thats a whole new ballgame.And so I do want to do a callout because the collaboration that we have with Epic is monumental because here, you have the system of record that most physicians, at least in the US, use. And theyre going to use an interface and theyre going to have an understanding of, hey, we know these are pain points, and so I think theres some really, really cool, you know, new innovations that are coming out of the relationship that we have with Epic. And certainly the audience may be familiar with those, that I think will start to knock off a lot of the things that you predicted in your book relatively soon.LEE: I think most of the listeners to this podcast will know what Epic is. But for those that are unfamiliar with the health industry, and especially the technology foundation, Epic is probably the largest provider of electronic health record systems. And, of course, in collaboration with you and your team, theyve been integrating generative AI quite a bit. Are there specific uses that Epic is making and deploying that get you particularly excited?LUNGREN: First of all, the ambient note generation, by the way, is integrated into Epic now. So like, you know, its not another screen, another thing for physicians. So thats a huge, huge unlock in terms of the translation.But then Epic themselves, so they have, I guess, on the last roadmap that they talked [about], more than 60, but the one thats kind of been used now is this inbox response.So again, maybe someone might not be familiar with, why is it such a big deal? Well, if youre a physician, you already have, you know, 20 patients to see that day and you got all those notes to do, and then Jevons paradox, right. So if you give me better access to my doctor, well, maybe I wont make an appointment. Im just going to send him a note and this is kind of this inbox, right.So then at the end of my day, I got to get all my notes done. And then I got to go through all the inbox messages Ive received from all of my patients and make sure that theyre not like having chest pain and theyre blowing it off or something.Now thats a lot of work and the cold start problem of like, OK, I to respond to them. So Epic has leveraged this system to say, Let me just draft a note for you, understanding the context of, you know, whats going on with the patient, etc. And you can edit that and sign it, right. So you can accelerate some of those so thats probably one Im most excited about. But theres so many right now.LEE: Well, I think I need to let you actually state the name of the clinical note-taking product that youre associated with. Would you like to do that? [LAUGHS]LUNGREN: [LAUGHS] Sure. Yeah, its called DAX Copilot [2]. And for the record, it is the fastest-growing copilot in the Microsoft ecosystem. Were very proud of that. Five hundred institutions already are using it, and millions of notes have already been created with it. And the feedback has been tremendous.LEE: So, you sort of referred to this a little bit, you know, this idea of AI being a second set of eyes. So, doctor makes some decisions in diagnosis or kind of working out potential treatments or medication decisions. And in the book, you know, we surmise that, well, AI might not replace the doctor doing those things. It could but might not. But AI could possibly reduce errors if doctors and nurses are making decisions by just looking at those decisions and just checking them out. Is that happening at all, and what do you see the future there?LUNGREN: Yeah, I would say, you know, thats kind of the jagged edge of innovation, right, where sometimes the capability gets ahead of the ability to, you know, operationalize that. You know, part of that is just related to the systems. The evidence has been interesting on this. So, like, you know this, our colleague Eric Horvitz has been doing a lot of work in sort of looking at physician, physician with GPT-4, lets say, and then GPT-4 alone for a whole variety of things. You know, weve been saying to the world for a long time, particularly in the narrow AI days, that AI plus human is better than either alone. Were not really seeing that bear out really that well yet in some of the research.But it is a signal to me and to the use case youre suggesting, which is that if we let this system, in the right way, kind of handle a lot of the safety-net aspects of what we do but then also potentially take on some of the things that maybe are not that challenging or at least somewhat simple.And of course, this is really an interesting use case in my world, in the vision world, which is that we know these models are multimodal, right. They can process images and text. And what does that look like for pathologists or radiologists, where we do have a certain percentage of the things we look at in a given day are normal, right? Or as close to normal as you can imagine. So is there a way to do that? And then also, by the way, have a safety net.And so I think that this is an extremely active area right now. I dont think weve figured out exactly how to have the human and AI model interact in this space yet. But I know that theres a lot of attempts at it right now.LEE: Yeah, I think, you know, this idea of a true copilot, you know, a true collaborator, you know, I think is still something thats coming. I think weve had a couple of decades of people being trained to think of computers as question-answering machines. Ask a question, get an answer. Provide a document, get a summary. And so on.But the idea that something might actually be this second set of eyes just assisting you all day continuously, I think, is a new mode of interaction. And we havent quite figured that out.Now, in preparation for this podcast, Matt, you said that you actually used AI to assist you in getting ready. [LAUGHS] Would you like to share what you learned by doing that?LUNGREN: Yeah, its very funny. So, like, you may have heard this term coined by Ethan Mollick called the secret cyborg, (opens in new tab) which is sort of referring to the phenomena of folks using GPT, realizing it can actually help them a ton in all kinds of parts of their work, but not necessarily telling anybody that theyre using it, right.And so in a similar secret cyborgish way, I was like, Well, listen, you know, I havent read your book in like a year. I recommend it to everybody. And [I need] just a refresher. So what I did was I took your book, I put it into GPT-4, OK, and asked it to sort of talk about the predictions that you made.And then I took that and put it in the stronger reasoning modelin this case, the deep research that you may have just seen or heard of and the audience from OpenAIand asked it to research all the current papers, you know, and blogs and whatever else and tell me like what was right, what was wrong in terms of the predictions. [LAUGHS]So it, actually, it was an incredible thing. Its a, like, what, six or seven pages. It probably would have taken me two weeks, frankly, to do this amount of work.LEE: Ill be looking forward to reading that in the New England Journal of Medicine shortly.LUNGREN: [LAUGHS] Thats right. Yeah, no, dont, before this podcast comes out, Ill submit it as an opinion piece. No. [LAUGHS] But, yeah, but I think on balance, incredibly insightful views. And I think part of that was, you know, your team that got together really had a lot of different angles on this. But, you know, and I think the only area that was, like, which Ive observed as well, its just, man, this can do a lot for education.We havent seen I dont think were looking at this as a tutor. To your point, were kind of looking at it as a transactional in and out. But as weve seen in all kinds of data, both in low-, middle-income countries and even in Harvard, using this as a tutor can really accelerate your knowledge and in profound ways.And so that is probably one area where I think your prediction was maybe slightly even further ahead of the curve because I dont think folks have really grokked that opportunity yet.LEE: Yeah, and for people who havent read the book, you know, the guess was that you might use this as a training aid if youre an aspiring doctor. For example, you can ask GPT-4 to pretend to be a patient that presents a certain way and that you are the doctor that this patient has come to see. And so you have an interaction. And then when you say end of encounter, you ask GPT-4 to assess how well you did. And we thought that this might be a great training aid, and to your point, it seems not to have materialized.LUNGREN: Theres some sparks. You know, with, like, communication, end-of-life conversations that no physician loves to have, right. Its very, very hard to train someone in those. Ive seen some work done, but youre right. Its not quite hit mainstream yet.LEE: On the subject of things that we missed, one thing that youve been very, very involved in in the last several months has been in shipping products that are multimodal. So that was something I think that we missed completely. What is the current state of affairs for multimodal, you know, healthcare AI, medical AI?LUNGREN: Yeah, the way I like to explain itand first of all, no fault to you, but this is not an area that, like, we were just so excited about the text use cases that I cant fault you. But yeah, I mean, so if we look at healthcare, right, how we take care of patients today, as you know, the vast majority of the data in terms of just data itself is actually not in text, right. Its going be in pathology and genomics and radiology, etc.And it seems like an opportunity here to watch this huge curve just goes straight up in the general reasoning and frankly medical competency and capabilities of the models that are coming and continue to come but then to see that its not as proficient for medical-specific imaging and video and, you know, other data types. And that gap is, kind of, what I describe as the multimodal medical AI gap.Were probably in GPT-2 land, right, for this other modality types versus the, you know, were now at o3, who knows where were going to go. At least in our view, we can innovate in that space.How do we help bring those innovations to the broader community to close that gap and see some of these use cases really start to accelerate in the multimodal world?And I think weve taken a pretty good crack at that. A lot of that is credit to the innovative work. I mean, MSR [Microsoft Research] was two or three years ahead of everyone else on a lot of this. And so how do we package that up in a way that the community can actually access and use? And so, we took a lot of what your group had done in, lets just say, radiology or pathology in particular, and say, OK, well, lets put this in an ecosystem of other models. Other groups can participate in this, but lets put it in a platform where maybe Im really competent in radiology or pathology. How do I connect those things together? How do I bring the general reasoner knowledge into a multimodal use case?And I think thats what weve done pretty well so far. We have a lot of work to do still, but this is very, very exciting. Were seeing just such a ton of interest in building with the tools that we put out there.LEE: Well, I think how rapidly thats advancing has been a surprise to me. So I think were running short on time. So two last questions to wrap up this conversation. The first one is, as we think ahead on AI in medicine, what do you think will be the biggest changes or make the biggest differences two years from now, five years from now, 10 years from now?LUNGREN: This is really tough. OK. I think the two-year timeframe, I think we will have some autonomous agent-based workflows for a lot of the what I would call undifferentiated heavy lifting in healthcare.And this is happening in, you know, the pharmaceutical industry, the payer every aspect is sort of looking at their operations at a macro level: where are these big bureaucratic processes that largely involve text and where can we shrink those down and really kind of unlock a lot of our workforce to do things that might be more meaningful to the business? I think thats my safe one.Going five years out, you know, I have a really difficult time grappling with this seemingly shrinking timeline to AGI [artificial general intelligence] that we hear from people who I would respect and certainly know more than me. And in that world, I think theres only been one paper that Ive seen that has attempted to say, what does that mean in healthcare (opens in new tab) when we have this?And the fact is, I actually dont know. [LAUGHS] I wonder whether therell still be a gap in some modalities. Maybe therell be the ability to do new science, and all kinds of interesting things will come of that.But then if you go all the way to your 10-year, I do feel like were going to have systems that are acting autonomously in a variety of capacities, if Im being honest.What I would like to see if I have any influence on some of this is, can we start to celebrate the closing of hospitals instead of opening them? Meaning that, can we actually start to addressat a personal, individual levelcare? And maybe thats outside the home, maybe thats, you know, in a way that doesnt have to use so many resources and, frankly, really be very reactive instead of proactive.I really want to see that. Thats been the vision of precision medicine for, geez, 20-plus years. I feel like were getting close to that being something we can really tackle.LEE: So, we talked about Geoff Hinton and his famous prediction that we would soon not have human radiologists. And of course, maybe he got the date wrong. So, lets reset the date to 2028. So, Matt, do you think Geoff is right or wrong?LUNGREN: [LAUGHS] Yeah, so the way Im not going to dodge the question, but let me just answer this a different way.We have a clear line of sight to go from images to draft reports. That is unmistakable. And thats now in 2025. How it will be implemented and what the implications of that will be, I think, will be heavily dependent on the health system or the incentive structure for where its deployed.So, if Im trying to take a step back, back to my global health days, man, that cant come fast enough. Because, you know, you have entire health systems, you know, in fact entire countries that have five, you know, medical imaging experts for the whole country, but they still need this to you know take care of patients.Zooming in on todays crisis in the US, right, we have the burnout crisis just as much as the doctors who are seeing patients and write notes. We cant keep up with the volume. In fact, were not training folks fast enough, so there is a push pull; there may be a flip to your point of autonomous reads across some segments of what we do.By 2028, I think thats a reasonable expectation that well have some form of that. Yes.LEE: I tend to agree, and I think things get reshaped, but it seems very likely that even far into the future well have humans wanting to take care of other humans and be taken care of by humans.Matt, this has been a fantastic conversation, and, you know, I feel its always a personal privilege to have a chance to work with someone like you so keep it up.[TRANSITION MUSIC]LUNGREN: Thank you so much, Peter. Thanks for having me.LEE: Im always so impressed when I talk to Matt, and I feel lucky that we get a chance to work together here at Microsoft. You know, one of the things that always strikes me whenever I talk to him is just how disruptive generative AI has been to a business like Nuance. Nuance has had clinical note-taking as part of their product portfolio for a long, long time. And so, you know, when generative AI comes along, its not only an opportunity for them, but also a threat because in a sense, it opens up the possibility of almost anyone being able to make clinical note-taking capabilities into products.Its really interesting how Matts product, DAX Copilot, which since the time that we had our conversation has expanded into a full healthcare workflow product called Dragon Copilot, has really taken off in the marketplace and how many new competing AI products have also hit the market, and all in just two years, because of generative AI.The other thing, you know, that I always think about is just how important it is for these kinds of systems to work together and especially how they integrate into the electronic health record systems. This is something that Carey, Zak, and I didnt really realize fully when we wrote our book. But you know, when you talk to both Matt and Seth, of course, we see how important it is to have that integration.Finally, what a great example of yet another person who is both a surgeon and a tech geek. [LAUGHS] People sometimes think of healthcare as moving very slowly when it comes to new technology, but people like Matt are actually making it happen much more quickly than most people might expect.Well, anyway, as I mentioned, we also had a chance to talk to Seth Hain, and so heres my conversation with Seth:LEE: Seth, thank you so much for joining.SETH HAIN: Well, Peter, its such an exciting time to sit down and talk about this topic. So much has changed in the last two years. Thanks for inviting me.LEE: Yeah, in fact, I think in a way both of our lives have been upended in many ways by the emergence of AI. [LAUGHTER]The traditional listeners of the Microsoft Research Podcast, I think for the most part, arent steeped in the healthcare industry. And so maybe we can just start with two things. One is, what is Epic, really? And then two, what is your job? What does the senior vice president for R&D at Epic do every day?HAIN: Yeah, well, lets start with that first question. So, what is Epic? Most people across the world experience Epic through something we call MyChart. They might use it to message their physician. They might use it to check the lab values after theyve gotten a recent test. But its an app on their phone, right, for connecting in with their doctors and nurses and really making them part of the care team.But the software we create here at Epic goes beyond that. Its what runs in the clinic, what runs at the bedside, in the back office to help facilitate those different pieces of care, from collecting vital information at the bedside to helping place orders if youre coming in for an outpatient visit, maybe with a kiddo with an earache, and capturing that note and record of what happened during that encounter, all the way through back-office encounters, back-office information for interacting with payers as an example.And so, we provide a suite of software that health systems and increasingly a broader set of the healthcare ecosystem, like payers and specialty diagnostic groups, use to connect with that patient at the center around their care.And my job is to help our applications across the company take advantage of those latest pieces of technology to help improve the efficiency of folks like clinicians in the exam room when you go in for a visit. Well get into, I imagine, some use cases like ambient conversations, capturing that conversation in the exam room to help drive some of that documentation.But then providing that platform for those teams to build those and then strategize around what to create next to help both the physicians be efficient and also the health systems. But then ultimately continuing to use those tools to advance the science of medicine.LEE: Right. You know, one thing that I explain to fellow technologists is that I think today health records are almost entirely digital. I think the last figures I saw is well over 99% of all health records are digital.But in the year 2001, fewer than 15% of health records were digital. They were literally in folders on paper in storerooms, and if youre old enough, you might even remember seeing those storerooms.So, its been quite a journey. Epic and Epics competitorsthough I think Epic is really the most important companyhave really moved the entire infrastructure of record keeping and other communications in healthcare to a digital foundation.And I think one thing well get into, of course, one of the issues that has really become, I think, a problem for doctors and nurses is the kind of clerical or paperwork, record-keeping, burden. And for that reason, Epic and Epic systems end up being a real focus of attention. And so, well get into that in a bit here.HAIN: And I think that hits, just to highlight it, on both sides. There is both the need to capture documentation; theres also the challenge in reviewing it.LEE: Yes.HAIN: The average medical record these days is somewhere between the length of Fahrenheit 451 and To Kill a Mockingbird. [LAUGHTER] So theres a fair amount of effort going in on that review side, as well.LEE: Yeah, indeed. So much to get into there. But I would like to talk about encounters with AI. So obviously, I think there are two eras here: before the emergence of ChatGPT and what we now call of as generative AI and afterwards. And so, lets take the former.Of course, youve been thinking about machine learning and health data probably for decades. Do you have a memory of how you got into this? Why did you get an interest in data analytics and machine learning in the first place?HAIN: Well, my background, as you noted, is in mathematics before I came to Epic. And the sort of patterns and what could emerge were always part of what drove that. Having done development and kind of always been around computers all my life, it was a natural transition as I came here.And I started by really focusing on, how do we scale systems for the very largest organizations, making sure they are highly available and also highly responsive? Time is critical in these contexts in regards to rapidly getting information to doctors and nurses.And then really in the, say, in the 2010s, there started to be an emergence of capabilities from a storage and compute perspective where we could begin to build predictive analytics models. And these were models that were very focused, right. It predicted the likelihood somebody would show up for an appointment. It predicted the likelihood that somebody may fall during an inpatient stay, as an example.And I think a key learning during that time period was thinking through the full workflow. What information was available at that point in time, right? At the moment somebody walks into the ED [emergency department], you dont have a full picture to predict the likelihood that they may deteriorate during an inpatient encounter.And in addition to what information was available was, what can you do about it? And a key part of that was how do we help get the right people in the right point in time at the bedside to make an assessment, right? It was a human-in-the-loop type of workflow where, for example, you would predict deterioration in advance and have a nurse come to the bedside or a physician come to the bedside to assess.And I think that combination of narrowly focused predictive models with an understanding that to have them make an impact you had to think through the full workflow of where a human would make a decision was a key piece.LEE: Obviously there is a positive human impact. And so, for sure, part of the thought process for these kinds of capabilities comes from that.But Epic is also a business, and you have to worry about, you know, what are doctors and clinics and healthcare systems willing to buy. And so how do you balance those two things, and do those two things ever come into conflict as youre imagining what kinds of new capabilities and features and products to create?HAIN: Two, sort of, two aspects I think really come to mind. First off, generally speaking, we see analytics and AI as a part of the application. So, in that sense, its not something we license separately. We think that those insights and those pieces of data are part of what makes the application meaningful and impactful.At the scale that many of these health systems operate and the number of patients that they care for, as well as having tens of thousands of users in the system daily, one needs to think about the compute overhead LEE: Yes.HAIN: that these things cause. And so, in that regard, there is always a ROI assessment that is taking place to some degree around, what happens if this runs at full scale? And in a way, that really got accelerated as we went into the generative AI era.LEE: Right. OK. So, you mentioned generative AI. What was the first encounter, and what was that experience for you?HAIN: So, in the winter of 22 and into 2023, I started experimenting alongside you with what we at that time called DV3, or Davinci 3, and eventually became GPT-4. And immediately, a few things became obvious. The tool was highly general purpose. One was able to, in putting in a prompt, have it sort of convert into the framing and context of a particular clinical circumstance and reason around that context. But I think the other thing that started to come to bear in that context was there was a fair amount of latent knowledge inside of it that was very, very different than anything wed seen before. And, you know, theres some examples from the Sparks of AGI paper from Microsoft Research, where a series of objects end up getting stacked together in the optimal way to build height. Just given the list of objects, it seems to have a understanding of physical space that it intuited from the training processes we hadnt seen anywhere. So that was an entirely new capability that programmers now had access to.LEE: Well fact, you know, I think that winter of 2022, and well get into this, one of your projects that youve been running for quite a few years is something called Cosmos (opens in new tab), which I find exceptionally interesting. And I was motivated to understand whether this type of technology could have an impact there.And so, I had to receive permission from both OpenAI and Microsoft to provide you with early access.When I did first show this technology to you, you must have had an emotional response, either skepticism or I cant imagine you just trusted, you know, trusted me to the extent of believing everything I was telling you.HAIN: I think theres always a question of, what is it actually, right? Its often easy to create demos. Its often easy to show things in a narrow circumstance. And it takes getting your hands on it and really spending your 10,000 hours digging in and probing it in different ways to see just how general purpose it was.And so, the skepticism was really around, how applicable can this be broadly? And I think the second questionand were starting to see this play out now in some of the later modelswas, is this just a language thing? Is it narrowly only focused on that? Or can we start to imagine other modalities really starting to factor into this? How will it impact basic sciences? Those sorts of things.On a personal note, I mean, I had, at that point, now theyre now 14 and 12, two kids that I wondered, what did this mean for them? What is the right thing for them to be studying? And so I remember sleepless nights on that topic, as well.LEE: OK, so now you get early access to this technology; youre able to do some experimentation. I think one of the things that impressed me is just less than four months later at the major health tech industry conference, HIMSS, which also happened timing-wise to take place just after the public disclosure of GPT-4, Epic showed off some early prototype applications of generative AI. And so, describe what those were, and how did you choose what to try to do there?HAIN: Yeah, and we were at that point, we actually had the very first users live on that prototype, on that early version.And the key thing wed focused onwe started this development in very, very late December, January of 2023was a problem that its origins really were during the pandemic.So, during the pandemic, we started to see patients increasingly messaging their providers, nurses, and clinicians through MyChart, that patient portal I mentioned with about 190 million folks on it. And as you can imagine, that was a great opportunity in the context of COVID to limit the amount of direct contact between providers and patients while still getting their questions answered.But what we found as we came out of the pandemic was that folks preferred it regardless. And that messaging volume had stayed very, very high and was a time-consuming effort for folks.And so, the first use case we came out with was a draft message in the context of the message from the patient and understanding of their medical history using that medical record that we talked about.And the nurse or physician using the tool had two options. They could either click to start with that draft and edit it and then hit send, or they could go back to the old workflow and start with a blank text box and write it from their own memory as they preferred.And so that was that very first use case. There were many more that we had started from a development perspective, but, yeah, we had that rolling out right in March of 2023 there with the first folks.LEE: So, I know from our occasional discussions that some things worked very well. In fact, this is a real product now for Epic. And it seems to be really a very, very popular feature now. I know from talking to you that a lot of things have been harder. And so, Id like to dive into that. As a developer, tech developer, you know, whats been easy, whats been hard, whats in your mind still is left to do in terms of the development of AI?HAIN: Yeah. You know, the first thing that comes to mind sort of starting foundationally, and we hinted at this earlier in our conversation, was at that point in time, it was kind of per a message, rather compute-intensive to run these. And so, there were always trade-offs we were making in regards to how many pieces of information we would send into the model and how much would we request back out of it.The result of that was that while kind of theoretically or even from a research perspective, we could achieve certain outcomes that were quite advanced, one had to think about, where you make those trade-offs from a scalability perspective as you wanted to roll that out to lot of folks. So LEE: Were you charging your customers more money for this feature?HAIN: Yeah, essentially the way that we handle that is theres compute thats required. As I mentioned, the feature is just part of our application. So, its just what they get with an upgrade.But that compute overhead is something that we needed to pass through to them. And so, it was something, particularly given both the staffing challenges, but also the margin pressures that health systems are feeling today, we wanted to be very cautious and careful about.LEE: And lets put that on the stack because I do want to get into, from the selling perspective, that challenge and how you perceive health systems as a customer making those trade-offs. But lets continue on the technical side here.HAIN: Yeah. On the technical side, it was a consideration, right. We needed to be thoughtful about how we used them. But going up a layer in the stack, at that time, theres a lot of conversation in the industry around something called RAG, or retrieval-augmented generation.And the idea was, could you pull the relevant bits, the relevant pieces of the chart, into that prompt, that information you shared with the generative AI model, to be able to increase the usefulness of the draft that was being created? And that approach ended up proving and continues to be to some degree, although the techniques have greatly improved, somewhat brittle, right. You have a general-purpose technology that is drafting the response.But in many ways, you needed to, for a variety of pragmatic reasons, have somewhat brittle capability in regards to what you pulled into that approach. It tended to be pretty static. And I think this becomes one of the things that, looking forward, as these models have gotten a lot more efficient, we are and will continue to improve upon because, as you get a richer and richer amount of information into the model, it does a better job of responding.I think the third thing, and I think this is going to be something were going to continue to work through as an industry, was helping users understand and adapt to these circumstances. So many folks when they hear AI think, it will just magically do everything perfectly.And particularly early on with some of those challenges were talking about, it doesnt. You know, if its helpful 85% of the time, thats great, but its not going to be 100% of the time. And its interesting as we started, we do something we call immersion, where we always make sure that developers are right there elbow to elbow with the users of the software.And one of the things that I realized through that experience with some of the very early organizations like UCSD [UC San Diego] or University of Wisconsin here in Madison was that even when Im responding to an email or a physician is responding to one of these messages from a patient, depending on the patient and depending on the person, they respond differently.In that context, theres opportunity to continue to mimic that behavior as we go forward more deeply. And so, you learn a lot about, kind of, human behavior as youre putting these use cases out into the world.LEE: So, you know, this increasing burden of electronic communications between doctors, nurses, and patients is centered in one part of Epic. I think thats called your in-basket application, if I understand correctly.HAIN: Thats correct.LEE: But that also creates, I think, a reputational risk and challenge for Epic because as doctors feel overburdened by this and theyre feeling burnt outand as we know, thats a big issuethen they point to, you know, Oh, Im just stuck in this Epic system.And I think a lot of the dissatisfaction about the day-to-day working lives of doctors and nurses then focuses on Epic. And so, to what extent do you see technologies like generative AI as, you know, a solution to that or contributing either positively or negatively to this?HAIN: You know, earlier I made the comment that in December, as we started to explore this technology, we realized there were a class of problems that now might have solutions that never did before.And as weve started to dig into thoseand we now have about 150 different use cases that are under development, many of which are live across weve got about 350 health systems using themone of the things weve started to find is that physicians, nurses, and others start to react to saying its helping them move forward with their job.And examples of this, obviously the draft of the in-basket message response is one, but using ambient voice recognition as a kind of new input into the software so that when a patient and a physician sit down in the exam room, the physician can start a recording and that conversation then ends up getting translated or summarized, if you will, including using medical jargon, into the note in the framework that the physician would typically write.Another one of those circumstances where they then review it, dont need to type it out from scratch, for example, LEE: Right.HAIN: and can quickly move forward.I think looking forward, you know, you brought up Cosmos earlier. Its a suite of applications, but at its core is a dataset of about 300 million de-identified patients. And so using generative AI, we built research tools on top of it. And I bring that up because its a precursor of how that type of deep analytics can be put into context at the point of care. Thats what we see this technology more deeply enabling in the future.LEE: Yeah, when you are creating so you said there are about 150 sort of integrations of generative AI going into different parts of Epics software products.When you are doing those developments and then youre making a decision that something is going to get deployed, one thing that people might worry about is, well, these AI systems hallucinate. They have biases. There are unclear accountabilities, you know, maybe patient expectations.For example, if theres a note drafted by AI thats sent to a patient, does the patient have a right to know what was written by AI and what was written by the human doctor? So, can we run through how you have thought about those things?HAIN: I think one thing that is important context to set here for folks, and I think its often a point of confusion when Im chatting with folks in public, is that their interaction with generative AI is typically through a chatbot, right. Its something like ChatGPT or Bing or one of these other products where theyre essentially having a back-and-forth conversation.LEE: Right.HAIN: And that is a dramatically different experience than how we think it makes sense to embed into an enterprise set of applications.So, an example use case may be in the back office, there are folks that are coding encounters. So, when a patient comes in, right, they have the conversation with the doctor, the doctor documents it, that encounter needs to be billed for, and those folks in the back-office associate to that encounter a series of codes that provide information about how that billing should occur.So, one of the things we did from a workflow perspective was add a selector pane to the screen that uses generative AI to suggest a likely code. Now, this suggestion runs the risk of hallucination. So, the question is, how do you build into the workflow additional checks that can help the user do that?And so in this context, we always include a citation back to the part of the medical record that justifies or supports that code. So quickly on hover, the user can see, does this make sense before selecting it? And its those types of workflow pieces that we think are critical to using this technology as an aid to helping people make decisions faster, right. Its similar to drafting documentation that we talked about earlier.And its interesting because theres a series of patterns that are going back to the AI Revolution book you folks wrote two years ago. Some of these are really highlighted there, right. This idea of things like a universal translator is a common pattern that we ended up applying across the applications. And in my mind, translation, this may sound a little bit strange, but summarization is an example of translating a very long series of information in a medical record into the context that an ED physician might care about, where they have three or four minutes to quick review that very long chart.And so, in that perspective, and back to your earlier comment, we added the summary into the workflow but always made sure that the full medical record was available to that user, as well. So, a lot of what weve done over the last couple of years has been to create a series of repeatable techniques in regards to both how to build the backend use cases, where to pull the information, feed it into the generative AI models.But then I think more importantly are the user experience design patterns to help mitigate those risks you talked about and to maintain consistency across the integrated suite of applications of how those are deployed.LEE: You might remember from our book, we had a whole chapter on reducing paperwork, and I think thats been a lot of what weve been talking about. I want to get beyond that, but before transitioning, lets get some numbers.So, you talked about messages drafted to patients, to be sent to patients. So, give a sense of the volume of whats happening right now.HAIN: Oh, we are seeing across the 300 and, I think its, 48 health systems that are now using generative AIand to be clear, we have about 500 health systems we have the privilege of working with, each with many, many hospitalsthere are tens of thousands of physicians and nurses using the software. That includes drafting million-plus, for example, notes a month at this point, as well as helping to generate in a similar ballpark that number of responses to patients.The thing Im increasingly excited about is the broader set of use cases that were seeing folks starting to deploy now. One of my favorites has been its natural that as part of, for example, a radiology workflow, in studying that image, the radiologist made note that it would be worth double checking, say in six to eight months, that the patient have this area scanned of their chest. Something looks a little bit fishy there, but theres not LEE: Theres not a definitive finding yet.HAIN: theres not a definitive finding at that point. Part of that workflow is that the patients physician place an order for that in the future. And so, were using generative AI to note that back to the physician. And with one click, allow them to place that order, helping that patient get better care.Thats one example of dozens of use cases that are now live, both to help improve the care patients are getting but also help the workforce. So going back to the translation-summarization example, a nurse at the end of their shift needs to write up a summary of that shift for the next nurse for each LEE: Right.HAIN: each patient that they care for. Well, theyve been documenting information in the chart over those eight or 12 hours, right.LEE: Yep, yep.HAIN: So, we can use that information to quickly draft that end-of-shift note for the nurse. They can verify it with those citations we talked about and make any additions or edits that they need and then complete their end of day far more efficiently.LEE: Right. OK. So now lets get to Cosmos, which has been one of these projects that I think has been your baby for many years and has been something that has had a profound impact on my thinking about possibilities. So first off, what is Cosmos?HAIN: Well, just as an aside, I appreciate the thoughtful comments. There is a whole team of folks here that are really driving these projects forward. And a large part of that has been, as you brought up, both Cosmos as a foundational capability but then beginning to integrate it into applications. And thats what those folks spend time on.Cosmos is this effort across hundreds of health systems that we have the privilege of working with to build out a de-identified dataset with todayand it climbs every daybut 300 million unique patient records in it.And one of the interesting things about that structure is that, for example, if I end up in a hospital in Seattle and have that encounter documented at a health system in Seattle, I stilla de-identified version of mestill only shows up once in Cosmos, stitching together both my information from here in Madison, Wisconsin, where Epic is at, with that extra data from Seattle. The result is these 300 million unique longitudinal records that have a deep history associated with them.LEE: And just to be clear, a patient record might have hundreds or even thousands of individual, I guess what you would call, clinical records or elements.HAIN: Thats exactly right. Its the breadth of information from orders and allergies and blood pressures collected, for example, in an outpatient setting to cancer staging information that might have come through as part of an oncology visit. And its coming from a variety of sources. We exchange information about 10 million times a day between different health systems. And that full picture is available within Cosmos in that way of the patient.LEE: So now why? Why Cosmos?HAIN: Why Cosmos? Well, the real ultimate aim is to put a deeply informed in-context perspective at the point of care. So, as a patient, if Im in the exam room, its helpful for the physician and me to know what have similar patients like me experienced in this context. What was the result of that line of treatment, for example?Or as a doctor, if Im looking and working through a relatively rare or strange case to me, I might be able to connect withthis as an example workflow we built called Look-Alikeswith another physician who has seen similar patients or within the workflow see a list of likely diagnoses based on patients that have been in a similar context. And so, the design of Cosmos is to put those insights into the point of care in the context of the patient.To facilitate those steps there, the first phase was building out a set of research tooling. So, we see dozens of papers a year being published by the health systems that we work with. Those that participate in Cosmos have access to it to do research on it. And so they use both a series of analytical and data science tools to do that analysis and then publish research. So, building up trust that way.LEE: The examples you gave are, like with Look-Alikes, its very easy, I think, for people outside of the healthcare world to imagine how that could be useful. So now why is GPT-4 or any generative AI relevant to this?HAIN: Well, so a couple of different pieces, right. Earlier we talked aboutand I think this is the most importanthow generative AI is able to cast things into a specific context. And so, in that way, we can use these tools to help both identify a cohort of patients similar to you when youre in the exam room. And then also help present that information back in a way that relates to other research and understandings from medical literature to understand what are those likely outcomes.I think more broadly, these tools and generative AI techniques in the transformer architecture envision a deeper understanding of sequences of events, sequences of words. And that starts to open up broader questions about what can really be understood about patterns and sequences of events in a patients journey.Which if you didnt know, the name Epic, just like a great long nations journey is told through an epic story, is a patients story. So thats where it came from.LEE: So, were running up against our time together. And I always like to end with a more provocative question.HAIN: Certainly.LEE: And for you, I wanted to raise a question that I think we had asked ourselves in the very earliest days that we were sharing Davinci 3, what we now know of as GPT-4, with each other, which is, is there a world in the future because of AI where we dont need electronic health records anymore? Is there a world in the future without EHR?HAIN: I think it depends on how you define EHR. I see a world coming where we need to manage a hybrid workforce, where there is a combination of humans and something folks are sometimes calling agents working in concert together to care for more and more of our of the country and of the world. And there is and will need to be a series of tools to help orchestrate that hybrid workforce. And I think things like EHRs will transform into helping that operate be operationally successful.But as a patient, I think theres a very different opportunity that starts to be presented. And weve talked about kind of understanding things deeply in context. Theres also a real acceleration happening in science right now. And the possibility of bringing that second- and third-order effects of generative AI to the point of care, be that through the real-world evidence we were talking about with Cosmos or maybe personalized therapies that really are well matched to that individual. These generative AI techniques open the door for that, as well as the full lifecycle of managing that from a healthcare perspective all the way through monitoring after the fact.And so, I think well still be recording peoples stories. Their stories are relevant to them, and they can help inform the bigger picture. But I think the real question is, how do you put those in a broader context? And these tools open the door for a lot more.LEE: Well, thats really a great vision for the future.[TRANSITION MUSIC]Seth, I always really learn so much talking to you, and thank you so much for this great chat.HAIN: Thank you for inviting me.LEE: I see Seth as someone on the very leading frontier of bringing generative AI to the clinic and into the healthcare back office and at the full scale of our massive healthcare system. Its always impressive to me how thoughtful Seth has had to be about how to deploy generative AI into a clinical setting.And, you know, one thing that sticks outand he made such a point of thisis, you know, generative AI in the clinical setting isnt just a chatbot. Theyve had to really think of other ways that will guarantee that the human stays in the loop. And thats of course exactly what Carey, Zak, and I had predicted in our book. In fact, we even had a full chapter of our book entitled Trust but Verify, which really spoke to the need in medicine to always have a human being directly involved in overseeing the process of healthcare delivery.One technical point that Carey, Zak, and I completely missed, on the other hand, in our book, was the idea of something that Seth brought up called RAG, which is retrieval augmented generation. Thats the idea of giving AI access to a database of information and allowing it to use that database as it constructs its answers. And we heard from Seth how fundamental RAG is to a lot of the use cases that Epic is deploying.And finally, I continue to find Seths project called Cosmos to be a source of inspiration, and Ive continued to urge every healthcare organization that has been collecting data to consider following a similar path.In our book, we spent a great deal of time focusing on the possibility that AI might be able to reduce or even eliminate a lot of the clerical drudgery that currently exists in the delivery of healthcare. We even had a chapter entitled The Paperwork Shredder. And we heard from both Matt and Seth that that has indeed been the early focus of their work.But we also saw in our book the possibility that AI could provide diagnoses, propose treatment options, be a second set of eyes to reduce medical errors, and in the research lab be a research assistant. And here in Epics Cosmos, we are seeing just the early glimpses that perhaps generative AI can actually provide new research possibilities in addition to assistance in clinical decision making and problem solving. On the other hand, that still seems to be for the most part in our future rather than something thats happening at any scale today.But looking ahead to the future, we can still see the potential of AI helping connect healthcare delivery experiences to the advancement of medical knowledge. As Seth would say, the ability to connect bedside to the back office to the bench. Thats a pretty wonderful future that will take a lot of work and tech breakthroughs to make it real. But the fact that we now have a credible chance of making that dream happen for real, I think thats pretty wonderful.[MUSIC TRANSITIONS TO THEME]Id like to say thank you again to Matt and Seth for sharing their experiences and insights. And to our listeners, thank you for joining us. We have some really great conversations planned for the coming episodes, including a look at how patients are using generative AI for their own healthcare, as well as an episode on the laws, norms, and ethics developing around AI and health, and more. We hope youll continue to tune in.Until next time.[MUSIC FADES]
English
Arabic
French
Spanish
Portuguese
Deutsch
Turkish
Dutch
Italiano
Russian
Romaian
Portuguese (Brazil)
Greek
