Transcript [MUSIC]      [BOOK PASSAGE]   PETER LEE: “In ‘The Little Black Bag,’ a classic science fiction story, a high-tech doctor’s kit of the future is accidentally transported back to the 1950s, into the shaky hands of a washed-up, alcoholic doctor. The ultimate medical tool, it redeems the doctor wielding it, allowing him to practice gratifyingly heroic medicine. … The tale ends badly for the doctor and his treacherous assistant, but it offered a picture of how advanced technology could transform medicine—powerful when it was written nearly 75 years ago and still so today. What would be the Al equivalent of that little black bag? At this moment when new capabilities are emerging, how do we imagine them into medicine?”   [END OF BOOK PASSAGE]     [THEME MUSIC]     This is The AI Revolution in Medicine, Revisited. I’m your host, Peter Lee.    Shortly after OpenAI’s GPT-4 was publicly released, Carey Goldberg, Dr. Zak Kohane, and I published The AI Revolution in Medicine to help educate the world of healthcare and medical research about the transformative impact this new generative AI technology could have. But because we wrote the book when GPT-4 was still a secret, we had to speculate. Now, two years later, what did we get right, and what did we get wrong?     In this series, we’ll talk to clinicians, patients, hospital administrators, and others to understand the reality of AI in the field and where we go from here.   [THEME MUSIC FADES] The book passage I read at the top is from “Chapter 10: The Big Black Bag.”  In imagining AI in medicine, Carey, Zak, and I included in our book two fictional accounts. In the first, a medical resident consults GPT-4 on her personal phone as the patient in front of her crashes. Within seconds, it offers an alternate response based on recent literature. In the second account, a 90-year-old woman with several chronic conditions is living independently and receiving near-constant medical support from an AI aide.    In our conversations with the guests we’ve spoken to so far, we’ve caught a glimpse of these predicted futures, seeing how clinicians and patients are actually using AI today and how developers are leveraging the technology in the healthcare products and services they’re creating. In fact, that first fictional account isn’t so fictional after all, as most of the doctors in the real world actually appear to be using AI at least occasionally—and sometimes much more than occasionally—to help in their daily clinical work. And as for the second fictional account, which is more of a science fiction account, it seems we are indeed on the verge of a new way of delivering and receiving healthcare, though the future is still very much open.  As we continue to examine the current state of AI in healthcare and its potential to transform the field, I’m pleased to welcome Bill Gates and Sébastien Bubeck.   Bill may be best known as the co-founder of Microsoft, having created the company with his childhood friend Paul Allen in 1975. He’s now the founder of Breakthrough Energy, which aims to advance clean energy innovation, and TerraPower, a company developing groundbreaking nuclear energy and science technologies. He also chairs the world’s largest philanthropic organization, the Gates Foundation, and focuses on solving a variety of health challenges around the globe and here at home.  Sébastien is a research lead at OpenAI. He was previously a distinguished scientist, vice president of AI, and a colleague of mine here at Microsoft, where his work included spearheading the development of the family of small language models known as Phi. While at Microsoft, he also coauthored the discussion-provoking 2023 paper “Sparks of Artificial General Intelligence,” which presented the results of early experiments with GPT-4 conducted by a small team from Microsoft Research.    [TRANSITION MUSIC]   Here’s my conversation with Bill Gates and Sébastien Bubeck.  LEE: Bill, welcome.  BILL GATES: Thank you.  LEE: Seb …  SÉBASTIEN BUBECK: Yeah. Hi, hi, Peter. Nice to be here.  LEE: You know, one of the things that I’ve been doing just to get the conversation warmed up is to talk about origin stories, and what I mean about origin stories is, you know, what was the first contact that you had with large language models or the concept of generative AI that convinced you or made you think that something really important was happening?  And so, Bill, I think I’ve heard the story about, you know, the time when the OpenAI folks—Sam Altman, Greg Brockman, and others—showed you something, but could we hear from you what those early encounters were like and what was going through your mind?   GATES: Well, I’d been visiting OpenAI soon after it was created to see things like GPT-2 and to see the little arm they had that was trying to match human manipulation and, you know, looking at their games like Dota that they were trying to get as good as human play. And honestly, I didn’t think the language model stuff they were doing, even when they got to GPT-3, would show the ability to learn, you know, in the same sense that a human reads a biology book and is able to take that knowledge and access it not only to pass a test but also to create new medicines.  And so my challenge to them was that if their LLM could get a five on the advanced placement biology test, then I would say, OK, it took biologic knowledge and encoded it in an accessible way and that I didn’t expect them to do that very quickly but it would be profound.   And it was only about six months after I challenged them to do that, that an early version of GPT-4 they brought up to a dinner at my house, and in fact, it answered most of the questions that night very well. The one it got totally wrong, we were … because it was so good, we kept thinking, Oh, we must be wrong. It turned out it was a math weakness [LAUGHTER] that, you know, we later understood that that was an area of, weirdly, of incredible weakness of those early models. But, you know, that was when I realized, OK, the age of cheap intelligence was at its beginning.  LEE: Yeah. So I guess it seems like you had something similar to me in that my first encounters, I actually harbored some skepticism. Is it fair to say you were skeptical before that?  GATES: Well, the idea that we’ve figured out how to encode and access knowledge in this very deep sense without even understanding the nature of the encoding, …  LEE: Right.   GATES: … that is a bit weird.   LEE: Yeah.  GATES: We have an algorithm that creates the computation, but even say, OK, where is the president’s birthday stored in there? Where is this fact stored in there? The fact that even now when we’re playing around, getting a little bit more sense of it, it’s opaque to us what the semantic encoding is, it’s, kind of, amazing to me. I thought the invention of knowledge storage would be an explicit way of encoding knowledge, not an implicit statistical training.  LEE: Yeah, yeah. All right. So, Seb, you know, on this same topic, you know, I got—as we say at Microsoft—I got pulled into the tent. [LAUGHS]  BUBECK: Yes.   LEE: Because this was a very secret project. And then, um, I had the opportunity to select a small number of researchers in MSR [Microsoft Research] to join and start investigating this thing seriously. And the first person I pulled in was you.  BUBECK: Yeah.  LEE: And so what were your first encounters? Because I actually don’t remember what happened then.  BUBECK: Oh, I remember it very well. [LAUGHS] My first encounter with GPT-4 was in a meeting with the two of you, actually. But my kind of first contact, the first moment where I realized that something was happening with generative AI, was before that. And I agree with Bill that I also wasn’t too impressed by GPT-3.  I though that it was kind of, you know, very naturally mimicking the web, sort of parroting what was written there in a nice way. Still in a way which seemed very impressive. But it wasn’t really intelligent in any way. But shortly after GPT-3, there was a model before GPT-4 that really shocked me, and this was the first image generation model, DALL-E 1.  So that was in 2021. And I will forever remember the press release of OpenAI where they had this prompt of an avocado chair and then you had this image of the avocado chair. [LAUGHTER] And what really shocked me is that clearly the model kind of “understood” what is a chair, what is an avocado, and was able to merge those concepts.  So this was really, to me, the first moment where I saw some understanding in those models.   LEE: So this was, just to get the timing right, that was before I pulled you into the tent.  BUBECK: That was before. That was like a year before.  LEE: Right.   BUBECK: And now I will tell you how, you know, we went from that moment to the meeting with the two of you and GPT-4.  So once I saw this kind of understanding, I thought, OK, fine. It understands concept, but it’s still not able to reason. It cannot—as, you know, Bill was saying—it cannot learn from your document. It cannot reason.   So I set out to try to prove that. You know, this is what I was in the business of at the time, trying to prove things in mathematics. So I was trying to prove that basically autoregressive transformers could never reason. So I was trying to prove this. And after a year of work, I had something reasonable to show. And so I had the meeting with the two of you, and I had this example where I wanted to say, there is no way that an LLM is going to be able to do x.  And then as soon as I … I don’t know if you remember, Bill. But as soon as I said that, you said, oh, but wait a second. I had, you know, the OpenAI crew at my house recently, and they showed me a new model. Why don’t we ask this new model this question?   LEE: Yeah. BUBECK: And we did, and it solved it on the spot. And that really, honestly, just changed my life. Like, you know, I had been working for a year trying to say that this was impossible. And just right there, it was shown to be possible.   LEE: [LAUGHS] One of the very first things I got interested in—because I was really thinking a lot about healthcare—was healthcare and medicine.  And I don’t know if the two of you remember, but I ended up doing a lot of tests. I ran through, you know, step one and step two of the US Medical Licensing Exam. Did a whole bunch of other things. I wrote this big report. It was, you know, I can’t remember … a couple hundred pages.   And I needed to share this with someone. I didn’t … there weren’t too many people I could share it with. So I sent, I think, a copy to you, Bill. Sent a copy to you, Seb.   I hardly slept for about a week putting that report together. And, yeah, and I kept working on it. But I was far from alone. I think everyone who was in the tent, so to speak, in those early days was going through something pretty similar. All right. So I think … of course, a lot of what I put in the report also ended up being examples that made it into the book.  But the main purpose of this conversation isn’t to reminisce about [LAUGHS] or indulge in those reminiscences but to talk about what’s happening in healthcare and medicine. And, you know, as I said, we wrote this book. We did it very, very quickly. Seb, you helped. Bill, you know, you provided a review and some endorsements.  But, you know, honestly, we didn’t know what we were talking about because no one had access to this thing. And so we just made a bunch of guesses. So really, the whole thing I wanted to probe with the two of you is, now with two years of experience out in the world, what, you know, what do we think is happening today?  You know, is AI actually having an impact, positive or negative, on healthcare and medicine? And what do we now think is going to happen in the next two years, five years, or 10 years? And so I realize it’s a little bit too abstract to just ask it that way. So let me just try to narrow the discussion and guide us a little bit.   Um, the kind of administrative and clerical work, paperwork, around healthcare—and we made a lot of guesses about that—that appears to be going well, but, you know, Bill, I know we’ve discussed that sometimes that you think there ought to be a lot more going on. Do you have a viewpoint on how AI is actually finding its way into reducing paperwork?  GATES: Well, I’m stunned … I don’t think there should be a patient-doctor meeting where the AI is not sitting in and both transcribing, offering to help with the paperwork, and even making suggestions, although the doctor will be the one, you know, who makes the final decision about the diagnosis and whatever prescription gets done.   It’s so helpful. You know, when that patient goes home and their, you know, son who wants to understand what happened has some questions, that AI should be available to continue that conversation. And the way you can improve that experience and streamline things and, you know, involve the people who advise you. I don’t understand why that’s not more adopted, because there you still have the human in the loop making that final decision.  But even for, like, follow-up calls to make sure the patient did things, to understand if they have concerns and knowing when to escalate back to the doctor, the benefit is incredible. And, you know, that thing is ready for prime time. That paradigm is ready for prime time, in my view.  LEE: Yeah, there are some good products, but it seems like the number one use right now—and we kind of got this from some of the previous guests in previous episodes—is the use of AI just to respond to emails from patients. [LAUGHTER] Does that make sense to you?  BUBECK: Yeah. So maybe I want to second what Bill was saying but maybe take a step back first. You know, two years ago, like, the concept of clinical scribes, which is one of the things that we’re talking about right now, it would have sounded, in fact, it sounded two years ago, borderline dangerous. Because everybody was worried about hallucinations. What happened if you have this AI listening in and then it transcribes, you know, something wrong?  Now, two years later, I think it’s mostly working. And in fact, it is not yet, you know, fully adopted. You’re right. But it is in production. It is used, you know, in many, many places. So this rate of progress is astounding because it wasn’t obvious that we would be able to overcome those obstacles of hallucination. It’s not to say that hallucinations are fully solved. In the case of the closed system, they are.   Now, I think more generally what’s going on in the background is that there is something that we, that certainly I, underestimated, which is this management overhead. So I think the reason why this is not adopted everywhere is really a training and teaching aspect. People need to be taught, like, those systems, how to interact with them.  And one example that I really like, a study that recently appeared where they tried to use ChatGPT for diagnosis and they were comparing doctors without and with ChatGPT (opens in new tab). And the amazing thing … so this was a set of cases where the accuracy of the doctors alone was around 75%. ChatGPT alone was 90%. So that’s already kind of mind blowing. But then the kicker is that doctors with ChatGPT was 80%.   Intelligence alone is not enough. It’s also how it’s presented, how you interact with it. And ChatGPT, it’s an amazing tool. Obviously, I absolutely love it. But it’s not … you don’t want a doctor to have to type in, you know, prompts and use it that way.  It should be, as Bill was saying, kind of running continuously in the background, sending you notifications. And you have to be really careful of the rate at which those notifications are being sent. Because if they are too frequent, then the doctor will learn to ignore them. So you have to … all of those things matter, in fact, at least as much as the level of intelligence of the machine.  LEE: One of the things I think about, Bill, in that scenario that you described, doctors do some thinking about the patient when they write the note. So, you know, I’m always a little uncertain whether it’s actually … you know, you wouldn’t necessarily want to fully automate this, I don’t think. Or at least there needs to be some prompt to the doctor to make sure that the doctor puts some thought into what happened in the encounter with the patient. Does that make sense to you at all?  GATES: At this stage, you know, I’d still put the onus on the doctor to write the conclusions and the summary and not delegate that.  The tradeoffs you make a little bit are somewhat dependent on the situation you’re in. If you’re in Africa, So, yes, the doctor’s still going to have to do a lot of work, but just the quality of letting the patient and the people around them interact and ask questions and have things explained, that alone is such a quality improvement. It’s mind blowing.   LEE: So since you mentioned, you know, Africa—and, of course, this touches on the mission and some of the priorities of the Gates Foundation and this idea of democratization of access to expert medical care—what’s the most interesting stuff going on right now? Are there people and organizations or technologies that are impressing you or that you’re tracking?  GATES: Yeah. So the Gates Foundation has given out a lot of grants to people in Africa doing education, agriculture but more healthcare examples than anything. And the way these things start off, they often start out either being patient-centric in a narrow situation, like, OK, I’m a pregnant woman; talk to me. Or, I have infectious disease symptoms; talk to me. Or they’re connected to a health worker where they’re helping that worker get their job done. And we have lots of pilots out, you know, in both of those cases.   The dream would be eventually to have the thing the patient consults be so broad that it’s like having a doctor available who understands the local things.   LEE: Right.   GATES: We’re not there yet. But over the next two or three years, you know, particularly given the worsening financial constraints against African health systems, where the withdrawal of money has been dramatic, you know, figuring out how to take this—what I sometimes call “free intelligence”—and build a quality health system around that, we will have to be more radical in low-income countries than any rich country is ever going to be.   LEE: Also, there’s maybe a different regulatory environment, so some of those things maybe are easier? Because right now, I think the world hasn’t figured out how to and whether to regulate, let’s say, an AI that might give a medical diagnosis or write a prescription for a medication.  BUBECK: Yeah. I think one issue with this, and it’s also slowing down the deployment of AI in healthcare more generally, is a lack of proper benchmark. Because, you know, you were mentioning the USMLE [United States Medical Licensing Examination], for example. That’s a great test to test human beings and their knowledge of healthcare and medicine. But it’s not a great test to give to an AI.  It’s not asking the right questions. So finding what are the right questions to test whether an AI system is ready to give diagnosis in a constrained setting, that’s a very, very important direction, which to my surprise, is not yet accelerating at the rate that I was hoping for.  LEE: OK, so that gives me an excuse to get more now into the core AI tech because something I’ve discussed with both of you is this issue of what are the right tests. And you both know the very first test I give to any new spin of an LLM is I present a patient, the results—a mythical patient—the results of my physical exam, my mythical physical exam. Maybe some results of some initial labs. And then I present or propose a differential diagnosis. And if you’re not in medicine, a differential diagnosis you can just think of as a prioritized list of the possible diagnoses that fit with all that data. And in that proposed differential, I always intentionally make two mistakes.  I make a textbook technical error in one of the possible elements of the differential diagnosis, and I have an error of omission. And, you know, I just want to know, does the LLM understand what I’m talking about? And all the good ones out there do now. But then I want to know, can it spot the errors? And then most importantly, is it willing to tell me I’m wrong, that I’ve made a mistake?   That last piece seems really hard for AI today. And so let me ask you first, Seb, because at the time of this taping, of course, there was a new spin of GPT-4o last week that became overly sycophantic. In other words, it was actually prone in that test of mine not only to not tell me I’m wrong, but it actually praised me for the creativity of my differential. [LAUGHTER] What’s up with that?  BUBECK: Yeah, I guess it’s a testament to the fact that training those models is still more of an art than a science. So it’s a difficult job. Just to be clear with the audience, we have rolled back that [LAUGHS] version of GPT-4o, so now we don’t have the sycophant version out there.  Yeah, no, it’s a really difficult question. It has to do … as you said, it’s very technical. It has to do with the post-training and how, like, where do you nudge the model? So, you know, there is this very classical by now technique called RLHF [reinforcement learning from human feedback], where you push the model in the direction of a certain reward model. So the reward model is just telling the model, you know, what behavior is good, what behavior is bad.  But this reward model is itself an LLM, and, you know, Bill was saying at the very beginning of the conversation that we don’t really understand how those LLMs deal with concepts like, you know, where is the capital of France located? Things like that. It is the same thing for this reward model. We don’t know why it says that it prefers one output to another, and whether this is correlated with some sycophancy is, you know, something that we discovered basically just now. That if you push too hard in optimization on this reward model, you will get a sycophant model.  So it’s kind of … what I’m trying to say is we became too good at what we were doing, and we ended up, in fact, in a trap of the reward model.  LEE: I mean, you do want … it’s a difficult balance because you do want models to follow your desires and …  BUBECK: It’s a very difficult, very difficult balance.  LEE: So this brings up then the following question for me, which is the extent to which we think we’ll need to have specially trained models for things. So let me start with you, Bill. Do you have a point of view on whether we will need to, you know, quote-unquote take AI models to med school? Have them specially trained? Like, if you were going to deploy something to give medical care in underserved parts of the world, do we need to do something special to create those models?  GATES: We certainly need to teach them the African languages and the unique dialects so that the multimedia interactions are very high quality. We certainly need to teach them the disease prevalence and unique disease patterns like, you know, neglected tropical diseases and malaria. So we need to gather a set of facts that somebody trying to go for a US customer base, you know, wouldn’t necessarily have that in there.  Those two things are actually very straightforward because the additional training time is small. I’d say for the next few years, we’ll also need to do reinforcement learning about the context of being a doctor and how important certain behaviors are. Humans learn over the course of their life to some degree that, I’m in a different context and the way I behave in terms of being willing to criticize or be nice, you know, how important is it? Who’s here? What’s my relationship to them?   Right now, these machines don’t have that broad social experience. And so if you know it’s going to be used for health things, a lot of reinforcement learning of the very best humans in that context would still be valuable. Eventually, the models will, having read all the literature of the world about good doctors, bad doctors, it’ll understand as soon as you say, “I want you to be a doctor diagnosing somebody.” All of the implicit reinforcement that fits that situation, you know, will be there. LEE: Yeah. GATES: And so I hope three years from now, we don’t have to do that reinforcement learning. But today, for any medical context, you would want a lot of data to reinforce tone, willingness to say things when, you know, there might be something significant at stake.  LEE: Yeah. So, you know, something Bill said, kind of, reminds me of another thing that I think we missed, which is, the context also … and the specialization also pertains to different, I guess, what we still call “modes,” although I don’t know if the idea of multimodal is the same as it was two years ago. But, you know, what do you make of all of the hubbub around—in fact, within Microsoft Research, this is a big deal, but I think we’re far from alone—you know, medical images and vision, video, proteins and molecules, cell, you know, cellular data and so on.  BUBECK: Yeah. OK. So there is a lot to say to everything … to the last, you know, couple of minutes. Maybe on the specialization aspect, you know, I think there is, hiding behind this, a really fundamental scientific question of whether eventually we have a singular AGI [artificial general intelligence] that kind of knows everything and you can just put, you know, explain your own context and it will just get it and understand everything.  That’s one vision. I have to say, I don’t particularly believe in this vision. In fact, we humans are not like that at all. I think, hopefully, we are general intelligences, yet we have to specialize a lot. And, you know, I did myself a lot of RL, reinforcement learning, on mathematics. Like, that’s what I did, you know, spent a lot of time doing that. And I didn’t improve on other aspects. You know, in fact, I probably degraded in other aspects. [LAUGHTER] So it’s … I think it’s an important example to have in mind.  LEE: I think I might disagree with you on that, though, because, like, doesn’t a model have to see both good science and bad science in order to be able to gain the ability to discern between the two?  BUBECK: Yeah, no, that absolutely. I think there is value in seeing the generality, in having a very broad base. But then you, kind of, specialize on verticals. And this is where also, you know, open-weights model, which we haven’t talked about yet, are really important because they allow you to provide this broad base to everyone. And then you can specialize on top of it.  LEE: So we have about three hours of stuff to talk about, but our time is actually running low. BUBECK: Yes, yes, yes.   LEE: So I think I want … there’s a more provocative question. It’s almost a silly question, but I need to ask it of the two of you, which is, is there a future, you know, where AI replaces doctors or replaces, you know, medical specialties that we have today? So what does the world look like, say, five years from now?  GATES: Well, it’s important to distinguish healthcare discovery activity from healthcare delivery activity. We focused mostly on delivery. I think it’s very much within the realm of possibility that the AI is not only accelerating healthcare discovery but substituting for a lot of the roles of, you know, I’m an organic chemist, or I run various types of assays. I can see those, which are, you know, testable-output-type jobs but with still very high value, I can see, you know, some replacement in those areas before the doctor.   The doctor, still understanding the human condition and long-term dialogues, you know, they’ve had a lifetime of reinforcement of that, particularly when you get into areas like mental health. So I wouldn’t say in five years, either people will choose to adopt it, but it will be profound that there’ll be this nearly free intelligence that can do follow-up, that can help you, you know, make sure you went through different possibilities.  And so I’d say, yes, we’ll have doctors, but I’d say healthcare will be massively transformed in its quality and in efficiency by AI in that time period.  LEE: Is there a comparison, useful comparison, say, between doctors and, say, programmers, computer programmers, or doctors and, I don’t know, lawyers?  GATES: Programming is another one that has, kind of, a mathematical correctness to it, you know, and so the objective function that you’re trying to reinforce to, as soon as you can understand the state machines, you can have something that’s “checkable”; that’s correct. So I think programming, you know, which is weird to say, that the machine will beat us at most programming tasks before we let it take over roles that have deep empathy, you know, physical presence and social understanding in them.  LEE: Yeah. By the way, you know, I fully expect in five years that AI will produce mathematical proofs that are checkable for validity, easily checkable, because they’ll be written in a proof-checking language like Lean or something but will be so complex that no human mathematician can understand them. I expect that to happen.   I can imagine in some fields, like cellular biology, we could have the same situation in the future because the molecular pathways, the chemistry, biochemistry of human cells or living cells is as complex as any mathematics, and so it seems possible that we may be in a state where in wet lab, we see, Oh yeah, this actually works, but no one can understand why.  BUBECK: Yeah, absolutely. I mean, I think I really agree with Bill’s distinction of the discovery and the delivery, and indeed, the discovery’s when you can check things, and at the end, there is an artifact that you can verify. You know, you can run the protocol in the wet lab and see [if you have] produced what you wanted. So I absolutely agree with that.   And in fact, you know, we don’t have to talk five years from now. I don’t know if you know, but just recently, there was a paper that was published on a scientific discovery using o3- mini (opens in new tab). So this is really amazing. And, you know, just very quickly, just so people know, it was about this statistical physics model, the frustrated Potts model, which has to do with coloring, and basically, the case of three colors, like, more than two colors was open for a long time, and o3 was able to reduce the case of three colors to two colors.   LEE: Yeah.  BUBECK: Which is just, like, astounding. And this is not … this is now. This is happening right now. So this is something that I personally didn’t expect it would happen so quickly, and it’s due to those reasoning models.   Now, on the delivery side, I would add something more to it for the reason why doctors and, in fact, lawyers and coders will remain for a long time, and it’s because we still don’t understand how those models generalize. Like, at the end of the day, we are not able to tell you when they are confronted with a really new, novel situation, whether they will work or not.  Nobody is able to give you that guarantee. And I think until we understand this generalization better, we’re not going to be willing to just let the system in the wild without human supervision.  LEE: But don’t human doctors, human specialists … so, for example, a cardiologist sees a patient in a certain way that a nephrologist …  BUBECK: Yeah. LEE: … or an endocrinologist might not. BUBECK: That’s right. But another cardiologist will understand and, kind of, expect a certain level of generalization from their peer. And this, we just don’t have it with AI models. Now, of course, you’re exactly right. That generalization is also hard for humans. Like, if you have a human trained for one task and you put them into another task, then you don’t … you often don’t know. LEE: OK. You know, the podcast is focused on what’s happened over the last two years. But now, I’d like one provocative prediction about what you think the world of AI and medicine is going to be at some point in the future. You pick your timeframe. I don’t care if it’s two years or 20 years from now, but, you know, what do you think will be different about AI in medicine in that future than today?  BUBECK: Yeah, I think the deployment is going to accelerate soon. Like, we’re really not missing very much. There is this enormous capability overhang. Like, even if progress completely stopped, with current systems, we can do a lot more than what we’re doing right now. So I think this will … this has to be realized, you know, sooner rather than later.  And I think it’s probably dependent on these benchmarks and proper evaluation and tying this with regulation. So these are things that take time in human society and for good reason. But now we already are at two years; you know, give it another two years and it should be really …   LEE: Will AI prescribe your medicines? Write your prescriptions?  BUBECK: I think yes. I think yes.  LEE: OK. Bill?  GATES: Well, I think the next two years, we’ll have massive pilots, and so the amount of use of the AI, still in a copilot-type mode, you know, we should get millions of patient visits, you know, both in general medicine and in the mental health side, as well. And I think that’s going to build up both the data and the confidence to give the AI some additional autonomy. You know, are you going to let it talk to you at night when you’re panicked about your mental health with some ability to escalate? And, you know, I’ve gone so far as to tell politicians with national health systems that if they deploy AI appropriately, that the quality of care, the overload of the doctors, the improvement in the economics will be enough that their voters will be stunned because they just don’t expect this, and, you know, they could be reelected [LAUGHTER] just on this one thing of fixing what is a very overloaded and economically challenged health system in these rich countries.  You know, my personal role is going to be to make sure that in the poorer countries, there isn’t some lag; in fact, in many cases, that we’ll be more aggressive because, you know, we’re comparing to having no access to doctors at all. And, you know, so I think whether it’s India or Africa, there’ll be lessons that are globally valuable because we need medical intelligence. And, you know, thank god AI is going to provide a lot of that.  LEE: Well, on that optimistic note, I think that’s a good way to end. Bill, Seb, really appreciate all of this.   I think the most fundamental prediction we made in the book is that AI would actually find its way into the practice of medicine, and I think that that at least has come true, maybe in different ways than we expected, but it’s come true, and I think it’ll only accelerate from here. So thanks again, both of you.  [TRANSITION MUSIC]  GATES: Yeah. Thanks, you guys.  BUBECK: Thank you, Peter. Thanks, Bill.  LEE: I just always feel such a sense of privilege to have a chance to interact and actually work with people like Bill and Sébastien.    With Bill, I’m always amazed at how practically minded he is. He’s really thinking about the nuts and bolts of what AI might be able to do for people, and his thoughts about underserved parts of the world, the idea that we might actually be able to empower people with access to expert medical knowledge, I think is both inspiring and amazing.   And then, Seb, Sébastien Bubeck, he’s just absolutely a brilliant mind. He has a really firm grip on the deep mathematics of artificial intelligence and brings that to bear in his research and development work. And where that mathematics takes him isn’t just into the nuts and bolts of algorithms but into philosophical questions about the nature of intelligence.   One of the things that Sébastien brought up was the state of evaluation of AI systems. And indeed, he was fairly critical in our conversation. But of course, the world of AI research and development is just moving so fast, and indeed, since we recorded our conversation, OpenAI, in fact, released a new evaluation metric that is directly relevant to medical applications, and that is something called HealthBench. And Microsoft Research also released a new evaluation approach or process called ADeLe.   HealthBench and ADeLe are examples of new approaches to evaluating AI models that are less about testing their knowledge and ability to pass multiple-choice exams and instead are evaluation approaches designed to assess how well AI models are able to complete tasks that actually arise every day in typical healthcare or biomedical research settings. These are examples of really important good work that speak to how well AI models work in the real world of healthcare and biomedical research and how well they can collaborate with human beings in those settings.  You know, I asked Bill and Seb to make some predictions about the future. You know, my own answer, I expect that we’re going to be able to use AI to change how we diagnose patients, change how we decide treatment options.   If you’re a doctor or a nurse and you encounter a patient, you’ll ask questions, do a physical exam, you know, call out for labs just like you do today, but then you’ll be able to engage with AI based on all of that data and just ask, you know, based on all the other people who have gone through the same experience, who have similar data, how were they diagnosed? How were they treated? What were their outcomes? And what does that mean for the patient I have right now? Some people call it the “patients like me” paradigm. And I think that’s going to become real because of AI within our lifetimes. That idea of really grounding the delivery in healthcare and medical practice through data and intelligence, I actually now don’t see any barriers to that future becoming real.  [THEME MUSIC]  I’d like to extend another big thank you to Bill and Sébastien for their time. And to our listeners, as always, it’s a pleasure to have you along for the ride. I hope you’ll join us for our remaining conversations, as well as a second coauthor roundtable with Carey and Zak.   Until next time.   [MUSIC FADES]

The nightmare was real, the situation was not.Revenge of the Savage Planet, an adventure spread across a number of distant — and quite savage! — planets, invites nonlinear exploration. To complete its missions and discover all of its secrets, you must leap into an unknown where the otherworldly flora, fauna, and even the inorganic material are primed to kill you. So, shortly after assembling an underwater scooter that allowed my robot sidekick to whisk me through the depths of alien oceans, I descended into a series of caverns under the Zenithian Rift to see what was going on down there. The specters of death I encountered below weren’t even designed to haunt me.In Raccoon Logic’s sequel to Journey to the Savage Planet, players are tasked with scanning every object in every nook and cranny to assemble an exhaustive log of materials located on each planet. At first, the task is a walk in the (overgrown killer) park: find a tree, scan a tree. Find a slobbering beastie, scan a slobbering beastie. But a counter on the map charting your scannables becomes the most daunting subtask — can I really find every single micro scannable? I found myself longing after completing the core missions. To really 100% this, there was even more reason to venture into the most uninviting spaces, including a dark underwater cave on Zenithian Rift that absolutely did not look like it contained any scannable items. But I couldn’t not go in there.It took about two seconds for me to realize… I had made a horrible mistake. While the cave was easily accessible from the water, there were no enemy or collectible breadcrumbs to suggest this was a place the folks at Raccoon Logic intended for me to. I was lured in by curiosity, but the joy of discovery in Revenge of the Savage Planet got the best of me. Now I was stuck. I had stumbled into a graphical anomaly, an in-game black hole that had an entrance but no apparent exit. In Revenge of the Savage Planet, you can’t beam back to starting locations on the fly or off yourself in order to respawn from your last save. In a clever but likely divisive design choice, the game forces you to navigate to transporters spread across the worlds in order to beam off to your next desired location, which forces traversal and new encounters. But it meant that while bumbling around in the dark, hoping to find a way out of my watery grave, I couldn’t simply die and move on. I was actually trapped, and in a scenario I haven’t experienced in quite some time, feeling IRL like I was actually trapped.I already don’t do well with underwater levels out of an intense fear of drowning. Luckily for me, most games will throw me the lifeline of a visual countdown to illustrate oxygen levels, ensuring (1) I surface in time and (2) I don’t hyperventilate over the stress of surfacing in time. Revenge of the Savage Planet doesn’t need that because there’s no punishment for enjoying the waters; you’re already in a spacesuit and the challenges you encounter via underwater scooter require a bunch of time-intensive back and forth. Doing it all on limited air would simply not be fun. But that meant, stuck in this tight underwater cave, I would never die. I was in limbo. Or maybe I was in hell.I spent far too long searching for a route out. Streaks of light bled in from a theoretical escape that I could never reach — any time I thought I was close, I bumped into a new rock and found myself jetting in the opposite direction. Not since I watched The Rescue, the riveting-yet-terrifying documentary about the team of divers who squeezed through cave passageways to free 12 trapped Thai soccer players, had my apparent claustrophobia had its way with my nerves. I can’t quite explain why I pushed myself over the edge to find an in-game solution to this unintentional challenge, except to say that I really wanted to do a good job at Revenge of the Savage Planet.Most glitches are considered errors by programmers, annoyances by players, and occasionally shortcuts for the speedrunner crowd. Revenge of the Savage Planet’s death cave might fall into the first two categories, but it’s a harrowing experience I ultimately appreciated, a unique screw up that could only happen in a game. I have never felt truly trapped in a film, despite the best efforts of 3D stereoscopic effects and 4DX rumble seats. After finally rebooting Revenge of the Savage Planet, I had to give myself a few minutes to let my heart rate die down before I grabbed the controller. But I got right back to it. Sure, this was a glitch, but in a game where exploration is everything, leaping into a true unknown — one that the creators of the game clearly didn’t intend me to find — was its own form of success.Revenge of the Savage Planet is currently available for PC, Playstation, and Xbox, and it’s currently on Game Pass.See More:

When YARA + DAVINA became mothers within a month of each other, they didn't step back from their practice – they stepped forward together. The socially engaged artist duo began collaborating as a job share, determined to remain visible in an industry that too often sidelines mothers. Their work – which spans everything from poetry to bronze public sculptures – is rooted in play, accessibility and political intent, often exploring themes of care, identity, and who art is really for. In this candid Q&A, they reflect on making time (and peace) with imperfection, the pressures of doing it all, and why motherhood has only deepened their creative drive. How has motherhood influenced your creative process or career choices? Brian Sewel, the art critic, said in 2008 that "Female artists fade away in their late 20s or 30s. Maybe it's something to do with bearing children". Yes, motherhood has had a profound influence on our career choices. We became a duo after becoming mothers, and we had a deep wellspring of desire to nurture our babies and also nurture our art practice, not letting it fade away. Being a mum gave us a wider range of being and a deeper depth of what it means to be human. It filled us with more ideas, not less… we had more determination and more creative ideas than ever. In 2016, our hack on motherhood was to start collaborating as a duo as a 'job share'. We had been friends and admirers of each other's work for 11 years prior, and both of us became parents within a month of each other. We realised we both wanted to be present mothers but also visible artists. We literally started working together so we could work part-time but have a full-time practice between us. We are driven to make powerful, playful and fun contemporary art, alongside being mothers, to challenge ideas like Tracy Emin, who said, "There are good artists that have children. They are called men." We are good artists, and we are not only women; we are mothers! Photo credit: Alice Horsley What's been the biggest challenge in balancing creativity and caregiving? Time and some guilt! Quite literally, there is little time to parent and make art. But we were both determined to be part-time mums, part-time artists and full-time friends. This duo works because we both understand the limitations of our free time: we are always there to step up when the other needs a break, has sick children, or craves space for ourselves. It was almost like an intuitive dance, where we had become in tune with each other's outside demands and rhythm. As artists, we often have to travel extensively for work, which can put considerable pressure on our partners and be unsettling for our children. So guilt slips in every now and then. Luckily, we both support each other in those times and remind each other that to parent and care for others effectively, it is essential that we make time for our art practice and thus ourselves. With time, we hope our children will respect and understand the role art played in our lives and why we had to find a balance that worked for us as creative beings. We work together every weekday, and we always find ourselves talking and supporting each other with our parenting alongside making work. We both deeply feel that we were put on this planet to create great art, to push the boundaries of what art is and can be, and with whom it can be made and for whom it can be made. And we can do this while raising children. Photo credit: Alice Horsley Have you felt pressure to 'do it all,' and how do you navigate that? Yes, we feel it all the time, from ourselves as much as from society! One of our mottos that we tell ourselves is 'Good Enough'. We have talked about getting these as matching tattoos ( we love to wear matching outfits). Essentially, it is impossible to do everything really well. We need to prioritise what things need to be brilliant and what things can just be good enough. These priorities shift daily, creating an interplay between our personal and professional lives. Being a duo allows us to pick up each other's slack when needed. Photo credit: Nick Turpin What changes would you like to see in the creative industry to better support mothers? In 2016, we participated in a British Council residency at Portland State University titled 'Motherhood: A Social Practice'. We explored motherhood in the creative world, and our passion was to create more family-focused residencies and secure funding to support childcare. We wanted programmers to consider simple things, such as family-accessible residences, and work around term times. Things have dramatically changed since then, but we need to see more changes. People like Lizzie Humber and her daylight collective are doing amazing things, thinking about programming parent-accessible daytime culture. When we live in a time where Evening Standard art critic Brian Sewell says things like, "Only men are capable of aesthetic greatness.", women, in general, have a tough time, never mind mums! We are passionate not only about supporting artist mothers but also about working with and for mothers. Our public artwork, WOMAN - WHOLE, was created alongside, with, and for mothers on the Regents Park Estate, commissioned by ODAC, Camden. We subverted the idea of manholes, creating a series of bronze-cast covers embedded in the pavements of Camden. These permanent public artworks playfully remind us that, as women, we are whole. Photo credit: Hugo Glendinning

More enterprises have moved to pay-for-performance salary and promotion models that measure progress toward goals -- but how do you measure goals for a maintenance programmer who barrels through a request backlog but delivers marginal value for the business, or for a business analyst whose success is predicated on forging intangibles like trust and cooperation with users so things can get done? It’s an age-old question facing companies, now that 77% of them use some type of pay-for-performance model. What are some popular pay-for-performance use cases? A factory doing piece work that pays employees based upon the number of items they assemble. A call center that pays agents based on how many calls they complete per day. A bank teller who gets rewarded for how many customers they sign up for credit cards. An IT project team that gets a bonus for completing a major project ahead of schedule. The IT example differs from the others, because it depends on team and not individual execution, but there nevertheless is something tangible to measure. The other use cases are more clearcut -- although they don’t account for pieces in the plant that were poorly assembled in haste to make quota and had to be reworked, or a call center agent who pushes calls off to someone else so they can end their calls in six minutes or less, or the teller who signs up X number of customers for credit cards, although two-thirds of them never use the credit card they signed up for. Related:In short, there are flaws in pay-for-performance models just as there are in other types of compensation models that organizations use. So, what’s the best path for IT for CIOs who want to implement pay for performance? One approach is to measure pay for performance based upon four key elements: hard results, effort, skill, and communications. The mix of these elements will vary, depending on the type of position each IT staff member performs. Here are two examples of pay per performance by position: 1. Computer maintenance programmers and help desk specialists Historically, IT departments have used hard numbers like how many open requests a computer maintenance programmer has closed, or how many calls a help desk employee has solved. There is merit in using hard results, and hard results should be factored into performance reviews for these individuals -- but hard numbers don’t tell the whole story.  For example, how many times has a help desk agent gone the extra mile with a difficult user or software bug, taking the time to see the entire process through until it is thoroughly solved? lf the issue was of a global nature, did the Help Desk agent follow up by letting others who use the application know that a bug was fixed? For the maintenance programmer who has completed the most open requests, which of these requests really solved a major business pain point? For both help desk and maintenance programming employees, were the changes and fixes properly documented and communicated to everyone with a need to know? And did these employees demonstrate the skills needed to solve their issues? Related:It’s difficult to capture hard results on elements like effort, communication and skills, but one way to go about it is to survey user departments on individual levels of service and effectiveness. From there, it’s up to IT managers to determinate the “mix” of hard results, effort, communication and skills on which the employee will be evaluated, and to communicate upfront to the employee what the pay for performance assessment will be based on. 2. Business analysts and trainers Business analysts and trainers are difficult to quantify in pay for performance models because so much of their success depends upon other people. A business analyst can know everything there is to know about a particular business area and its systems, but if the analyst is working with unresponsive users, or lacks the soft skills needed to communicate with users, the pay for performance can’t be based upon the technology skillset alone.  Related:IT trainers face a somewhat different dilemma when it  comes to performance evaluation: they can produce the training that new staff members need before staff is deployed on key projects,  but if a project gets delayed and this causes trainees to lose the knowledge that they learned, there is little the trainer can do aside from offering a refresher course. Can pay for performance be used for positions like these? It’s a mixed answer. Yes, pay per performance can be used for trainers, based upon how many individuals the trainer trains and how many new courses the trainer obtains or develops. These are the hard results. However, since so much of training’s execution depends upon other people downstream, like project managers who must start projects on time so new skills aren’t lost,  managers of training should also consider pay for performance elements such as effort (has the trainer consistently gone the extra mile to make things work?), skills and communication.  In sum, for both business analysts and trainers, there are hard results that can be factored into a pay for performance formula, but there is also a need to survey each position’s “customers” -- those individuals (and their managers) who utilized the business analyst’s or trainer’s skills and products to accomplish their respective objectives in projects and training. Were these user-customers satisfied?  Summary Remarks The value that IT employees contribute to overall IT and to the business at large is a combination of tangible and intangible results. Pay for performance models are well suited to gauge tangible outcomes, but they fall short when it comes to the intangibles that could be just as important. Many years ago, when Pat Riley was coaching the Los Angeles Lakers, an interviewer asked what type of metrics he used when he measured the effectiveness of individual players on the basketball court. Was it the number of points, rebounds, or assists? Riley said he used an “effort" index. For example, how many times did a player go up to get a rebound, even if he didn’t end up with the ball? Riley said the effort individual players exhibited mattered, because even if they didn’t get the rebound, they were creating situations so someone else on the team could. IT is similar. It’s why OKR International, a performance consultancy, stated “Intangibles often create or destroy value quietly -- until their impact is too big to ignore. In the long run, they are the unseen levers that determine whether strategy thrives or withers.”  What CIOs and IT leadership can do when they use pay for performance is to assure that hard results, effort, communications and skills are appropriately blended for each IT staff position, and its responsibilities and realities -- because you can’t attach a numerical measurement to everything -- but you can observe visible changes that begin to manifest when a business analyst turns around what has been a hostile relationship with a user department and you begin to get things done. 

The shoehorning of AI into everything has programmers at Amazon feeling less like the tedious parts of their jobs are being smoothly automated, and more like their work is beginning to resemble the drudgery of toiling away in one of the e-commerce giant's vast warehouses.That's the bleak picture painted in new reporting from the New York Times, in which Amazon leadership — as is the case at so many other companies — is convinced that AI will marvelously jack up productivity. Tasked with conjuring the tech's mystic properties, of course, are our beleaguered keyboard-clackers.Today, there's no shortage of coding AI assistants to choose from. Google and Meta are making heavy use of them, as is Microsoft. Satya Nadella, CEO of the Redmond giant, estimates that as much as 30 percent of the company's code is now written with AI. If Amazon's to keep up with the competition, it needs to follow suit. CEO Andy Jassy echoed this in a recent letter to shareholders, cited by the NYT, emphasizing the need to give customers what they want as "quickly as possible," before upholding programming as a field in which AI would "change the norms."And that it has — though this is less due to the merits of AI and more the result of the over-eager opportunism of the company's management. Three Amazon engineers told the NYT that their bosses have increasingly pushed them to use AI in their work over the past year. And with that came increased output goals and even tighter deadlines. One engineer said that his team was reduced to roughly half the size it was last year — but it was still expected to produce the same amount of code by using AI.In short, new automating technology is being used to justify placing increased demands at their jobs."Things look like a speed-up for knowledge workers," Lawrence Katz, a labor economist at Harvard University, told the NYT, citing ongoing research. "There is a sense that the employer can pile on more stuff."Adopting AI was ostensibly optional for the Amazon programmers, but the choice was all but made for them. One engineer told the newspaper that they're now expected to finish building new website features in just a few days, whereas before they had several weeks. This ludicrous ramp up is only made possible by using AI to automate some of the coding, and comes at the expense of quality: there's less time for consulting with colleagues to get feedback and bounce ideas around.Above all, AI is sapping all the joy out of their profession. AI-amalgamated code requires extensive double checking — a prominent critique that can't be ignored here and is one of the main reasons skeptics question whether these programming assistants actually produce gains in efficiency. And when you're reduced to proofreading a machine, there's little room for creativity, and an even more diminished sense of control."It's more fun to write code than to read code," Simon Willison, a programmer and blogger who's both an enthusiast of AI and a frequent critic of the tech, told the NYT, playing devil's advocate. "If you're told you have to do a code review, it's never a fun part of the job. When you're working with these tools, it's most of the job."Amazon, for its part, maintains that it conducts regular reviews to ensure that its teams are adequately staffed. "We'll continue to adapt how we incorporate Gen AI into our processes," an Amazon spokesman told the NYT.More on AI: AI Is Replacing Women's Jobs SpecificallyShare This Article

Game Designer (gn) - Combat & ControlsKeen GamesFrankfurt am Main 60594 DE1 hour agoApplyJoin a team of super dedicated and driven people!Keen Games is an independent developer and publisher with a long history of developing games for PC, consoles, and handhelds. Working at Keen Games means working with super experienced, passionate, and humble people.In 2024, we successfully released Enshrouded in Early Access and are now preparing for our 1.0 launch. We're looking for a Game Designer who’s passionate about responsive combat, fluid controls, and engaging class gameplay. You actively seek out player and team feedback and use it to improve the moment-to-moment experience. From fine-tuning combat feel to refining enemy encounters, you’ll help us deliver a polished and satisfying game for 1.0 and beyond.Your Job Opportunity as a Game Designer!Are you ready to shape thrilling combat experiences and make every battle feel unforgettable? Join our experienced design team at Keen Games and take on the challenge of shaping the future of gaming!What you’ll do:You help shape and refine core combat experiences - including player controls, enemy AI, class gameplay, and encounter setups in the world editorYou collaborate with our 3Cs programmers and other designers to improve responsiveness, snappiness, and overall combat feelYou develop innovative gameplay systems and evolve existing ones while aligning with genre conventions for frictionless and engaging long-term playYou grow and learn together alongside our dynamic projects to create the best possible experience for playersWhat you bring to the adventure:Know how to make a game shine!You’re experienced working on both moment-to-moment gameplay (like player controls polishing and combat tuning)Keep Players coming back for more! You are experienced in combat balancing and understand how to make combat feel exciting, responsive, and varied through iteration and attention to detailEmpower players! You have the ability to design meaningful, engaging player choices in a sandbox environment.Work within our team! You have strong collaboration and communication skills to work seamlessly with diverse teams and ensure your design visions and new systems are clearly understood and realizedCollaborate effectively! You have the adaptability to iterate based on community feedback and internal playtests.What we have to offer:Be part of a dynamic and highly creative team that inspires innovationEnjoy a competitive salary that recognizes your talentGrow your skills and career with exciting development opportunitiesRecharge with paid vacation days - balance matters!Stay healthy and covered with comprehensive health care and benefitsCelebrate success with epic events and unforgettable partiesWe value your personality and passion!We prioritize personality and passion over formal education. You'll be part of a supportive, diverse team where creativity and independence are celebrated. Create Your Profile — Game companies can contact you with their relevant job openings. Apply

In Unity 2021 LTS, UI Toolkit offers a collection of features, resources, and tools to help you build and debug adaptive runtime UIs on a wide range of game applications and Editor extensions. Its intuitive workflow enables Unity creators in different roles – artists, programmers, and designers alike – to get started with UI development as quickly as possible.See our earlier blog post for an explanation of UI Toolkit’s main benefits, such as enhanced scalability and performance, already being leveraged by studios like Mechanistry for their game, Timberborn.While Unity UI remains the go-to solution for positioning and lighting UI in a 3D world or integrating with other Unity systems, UI Toolkit for runtime UI can already benefit game productions seeking performance and scalability as of Unity 2021 LTS. It’s particularly effective for Screen Space – Overlay UI, and scales well on a variety of screen resolutions.That’s why we’re excited to announce two new learning resources to better support UI development with UI Toolkit:UI Toolkit sample – Dragon Crashers: The demo is now available to download for free from the Asset Store.User interface design and implementation in Unity: This free e-book can be download from hereRead on to learn about some key features part of the UI Toolkit sample project.The UI Toolkit sample demonstrates how you can leverage UI Toolkit for your own applications. This demo involves a full-featured interface over a slice of the 2D project Dragon Crashers, a mini RPG, using the Unity 2021 LTS UI Toolkit workflow at runtime.Some of the actions illustrated in the sample project show you how to:Style with selectors in Unity style sheet (USS) files and use UXML templatesCreate custom controls, such as a circular meter or tabbed viewsCustomize the appearance of elements like sliders and toggle buttonsUse Render Texture for UI overlay effects, USS animations, seasonal themes, and moreTo try out the project after adding it to your assets, enter Play mode. Please note that UI Toolkit interfaces do not appear in the Scene view. Instead, you can view them in the Game view or UI Builder.The menu on the left helps you navigate the modal main menu screens. This vertical column of buttons provides access to the five modal screens that comprise the main menu (they stay active while switching between screens).While some interactivity is possible, such as healing the characters by dragging available potions in the scene, gameplay has been kept to a minimum to ensure continued focus on the UI examples.Let’s take a closer look at the UIs in the menu bar:The home screen serves as a landing pad when launching the application. You can use this screen to play the game or receive simulated chat messages.The character screen involves a mix of GameObjects and UI elements. This is where you can explore each of the four Dragon Crashers characters. Use the stats, skills, and bio tabs to read the specific character details, and click on the inventory slots to add or remove items. The preview area shows a 2D lit and rigged character over a tiled background.The resources screen links to documentation, the forum, and other resources for making the most of UI Toolkit.The shop screen simulates an in-game store where you can purchase hard and soft currency, such as gold or gems, as well as virtual goods like healing potions. Each item in the shop screen is a separate VisualTreeAsset. UI Toolkit instantiates these assets at runtime; one for each ScriptableObject in the Resources/GameData.The mail screen is a front-end reader of fictitious messages that uses a tabbed menu to separate the inbox and deleted messages.The game screen is a mini version of the Dragon Crashers project that starts playing automatically. In this project, you’ll notice a few revised elements with UI Toolkit, such as a pause button, health bars, and the capacity to drag a healing potion element to your characters when they take damage.UI Toolkit enables you to build stable and consistent UIs for your entire project. At the same time, it provides flexible tools for adding your own design flourishes and details to further flesh out the game’s theme and style.Let’s go over some of the features used to refine the UI designs in the sample:Render Textures:UI Toolkit interfaces are rendered last in the render queue, meaning you can’t overlay other game graphics on top of a UI Toolkit UI. Render Textures provide a workaround to this limitation, making it possible to integrate in-game effects into UI Toolkit UIs. While these effects based on Render Textures should be used sparingly, you’ll still be able to afford sharp effects within the context of a fullscreen UI, without running gameplay. The following images show a number of Render Textures from the demo.Themes with Theme style sheets (TSS): TSS files are Asset files that are similar to regular USS files. They serve as a starting point for defining your own custom theme via USS selectors as well as property and variable settings. In the demo, we duplicated the default theme files and modified the copies to offer seasonal variations.Custom UI elements: Since designers are trained to think outside the box, UI Toolkit gives you plenty of room to customize or extend the standard library. The demo project highlights a few custom-built elements in the tabbed menus, slide toggles, and drop-down lists, plus a radial counter to demonstrate what UI artists are capable of alongside developers.USS transitions for animated UI state changes: Adding transitions to the menu screens can polish and smooth out your visuals. UI Toolkit makes this more straightforward with the Transition Animations property, part of the UI Builder’s Inspector. Adjust the Property, Duration, Easing, and Delay properties to set up the animation. Then simply change styles for UI Toolkit to apply the animated transition at runtime.Post-processing volume for a background blur: A popular effect in games is to blur a crowded gameplay scene to draw the player’s attention to a particular pop-up message or dialog window. You can achieve this effect by enabling Depth of Field in the Volume framework (available in the Universal Render Pipeline).We made sure that efficient workflows were used to fortify the UI. Here are a few recommendations for keeping the project well-organized:Consistent naming conventions: It’s important to adopt naming conventions that align with your visual elements and style sheets. Clear naming conventions not only maintain the hierarchy’s organization in UI Builder, they make it more accessible to your teammates, and keep the code clean and readable. More specifically, we suggest the Block Element Modifier (BEM) naming convention for visual elements and style sheets. Just at a glance, an element’s BEM naming can tell you what it does, how it appears, and how it relates to the other elements around it. See the following BEM naming examples:Responsive UI layout: Similar to web technologies, UI Toolkit offers the possibility of creating layouts where “child” visual elements adapt to the current available size inside their “parent” visual elements, and others where each element has an absolute position anchored to a reference point, akin to the Unity UI system. The sample uses both options as needed through the visual elements of the UI.PSD Importer: One of the most effective tools for creating the demo, PSD Importer allows artists to work in a master document without having to manually export every sprite separately. When changes are needed, they can be done in the original PSD file and updated automatically in Unity.ScriptableObjects: In order to focus on UI design and implementation, the sample project simulates backend data, such as in-app purchases and mail messages, using ScriptableObjects. You can conveniently customize this stand-in data from the Resources/GameData folder and use the example to create similar data assets, like inventory items and character or dialog data in UI Toolkit.Remember that with UI Toolkit, UI layouts and styles are decoupled from code. This means that rewriting the backend data can occur independently from the UI design. If your development team replaces those systems, the interface should continue to work.Additional tools used in the demo include particle systems created with the Built-in Particle System for special effects, and the 2D toolset, among others. Feel free to review the project via the Inspector to see how these different elements come into play.You can find reference art made by the UI artists under UI/Reference, as replicated in UI Builder. The whole process, from mockups to wireframes, is also documented in the e-book. Finally, all of the content in the sample can be added to your own Unity project.You can download the UI Toolkit sample – Dragon Crashers from the Asset Store. Once you’ve explored its different UI designs, please provide your feedback on the forum.Then be sure to check out our e-book, User interface design and implementation in Unity. Download

Chinese AI startup DeepSeek has released an update to the R1 reasoning AI model that took the tech world by storm when it was launched at the beginning of the year. The open-source model sent shockwaves through the AI industry as its efficient use of compute and memory resources helped it match leading US models’ performance at a fraction of the cost. The new version, DeepSeek-R1-0528, has a whopping 685 billion parameters, meaning it can perform on par with competitors such as o3 from OpenAI and Gemini 2.5 Pro from Google. The company says the model’s accuracy has increased from 70% to 87.5%, which means that the risk of hallucinations has been significantly reduced. Furthermore, DeepSeek has reportedly become better at so-called vibe coding, where AI tools generate code based on natural-language conversations with programmers. TechCrunch notes that Deepseek-R1-0528 likely cannot run on consumer computers without modification, which signals that the target audience is professional users. The new version of DeepSeek can be downloaded from the Hugging Face platform. Detailed information on the changes is also available there.