In early-stage start-ups, the pitch deck is often the first thing investors see. Sometimes, it’s the only thing. And yet, it rarely gets the same attention as the website or the socials. Most decks are pulled together last minute, with slides that feel rushed, messy, or just off. That’s where designers can really make a difference. The deck might seem like just another task, but it’s a chance to work on something strategic early on and help shape how the company is understood. It offers a rare opportunity to collaborate closely with copywriters, strategists and the founders to turn their vision into a clear and convincing story. Founders bring the vision, but more and more, design and brand teams are being asked to shape how that vision is told, and sold. So here are five handy things we’ve learned at SIDE ST for the next time you’re asked to design a deck. Think in context Designers stepping into pitch work should begin by understanding the full picture – who the deck is for, what outcomes it’s meant to drive and how it fits into the broader brand and business context. Their role isn’t just to make things look good, but to prioritise clarity over surface-level aesthetics. It’s about getting into the founders’ mindset, shaping visuals and copy around the message, and connecting with the intended audience. Every decision, from slide hierarchy to image selection, should reinforce the business goals behind the deck. Support the narrative Visuals are more subjective than words, and that’s exactly what gives them power. The right image can suggest an idea, reinforce a value, or subtly shift perception without a single word. Whether it’s hinting at accessibility, signalling innovation, or grounding the product in context, design plays a strategic role in how a company is understood. It gives designers the opportunity to take centre stage in the storytelling, shaping how the company is understood through visual choices. But that influence works both ways. Used thoughtlessly, visuals can distort the story, suggesting the wrong market, implying a different stage of maturity, or confusing people about the product itself. When used with care, they become a powerful design tool to sharpen the narrative and spark interest from the very first slide. Keep it real Stock photos can be tempting. They’re high-quality and easy to drop in, especially when the real images a start-up has can be grainy, unfinished, or simply not there yet. But in early-stage pitch decks, they often work against your client. Instead of supporting the story, they flatten it, and rarely reflect the actual team, product, or context. This is your chance as a designer to lean into what’s real, even if it’s a bit rough. Designers can elevate even scrappy assets with thoughtful framing and treatment, turning rough imagery into a strength. In early-stage storytelling, “real” often resonates more than “perfect.” Pay attention to the format Even if you’re brought in just to design the deck, don’t treat it as a standalone piece. It’s often the first brand touchpoint investors will see—but it won’t be the last. They’ll go on to check the website, scroll through social posts, and form an impression based on how it all fits together. Early-stage startups might not have full brand guidelines in place yet, but that doesn’t mean there’s no need for consistency. In fact, it gives designers a unique opportunity to lay the foundation. A strong, thoughtful deck can help shape the early visual language and give the team something to build on as the brand grows. Before you hit export For designers, the deck isn’t just another deliverable. It’s an early tool that shapes and impacts investor perception, internal alignment and founder confidence. It’s a strategic design moment to influence the trajectory of a company before it’s fully formed. Designers who understand the pressure, pace and uncertainty founders face at this stage are better equipped to deliver work that resonates. This is about more than simply polishing slides, it’s about helping early-stage teams tell a sharper, more human story when it matters most. Maor Ofek is founder of SIDE ST, a brand consultancy that works mainly with start-ups. 

A coiled giant anaconda. They are the largest snake species in Brazil and play a major role in legends including the ‘Boiuna’ and the ‘Cobra Grande.’ CREDIT: Beatriz Cosendey. Get the Popular Science daily newsletter💡 Breakthroughs, discoveries, and DIY tips sent every weekday. South America’s lush Amazon region is a biodiversity hotspot, which means that every living thing must find a way to co-exist. Even some of the most feared snakes on the planet–anacondas. In a paper published June 16 in the journal Frontiers in Amphibian and Reptile Science, conservation biologists Beatriz Cosendey and Juarez Carlos Brito Pezzuti from the Federal University of Pará’s Center for Amazonian Studies in Brazil, analyze the key points behind the interactions between humans and the local anaconda populations. Ahead of the paper’s publication, the team at Frontiers conducted this wide-ranging Q&A with Conesday. It has not been altered. Frontiers: What inspired you to become a researcher? Beatriz Cosendey: As a child, I was fascinated by reports and documentaries about field research and often wondered what it took to be there and what kind of knowledge was being produced. Later, as an ecologist, I felt the need for approaches that better connected scientific research with real-world contexts. I became especially interested in perspectives that viewed humans not as separate from nature, but as part of ecological systems. This led me to explore integrative methods that incorporate local and traditional knowledge, aiming to make research more relevant and accessible to the communities involved. F: Can you tell us about the research you’re currently working on? BC: My research focuses on ethnobiology, an interdisciplinary field intersecting ecology, conservation, and traditional knowledge. We investigate not only the biodiversity of an area but also the relationship local communities have with surrounding species, providing a better understanding of local dynamics and areas needing special attention for conservation. After all, no one knows a place better than those who have lived there for generations. This deep familiarity allows for early detection of changes or environmental shifts. Additionally, developing a collaborative project with residents generates greater engagement, as they recognize themselves as active contributors; and collective participation is essential for effective conservation. Local boating the Amazon River. CREDIT: Beatriz Cosendey. F: Could you tell us about one of the legends surrounding anacondas? BC: One of the greatest myths is about the Great Snake—a huge snake that is said to inhabit the Amazon River and sleep beneath the town. According to the dwellers, the Great Snake is an anaconda that has grown too large; its movements can shake the river’s waters, and its eyes look like fire in the darkness of night. People say anacondas can grow so big that they can swallow large animals—including humans or cattle—without difficulty. F: What could be the reasons why the traditional role of anacondas as a spiritual and mythological entity has changed? Do you think the fact that fewer anacondas have been seen in recent years contributes to their diminished importance as an mythological entity? BC: Not exactly. I believe the two are related, but not in a direct way. The mythology still exists, but among Aritapera dwellers, there’s a more practical, everyday concern—mainly the fear of losing their chickens. As a result, anacondas have come to be seen as stealthy thieves. These traits are mostly associated with smaller individuals (up to around 2–2.5 meters), while the larger ones—which may still carry the symbolic weight of the ‘Great Snake’—tend to retreat to more sheltered areas; because of the presence of houses, motorized boats, and general noise, they are now seen much less frequently. A giant anaconda is being measured. Credit: Pedro Calazans. F: Can you share some of the quotes you’ve collected in interviews that show the attitude of community members towards anacondas? How do chickens come into play? BC: When talking about anacondas, one thing always comes up: chickens. “Chicken is her [the anaconda’s] favorite dish. If one clucks, she comes,” said one dweller. This kind of remark helps explain why the conflict is often framed in economic terms. During the interviews and conversations with local dwellers, many emphasized the financial impact of losing their animals: “The biggest loss is that they keep taking chicks and chickens…” or “You raise the chicken—you can’t just let it be eaten for free, right?” For them, it’s a loss of investment, especially since corn, which is used as chicken feed, is expensive. As one person put it: “We spend time feeding and raising the birds, and then the snake comes and takes them.” One dweller shared that, in an attempt to prevent another loss, he killed the anaconda and removed the last chicken it had swallowed from its belly—”it was still fresh,” he said—and used it for his meal, cooking the chicken for lunch so it wouldn’t go to waste. One of the Amazonas communities where the researchers conducted their research. CREDIT: Beatriz Cosendey. Some interviewees reported that they had to rebuild their chicken coops and pigsties because too many anacondas were getting in. Participants would point out where the anaconda had entered and explained that they came in through gaps or cracks but couldn’t get out afterwards because they ‘tufavam’ — a local term referring to the snake’s body swelling after ingesting prey. We saw chicken coops made with mesh, with nylon, some that worked and some that didn’t. Guided by the locals’ insights, we concluded that the best solution to compensate for the gaps between the wooden slats is to line the coop with a fine nylon mesh (to block smaller animals), and on the outside, a layer of wire mesh, which protects the inner mesh and prevents the entry of larger animals. F: Are there any common misconceptions about this area of research? How would you address them? BC: Yes, very much. Although ethnobiology is an old science, it’s still underexplored and often misunderstood. In some fields, there are ongoing debates about the robustness and scientific validity of the field and related areas. This is largely because the findings don’t always rely only on hard statistical data. However, like any other scientific field, it follows standardized methodologies, and no result is accepted without proper grounding. What happens is that ethnobiology leans more toward the human sciences, placing human beings and traditional knowledge as key variables within its framework. To address these misconceptions, I believe it’s important to emphasize that ethnobiology produces solid and relevant knowledge—especially in the context of conservation and sustainable development. It offers insights that purely biological approaches might overlook and helps build bridges between science and society. The study focused on the várzea regions of the Lower Amazon River. CREDIT: Beatriz Cosendey. F: What are some of the areas of research you’d like to see tackled in the years ahead? BC: I’d like to see more conservation projects that include local communities as active participants rather than as passive observers. Incorporating their voices, perspectives, and needs not only makes initiatives more effective, but also more just. There is also great potential in recognizing and valuing traditional knowledge. Beyond its cultural significance, certain practices—such as the use of natural compounds—could become practical assets for other vulnerable regions. Once properly documented and understood, many of these approaches offer adaptable forms of environmental management and could help inform broader conservation strategies elsewhere. F: How has open science benefited the reach and impact of your research? BC: Open science is crucial for making research more accessible. By eliminating access barriers, it facilitates a broader exchange of knowledge—important especially for interdisciplinary research like mine which draws on multiple knowledge systems and gains value when shared widely. For scientific work, it ensures that knowledge reaches a wider audience, including practitioners and policymakers. This openness fosters dialogue across different sectors, making research more inclusive and encouraging greater collaboration among diverse groups. The Q&A can also be read here.

Casa Sofia | © Fernando Guerra / FG+SG Located in the historic heart of Lagos, Portugal, Casa Sofia by Mário Martins Atelier is a thoughtful exercise in urban integration and contemporary reinterpretation. Occupying a site once held by a modest two-story house, the project is situated on the corner of a block facing the Church of St Sebastião. With its commanding presence, this national monument set a formidable challenge for the architects: introducing a new residence that respects the weight of history while offering a clear, contemporary expression. Casa Sofia Technical Information Architects1-4: Mário Martins Atelier Location: Lagos, Portugal Project Completion Years: 2023 Photographs: © Fernando Guerra / FG+SG It is therefore important to design a building to fit into and complete the block. A house that is quiet and solid, with rhythmic metrics, whose new design brings an identity, with the weight and scent of the times, to a city that has existed for many centuries. – Mário Martins Atelier Casa Sofia Photographs © Fernando Guerra / FG+SG © Fernando Guerra / FG+SG © Fernando Guerra / FG+SG © Fernando Guerra / FG+SG © Fernando Guerra / FG+SG © Fernando Guerra / FG+SG © Fernando Guerra / FG+SG © Fernando Guerra / FG+SG © Fernando Guerra / FG+SG © Fernando Guerra / FG+SG © Fernando Guerra / FG+SG Spatial Organization and Circulation The design’s ambition is anchored in reconciling modern residential needs with the dense urban fabric that defines the walled city. Rather than imposing a bold or disruptive form, the project embraces the existing rhythms and textures of the surrounding architecture. The result is a building that both defers to and elevates the neighborhood’s character. Its restrained profile and carefully modulated facade echo the massing and articulation of the original house while introducing an identity that is clearly of its time. At the core of Casa Sofia’s spatial organization is a deliberate hierarchy of spaces that transitions seamlessly between public, semi-public, and private domains. Entry from the street occurs through a modest set of steps leading to an exterior atrium. This threshold mediates the relationship between the public realm and the interior, grounding the house in its urban context. Once inside, an open hall reveals the vertical flow of the building, dominated by a staircase that appears to float, linking the house’s various levels while maintaining visual continuity throughout. The ground floor houses three bedrooms, each with an ensuite bathroom, radiating from the central hall. This level also contains a small basement for technical support, reinforcing the discreet layering of functional and domestic spaces. Midway up the staircase, the house opens onto a garage, a laundry room, and an intimate courtyard. These areas, essential for daily life, are seamlessly integrated into the overall composition, contributing to a spatial richness that is both pragmatic and sensorial. On the first floor, an open-plan arrangement accommodates the main living spaces. Around a central void, the living and dining areas, kitchen, and master suite are arranged to encourage visual interplay and shared light. This configuration enhances the spatial porosity, ensuring that despite the density of the historic center, the house retains a sense of openness and fluidity. Above, a recessed roof level recedes from the street, culminating in a panoramic terrace with a swimming pool. Here, the building dissolves into the sky, offering expansive views and light-filled leisure spaces that contrast with the more enclosed lower floors. Materiality and Craftsmanship Materiality plays a decisive role in mediating the building’s relationship with its context. White-painted plaster, a familiar element in the region, is punctuated by deep limestone moldings. These details create a play of light and shadow that emphasizes the facade’s verticality and rhythm. The generous thickness of the walls, carried over from the site’s earlier construction, lends a sense of solidity and permanence to the house, recalling the tactile traditions of the Algarve’s architecture. The interior and exterior detailing is characterized by an economy of means, where each material is selected for its ability to reinforce the house’s quiet presence. Local materials and craftsmanship ground the project in its immediate context while responding to environmental imperatives. High thermal comfort is achieved through careful orientation and passive design strategies, complemented by the integration of solar control and water conservation measures. These considerations underscore the project’s commitment to sustainability without resorting to superficial gestures. Broader Urban and Cultural Implications Beyond its immediate function as a family home, Casa Sofia engages in a broader dialogue with its urban and cultural surroundings. The project exemplifies a measured response to the question of how to build within a historical setting without resorting to nostalgia or pastiche. It demonstrates that contemporary architecture can find resonance within heritage contexts by prioritizing the values of continuity, scale, and material authenticity. In its measured dialogue with the Church of St Sebastião and the centuries-old urban landscape of Lagos, Casa Sofia illustrates the potential for architecture to enrich the experience of place through quiet, rigorous interventions. It is a project that reaffirms architecture’s capacity to negotiate between past and present, crafting spaces that are at once deeply contextual and unambiguously of their moment. Casa Sofia Plans Sketch | © Mário Martins Atelier Ground Level | © Mário Martins Atelier Level 1 | © Mário Martins Atelier Level 2 | © Mário Martins Atelier Roof Plan | © Mário Martins Atelier Section | © Mário Martins Atelier Casa Sofia Image Gallery About Mário Martins Atelier Mário Martins Atelier is a Portuguese architecture and urbanism practice founded in 2000 by architect Mário Martins, who holds a degree from the Faculty of Architecture at the Technical University of Lisbon (1988). Headquartered in Lagos with a secondary office in Lisbon, the firm operates with a dedicated multidisciplinary team. The office has developed a broad spectrum of work, from single-family homes and collective housing to public buildings and urban regeneration, distinguished by technical precision, contextual sensitivity, and sustainable strategies. Credits and Additional Notes Lead Architect: Mário Martins, arq. Project Team: Rita Rocha, Sónia Fialho, Susana Caetano, Susana Jóia, Ana Graça Engineering: Nuno Grave Engenharia Building: Marques Antunes Engenharia Lda

8 Stunning Sunset Color Palettes Zoe Santoro •  In this article:See more ▼Post may contain affiliate links which give us commissions at no cost to you.There’s something absolutely magical about watching the sun dip below the horizon, painting the sky in breathtaking hues that seem almost too beautiful to be real. As a designer, I find myself constantly inspired by these natural masterpieces that unfold before us every evening. The way warm oranges melt into soft pinks, how deep purples blend seamlessly with golden yellows – it’s like nature’s own masterclass in color theory. If you’re looking to infuse your next project with the warmth, romance, and natural beauty of a perfect sunset, you’ve come to the right place. I’ve curated eight of the most captivating sunset color palettes that will bring that golden hour magic directly into your designs. 👋 Psst... Did you know you can get unlimited downloads of 59,000+ fonts and millions of other creative assets for just $16.95/mo? Learn more »The 8 Most Breathtaking Sunset Color Palettes 1. Golden Hour Glow #FFD700 #FF8C00 #FF6347 #CD5C5C Download this color palette 735×1102 Pinterest image 2160×3840 Vertical wallpaper 900×900 Square 3840×2160 4K Wallpaper This palette captures that perfect moment when everything seems to be touched by liquid gold. The warm yellows transition beautifully into rich oranges and soft coral reds, creating a sense of warmth and optimism that’s impossible to ignore. I find this combination works wonderfully for brands that want to evoke feelings of happiness, energy, and positivity. 2. Tropical Paradise #FF69B4 #FF1493 #FF8C00 #FFD700 Download this color palette 735×1102 Pinterest image 2160×3840 Vertical wallpaper 900×900 Square 3840×2160 4K Wallpaper Inspired by those incredible sunsets you see in tropical destinations, this vibrant palette combines hot pinks with brilliant oranges and golden yellows. It’s bold, it’s energetic, and it’s perfect for projects that need to make a statement. I love using these colors for summer campaigns or anything that needs to capture that vacation feeling. 3. Desert Dreams #CD853F #D2691E #B22222 #8B0000 Download this color palette 735×1102 Pinterest image 2160×3840 Vertical wallpaper 900×900 Square 3840×2160 4K Wallpaper Get 300+ Fonts for FREEEnter your email to download our 100% free "Font Lover's Bundle". For commercial & personal use. No royalties. No fees. No attribution. 100% free to use anywhere. The American Southwest produces some of the most spectacular sunsets on earth, and this palette pays homage to those incredible desert skies. The earthy browns blend into warm oranges before deepening into rich reds and burgundies. This combination brings a sense of grounding and authenticity that works beautifully for rustic or heritage brands. 4. Pastel Evening #FFE4E1 #FFA07A #F0E68C #DDA0DD Download this color palette 735×1102 Pinterest image 2160×3840 Vertical wallpaper 900×900 Square 3840×2160 4K Wallpaper Not every sunset needs to be bold and dramatic. This softer palette captures those gentle, dreamy evenings when the sky looks like it’s been painted with watercolors. The delicate pinks, peaches, and lavenders create a romantic, ethereal feeling that’s perfect for wedding designs, beauty brands, or any project that needs a touch of feminine elegance. 5. Coastal Sunset #fae991 #FF7F50 #FF6347 #4169E1 #1E90FF Download this color palette 735×1102 Pinterest image 2160×3840 Vertical wallpaper 900×900 Square 3840×2160 4K Wallpaper There’s something special about watching the sun set over the ocean, where warm oranges and corals meet the deep blues of the sea and sky. This palette captures that perfect contrast between warm and cool tones. I find it creates a sense of adventure and wanderlust that’s ideal for travel brands or outdoor companies. 6. Urban Twilight #ffeda3 #fdad52 #fc8a6e #575475 #111f2a Download this color palette 735×1102 Pinterest image 2160×3840 Vertical wallpaper 900×900 Square 3840×2160 4K Wallpaper As the sun sets behind city skylines, you get these incredible contrasts between deep purples and vibrant oranges. This sophisticated palette brings together the mystery of twilight with the warmth of the setting sun. It’s perfect for creating designs that feel both modern and dramatic. 7. Autumn Harvest #FF4500 #FF8C00 #DAA520 #8B4513 Download this color palette 735×1102 Pinterest image 2160×3840 Vertical wallpaper 900×900 Square 3840×2160 4K Wallpaper This palette captures those perfect fall evenings when the sunset seems to echo the changing leaves. The deep oranges and golden yellows create a cozy, inviting feeling that’s perfect for seasonal campaigns or brands that want to evoke comfort and tradition. 8. Fire Sky #652220 #DC143C #FF0000 #FF4500 #FF8C00 Download this color palette 735×1102 Pinterest image 2160×3840 Vertical wallpaper 900×900 Square 3840×2160 4K Wallpaper Sometimes nature puts on a show that’s so intense it takes your breath away. This bold, fiery palette captures those dramatic sunsets that look like the sky is literally on fire. It’s not for the faint of heart, but when you need maximum impact and energy, these colors deliver in spades. Why Sunset Colors Never Go Out of Style Before we explore how to use these palettes effectively, let’s talk about why sunset colors have such enduring appeal in design. There’s something deeply ingrained in human psychology that responds to these warm, glowing hues. They remind us of endings and beginnings, of peaceful moments and natural beauty. From a design perspective, sunset colors offer incredible versatility. They can be bold and energetic or soft and romantic. They work equally well for corporate branding and personal projects. And perhaps most importantly, they’re inherently optimistic – they make people feel good. I’ve found that incorporating sunset-inspired colors into modern projects adds an instant sense of warmth and approachability that resonates with audiences across all demographics. Whether you’re working on packaging design, web interfaces, or environmental graphics, these palettes can help create an emotional connection that goes beyond mere aesthetics. How to Master Sunset Palettes in Contemporary Design Using sunset colors effectively requires more than just picking pretty hues and hoping for the best. Here are some strategies I’ve developed for incorporating these palettes into modern design work: Start with Temperature Balance One of the most important aspects of working with sunset palettes is understanding color temperature. Most sunset combinations naturally include both warm and cool elements – the warm oranges and yellows of the sun itself, balanced by the cooler purples and blues of the surrounding sky. Maintaining this temperature balance keeps your designs from feeling flat or monotonous. Layer for Depth Real sunsets have incredible depth and dimension, with colors layering and blending into each other. Try to recreate this in your designs by using gradients, overlays, or layered elements rather than flat blocks of color. This approach creates visual interest and mimics the natural way these colors appear in nature. Consider Context and Contrast While sunset colors are beautiful, they need to work within the context of your overall design. Pay attention to readability – text needs sufficient contrast against sunset backgrounds. Consider using neutrals like deep charcoal or cream to provide breathing room and ensure your message remains clear. Embrace Gradual Transitions The magic of a sunset lies in how colors flow seamlessly from one to another. Incorporate this principle into your designs through smooth gradients, subtle color shifts, or elements that bridge between different hues in your palette. The Science Behind Our Sunset Obsession As someone who’s spent years studying color psychology, I’m fascinated by why sunset colors have such universal appeal. Research suggests that warm colors like those found in sunsets trigger positive emotional responses and can even increase feelings of comfort and security. There’s also the association factor – sunsets are linked in our minds with relaxation, beauty, and positive experiences. When we see these colors in design, we unconsciously associate them with those same positive feelings. This makes sunset palettes particularly effective for brands that want to create emotional connections with their audiences. The cyclical nature of sunsets also plays a role. They happen every day, marking the transition from activity to rest, from work to leisure. This gives sunset colors a sense of familiarity and comfort that few other color combinations can match. Applying Sunset Palettes Across Design Disciplines One of the things I love most about sunset color palettes is how adaptable they are across different types of design work: Brand Identity Design Sunset colors can help brands convey warmth, optimism, and approachability. I’ve used variations of these palettes for everything from artisanal food companies to wellness brands. The key is choosing the right intensity level for your brand’s personality – softer palettes for more refined brands, bolder combinations for companies that want to make a statement. Digital Design In web and app design, sunset colors can create interfaces that feel warm and inviting rather than cold and clinical. I often use these palettes for backgrounds, accent elements, or call-to-action buttons. The natural flow between colors makes them perfect for creating smooth user experiences that guide the eye naturally through content. Print and Packaging Sunset palettes really shine in print applications where you can take advantage of rich, saturated colors. They work beautifully for packaging design, particularly for products associated with warmth, comfort, or natural ingredients. The key is ensuring your color reproduction is accurate – sunset colors can look muddy if not handled properly in print. Environmental Design In spaces, sunset colors can create incredibly welcoming environments. I’ve seen these palettes used effectively in restaurants, retail spaces, and even corporate offices where the goal is to create a sense of warmth and community. Seasonal Considerations and Trending Applications While sunset colors are timeless, they do have natural seasonal associations that smart designers can leverage. The warmer, more intense sunset palettes work beautifully for fall and winter campaigns, while the softer, more pastel variations are perfect for spring and summer applications. I’ve noticed a growing trend toward using sunset palettes in unexpected contexts – tech companies embracing warm gradients, financial services using sunset colors to appear more approachable, and healthcare brands incorporating these hues to create more comforting environments. Conclusion: Bringing Natural Beauty Into Modern Design As we’ve explored these eight stunning sunset color palettes, I hope you’ve gained new appreciation for the incredible design potential that nature provides us every single day. These colors aren’t just beautiful – they’re powerful tools for creating emotional connections, conveying brand values, and making designs that truly resonate with people. The secret to successfully using sunset palettes lies in understanding both their emotional impact and their technical requirements. Don’t be afraid to experiment with different combinations and intensities, but always keep your audience and context in mind. Remember, the best sunset colors aren’t just about picking the prettiest hues – they’re about capturing the feeling of those magical moments when day transitions to night. Whether you’re creating a logo that needs to convey warmth and trust, designing a website that should feel welcoming and approachable, or developing packaging that needs to stand out on crowded shelves, these sunset-inspired palettes offer endless possibilities. So the next time you catch yourself stopped in your tracks by a particularly stunning sunset, take a moment to really study those colors. Notice how they blend and flow, how they make you feel, and how they change as the light shifts. Then bring that natural magic into your next design project. After all, if nature can create such breathtaking color combinations every single day, imagine what we can achieve when we learn from the master. Happy designing! Zoe Santoro Zoe is an art student and graphic designer with a passion for creativity and adventure. Whether she’s sketching in a cozy café or capturing inspiration from vibrant cityscapes, she finds beauty in every corner of the world. With a love for bold colors, clean design, and storytelling through visuals, Zoe blends her artistic skills with her wanderlust to create stunning, travel-inspired designs. Follow her journey as she explores new places, discovers fresh inspiration, and shares her creative process along the way. 10 Warm Color Palettes That’ll Brighten Your DayThere’s nothing quite like the embracing quality of warm colors to make a design feel inviting and alive. As someone...These 1920s Color Palettes are ‘Greater than Gatsby’There’s something undeniably captivating about the color schemes of the Roaring Twenties. As a designer with a passion for historical...How Fonts Influence Tone and Clarity in Animated VideosAudiences interact differently with messages based on which fonts designers choose to use within a text presentation. Fonts shape how...

Transcript [MUSIC]      [BOOK PASSAGE]   PETER LEE: “In ‘The Little Black Bag,’ a classic science fiction story, a high-tech doctor’s kit of the future is accidentally transported back to the 1950s, into the shaky hands of a washed-up, alcoholic doctor. The ultimate medical tool, it redeems the doctor wielding it, allowing him to practice gratifyingly heroic medicine. … The tale ends badly for the doctor and his treacherous assistant, but it offered a picture of how advanced technology could transform medicine—powerful when it was written nearly 75 years ago and still so today. What would be the Al equivalent of that little black bag? At this moment when new capabilities are emerging, how do we imagine them into medicine?”   [END OF BOOK PASSAGE]     [THEME MUSIC]     This is The AI Revolution in Medicine, Revisited. I’m your host, Peter Lee.    Shortly after OpenAI’s GPT-4 was publicly released, Carey Goldberg, Dr. Zak Kohane, and I published The AI Revolution in Medicine to help educate the world of healthcare and medical research about the transformative impact this new generative AI technology could have. But because we wrote the book when GPT-4 was still a secret, we had to speculate. Now, two years later, what did we get right, and what did we get wrong?     In this series, we’ll talk to clinicians, patients, hospital administrators, and others to understand the reality of AI in the field and where we go from here.   [THEME MUSIC FADES] The book passage I read at the top is from “Chapter 10: The Big Black Bag.”  In imagining AI in medicine, Carey, Zak, and I included in our book two fictional accounts. In the first, a medical resident consults GPT-4 on her personal phone as the patient in front of her crashes. Within seconds, it offers an alternate response based on recent literature. In the second account, a 90-year-old woman with several chronic conditions is living independently and receiving near-constant medical support from an AI aide.    In our conversations with the guests we’ve spoken to so far, we’ve caught a glimpse of these predicted futures, seeing how clinicians and patients are actually using AI today and how developers are leveraging the technology in the healthcare products and services they’re creating. In fact, that first fictional account isn’t so fictional after all, as most of the doctors in the real world actually appear to be using AI at least occasionally—and sometimes much more than occasionally—to help in their daily clinical work. And as for the second fictional account, which is more of a science fiction account, it seems we are indeed on the verge of a new way of delivering and receiving healthcare, though the future is still very much open.  As we continue to examine the current state of AI in healthcare and its potential to transform the field, I’m pleased to welcome Bill Gates and Sébastien Bubeck.   Bill may be best known as the co-founder of Microsoft, having created the company with his childhood friend Paul Allen in 1975. He’s now the founder of Breakthrough Energy, which aims to advance clean energy innovation, and TerraPower, a company developing groundbreaking nuclear energy and science technologies. He also chairs the world’s largest philanthropic organization, the Gates Foundation, and focuses on solving a variety of health challenges around the globe and here at home.  Sébastien is a research lead at OpenAI. He was previously a distinguished scientist, vice president of AI, and a colleague of mine here at Microsoft, where his work included spearheading the development of the family of small language models known as Phi. While at Microsoft, he also coauthored the discussion-provoking 2023 paper “Sparks of Artificial General Intelligence,” which presented the results of early experiments with GPT-4 conducted by a small team from Microsoft Research.    [TRANSITION MUSIC]   Here’s my conversation with Bill Gates and Sébastien Bubeck.  LEE: Bill, welcome.  BILL GATES: Thank you.  LEE: Seb …  SÉBASTIEN BUBECK: Yeah. Hi, hi, Peter. Nice to be here.  LEE: You know, one of the things that I’ve been doing just to get the conversation warmed up is to talk about origin stories, and what I mean about origin stories is, you know, what was the first contact that you had with large language models or the concept of generative AI that convinced you or made you think that something really important was happening?  And so, Bill, I think I’ve heard the story about, you know, the time when the OpenAI folks—Sam Altman, Greg Brockman, and others—showed you something, but could we hear from you what those early encounters were like and what was going through your mind?   GATES: Well, I’d been visiting OpenAI soon after it was created to see things like GPT-2 and to see the little arm they had that was trying to match human manipulation and, you know, looking at their games like Dota that they were trying to get as good as human play. And honestly, I didn’t think the language model stuff they were doing, even when they got to GPT-3, would show the ability to learn, you know, in the same sense that a human reads a biology book and is able to take that knowledge and access it not only to pass a test but also to create new medicines.  And so my challenge to them was that if their LLM could get a five on the advanced placement biology test, then I would say, OK, it took biologic knowledge and encoded it in an accessible way and that I didn’t expect them to do that very quickly but it would be profound.   And it was only about six months after I challenged them to do that, that an early version of GPT-4 they brought up to a dinner at my house, and in fact, it answered most of the questions that night very well. The one it got totally wrong, we were … because it was so good, we kept thinking, Oh, we must be wrong. It turned out it was a math weakness [LAUGHTER] that, you know, we later understood that that was an area of, weirdly, of incredible weakness of those early models. But, you know, that was when I realized, OK, the age of cheap intelligence was at its beginning.  LEE: Yeah. So I guess it seems like you had something similar to me in that my first encounters, I actually harbored some skepticism. Is it fair to say you were skeptical before that?  GATES: Well, the idea that we’ve figured out how to encode and access knowledge in this very deep sense without even understanding the nature of the encoding, …  LEE: Right.   GATES: … that is a bit weird.   LEE: Yeah.  GATES: We have an algorithm that creates the computation, but even say, OK, where is the president’s birthday stored in there? Where is this fact stored in there? The fact that even now when we’re playing around, getting a little bit more sense of it, it’s opaque to us what the semantic encoding is, it’s, kind of, amazing to me. I thought the invention of knowledge storage would be an explicit way of encoding knowledge, not an implicit statistical training.  LEE: Yeah, yeah. All right. So, Seb, you know, on this same topic, you know, I got—as we say at Microsoft—I got pulled into the tent. [LAUGHS]  BUBECK: Yes.   LEE: Because this was a very secret project. And then, um, I had the opportunity to select a small number of researchers in MSR [Microsoft Research] to join and start investigating this thing seriously. And the first person I pulled in was you.  BUBECK: Yeah.  LEE: And so what were your first encounters? Because I actually don’t remember what happened then.  BUBECK: Oh, I remember it very well. [LAUGHS] My first encounter with GPT-4 was in a meeting with the two of you, actually. But my kind of first contact, the first moment where I realized that something was happening with generative AI, was before that. And I agree with Bill that I also wasn’t too impressed by GPT-3.  I though that it was kind of, you know, very naturally mimicking the web, sort of parroting what was written there in a nice way. Still in a way which seemed very impressive. But it wasn’t really intelligent in any way. But shortly after GPT-3, there was a model before GPT-4 that really shocked me, and this was the first image generation model, DALL-E 1.  So that was in 2021. And I will forever remember the press release of OpenAI where they had this prompt of an avocado chair and then you had this image of the avocado chair. [LAUGHTER] And what really shocked me is that clearly the model kind of “understood” what is a chair, what is an avocado, and was able to merge those concepts.  So this was really, to me, the first moment where I saw some understanding in those models.   LEE: So this was, just to get the timing right, that was before I pulled you into the tent.  BUBECK: That was before. That was like a year before.  LEE: Right.   BUBECK: And now I will tell you how, you know, we went from that moment to the meeting with the two of you and GPT-4.  So once I saw this kind of understanding, I thought, OK, fine. It understands concept, but it’s still not able to reason. It cannot—as, you know, Bill was saying—it cannot learn from your document. It cannot reason.   So I set out to try to prove that. You know, this is what I was in the business of at the time, trying to prove things in mathematics. So I was trying to prove that basically autoregressive transformers could never reason. So I was trying to prove this. And after a year of work, I had something reasonable to show. And so I had the meeting with the two of you, and I had this example where I wanted to say, there is no way that an LLM is going to be able to do x.  And then as soon as I … I don’t know if you remember, Bill. But as soon as I said that, you said, oh, but wait a second. I had, you know, the OpenAI crew at my house recently, and they showed me a new model. Why don’t we ask this new model this question?   LEE: Yeah. BUBECK: And we did, and it solved it on the spot. And that really, honestly, just changed my life. Like, you know, I had been working for a year trying to say that this was impossible. And just right there, it was shown to be possible.   LEE: [LAUGHS] One of the very first things I got interested in—because I was really thinking a lot about healthcare—was healthcare and medicine.  And I don’t know if the two of you remember, but I ended up doing a lot of tests. I ran through, you know, step one and step two of the US Medical Licensing Exam. Did a whole bunch of other things. I wrote this big report. It was, you know, I can’t remember … a couple hundred pages.   And I needed to share this with someone. I didn’t … there weren’t too many people I could share it with. So I sent, I think, a copy to you, Bill. Sent a copy to you, Seb.   I hardly slept for about a week putting that report together. And, yeah, and I kept working on it. But I was far from alone. I think everyone who was in the tent, so to speak, in those early days was going through something pretty similar. All right. So I think … of course, a lot of what I put in the report also ended up being examples that made it into the book.  But the main purpose of this conversation isn’t to reminisce about [LAUGHS] or indulge in those reminiscences but to talk about what’s happening in healthcare and medicine. And, you know, as I said, we wrote this book. We did it very, very quickly. Seb, you helped. Bill, you know, you provided a review and some endorsements.  But, you know, honestly, we didn’t know what we were talking about because no one had access to this thing. And so we just made a bunch of guesses. So really, the whole thing I wanted to probe with the two of you is, now with two years of experience out in the world, what, you know, what do we think is happening today?  You know, is AI actually having an impact, positive or negative, on healthcare and medicine? And what do we now think is going to happen in the next two years, five years, or 10 years? And so I realize it’s a little bit too abstract to just ask it that way. So let me just try to narrow the discussion and guide us a little bit.   Um, the kind of administrative and clerical work, paperwork, around healthcare—and we made a lot of guesses about that—that appears to be going well, but, you know, Bill, I know we’ve discussed that sometimes that you think there ought to be a lot more going on. Do you have a viewpoint on how AI is actually finding its way into reducing paperwork?  GATES: Well, I’m stunned … I don’t think there should be a patient-doctor meeting where the AI is not sitting in and both transcribing, offering to help with the paperwork, and even making suggestions, although the doctor will be the one, you know, who makes the final decision about the diagnosis and whatever prescription gets done.   It’s so helpful. You know, when that patient goes home and their, you know, son who wants to understand what happened has some questions, that AI should be available to continue that conversation. And the way you can improve that experience and streamline things and, you know, involve the people who advise you. I don’t understand why that’s not more adopted, because there you still have the human in the loop making that final decision.  But even for, like, follow-up calls to make sure the patient did things, to understand if they have concerns and knowing when to escalate back to the doctor, the benefit is incredible. And, you know, that thing is ready for prime time. That paradigm is ready for prime time, in my view.  LEE: Yeah, there are some good products, but it seems like the number one use right now—and we kind of got this from some of the previous guests in previous episodes—is the use of AI just to respond to emails from patients. [LAUGHTER] Does that make sense to you?  BUBECK: Yeah. So maybe I want to second what Bill was saying but maybe take a step back first. You know, two years ago, like, the concept of clinical scribes, which is one of the things that we’re talking about right now, it would have sounded, in fact, it sounded two years ago, borderline dangerous. Because everybody was worried about hallucinations. What happened if you have this AI listening in and then it transcribes, you know, something wrong?  Now, two years later, I think it’s mostly working. And in fact, it is not yet, you know, fully adopted. You’re right. But it is in production. It is used, you know, in many, many places. So this rate of progress is astounding because it wasn’t obvious that we would be able to overcome those obstacles of hallucination. It’s not to say that hallucinations are fully solved. In the case of the closed system, they are.   Now, I think more generally what’s going on in the background is that there is something that we, that certainly I, underestimated, which is this management overhead. So I think the reason why this is not adopted everywhere is really a training and teaching aspect. People need to be taught, like, those systems, how to interact with them.  And one example that I really like, a study that recently appeared where they tried to use ChatGPT for diagnosis and they were comparing doctors without and with ChatGPT (opens in new tab). And the amazing thing … so this was a set of cases where the accuracy of the doctors alone was around 75%. ChatGPT alone was 90%. So that’s already kind of mind blowing. But then the kicker is that doctors with ChatGPT was 80%.   Intelligence alone is not enough. It’s also how it’s presented, how you interact with it. And ChatGPT, it’s an amazing tool. Obviously, I absolutely love it. But it’s not … you don’t want a doctor to have to type in, you know, prompts and use it that way.  It should be, as Bill was saying, kind of running continuously in the background, sending you notifications. And you have to be really careful of the rate at which those notifications are being sent. Because if they are too frequent, then the doctor will learn to ignore them. So you have to … all of those things matter, in fact, at least as much as the level of intelligence of the machine.  LEE: One of the things I think about, Bill, in that scenario that you described, doctors do some thinking about the patient when they write the note. So, you know, I’m always a little uncertain whether it’s actually … you know, you wouldn’t necessarily want to fully automate this, I don’t think. Or at least there needs to be some prompt to the doctor to make sure that the doctor puts some thought into what happened in the encounter with the patient. Does that make sense to you at all?  GATES: At this stage, you know, I’d still put the onus on the doctor to write the conclusions and the summary and not delegate that.  The tradeoffs you make a little bit are somewhat dependent on the situation you’re in. If you’re in Africa, So, yes, the doctor’s still going to have to do a lot of work, but just the quality of letting the patient and the people around them interact and ask questions and have things explained, that alone is such a quality improvement. It’s mind blowing.   LEE: So since you mentioned, you know, Africa—and, of course, this touches on the mission and some of the priorities of the Gates Foundation and this idea of democratization of access to expert medical care—what’s the most interesting stuff going on right now? Are there people and organizations or technologies that are impressing you or that you’re tracking?  GATES: Yeah. So the Gates Foundation has given out a lot of grants to people in Africa doing education, agriculture but more healthcare examples than anything. And the way these things start off, they often start out either being patient-centric in a narrow situation, like, OK, I’m a pregnant woman; talk to me. Or, I have infectious disease symptoms; talk to me. Or they’re connected to a health worker where they’re helping that worker get their job done. And we have lots of pilots out, you know, in both of those cases.   The dream would be eventually to have the thing the patient consults be so broad that it’s like having a doctor available who understands the local things.   LEE: Right.   GATES: We’re not there yet. But over the next two or three years, you know, particularly given the worsening financial constraints against African health systems, where the withdrawal of money has been dramatic, you know, figuring out how to take this—what I sometimes call “free intelligence”—and build a quality health system around that, we will have to be more radical in low-income countries than any rich country is ever going to be.   LEE: Also, there’s maybe a different regulatory environment, so some of those things maybe are easier? Because right now, I think the world hasn’t figured out how to and whether to regulate, let’s say, an AI that might give a medical diagnosis or write a prescription for a medication.  BUBECK: Yeah. I think one issue with this, and it’s also slowing down the deployment of AI in healthcare more generally, is a lack of proper benchmark. Because, you know, you were mentioning the USMLE [United States Medical Licensing Examination], for example. That’s a great test to test human beings and their knowledge of healthcare and medicine. But it’s not a great test to give to an AI.  It’s not asking the right questions. So finding what are the right questions to test whether an AI system is ready to give diagnosis in a constrained setting, that’s a very, very important direction, which to my surprise, is not yet accelerating at the rate that I was hoping for.  LEE: OK, so that gives me an excuse to get more now into the core AI tech because something I’ve discussed with both of you is this issue of what are the right tests. And you both know the very first test I give to any new spin of an LLM is I present a patient, the results—a mythical patient—the results of my physical exam, my mythical physical exam. Maybe some results of some initial labs. And then I present or propose a differential diagnosis. And if you’re not in medicine, a differential diagnosis you can just think of as a prioritized list of the possible diagnoses that fit with all that data. And in that proposed differential, I always intentionally make two mistakes.  I make a textbook technical error in one of the possible elements of the differential diagnosis, and I have an error of omission. And, you know, I just want to know, does the LLM understand what I’m talking about? And all the good ones out there do now. But then I want to know, can it spot the errors? And then most importantly, is it willing to tell me I’m wrong, that I’ve made a mistake?   That last piece seems really hard for AI today. And so let me ask you first, Seb, because at the time of this taping, of course, there was a new spin of GPT-4o last week that became overly sycophantic. In other words, it was actually prone in that test of mine not only to not tell me I’m wrong, but it actually praised me for the creativity of my differential. [LAUGHTER] What’s up with that?  BUBECK: Yeah, I guess it’s a testament to the fact that training those models is still more of an art than a science. So it’s a difficult job. Just to be clear with the audience, we have rolled back that [LAUGHS] version of GPT-4o, so now we don’t have the sycophant version out there.  Yeah, no, it’s a really difficult question. It has to do … as you said, it’s very technical. It has to do with the post-training and how, like, where do you nudge the model? So, you know, there is this very classical by now technique called RLHF [reinforcement learning from human feedback], where you push the model in the direction of a certain reward model. So the reward model is just telling the model, you know, what behavior is good, what behavior is bad.  But this reward model is itself an LLM, and, you know, Bill was saying at the very beginning of the conversation that we don’t really understand how those LLMs deal with concepts like, you know, where is the capital of France located? Things like that. It is the same thing for this reward model. We don’t know why it says that it prefers one output to another, and whether this is correlated with some sycophancy is, you know, something that we discovered basically just now. That if you push too hard in optimization on this reward model, you will get a sycophant model.  So it’s kind of … what I’m trying to say is we became too good at what we were doing, and we ended up, in fact, in a trap of the reward model.  LEE: I mean, you do want … it’s a difficult balance because you do want models to follow your desires and …  BUBECK: It’s a very difficult, very difficult balance.  LEE: So this brings up then the following question for me, which is the extent to which we think we’ll need to have specially trained models for things. So let me start with you, Bill. Do you have a point of view on whether we will need to, you know, quote-unquote take AI models to med school? Have them specially trained? Like, if you were going to deploy something to give medical care in underserved parts of the world, do we need to do something special to create those models?  GATES: We certainly need to teach them the African languages and the unique dialects so that the multimedia interactions are very high quality. We certainly need to teach them the disease prevalence and unique disease patterns like, you know, neglected tropical diseases and malaria. So we need to gather a set of facts that somebody trying to go for a US customer base, you know, wouldn’t necessarily have that in there.  Those two things are actually very straightforward because the additional training time is small. I’d say for the next few years, we’ll also need to do reinforcement learning about the context of being a doctor and how important certain behaviors are. Humans learn over the course of their life to some degree that, I’m in a different context and the way I behave in terms of being willing to criticize or be nice, you know, how important is it? Who’s here? What’s my relationship to them?   Right now, these machines don’t have that broad social experience. And so if you know it’s going to be used for health things, a lot of reinforcement learning of the very best humans in that context would still be valuable. Eventually, the models will, having read all the literature of the world about good doctors, bad doctors, it’ll understand as soon as you say, “I want you to be a doctor diagnosing somebody.” All of the implicit reinforcement that fits that situation, you know, will be there. LEE: Yeah. GATES: And so I hope three years from now, we don’t have to do that reinforcement learning. But today, for any medical context, you would want a lot of data to reinforce tone, willingness to say things when, you know, there might be something significant at stake.  LEE: Yeah. So, you know, something Bill said, kind of, reminds me of another thing that I think we missed, which is, the context also … and the specialization also pertains to different, I guess, what we still call “modes,” although I don’t know if the idea of multimodal is the same as it was two years ago. But, you know, what do you make of all of the hubbub around—in fact, within Microsoft Research, this is a big deal, but I think we’re far from alone—you know, medical images and vision, video, proteins and molecules, cell, you know, cellular data and so on.  BUBECK: Yeah. OK. So there is a lot to say to everything … to the last, you know, couple of minutes. Maybe on the specialization aspect, you know, I think there is, hiding behind this, a really fundamental scientific question of whether eventually we have a singular AGI [artificial general intelligence] that kind of knows everything and you can just put, you know, explain your own context and it will just get it and understand everything.  That’s one vision. I have to say, I don’t particularly believe in this vision. In fact, we humans are not like that at all. I think, hopefully, we are general intelligences, yet we have to specialize a lot. And, you know, I did myself a lot of RL, reinforcement learning, on mathematics. Like, that’s what I did, you know, spent a lot of time doing that. And I didn’t improve on other aspects. You know, in fact, I probably degraded in other aspects. [LAUGHTER] So it’s … I think it’s an important example to have in mind.  LEE: I think I might disagree with you on that, though, because, like, doesn’t a model have to see both good science and bad science in order to be able to gain the ability to discern between the two?  BUBECK: Yeah, no, that absolutely. I think there is value in seeing the generality, in having a very broad base. But then you, kind of, specialize on verticals. And this is where also, you know, open-weights model, which we haven’t talked about yet, are really important because they allow you to provide this broad base to everyone. And then you can specialize on top of it.  LEE: So we have about three hours of stuff to talk about, but our time is actually running low. BUBECK: Yes, yes, yes.   LEE: So I think I want … there’s a more provocative question. It’s almost a silly question, but I need to ask it of the two of you, which is, is there a future, you know, where AI replaces doctors or replaces, you know, medical specialties that we have today? So what does the world look like, say, five years from now?  GATES: Well, it’s important to distinguish healthcare discovery activity from healthcare delivery activity. We focused mostly on delivery. I think it’s very much within the realm of possibility that the AI is not only accelerating healthcare discovery but substituting for a lot of the roles of, you know, I’m an organic chemist, or I run various types of assays. I can see those, which are, you know, testable-output-type jobs but with still very high value, I can see, you know, some replacement in those areas before the doctor.   The doctor, still understanding the human condition and long-term dialogues, you know, they’ve had a lifetime of reinforcement of that, particularly when you get into areas like mental health. So I wouldn’t say in five years, either people will choose to adopt it, but it will be profound that there’ll be this nearly free intelligence that can do follow-up, that can help you, you know, make sure you went through different possibilities.  And so I’d say, yes, we’ll have doctors, but I’d say healthcare will be massively transformed in its quality and in efficiency by AI in that time period.  LEE: Is there a comparison, useful comparison, say, between doctors and, say, programmers, computer programmers, or doctors and, I don’t know, lawyers?  GATES: Programming is another one that has, kind of, a mathematical correctness to it, you know, and so the objective function that you’re trying to reinforce to, as soon as you can understand the state machines, you can have something that’s “checkable”; that’s correct. So I think programming, you know, which is weird to say, that the machine will beat us at most programming tasks before we let it take over roles that have deep empathy, you know, physical presence and social understanding in them.  LEE: Yeah. By the way, you know, I fully expect in five years that AI will produce mathematical proofs that are checkable for validity, easily checkable, because they’ll be written in a proof-checking language like Lean or something but will be so complex that no human mathematician can understand them. I expect that to happen.   I can imagine in some fields, like cellular biology, we could have the same situation in the future because the molecular pathways, the chemistry, biochemistry of human cells or living cells is as complex as any mathematics, and so it seems possible that we may be in a state where in wet lab, we see, Oh yeah, this actually works, but no one can understand why.  BUBECK: Yeah, absolutely. I mean, I think I really agree with Bill’s distinction of the discovery and the delivery, and indeed, the discovery’s when you can check things, and at the end, there is an artifact that you can verify. You know, you can run the protocol in the wet lab and see [if you have] produced what you wanted. So I absolutely agree with that.   And in fact, you know, we don’t have to talk five years from now. I don’t know if you know, but just recently, there was a paper that was published on a scientific discovery using o3- mini (opens in new tab). So this is really amazing. And, you know, just very quickly, just so people know, it was about this statistical physics model, the frustrated Potts model, which has to do with coloring, and basically, the case of three colors, like, more than two colors was open for a long time, and o3 was able to reduce the case of three colors to two colors.   LEE: Yeah.  BUBECK: Which is just, like, astounding. And this is not … this is now. This is happening right now. So this is something that I personally didn’t expect it would happen so quickly, and it’s due to those reasoning models.   Now, on the delivery side, I would add something more to it for the reason why doctors and, in fact, lawyers and coders will remain for a long time, and it’s because we still don’t understand how those models generalize. Like, at the end of the day, we are not able to tell you when they are confronted with a really new, novel situation, whether they will work or not.  Nobody is able to give you that guarantee. And I think until we understand this generalization better, we’re not going to be willing to just let the system in the wild without human supervision.  LEE: But don’t human doctors, human specialists … so, for example, a cardiologist sees a patient in a certain way that a nephrologist …  BUBECK: Yeah. LEE: … or an endocrinologist might not. BUBECK: That’s right. But another cardiologist will understand and, kind of, expect a certain level of generalization from their peer. And this, we just don’t have it with AI models. Now, of course, you’re exactly right. That generalization is also hard for humans. Like, if you have a human trained for one task and you put them into another task, then you don’t … you often don’t know. LEE: OK. You know, the podcast is focused on what’s happened over the last two years. But now, I’d like one provocative prediction about what you think the world of AI and medicine is going to be at some point in the future. You pick your timeframe. I don’t care if it’s two years or 20 years from now, but, you know, what do you think will be different about AI in medicine in that future than today?  BUBECK: Yeah, I think the deployment is going to accelerate soon. Like, we’re really not missing very much. There is this enormous capability overhang. Like, even if progress completely stopped, with current systems, we can do a lot more than what we’re doing right now. So I think this will … this has to be realized, you know, sooner rather than later.  And I think it’s probably dependent on these benchmarks and proper evaluation and tying this with regulation. So these are things that take time in human society and for good reason. But now we already are at two years; you know, give it another two years and it should be really …   LEE: Will AI prescribe your medicines? Write your prescriptions?  BUBECK: I think yes. I think yes.  LEE: OK. Bill?  GATES: Well, I think the next two years, we’ll have massive pilots, and so the amount of use of the AI, still in a copilot-type mode, you know, we should get millions of patient visits, you know, both in general medicine and in the mental health side, as well. And I think that’s going to build up both the data and the confidence to give the AI some additional autonomy. You know, are you going to let it talk to you at night when you’re panicked about your mental health with some ability to escalate? And, you know, I’ve gone so far as to tell politicians with national health systems that if they deploy AI appropriately, that the quality of care, the overload of the doctors, the improvement in the economics will be enough that their voters will be stunned because they just don’t expect this, and, you know, they could be reelected [LAUGHTER] just on this one thing of fixing what is a very overloaded and economically challenged health system in these rich countries.  You know, my personal role is going to be to make sure that in the poorer countries, there isn’t some lag; in fact, in many cases, that we’ll be more aggressive because, you know, we’re comparing to having no access to doctors at all. And, you know, so I think whether it’s India or Africa, there’ll be lessons that are globally valuable because we need medical intelligence. And, you know, thank god AI is going to provide a lot of that.  LEE: Well, on that optimistic note, I think that’s a good way to end. Bill, Seb, really appreciate all of this.   I think the most fundamental prediction we made in the book is that AI would actually find its way into the practice of medicine, and I think that that at least has come true, maybe in different ways than we expected, but it’s come true, and I think it’ll only accelerate from here. So thanks again, both of you.  [TRANSITION MUSIC]  GATES: Yeah. Thanks, you guys.  BUBECK: Thank you, Peter. Thanks, Bill.  LEE: I just always feel such a sense of privilege to have a chance to interact and actually work with people like Bill and Sébastien.    With Bill, I’m always amazed at how practically minded he is. He’s really thinking about the nuts and bolts of what AI might be able to do for people, and his thoughts about underserved parts of the world, the idea that we might actually be able to empower people with access to expert medical knowledge, I think is both inspiring and amazing.   And then, Seb, Sébastien Bubeck, he’s just absolutely a brilliant mind. He has a really firm grip on the deep mathematics of artificial intelligence and brings that to bear in his research and development work. And where that mathematics takes him isn’t just into the nuts and bolts of algorithms but into philosophical questions about the nature of intelligence.   One of the things that Sébastien brought up was the state of evaluation of AI systems. And indeed, he was fairly critical in our conversation. But of course, the world of AI research and development is just moving so fast, and indeed, since we recorded our conversation, OpenAI, in fact, released a new evaluation metric that is directly relevant to medical applications, and that is something called HealthBench. And Microsoft Research also released a new evaluation approach or process called ADeLe.   HealthBench and ADeLe are examples of new approaches to evaluating AI models that are less about testing their knowledge and ability to pass multiple-choice exams and instead are evaluation approaches designed to assess how well AI models are able to complete tasks that actually arise every day in typical healthcare or biomedical research settings. These are examples of really important good work that speak to how well AI models work in the real world of healthcare and biomedical research and how well they can collaborate with human beings in those settings.  You know, I asked Bill and Seb to make some predictions about the future. You know, my own answer, I expect that we’re going to be able to use AI to change how we diagnose patients, change how we decide treatment options.   If you’re a doctor or a nurse and you encounter a patient, you’ll ask questions, do a physical exam, you know, call out for labs just like you do today, but then you’ll be able to engage with AI based on all of that data and just ask, you know, based on all the other people who have gone through the same experience, who have similar data, how were they diagnosed? How were they treated? What were their outcomes? And what does that mean for the patient I have right now? Some people call it the “patients like me” paradigm. And I think that’s going to become real because of AI within our lifetimes. That idea of really grounding the delivery in healthcare and medical practice through data and intelligence, I actually now don’t see any barriers to that future becoming real.  [THEME MUSIC]  I’d like to extend another big thank you to Bill and Sébastien for their time. And to our listeners, as always, it’s a pleasure to have you along for the ride. I hope you’ll join us for our remaining conversations, as well as a second coauthor roundtable with Carey and Zak.   Until next time.   [MUSIC FADES]

Join the event trusted by enterprise leaders for nearly two decades. VB Transform brings together the people building real enterprise AI strategy. Learn more When DeepSeek released its R1 model this January, it wasn’t just another AI announcement. It was a watershed moment that sent shockwaves through the tech industry, forcing industry leaders to reconsider their fundamental approaches to AI development. What makes DeepSeek’s accomplishment remarkable isn’t that the company developed novel capabilities; rather, it was how it achieved comparable results to those delivered by tech heavyweights at a fraction of the cost. In reality, DeepSeek didn’t do anything that hadn’t been done before; its innovation stemmed from pursuing different priorities. As a result, we are now experiencing rapid-fire development along two parallel tracks: efficiency and compute.  As DeepSeek prepares to release its R2 model, and as it concurrently faces the potential of even greater chip restrictions from the U.S., it’s important to look at how it captured so much attention. Engineering around constraints DeepSeek’s arrival, as sudden and dramatic as it was, captivated us all because it showcased the capacity for innovation to thrive even under significant constraints. Faced with U.S. export controls limiting access to cutting-edge AI chips, DeepSeek was forced to find alternative pathways to AI advancement. While U.S. companies pursued performance gains through more powerful hardware, bigger models and better data, DeepSeek focused on optimizing what was available. It implemented known ideas with remarkable execution — and there is novelty in executing what’s known and doing it well. This efficiency-first mindset yielded incredibly impressive results. DeepSeek’s R1 model reportedly matches OpenAI’s capabilities at just 5 to 10% of the operating cost. According to reports, the final training run for DeepSeek’s V3 predecessor cost a mere $6 million — which was described by former Tesla AI scientist Andrej Karpathy as “a joke of a budget” compared to the tens or hundreds of millions spent by U.S. competitors. More strikingly, while OpenAI reportedly spent $500 million training its recent “Orion” model, DeepSeek achieved superior benchmark results for just $5.6 million — less than 1.2% of OpenAI’s investment. If you get starry eyed believing these incredible results were achieved even as DeepSeek was at a severe disadvantage based on its inability to access advanced AI chips, I hate to tell you, but that narrative isn’t entirely accurate (even though it makes a good story). Initial U.S. export controls focused primarily on compute capabilities, not on memory and networking — two crucial components for AI development. That means that the chips DeepSeek had access to were not poor quality chips; their networking and memory capabilities allowed DeepSeek to parallelize operations across many units, a key strategy for running their large model efficiently. This, combined with China’s national push toward controlling the entire vertical stack of AI infrastructure, resulted in accelerated innovation that many Western observers didn’t anticipate. DeepSeek’s advancements were an inevitable part of AI development, but they brought known advancements forward a few years earlier than would have been possible otherwise, and that’s pretty amazing. Pragmatism over process Beyond hardware optimization, DeepSeek’s approach to training data represents another departure from conventional Western practices. Rather than relying solely on web-scraped content, DeepSeek reportedly leveraged significant amounts of synthetic data and outputs from other proprietary models. This is a classic example of model distillation, or the ability to learn from really powerful models. Such an approach, however, raises questions about data privacy and governance that might concern Western enterprise customers. Still, it underscores DeepSeek’s overall pragmatic focus on results over process. The effective use of synthetic data is a key differentiator. Synthetic data can be very effective when it comes to training large models, but you have to be careful; some model architectures handle synthetic data better than others. For instance, transformer-based models with mixture of experts (MoE) architectures like DeepSeek’s tend to be more robust when incorporating synthetic data, while more traditional dense architectures like those used in early Llama models can experience performance degradation or even “model collapse” when trained on too much synthetic content. This architectural sensitivity matters because synthetic data introduces different patterns and distributions compared to real-world data. When a model architecture doesn’t handle synthetic data well, it may learn shortcuts or biases present in the synthetic data generation process rather than generalizable knowledge. This can lead to reduced performance on real-world tasks, increased hallucinations or brittleness when facing novel situations.  Still, DeepSeek’s engineering teams reportedly designed their model architecture specifically with synthetic data integration in mind from the earliest planning stages. This allowed the company to leverage the cost benefits of synthetic data without sacrificing performance. Market reverberations Why does all of this matter? Stock market aside, DeepSeek’s emergence has triggered substantive strategic shifts among industry leaders. Case in point: OpenAI. Sam Altman recently announced plans to release the company’s first “open-weight” language model since 2019. This is a pretty notable pivot for a company that built its business on proprietary systems. It seems DeepSeek’s rise, on top of Llama’s success, has hit OpenAI’s leader hard. Just a month after DeepSeek arrived on the scene, Altman admitted that OpenAI had been “on the wrong side of history” regarding open-source AI.  With OpenAI reportedly spending $7 to 8 billion annually on operations, the economic pressure from efficient alternatives like DeepSeek has become impossible to ignore. As AI scholar Kai-Fu Lee bluntly put it: “You’re spending $7 billion or $8 billion a year, making a massive loss, and here you have a competitor coming in with an open-source model that’s for free.” This necessitates change. This economic reality prompted OpenAI to pursue a massive $40 billion funding round that valued the company at an unprecedented $300 billion. But even with a war chest of funds at its disposal, the fundamental challenge remains: OpenAI’s approach is dramatically more resource-intensive than DeepSeek’s. Beyond model training Another significant trend accelerated by DeepSeek is the shift toward “test-time compute” (TTC). As major AI labs have now trained their models on much of the available public data on the internet, data scarcity is slowing further improvements in pre-training. To get around this, DeepSeek announced a collaboration with Tsinghua University to enable “self-principled critique tuning” (SPCT). This approach trains AI to develop its own rules for judging content and then uses those rules to provide detailed critiques. The system includes a built-in “judge” that evaluates the AI’s answers in real-time, comparing responses against core rules and quality standards. The development is part of a movement towards autonomous self-evaluation and improvement in AI systems in which models use inference time to improve results, rather than simply making models larger during training. DeepSeek calls its system “DeepSeek-GRM” (generalist reward modeling). But, as with its model distillation approach, this could be considered a mix of promise and risk. For example, if the AI develops its own judging criteria, there’s a risk those principles diverge from human values, ethics or context. The rules could end up being overly rigid or biased, optimizing for style over substance, and/or reinforce incorrect assumptions or hallucinations. Additionally, without a human in the loop, issues could arise if the “judge” is flawed or misaligned. It’s a kind of AI talking to itself, without robust external grounding. On top of this, users and developers may not understand why the AI reached a certain conclusion — which feeds into a bigger concern: Should an AI be allowed to decide what is “good” or “correct” based solely on its own logic? These risks shouldn’t be discounted. At the same time, this approach is gaining traction, as again DeepSeek builds on the body of work of others (think OpenAI’s “critique and revise” methods, Anthropic’s constitutional AI or research on self-rewarding agents) to create what is likely the first full-stack application of SPCT in a commercial effort. This could mark a powerful shift in AI autonomy, but there still is a need for rigorous auditing, transparency and safeguards. It’s not just about models getting smarter, but that they remain aligned, interpretable, and trustworthy as they begin critiquing themselves without human guardrails. Moving into the future So, taking all of this into account, the rise of DeepSeek signals a broader shift in the AI industry toward parallel innovation tracks. While companies continue building more powerful compute clusters for next-generation capabilities, there will also be intense focus on finding efficiency gains through software engineering and model architecture improvements to offset the challenges of AI energy consumption, which far outpaces power generation capacity.  Companies are taking note. Microsoft, for example, has halted data center development in multiple regions globally, recalibrating toward a more distributed, efficient infrastructure approach. While still planning to invest approximately $80 billion in AI infrastructure this fiscal year, the company is reallocating resources in response to the efficiency gains DeepSeek introduced to the market. Meta has also responded, With so much movement in such a short time, it becomes somewhat ironic that the U.S. sanctions designed to maintain American AI dominance may have instead accelerated the very innovation they sought to contain. By constraining access to materials, DeepSeek was forced to blaze a new trail. Moving forward, as the industry continues to evolve globally, adaptability for all players will be key. Policies, people and market reactions will continue to shift the ground rules — whether it’s eliminating the AI diffusion rule, a new ban on technology purchases or something else entirely. It’s what we learn from one another and how we respond that will be worth watching. Jae Lee is CEO and co-founder of TwelveLabs. Daily insights on business use cases with VB Daily If you want to impress your boss, VB Daily has you covered. We give you the inside scoop on what companies are doing with generative AI, from regulatory shifts to practical deployments, so you can share insights for maximum ROI. Read our Privacy Policy Thanks for subscribing. Check out more VB newsletters here. An error occured.

Few roles evolve as quickly as that of the modern CIO. A great way to prepare for a future that is largely unknown is to build your adaptability skills through diverse work experiences, says Bhavik Rao, CIO for the Americas at Vertiv. Learn from your wins and your losses and carry on. Stay free of comfort zones and run towards the chaos. Leaders are born of challenges and not from comfort.Bhavik shares what he’s facing now, how he’s navigating it, and the hard-won lessons that helped shape his approach to IT leadership.Here’s what he had to say:What has your career path looked like so far? I actually started my career as a techno-functional consultant working with the public sector. That early experience gave me a solid grounding in both the technical and process side of enterprise systems. From there, I moved into consulting, which really opened up my world. I had the opportunity to work across multiple industries, leading everything from mobile app development and eCommerce deployments to omnichannel initiatives, data platforms, ERP rollouts, and ultimately large-scale digital transformation and IT strategy programs. It was fast paced, challenging, and incredibly rewarding.  That diversity shaped the way I think today. I learned how to adapt quickly, connect dots across domains, and communicate with everyone from developers to CXOs. Eventually, that path led me to Vertiv, where I now serve as the CIO for the Americas, in addition to leading a couple of global towers, such as data/AI and engineering systems, for example. I’ve been fortunate to lead initiatives that drive operational efficiency, scale GenAI adoption, and turn technology into a true business enabler.   Related:What are the highlights along your career path? There have been several defining moments, both wins and challenges, that have shaped how I lead today. One of the most pivotal chapters has been my time at Vertiv. I joined when the company was still owned by private equity. It was an intense, roll-up-your-sleeves kind of environment. Then, in 2020, we went public -- a huge milestone. But just as we were ramping up our digital transformation, COVID hit, and with it came massive supply chain disruptions. In the middle of all that chaos, I was asked to take over a large-scale transformation program that was struggling. bhBhavik RaoIt wasn’t easy. There were legacy challenges, resistance to change, and real execution pressure. But we rallied, restructured the program, and launched it. That experience taught me a lot about leading under pressure, aligning teams around outcomes, and staying focused even when everything feels like it’s shifting. Related:Another major learning moment was earlier in my career when I lost a large national account I’d spent over seven years building. That was a tough one, but it taught me resilience. I learned not to attach my identity to any one outcome and to keep moving forward with purpose. Then, there are the moments of creation, like launching VeGA, our internal GenAI platform at Vertiv. Seeing it go from idea to impact, with thousands of users and 100+ applications, has been incredibly energizing. It reminded me how powerful it is when innovation meets execution. I’ve also learned the power of being a “player-coach.” I don’t believe in leading from a distance. I get involved, understand the challenges on the ground, and then help teams move forward together.  What’s your vision for the future of sovereign AI? For me, sovereign AI isn’t just a regulatory checkbox; it’s about strategic autonomy. At our company, we are trying to be very intentional about how we scale AI responsibly across our global footprint. So, when I think about sovereign AI, I define it as the ability to control how, where, and why AI is built and deployed with full alignment to your business needs, risk posture, and data boundaries. Related:I’ve seen firsthand how AI becomes a competitive advantage only when you have governance, infrastructure flexibility, and contextual intelligence built in. Our work with VeGA, for example, has shown that employees adopt AI much faster when it’s embedded into secure, business-aligned workflows and not just bolted on from the outside. For CIOs, the shift to sovereign AI means: Designing AI infrastructure that can flex whether it’s hosted internally, cloud-based, or hybrid Building internal AI fluency so your teams aren't fully reliant on black-box solutions Creating a framework for trust and explainability, especially as AI touches regulated and legal processes It’s not about doing everything in-house, but it is about knowing what’s mission-critical to control. In my view, sovereign AI is less about isolation and more about intentional ownership. What do you do for fun or to relax? Golf is my go-to. It keeps me grounded and humble! It’s one of those games that’s as much about mindset as it is about mechanics. I try to work out regularly when I am not traveling for work.  I also enjoy traveling with my family and listening to podcasts.   What advice would you give to young people considering a leadership path in IT? Be curious, stay hands-on, don’t rush the title, and focus on impact. Learn the business, not just the tech. Some of the best technologists I’ve worked with are the ones who understand how a supply chain works or how a sale actually closes. Also, don’t be afraid to take on messy, undefined problems. Run toward the chaos. That’s where leadership is born. And finally, surround yourself with people smarter than you. Build teams that challenge you. That’s where real growth happens.