• So, summer is here, and it seems like the game releases have hit a dull patch. Kotaku’s Weekend Guide highlights some games that are worth revisiting, even if it feels like there's not much exciting happening right now. Just a few titles to keep us occupied until the holiday releases start rolling in. Honestly, it’s nice to take a break and look back at some old favorites instead of waiting for something new.

    #Gaming #WeekendGuide #SummerLull #Kotaku #GameRecommendations
    So, summer is here, and it seems like the game releases have hit a dull patch. Kotaku’s Weekend Guide highlights some games that are worth revisiting, even if it feels like there's not much exciting happening right now. Just a few titles to keep us occupied until the holiday releases start rolling in. Honestly, it’s nice to take a break and look back at some old favorites instead of waiting for something new. #Gaming #WeekendGuide #SummerLull #Kotaku #GameRecommendations
    KOTAKU.COM
    Kotaku’s Weekend Guide: 5 Great Games We Can't Wait To Get Back To
    We’ve hit peak summer and probably the biggest lull in the release calendar we’re going to get until the holiday. That doesn’t mean a lot of great stuff didn’t still come out this week. What it does mean is that we have time to highlight things that
    Like
    Love
    Wow
    Sad
    Angry
    104
    1 Comments 0 Shares 0 Reviews
  • After two decades, the incredible journey of David Gallagher, the charming voice behind Riku in Kingdom Hearts, takes a thrilling turn! At 40 years old, he’s finally jumping into the magical worlds of Disney RPGs that he’s helped bring to life. It’s never too late to embrace new adventures, to rediscover joy, and to dive into the experiences that shaped our childhoods!

    Let this remind us all that life is a continuous adventure filled with opportunities to explore and grow. Whether it’s revisiting old favorites or trying something completely new, let’s keep our spirits high and our hearts open!

    Embrace your passions and inspire others to do the same!

    #KingdomHearts #
    After two decades, the incredible journey of David Gallagher, the charming voice behind Riku in Kingdom Hearts, takes a thrilling turn! 🌟 At 40 years old, he’s finally jumping into the magical worlds of Disney RPGs that he’s helped bring to life. It’s never too late to embrace new adventures, to rediscover joy, and to dive into the experiences that shaped our childhoods! 🎮✨ Let this remind us all that life is a continuous adventure filled with opportunities to explore and grow. Whether it’s revisiting old favorites or trying something completely new, let’s keep our spirits high and our hearts open! 💖 Embrace your passions and inspire others to do the same! #KingdomHearts #
    KOTAKU.COM
    After Two Decades, One Of The Stars Of Kingdom Hearts Is Finally Playing The Games
    The original Kingdom Hearts came to the PlayStation 2 in 2002 when David Gallagher was 17 years old. The actor, now 40, has played deuteragonist and noted “sexy guy” who “the gamer girls” go crazy for, Riku, for more than half his lifetime, but someh
    1 Comments 0 Shares 0 Reviews
  • Aga Khan Award for Architecture 2025 announces 19 shortlisted projects from 15 countries

    html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" ";
    19 shortlisted projects for the 2025 Award cycle were revealed by the Aga Khan Award for Architecture. A portion of the million prize, one of the biggest in architecture, will be awarded to the winning proposals. Out of the 369 projects nominated for the 16th Award Cycle, an independent Master Jury chose the 19 shortlisted projects from 15 countries.The nine members of the Master Jury for the 16th Award cycle include Azra Akšamija, Noura Al-Sayeh Holtrop, Lucia Allais, David Basulto, Yvonne Farrell, Kabage Karanja, Yacouba Konaté, Hassan Radoine, and Mun Summ Wong.His Late Highness Prince Karim Aga Khan IV created the Aga Khan Award for Architecture in 1977 to recognize and promote architectural ideas that effectively meet the needs and goals of communities where Muslims are a major population. Nearly 10,000 construction projects have been documented since the award's inception 48 years ago, and 128 projects have been granted it. The AKAA's selection method places a strong emphasis on architecture that stimulates and responds to people's cultural ambitions in addition to meeting their physical, social, and economic demands.The Aga Khan Award for Architecture is governed by a Steering Committee chaired by His Highness the Aga Khan. The other members of the Steering Committee are Meisa Batayneh, Principal Architect, Founder, maisam architects and engineers, Amman, Jordan; Souleymane Bachir Diagne, Professor of Philosophy and Francophone Studies, Columbia University, New York, United States of America; Lesley Lokko, Founder & Director, African Futures Institute, Accra, Ghana; Gülru Necipoğlu, Director and Professor, Aga Khan Program for Islamic Architecture, Harvard University, Cambridge, United States of America; Hashim Sarkis, Founder & Principal, Hashim Sarkis Studios; Dean, School of Architecture and Planning, Massachusetts Institute of Technology, Cambridge, United States of America; and Sarah M. Whiting, Partner, WW Architecture; Dean and Josep Lluís Sert Professor of Architecture, Graduate School of Design, Harvard University, Cambridge, United States of America. Farrokh Derakhshani is the Director of the Award.Examples of outstanding architecture in the areas of modern design, social housing, community development and enhancement, historic preservation, reuse and area conservation, landscape design, and environmental enhancement are recognized by the Aga Khan Award for Architecture.Building plans that creatively utilize local resources and relevant technologies, as well as initiatives that could spur such initiatives abroad, are given special consideration. It should be mentioned that in addition to honoring architects, the Award also recognizes towns, builders, clients, master craftspeople, and engineers who have contributed significantly to the project.Projects had to be completed between January 1, 2018, and December 31, 2023, and they had to have been operational for a minimum of one year in order to be eligible for consideration in the 2025 Award cycle. The Award is not available for projects that His Highness the Aga Khan or any of the Aga Khan Development Networkinstitutions have commissioned.See the 19 shortlisted projects with their short project descriptions competing for the 2025 Award Cycle:Khudi Bari. Image © Aga Khan Trust for Culture / City SyntaxBangladeshKhudi Bari, in various locations, by Marina Tabassum ArchitectsMarina Tabassum Architects' Khudi Bari, which can be readily disassembled and reassembled to suit the needs of the users, is a replicable solution for displaced communities impacted by geographic and climatic changes.West Wusutu Village Community Centre. Image © Aga Khan Trust for Culture / Dou YujunChinaWest Wusutu Village Community Centre, Hohhot, Inner Mongolia, by Zhang PengjuIn addition to meeting the religious demands of the local Hui Muslims, Zhang Pengju's West Wusutu Village Community Centre in Hohhot, Inner Mongolia, offers social and cultural spaces for locals and artists. Constructed from recycled bricks, it features multipurpose indoor and outdoor areas that promote communal harmony.Revitalisation of Historic Esna. Image © Aga Khan Trust for Culture / Ahmed SalemEgyptRevitalisation of Historic Esna, by Takween Integrated Community DevelopmentBy using physical interventions, socioeconomic projects, and creative urban planning techniques, Takween Integrated Community Development's Revitalization of Historic Esna tackles the issues of cultural tourism in Upper Egypt and turns the once-forgotten area around the Temple of Khnum into a thriving historic city.The Arc at Green School. Image © Aga Khan Trust for Culture / Andreas Perbowo WidityawanIndonesiaThe Arc at Green School, in Bali, by IBUKU / Elora HardyAfter 15 years of bamboo experimenting at the Green School Bali, IBUKU/Elora Hardy created The Arc at Green School. The Arc is a brand-new community wellness facility built on the foundations of a temporary gym. High-precision engineering and regional handicraft are combined in this construction.Islamic Centre Nurul Yaqin Mosque. Image © Aga Khan Trust for Culture / Andreas Perbowo WidityawanIndonesiaIslamic Centre Nurul Yaqin Mosque, in Palu, Central Sulawesi, by Dave Orlando and Fandy GunawanDave Orlando and Fandy Gunawan built the Islamic Center Nurul Yaqin Mosque in Palu, Central Sulawesi, on the location of a previous mosque that was damaged by a 2018 tsunami. There is a place for worship and assembly at the new Islamic Center. Surrounded by a shallow reflecting pool that may be drained to make room for more guests, it is open to the countryside.Microlibrary Warak Kayu. Image © Aga Khan Trust for Culture / Andreas Perbowo WidityawanIndonesiaMicrolibraries in various cities, by SHAU / Daliana Suryawinata, Florian HeinzelmannFlorian Heinzelmann, the project's initiator, works with stakeholders at all levels to provide high-quality public spaces in a number of Indonesian parks and kampungs through microlibraries in different towns run by SHAU/Daliana Suryawinata. So far, six have been constructed, and by 2045, 100 are planned.Majara Residence. Image © Aga Khan Trust for Culture / Deed StudioIranMajara Complex and Community Redevelopment, in Hormuz Island by ZAV Architects / Mohamadreza GhodousiThe Majara Complex and Community Redevelopment on Hormuz Island, designed by ZAV Architects and Mohamadreza Ghodousi, is well-known for its vibrant domes that offer eco-friendly lodging for visitors visiting Hormuz's distinctive scenery. In addition to providing new amenities for the islanders who visit to socialize, pray, or utilize the library, it was constructed by highly trained local laborers.Jahad Metro Plaza. Image © Aga Khan Trust for Culture / Deed StudioIranJahad Metro Plaza in Tehran, by KA Architecture StudioKA Architecture Studio's Jahad Metro Plaza in Tehran was constructed to replace the dilapidated old buildings. It turned the location into a beloved pedestrian-friendly landmark. The arched vaults, which are covered in locally manufactured brick, vary in height to let air and light into the area they are protecting.Khan Jaljulia Restoration. Image © Aga Khan Trust for Culture / Mikaela BurstowIsraelKhan Jaljulia Restoration in Jaljulia by Elias KhuriElias Khuri's Khan Jaljulia Restoration is a cost-effective intervention set amidst the remnants of a 14th-century Khan in Jaljulia. By converting the abandoned historical location into a bustling public area for social gatherings, it helps the locals rediscover their cultural history.Campus Startup Lions. Image © Aga Khan Trust for Culture / Christopher Wilton-SteerKenyaCampus Startup Lions, in Turkana by Kéré ArchitectsKéré Architecture's Campus Startup Lions in Turkana is an educational and entrepreneurial center that offers a venue for community involvement, business incubation, and technology-driven education. The design incorporates solar energy, rainwater harvesting, and tall ventilation towers that resemble the nearby termite mounds, and it was constructed using local volcanic stone.Lalla Yeddouna Square. Image © Aga Khan Trust for Culture / Amine HouariMoroccoRevitalisation of Lalla Yeddouna Square in the medina of Fez, by Mossessian Architecture and Yassir Khalil StudioMossessian Architecture and Yassir Khalil Studio's revitalization of Lalla Yeddouna Square in the Fez medina aims to improve pedestrian circulation and reestablish a connection to the waterfront. For the benefit of locals, craftspeople, and tourists from around the globe, existing buildings were maintained and new areas created.Vision Pakistan. Image © Aga Khan Trust for Culture / Usman Saqib ZuberiPakistanVision Pakistan, in Islamabad by DB Studios / Mohammad Saifullah SiddiquiA tailoring training center run by Vision Pakistan, a nonprofit organization dedicated to empowering underprivileged adolescents, is located in Islamabad by DB Studios/Mohammad Saifullah Siddiqui. Situated in a crowded neighborhood, this multi-story building features flashy jaalis influenced by Arab and Pakistani crafts, echoing the city's 1960s design.Denso Hall Rahguzar Project. Image © Aga Khan Trust for Culture / Usman Saqib ZuberiPakistanDenso Hall Rahguzar Project, in Karachi by Heritage Foundation Pakistan / Yasmeen LariThe Heritage Foundation of Pakistan/Yasmeen Lari's Denso Hall Rahguzar Project in Karachi is a heritage-led eco-urban enclave that was built with low-carbon materials in response to the city's severe climate, which is prone to heat waves and floods. The freshly planted "forests" are irrigated by the handcrafted terracotta cobbles, which absorb rainfall and cool and purify the air.Wonder Cabinet. Image © Aga Khan Trust for Culture / Mikaela BurstowPalestineWonder Cabinet, in Bethlehem by AAU AnastasThe architects at AAU Anastas established Wonder Cabinet, a multifunctional, nonprofit exhibition and production venue in Bethlehem. The three-story concrete building was constructed with the help of regional contractors and artisans, and it is quickly emerging as a major center for learning, design, craft, and innovation.The Ned. Image © Aga Khan Trust for Culture / Cemal EmdenQatarThe Ned Hotel, in Doha by David Chipperfield ArchitectsThe Ministry of Interior was housed in the Ned Hotel in Doha, which was designed by David Chipperfield Architects. Its Middle Eastern brutalist building was meticulously transformed into a 90-room boutique hotel, thereby promoting architectural revitalization in the region.Shamalat Cultural Centre. Image © Aga Khan Trust for Culture / Hassan Al ShattiSaudi ArabiaShamalat Cultural Centre, in Riyadh, by Syn Architects / Sara Alissa, Nojoud AlsudairiOn the outskirts of Diriyah, the Shamalat Cultural Centre in Riyadh was created by Syn Architects/Sara Alissa, Nojoud Alsudairi. It was created from an old mud home that artist Maha Malluh had renovated. The center, which aims to incorporate historic places into daily life, provides a sensitive viewpoint on heritage conservation in the area by contrasting the old and the contemporary.Rehabilitation and Extension of Dakar Railway Station. Image © Aga Khan Trust for Culture / Sylvain CherkaouiSenegalRehabilitation and Extension of Dakar Railway Station, in Dakar by Ga2DIn order to accommodate the passengers of a new express train line, Ga2D extended and renovated Dakar train Station, which purposefully contrasts the old and modern buildings. The forecourt was once again open to pedestrian traffic after vehicular traffic was limited to the rear of the property.Rami Library. Image © Aga Khan Trust for Culture / Cemal EmdenTürkiyeRami Library, by Han Tümertekin Design & ConsultancyThe largest library in Istanbul is the Rami Library, designed by Han Tümertekin Design & Consultancy. It occupied the former Rami Barracks, a sizable, single-story building with enormous volumes that was constructed in the eighteenth century. In order to accommodate new library operations while maintaining the structure's original spatial features, a minimal intervention method was used.Morocco Pavilion Expo Dubai 2020. Image © Aga Khan Trust for Culture / Deed StudioUnited Arab EmiratesMorocco Pavilion Expo Dubai 2020, by Oualalou + ChoiOualalou + Choi's Morocco Pavilion Expo Dubai 2020 is intended to last beyond Expo 2020 and be transformed into a cultural center. The pavilion is a trailblazer in the development of large-scale rammed earth building techniques. Its use of passive cooling techniques, which minimize the need for mechanical air conditioning, earned it the gold LEED accreditation.At each project location, independent professionals such as architects, conservation specialists, planners, and structural engineers have conducted thorough evaluations of the nominated projects. This summer, the Master Jury convenes once more to analyze the on-site evaluations and choose the ultimate Award winners.The top image in the article: The Arc at Green School. Image © Aga Khan Trust for Culture / Andreas Perbowo Widityawan.> via Aga Khan Award for Architecture
    #aga #khan #award #architecture #announces
    Aga Khan Award for Architecture 2025 announces 19 shortlisted projects from 15 countries
    html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "; 19 shortlisted projects for the 2025 Award cycle were revealed by the Aga Khan Award for Architecture. A portion of the million prize, one of the biggest in architecture, will be awarded to the winning proposals. Out of the 369 projects nominated for the 16th Award Cycle, an independent Master Jury chose the 19 shortlisted projects from 15 countries.The nine members of the Master Jury for the 16th Award cycle include Azra Akšamija, Noura Al-Sayeh Holtrop, Lucia Allais, David Basulto, Yvonne Farrell, Kabage Karanja, Yacouba Konaté, Hassan Radoine, and Mun Summ Wong.His Late Highness Prince Karim Aga Khan IV created the Aga Khan Award for Architecture in 1977 to recognize and promote architectural ideas that effectively meet the needs and goals of communities where Muslims are a major population. Nearly 10,000 construction projects have been documented since the award's inception 48 years ago, and 128 projects have been granted it. The AKAA's selection method places a strong emphasis on architecture that stimulates and responds to people's cultural ambitions in addition to meeting their physical, social, and economic demands.The Aga Khan Award for Architecture is governed by a Steering Committee chaired by His Highness the Aga Khan. The other members of the Steering Committee are Meisa Batayneh, Principal Architect, Founder, maisam architects and engineers, Amman, Jordan; Souleymane Bachir Diagne, Professor of Philosophy and Francophone Studies, Columbia University, New York, United States of America; Lesley Lokko, Founder & Director, African Futures Institute, Accra, Ghana; Gülru Necipoğlu, Director and Professor, Aga Khan Program for Islamic Architecture, Harvard University, Cambridge, United States of America; Hashim Sarkis, Founder & Principal, Hashim Sarkis Studios; Dean, School of Architecture and Planning, Massachusetts Institute of Technology, Cambridge, United States of America; and Sarah M. Whiting, Partner, WW Architecture; Dean and Josep Lluís Sert Professor of Architecture, Graduate School of Design, Harvard University, Cambridge, United States of America. Farrokh Derakhshani is the Director of the Award.Examples of outstanding architecture in the areas of modern design, social housing, community development and enhancement, historic preservation, reuse and area conservation, landscape design, and environmental enhancement are recognized by the Aga Khan Award for Architecture.Building plans that creatively utilize local resources and relevant technologies, as well as initiatives that could spur such initiatives abroad, are given special consideration. It should be mentioned that in addition to honoring architects, the Award also recognizes towns, builders, clients, master craftspeople, and engineers who have contributed significantly to the project.Projects had to be completed between January 1, 2018, and December 31, 2023, and they had to have been operational for a minimum of one year in order to be eligible for consideration in the 2025 Award cycle. The Award is not available for projects that His Highness the Aga Khan or any of the Aga Khan Development Networkinstitutions have commissioned.See the 19 shortlisted projects with their short project descriptions competing for the 2025 Award Cycle:Khudi Bari. Image © Aga Khan Trust for Culture / City SyntaxBangladeshKhudi Bari, in various locations, by Marina Tabassum ArchitectsMarina Tabassum Architects' Khudi Bari, which can be readily disassembled and reassembled to suit the needs of the users, is a replicable solution for displaced communities impacted by geographic and climatic changes.West Wusutu Village Community Centre. Image © Aga Khan Trust for Culture / Dou YujunChinaWest Wusutu Village Community Centre, Hohhot, Inner Mongolia, by Zhang PengjuIn addition to meeting the religious demands of the local Hui Muslims, Zhang Pengju's West Wusutu Village Community Centre in Hohhot, Inner Mongolia, offers social and cultural spaces for locals and artists. Constructed from recycled bricks, it features multipurpose indoor and outdoor areas that promote communal harmony.Revitalisation of Historic Esna. Image © Aga Khan Trust for Culture / Ahmed SalemEgyptRevitalisation of Historic Esna, by Takween Integrated Community DevelopmentBy using physical interventions, socioeconomic projects, and creative urban planning techniques, Takween Integrated Community Development's Revitalization of Historic Esna tackles the issues of cultural tourism in Upper Egypt and turns the once-forgotten area around the Temple of Khnum into a thriving historic city.The Arc at Green School. Image © Aga Khan Trust for Culture / Andreas Perbowo WidityawanIndonesiaThe Arc at Green School, in Bali, by IBUKU / Elora HardyAfter 15 years of bamboo experimenting at the Green School Bali, IBUKU/Elora Hardy created The Arc at Green School. The Arc is a brand-new community wellness facility built on the foundations of a temporary gym. High-precision engineering and regional handicraft are combined in this construction.Islamic Centre Nurul Yaqin Mosque. Image © Aga Khan Trust for Culture / Andreas Perbowo WidityawanIndonesiaIslamic Centre Nurul Yaqin Mosque, in Palu, Central Sulawesi, by Dave Orlando and Fandy GunawanDave Orlando and Fandy Gunawan built the Islamic Center Nurul Yaqin Mosque in Palu, Central Sulawesi, on the location of a previous mosque that was damaged by a 2018 tsunami. There is a place for worship and assembly at the new Islamic Center. Surrounded by a shallow reflecting pool that may be drained to make room for more guests, it is open to the countryside.Microlibrary Warak Kayu. Image © Aga Khan Trust for Culture / Andreas Perbowo WidityawanIndonesiaMicrolibraries in various cities, by SHAU / Daliana Suryawinata, Florian HeinzelmannFlorian Heinzelmann, the project's initiator, works with stakeholders at all levels to provide high-quality public spaces in a number of Indonesian parks and kampungs through microlibraries in different towns run by SHAU/Daliana Suryawinata. So far, six have been constructed, and by 2045, 100 are planned.Majara Residence. Image © Aga Khan Trust for Culture / Deed StudioIranMajara Complex and Community Redevelopment, in Hormuz Island by ZAV Architects / Mohamadreza GhodousiThe Majara Complex and Community Redevelopment on Hormuz Island, designed by ZAV Architects and Mohamadreza Ghodousi, is well-known for its vibrant domes that offer eco-friendly lodging for visitors visiting Hormuz's distinctive scenery. In addition to providing new amenities for the islanders who visit to socialize, pray, or utilize the library, it was constructed by highly trained local laborers.Jahad Metro Plaza. Image © Aga Khan Trust for Culture / Deed StudioIranJahad Metro Plaza in Tehran, by KA Architecture StudioKA Architecture Studio's Jahad Metro Plaza in Tehran was constructed to replace the dilapidated old buildings. It turned the location into a beloved pedestrian-friendly landmark. The arched vaults, which are covered in locally manufactured brick, vary in height to let air and light into the area they are protecting.Khan Jaljulia Restoration. Image © Aga Khan Trust for Culture / Mikaela BurstowIsraelKhan Jaljulia Restoration in Jaljulia by Elias KhuriElias Khuri's Khan Jaljulia Restoration is a cost-effective intervention set amidst the remnants of a 14th-century Khan in Jaljulia. By converting the abandoned historical location into a bustling public area for social gatherings, it helps the locals rediscover their cultural history.Campus Startup Lions. Image © Aga Khan Trust for Culture / Christopher Wilton-SteerKenyaCampus Startup Lions, in Turkana by Kéré ArchitectsKéré Architecture's Campus Startup Lions in Turkana is an educational and entrepreneurial center that offers a venue for community involvement, business incubation, and technology-driven education. The design incorporates solar energy, rainwater harvesting, and tall ventilation towers that resemble the nearby termite mounds, and it was constructed using local volcanic stone.Lalla Yeddouna Square. Image © Aga Khan Trust for Culture / Amine HouariMoroccoRevitalisation of Lalla Yeddouna Square in the medina of Fez, by Mossessian Architecture and Yassir Khalil StudioMossessian Architecture and Yassir Khalil Studio's revitalization of Lalla Yeddouna Square in the Fez medina aims to improve pedestrian circulation and reestablish a connection to the waterfront. For the benefit of locals, craftspeople, and tourists from around the globe, existing buildings were maintained and new areas created.Vision Pakistan. Image © Aga Khan Trust for Culture / Usman Saqib ZuberiPakistanVision Pakistan, in Islamabad by DB Studios / Mohammad Saifullah SiddiquiA tailoring training center run by Vision Pakistan, a nonprofit organization dedicated to empowering underprivileged adolescents, is located in Islamabad by DB Studios/Mohammad Saifullah Siddiqui. Situated in a crowded neighborhood, this multi-story building features flashy jaalis influenced by Arab and Pakistani crafts, echoing the city's 1960s design.Denso Hall Rahguzar Project. Image © Aga Khan Trust for Culture / Usman Saqib ZuberiPakistanDenso Hall Rahguzar Project, in Karachi by Heritage Foundation Pakistan / Yasmeen LariThe Heritage Foundation of Pakistan/Yasmeen Lari's Denso Hall Rahguzar Project in Karachi is a heritage-led eco-urban enclave that was built with low-carbon materials in response to the city's severe climate, which is prone to heat waves and floods. The freshly planted "forests" are irrigated by the handcrafted terracotta cobbles, which absorb rainfall and cool and purify the air.Wonder Cabinet. Image © Aga Khan Trust for Culture / Mikaela BurstowPalestineWonder Cabinet, in Bethlehem by AAU AnastasThe architects at AAU Anastas established Wonder Cabinet, a multifunctional, nonprofit exhibition and production venue in Bethlehem. The three-story concrete building was constructed with the help of regional contractors and artisans, and it is quickly emerging as a major center for learning, design, craft, and innovation.The Ned. Image © Aga Khan Trust for Culture / Cemal EmdenQatarThe Ned Hotel, in Doha by David Chipperfield ArchitectsThe Ministry of Interior was housed in the Ned Hotel in Doha, which was designed by David Chipperfield Architects. Its Middle Eastern brutalist building was meticulously transformed into a 90-room boutique hotel, thereby promoting architectural revitalization in the region.Shamalat Cultural Centre. Image © Aga Khan Trust for Culture / Hassan Al ShattiSaudi ArabiaShamalat Cultural Centre, in Riyadh, by Syn Architects / Sara Alissa, Nojoud AlsudairiOn the outskirts of Diriyah, the Shamalat Cultural Centre in Riyadh was created by Syn Architects/Sara Alissa, Nojoud Alsudairi. It was created from an old mud home that artist Maha Malluh had renovated. The center, which aims to incorporate historic places into daily life, provides a sensitive viewpoint on heritage conservation in the area by contrasting the old and the contemporary.Rehabilitation and Extension of Dakar Railway Station. Image © Aga Khan Trust for Culture / Sylvain CherkaouiSenegalRehabilitation and Extension of Dakar Railway Station, in Dakar by Ga2DIn order to accommodate the passengers of a new express train line, Ga2D extended and renovated Dakar train Station, which purposefully contrasts the old and modern buildings. The forecourt was once again open to pedestrian traffic after vehicular traffic was limited to the rear of the property.Rami Library. Image © Aga Khan Trust for Culture / Cemal EmdenTürkiyeRami Library, by Han Tümertekin Design & ConsultancyThe largest library in Istanbul is the Rami Library, designed by Han Tümertekin Design & Consultancy. It occupied the former Rami Barracks, a sizable, single-story building with enormous volumes that was constructed in the eighteenth century. In order to accommodate new library operations while maintaining the structure's original spatial features, a minimal intervention method was used.Morocco Pavilion Expo Dubai 2020. Image © Aga Khan Trust for Culture / Deed StudioUnited Arab EmiratesMorocco Pavilion Expo Dubai 2020, by Oualalou + ChoiOualalou + Choi's Morocco Pavilion Expo Dubai 2020 is intended to last beyond Expo 2020 and be transformed into a cultural center. The pavilion is a trailblazer in the development of large-scale rammed earth building techniques. Its use of passive cooling techniques, which minimize the need for mechanical air conditioning, earned it the gold LEED accreditation.At each project location, independent professionals such as architects, conservation specialists, planners, and structural engineers have conducted thorough evaluations of the nominated projects. This summer, the Master Jury convenes once more to analyze the on-site evaluations and choose the ultimate Award winners.The top image in the article: The Arc at Green School. Image © Aga Khan Trust for Culture / Andreas Perbowo Widityawan.> via Aga Khan Award for Architecture #aga #khan #award #architecture #announces
    WORLDARCHITECTURE.ORG
    Aga Khan Award for Architecture 2025 announces 19 shortlisted projects from 15 countries
    html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd" 19 shortlisted projects for the 2025 Award cycle were revealed by the Aga Khan Award for Architecture (AKAA). A portion of the $1 million prize, one of the biggest in architecture, will be awarded to the winning proposals. Out of the 369 projects nominated for the 16th Award Cycle (2023-2025), an independent Master Jury chose the 19 shortlisted projects from 15 countries.The nine members of the Master Jury for the 16th Award cycle include Azra Akšamija, Noura Al-Sayeh Holtrop, Lucia Allais, David Basulto, Yvonne Farrell, Kabage Karanja, Yacouba Konaté, Hassan Radoine, and Mun Summ Wong.His Late Highness Prince Karim Aga Khan IV created the Aga Khan Award for Architecture in 1977 to recognize and promote architectural ideas that effectively meet the needs and goals of communities where Muslims are a major population. Nearly 10,000 construction projects have been documented since the award's inception 48 years ago, and 128 projects have been granted it. The AKAA's selection method places a strong emphasis on architecture that stimulates and responds to people's cultural ambitions in addition to meeting their physical, social, and economic demands.The Aga Khan Award for Architecture is governed by a Steering Committee chaired by His Highness the Aga Khan. The other members of the Steering Committee are Meisa Batayneh, Principal Architect, Founder, maisam architects and engineers, Amman, Jordan; Souleymane Bachir Diagne, Professor of Philosophy and Francophone Studies, Columbia University, New York, United States of America; Lesley Lokko, Founder & Director, African Futures Institute, Accra, Ghana; Gülru Necipoğlu, Director and Professor, Aga Khan Program for Islamic Architecture, Harvard University, Cambridge, United States of America; Hashim Sarkis, Founder & Principal, Hashim Sarkis Studios (HSS); Dean, School of Architecture and Planning, Massachusetts Institute of Technology, Cambridge, United States of America; and Sarah M. Whiting, Partner, WW Architecture; Dean and Josep Lluís Sert Professor of Architecture, Graduate School of Design, Harvard University, Cambridge, United States of America. Farrokh Derakhshani is the Director of the Award.Examples of outstanding architecture in the areas of modern design, social housing, community development and enhancement, historic preservation, reuse and area conservation, landscape design, and environmental enhancement are recognized by the Aga Khan Award for Architecture.Building plans that creatively utilize local resources and relevant technologies, as well as initiatives that could spur such initiatives abroad, are given special consideration. It should be mentioned that in addition to honoring architects, the Award also recognizes towns, builders, clients, master craftspeople, and engineers who have contributed significantly to the project.Projects had to be completed between January 1, 2018, and December 31, 2023, and they had to have been operational for a minimum of one year in order to be eligible for consideration in the 2025 Award cycle. The Award is not available for projects that His Highness the Aga Khan or any of the Aga Khan Development Network (AKDN) institutions have commissioned.See the 19 shortlisted projects with their short project descriptions competing for the 2025 Award Cycle:Khudi Bari. Image © Aga Khan Trust for Culture / City Syntax (F. M. Faruque Abdullah Shawon, H. M. Fozla Rabby Apurbo)BangladeshKhudi Bari, in various locations, by Marina Tabassum ArchitectsMarina Tabassum Architects' Khudi Bari, which can be readily disassembled and reassembled to suit the needs of the users, is a replicable solution for displaced communities impacted by geographic and climatic changes.West Wusutu Village Community Centre. Image © Aga Khan Trust for Culture / Dou Yujun (photographer)ChinaWest Wusutu Village Community Centre, Hohhot, Inner Mongolia, by Zhang PengjuIn addition to meeting the religious demands of the local Hui Muslims, Zhang Pengju's West Wusutu Village Community Centre in Hohhot, Inner Mongolia, offers social and cultural spaces for locals and artists. Constructed from recycled bricks, it features multipurpose indoor and outdoor areas that promote communal harmony.Revitalisation of Historic Esna. Image © Aga Khan Trust for Culture / Ahmed Salem (photographer)EgyptRevitalisation of Historic Esna, by Takween Integrated Community DevelopmentBy using physical interventions, socioeconomic projects, and creative urban planning techniques, Takween Integrated Community Development's Revitalization of Historic Esna tackles the issues of cultural tourism in Upper Egypt and turns the once-forgotten area around the Temple of Khnum into a thriving historic city.The Arc at Green School. Image © Aga Khan Trust for Culture / Andreas Perbowo Widityawan (photographer)IndonesiaThe Arc at Green School, in Bali, by IBUKU / Elora HardyAfter 15 years of bamboo experimenting at the Green School Bali, IBUKU/Elora Hardy created The Arc at Green School. The Arc is a brand-new community wellness facility built on the foundations of a temporary gym. High-precision engineering and regional handicraft are combined in this construction.Islamic Centre Nurul Yaqin Mosque. Image © Aga Khan Trust for Culture / Andreas Perbowo Widityawan (photographer)IndonesiaIslamic Centre Nurul Yaqin Mosque, in Palu, Central Sulawesi, by Dave Orlando and Fandy GunawanDave Orlando and Fandy Gunawan built the Islamic Center Nurul Yaqin Mosque in Palu, Central Sulawesi, on the location of a previous mosque that was damaged by a 2018 tsunami. There is a place for worship and assembly at the new Islamic Center. Surrounded by a shallow reflecting pool that may be drained to make room for more guests, it is open to the countryside.Microlibrary Warak Kayu. Image © Aga Khan Trust for Culture / Andreas Perbowo Widityawan (photographer)IndonesiaMicrolibraries in various cities, by SHAU / Daliana Suryawinata, Florian HeinzelmannFlorian Heinzelmann, the project's initiator, works with stakeholders at all levels to provide high-quality public spaces in a number of Indonesian parks and kampungs through microlibraries in different towns run by SHAU/Daliana Suryawinata. So far, six have been constructed, and by 2045, 100 are planned.Majara Residence. Image © Aga Khan Trust for Culture / Deed Studio (photographer)IranMajara Complex and Community Redevelopment, in Hormuz Island by ZAV Architects / Mohamadreza GhodousiThe Majara Complex and Community Redevelopment on Hormuz Island, designed by ZAV Architects and Mohamadreza Ghodousi, is well-known for its vibrant domes that offer eco-friendly lodging for visitors visiting Hormuz's distinctive scenery. In addition to providing new amenities for the islanders who visit to socialize, pray, or utilize the library, it was constructed by highly trained local laborers.Jahad Metro Plaza. Image © Aga Khan Trust for Culture / Deed Studio (photographer)IranJahad Metro Plaza in Tehran, by KA Architecture StudioKA Architecture Studio's Jahad Metro Plaza in Tehran was constructed to replace the dilapidated old buildings. It turned the location into a beloved pedestrian-friendly landmark. The arched vaults, which are covered in locally manufactured brick, vary in height to let air and light into the area they are protecting.Khan Jaljulia Restoration. Image © Aga Khan Trust for Culture / Mikaela Burstow (photographer)IsraelKhan Jaljulia Restoration in Jaljulia by Elias KhuriElias Khuri's Khan Jaljulia Restoration is a cost-effective intervention set amidst the remnants of a 14th-century Khan in Jaljulia. By converting the abandoned historical location into a bustling public area for social gatherings, it helps the locals rediscover their cultural history.Campus Startup Lions. Image © Aga Khan Trust for Culture / Christopher Wilton-Steer (photographer)KenyaCampus Startup Lions, in Turkana by Kéré ArchitectsKéré Architecture's Campus Startup Lions in Turkana is an educational and entrepreneurial center that offers a venue for community involvement, business incubation, and technology-driven education. The design incorporates solar energy, rainwater harvesting, and tall ventilation towers that resemble the nearby termite mounds, and it was constructed using local volcanic stone.Lalla Yeddouna Square. Image © Aga Khan Trust for Culture / Amine Houari (photographer)MoroccoRevitalisation of Lalla Yeddouna Square in the medina of Fez, by Mossessian Architecture and Yassir Khalil StudioMossessian Architecture and Yassir Khalil Studio's revitalization of Lalla Yeddouna Square in the Fez medina aims to improve pedestrian circulation and reestablish a connection to the waterfront. For the benefit of locals, craftspeople, and tourists from around the globe, existing buildings were maintained and new areas created.Vision Pakistan. Image © Aga Khan Trust for Culture / Usman Saqib Zuberi (photographer)PakistanVision Pakistan, in Islamabad by DB Studios / Mohammad Saifullah SiddiquiA tailoring training center run by Vision Pakistan, a nonprofit organization dedicated to empowering underprivileged adolescents, is located in Islamabad by DB Studios/Mohammad Saifullah Siddiqui. Situated in a crowded neighborhood, this multi-story building features flashy jaalis influenced by Arab and Pakistani crafts, echoing the city's 1960s design.Denso Hall Rahguzar Project. Image © Aga Khan Trust for Culture / Usman Saqib Zuberi (photographer)PakistanDenso Hall Rahguzar Project, in Karachi by Heritage Foundation Pakistan / Yasmeen LariThe Heritage Foundation of Pakistan/Yasmeen Lari's Denso Hall Rahguzar Project in Karachi is a heritage-led eco-urban enclave that was built with low-carbon materials in response to the city's severe climate, which is prone to heat waves and floods. The freshly planted "forests" are irrigated by the handcrafted terracotta cobbles, which absorb rainfall and cool and purify the air.Wonder Cabinet. Image © Aga Khan Trust for Culture / Mikaela Burstow (photographer)PalestineWonder Cabinet, in Bethlehem by AAU AnastasThe architects at AAU Anastas established Wonder Cabinet, a multifunctional, nonprofit exhibition and production venue in Bethlehem. The three-story concrete building was constructed with the help of regional contractors and artisans, and it is quickly emerging as a major center for learning, design, craft, and innovation.The Ned. Image © Aga Khan Trust for Culture / Cemal Emden (photographer)QatarThe Ned Hotel, in Doha by David Chipperfield ArchitectsThe Ministry of Interior was housed in the Ned Hotel in Doha, which was designed by David Chipperfield Architects. Its Middle Eastern brutalist building was meticulously transformed into a 90-room boutique hotel, thereby promoting architectural revitalization in the region.Shamalat Cultural Centre. Image © Aga Khan Trust for Culture / Hassan Al Shatti (photographer)Saudi ArabiaShamalat Cultural Centre, in Riyadh, by Syn Architects / Sara Alissa, Nojoud AlsudairiOn the outskirts of Diriyah, the Shamalat Cultural Centre in Riyadh was created by Syn Architects/Sara Alissa, Nojoud Alsudairi. It was created from an old mud home that artist Maha Malluh had renovated. The center, which aims to incorporate historic places into daily life, provides a sensitive viewpoint on heritage conservation in the area by contrasting the old and the contemporary.Rehabilitation and Extension of Dakar Railway Station. Image © Aga Khan Trust for Culture / Sylvain Cherkaoui (photographer)SenegalRehabilitation and Extension of Dakar Railway Station, in Dakar by Ga2DIn order to accommodate the passengers of a new express train line, Ga2D extended and renovated Dakar train Station, which purposefully contrasts the old and modern buildings. The forecourt was once again open to pedestrian traffic after vehicular traffic was limited to the rear of the property.Rami Library. Image © Aga Khan Trust for Culture / Cemal Emden (photographer)TürkiyeRami Library, by Han Tümertekin Design & ConsultancyThe largest library in Istanbul is the Rami Library, designed by Han Tümertekin Design & Consultancy. It occupied the former Rami Barracks, a sizable, single-story building with enormous volumes that was constructed in the eighteenth century. In order to accommodate new library operations while maintaining the structure's original spatial features, a minimal intervention method was used.Morocco Pavilion Expo Dubai 2020. Image © Aga Khan Trust for Culture / Deed Studio (photographer)United Arab EmiratesMorocco Pavilion Expo Dubai 2020, by Oualalou + ChoiOualalou + Choi's Morocco Pavilion Expo Dubai 2020 is intended to last beyond Expo 2020 and be transformed into a cultural center. The pavilion is a trailblazer in the development of large-scale rammed earth building techniques. Its use of passive cooling techniques, which minimize the need for mechanical air conditioning, earned it the gold LEED accreditation.At each project location, independent professionals such as architects, conservation specialists, planners, and structural engineers have conducted thorough evaluations of the nominated projects. This summer, the Master Jury convenes once more to analyze the on-site evaluations and choose the ultimate Award winners.The top image in the article: The Arc at Green School. Image © Aga Khan Trust for Culture / Andreas Perbowo Widityawan (photographer).> via Aga Khan Award for Architecture
    Like
    Love
    Wow
    Sad
    Angry
    531
    2 Comments 0 Shares 0 Reviews
  • Alec Haase Q&A: Customer Engagement Book Interview

    Reading Time: 6 minutes
    What is marketing without data? Assumptions. Guesses. Fluff.
    For Chapter 6 of our book, “The Customer Engagement Book: Adapt or Die,” we spoke with Alec Haase, Product GTM Lead, Commerce and AI at Hightouch, to explore how engagement data can truly inform critical business decisions. 
    Alec discusses the different types of customer behaviors that matter most, how to separate meaningful information from the rest, and the role of systems that learn over time to create tailored customer experiences.
    This interview provides insights into using data for real-time actions and shaping the future of marketing. Prepare to learn about AI decision-making and how a focus on data is changing how we engage with customers.

     
    Alec Haase Q&A Interview
    1. What types of customer engagement data are most valuable for making strategic business decisions?
    It’s a culmination of everything.
    Behavioral signals — the actual conversions and micro-conversions that users take within your product or website.
    Obviously, that’s things like purchases. But there are also other behavioral signals marketers should be using and thinking about. Things like micro-conversions — maybe that’s shopping for a product, clicking to learn more about a product, or visiting a certain page on your website.
    Behind that, you also need to have all your user data to tie that to.

    So I know someone took said action; I can follow up with them in email or out on paid social. I need the user identifiers to do that.

    2. How do you distinguish between data that is actionable versus data that is just noise?
    Data that’s actionable includes the conversions and micro-conversions — very clear instances of “someone did this.” I can react to or measure those.
    What’s becoming a bit of a challenge for marketers is understanding that there’s other data that is valuable for machine learning or reinforcement learning models, things like tags on the types of products customers are interacting with.
    Maybe there’s category information about that product, or color information. That would otherwise look like noise to the average marketer. But behind the scenes, it can be used for reinforcement learning.

    There is definitely the “clear-cut” actionable data, but marketers shouldn’t be quick to classify things as noise because the rise in machine learning and reinforcement learning will make that data more valuable.

    3. How can customer engagement data be used to identify and prioritize new business opportunities?
    At Hightouch, we don’t necessarily think about retroactive analysis. We have a system where we have customer engagement data firing in that we then have real-time scores reacting to.
    An interesting example is when you have machine learning and reinforcement learning models running. In the pet retailer example I gave you, the system is able to figure out what to prioritize.
    The concept of reinforcement learning is not a marketer making rules to say, “I know this type of thing works well on this type of audience.”

    It’s the machine itself using the data to determine what attribute responds well to which offer, recommendation, or marketing campaign.

    4. How can marketers ensure their use of customer engagement data aligns with the broader business objectives?
    It starts with the objectives. It’s starting with the desired outcome and working your way back. That whole flip of the paradigm is starting with outcomes and letting the system optimize. What are you trying to drive, and then back into the types of experiences that can make that happen?
    There’s personalization.
    When we talk about data-driven experiences and personalization, Spotify Wrapped is the North Star. For Spotify Wrapped, you want to drive customer stickiness and create a brand. To make that happen, you want to send a personalized email. What components do you want in that email?

    Maybe it’s top five songs, top five artists, and then you can back into the actual event data you need to make that happen.

    5. What role does engagement data play in influencing cross-functional decisions such as those in product development, sales, or customer service?
    For product development, it’s product analytics — knowing what features users are using, or seeing in heat maps where users are clicking.
    Sales is similar. We’re using behavioral signals like what types of content they’re reading on the site to help inform what they would be interested in — the types of products or the types of use cases.

    For customer service, you can look at errors they’ve run into in the past or specific purchases they’ve made, so that when you’re helping them the next time they engage with you, you know exactly what their past behaviors were and what products they could be calling about.

    6. What are some challenges marketers face when trying to translate customer engagement data into actionable insights?
    Access to data is one challenge. You might not know what data you have because marketers historically may not have been used to the systems where data is stored.
    Historically, that’s been pretty siloed away from them. Rich behavioral data and other data across the business was stored somewhere else.
    Now, as more companies embrace the data warehouse at the center of their business, it gives everyone a true single place where data can be stored.

    Marketers are working more with data teams, understanding more about the data they have, and using that data to power downstream use cases, personalization, reinforcement learning, or general business insights.

    7. How do you address skepticism or resistance from stakeholders when presenting data-driven recommendations?
    As a marketer, I think proof is key. The best thing is if you’ve actually run a test. “I think we should do this. I ran a small test, and it’s showing that this is actually proving out.” Being able to clearly explain and justify your reasoning with data is super important.

    8. What technology or tools have you found most effective for gathering and analyzing customer engagement data?
    Any type of behavioral event collection, specifically ones that write to the cloud data warehouse, is the critical component. Your data team is operating off the data warehouse.
    Having an event collection product that stores data in that central spot is really important if you want to use the other data when making recommendations.
    You want to get everything into the data warehouse where it can be used both for insights and for putting into action.

    For Spotify Wrapped, you want to collect behavioral event signals like songs listened to or concerts attended, writing to the warehouse so that you can get insights back — how many songs were played this year, projections for next month — but then you can also use those behavioral events in downstream platforms to fire off personalized emails with product recommendations or Spotify Wrapped-style experiences.

    9. How do you see the role of customer engagement data evolving in shaping business strategies over the next five years?

    What we’re excited about is the concept of AI Decisioning — having AI agents actually using customer data to train their own models and decision-making to create personalized experiences.
    We’re sitting on top of all this behavioral data, engagement data, and user attributes, and our system is learning from all of that to make the best decisions across downstream systems.
    Whether that’s as simple as driving a loyalty program and figuring out what emails to send or what on-site experiences to show, or exposing insights that might lead you to completely change your business strategy, we see engagement data as the fuel to the engine of reinforcement learning, machine learning, AI agents, this whole next wave of Martech that’s just now coming.
    But it all starts with having the data to train those systems.

    I think that behavioral data is the fuel of modern Martech, and that only holds more true as Martech platforms adopt these decisioning and AI capabilities, because they’re only as good as the data that’s training the models.

     

     
    This interview Q&A was hosted with Alec Haase, Product GTM Lead, Commerce and AI at Hightouch, for Chapter 6 of The Customer Engagement Book: Adapt or Die.
    Download the PDF or request a physical copy of the book here.
    The post Alec Haase Q&A: Customer Engagement Book Interview appeared first on MoEngage.
    #alec #haase #qampampa #customer #engagement
    Alec Haase Q&A: Customer Engagement Book Interview
    Reading Time: 6 minutes What is marketing without data? Assumptions. Guesses. Fluff. For Chapter 6 of our book, “The Customer Engagement Book: Adapt or Die,” we spoke with Alec Haase, Product GTM Lead, Commerce and AI at Hightouch, to explore how engagement data can truly inform critical business decisions.  Alec discusses the different types of customer behaviors that matter most, how to separate meaningful information from the rest, and the role of systems that learn over time to create tailored customer experiences. This interview provides insights into using data for real-time actions and shaping the future of marketing. Prepare to learn about AI decision-making and how a focus on data is changing how we engage with customers.   Alec Haase Q&A Interview 1. What types of customer engagement data are most valuable for making strategic business decisions? It’s a culmination of everything. Behavioral signals — the actual conversions and micro-conversions that users take within your product or website. Obviously, that’s things like purchases. But there are also other behavioral signals marketers should be using and thinking about. Things like micro-conversions — maybe that’s shopping for a product, clicking to learn more about a product, or visiting a certain page on your website. Behind that, you also need to have all your user data to tie that to. So I know someone took said action; I can follow up with them in email or out on paid social. I need the user identifiers to do that. 2. How do you distinguish between data that is actionable versus data that is just noise? Data that’s actionable includes the conversions and micro-conversions — very clear instances of “someone did this.” I can react to or measure those. What’s becoming a bit of a challenge for marketers is understanding that there’s other data that is valuable for machine learning or reinforcement learning models, things like tags on the types of products customers are interacting with. Maybe there’s category information about that product, or color information. That would otherwise look like noise to the average marketer. But behind the scenes, it can be used for reinforcement learning. There is definitely the “clear-cut” actionable data, but marketers shouldn’t be quick to classify things as noise because the rise in machine learning and reinforcement learning will make that data more valuable. 3. How can customer engagement data be used to identify and prioritize new business opportunities? At Hightouch, we don’t necessarily think about retroactive analysis. We have a system where we have customer engagement data firing in that we then have real-time scores reacting to. An interesting example is when you have machine learning and reinforcement learning models running. In the pet retailer example I gave you, the system is able to figure out what to prioritize. The concept of reinforcement learning is not a marketer making rules to say, “I know this type of thing works well on this type of audience.” It’s the machine itself using the data to determine what attribute responds well to which offer, recommendation, or marketing campaign. 4. How can marketers ensure their use of customer engagement data aligns with the broader business objectives? It starts with the objectives. It’s starting with the desired outcome and working your way back. That whole flip of the paradigm is starting with outcomes and letting the system optimize. What are you trying to drive, and then back into the types of experiences that can make that happen? There’s personalization. When we talk about data-driven experiences and personalization, Spotify Wrapped is the North Star. For Spotify Wrapped, you want to drive customer stickiness and create a brand. To make that happen, you want to send a personalized email. What components do you want in that email? Maybe it’s top five songs, top five artists, and then you can back into the actual event data you need to make that happen. 5. What role does engagement data play in influencing cross-functional decisions such as those in product development, sales, or customer service? For product development, it’s product analytics — knowing what features users are using, or seeing in heat maps where users are clicking. Sales is similar. We’re using behavioral signals like what types of content they’re reading on the site to help inform what they would be interested in — the types of products or the types of use cases. For customer service, you can look at errors they’ve run into in the past or specific purchases they’ve made, so that when you’re helping them the next time they engage with you, you know exactly what their past behaviors were and what products they could be calling about. 6. What are some challenges marketers face when trying to translate customer engagement data into actionable insights? Access to data is one challenge. You might not know what data you have because marketers historically may not have been used to the systems where data is stored. Historically, that’s been pretty siloed away from them. Rich behavioral data and other data across the business was stored somewhere else. Now, as more companies embrace the data warehouse at the center of their business, it gives everyone a true single place where data can be stored. Marketers are working more with data teams, understanding more about the data they have, and using that data to power downstream use cases, personalization, reinforcement learning, or general business insights. 7. How do you address skepticism or resistance from stakeholders when presenting data-driven recommendations? As a marketer, I think proof is key. The best thing is if you’ve actually run a test. “I think we should do this. I ran a small test, and it’s showing that this is actually proving out.” Being able to clearly explain and justify your reasoning with data is super important. 8. What technology or tools have you found most effective for gathering and analyzing customer engagement data? Any type of behavioral event collection, specifically ones that write to the cloud data warehouse, is the critical component. Your data team is operating off the data warehouse. Having an event collection product that stores data in that central spot is really important if you want to use the other data when making recommendations. You want to get everything into the data warehouse where it can be used both for insights and for putting into action. For Spotify Wrapped, you want to collect behavioral event signals like songs listened to or concerts attended, writing to the warehouse so that you can get insights back — how many songs were played this year, projections for next month — but then you can also use those behavioral events in downstream platforms to fire off personalized emails with product recommendations or Spotify Wrapped-style experiences. 9. How do you see the role of customer engagement data evolving in shaping business strategies over the next five years? What we’re excited about is the concept of AI Decisioning — having AI agents actually using customer data to train their own models and decision-making to create personalized experiences. We’re sitting on top of all this behavioral data, engagement data, and user attributes, and our system is learning from all of that to make the best decisions across downstream systems. Whether that’s as simple as driving a loyalty program and figuring out what emails to send or what on-site experiences to show, or exposing insights that might lead you to completely change your business strategy, we see engagement data as the fuel to the engine of reinforcement learning, machine learning, AI agents, this whole next wave of Martech that’s just now coming. But it all starts with having the data to train those systems. I think that behavioral data is the fuel of modern Martech, and that only holds more true as Martech platforms adopt these decisioning and AI capabilities, because they’re only as good as the data that’s training the models.     This interview Q&A was hosted with Alec Haase, Product GTM Lead, Commerce and AI at Hightouch, for Chapter 6 of The Customer Engagement Book: Adapt or Die. Download the PDF or request a physical copy of the book here. The post Alec Haase Q&A: Customer Engagement Book Interview appeared first on MoEngage. #alec #haase #qampampa #customer #engagement
    WWW.MOENGAGE.COM
    Alec Haase Q&A: Customer Engagement Book Interview
    Reading Time: 6 minutes What is marketing without data? Assumptions. Guesses. Fluff. For Chapter 6 of our book, “The Customer Engagement Book: Adapt or Die,” we spoke with Alec Haase, Product GTM Lead, Commerce and AI at Hightouch, to explore how engagement data can truly inform critical business decisions.  Alec discusses the different types of customer behaviors that matter most, how to separate meaningful information from the rest, and the role of systems that learn over time to create tailored customer experiences. This interview provides insights into using data for real-time actions and shaping the future of marketing. Prepare to learn about AI decision-making and how a focus on data is changing how we engage with customers.   Alec Haase Q&A Interview 1. What types of customer engagement data are most valuable for making strategic business decisions? It’s a culmination of everything. Behavioral signals — the actual conversions and micro-conversions that users take within your product or website. Obviously, that’s things like purchases. But there are also other behavioral signals marketers should be using and thinking about. Things like micro-conversions — maybe that’s shopping for a product, clicking to learn more about a product, or visiting a certain page on your website. Behind that, you also need to have all your user data to tie that to. So I know someone took said action; I can follow up with them in email or out on paid social. I need the user identifiers to do that. 2. How do you distinguish between data that is actionable versus data that is just noise? Data that’s actionable includes the conversions and micro-conversions — very clear instances of “someone did this.” I can react to or measure those. What’s becoming a bit of a challenge for marketers is understanding that there’s other data that is valuable for machine learning or reinforcement learning models, things like tags on the types of products customers are interacting with. Maybe there’s category information about that product, or color information. That would otherwise look like noise to the average marketer. But behind the scenes, it can be used for reinforcement learning. There is definitely the “clear-cut” actionable data, but marketers shouldn’t be quick to classify things as noise because the rise in machine learning and reinforcement learning will make that data more valuable. 3. How can customer engagement data be used to identify and prioritize new business opportunities? At Hightouch, we don’t necessarily think about retroactive analysis. We have a system where we have customer engagement data firing in that we then have real-time scores reacting to. An interesting example is when you have machine learning and reinforcement learning models running. In the pet retailer example I gave you, the system is able to figure out what to prioritize. The concept of reinforcement learning is not a marketer making rules to say, “I know this type of thing works well on this type of audience.” It’s the machine itself using the data to determine what attribute responds well to which offer, recommendation, or marketing campaign. 4. How can marketers ensure their use of customer engagement data aligns with the broader business objectives? It starts with the objectives. It’s starting with the desired outcome and working your way back. That whole flip of the paradigm is starting with outcomes and letting the system optimize. What are you trying to drive, and then back into the types of experiences that can make that happen? There’s personalization. When we talk about data-driven experiences and personalization, Spotify Wrapped is the North Star. For Spotify Wrapped, you want to drive customer stickiness and create a brand. To make that happen, you want to send a personalized email. What components do you want in that email? Maybe it’s top five songs, top five artists, and then you can back into the actual event data you need to make that happen. 5. What role does engagement data play in influencing cross-functional decisions such as those in product development, sales, or customer service? For product development, it’s product analytics — knowing what features users are using, or seeing in heat maps where users are clicking. Sales is similar. We’re using behavioral signals like what types of content they’re reading on the site to help inform what they would be interested in — the types of products or the types of use cases. For customer service, you can look at errors they’ve run into in the past or specific purchases they’ve made, so that when you’re helping them the next time they engage with you, you know exactly what their past behaviors were and what products they could be calling about. 6. What are some challenges marketers face when trying to translate customer engagement data into actionable insights? Access to data is one challenge. You might not know what data you have because marketers historically may not have been used to the systems where data is stored. Historically, that’s been pretty siloed away from them. Rich behavioral data and other data across the business was stored somewhere else. Now, as more companies embrace the data warehouse at the center of their business, it gives everyone a true single place where data can be stored. Marketers are working more with data teams, understanding more about the data they have, and using that data to power downstream use cases, personalization, reinforcement learning, or general business insights. 7. How do you address skepticism or resistance from stakeholders when presenting data-driven recommendations? As a marketer, I think proof is key. The best thing is if you’ve actually run a test. “I think we should do this. I ran a small test, and it’s showing that this is actually proving out.” Being able to clearly explain and justify your reasoning with data is super important. 8. What technology or tools have you found most effective for gathering and analyzing customer engagement data? Any type of behavioral event collection, specifically ones that write to the cloud data warehouse, is the critical component. Your data team is operating off the data warehouse. Having an event collection product that stores data in that central spot is really important if you want to use the other data when making recommendations. You want to get everything into the data warehouse where it can be used both for insights and for putting into action. For Spotify Wrapped, you want to collect behavioral event signals like songs listened to or concerts attended, writing to the warehouse so that you can get insights back — how many songs were played this year, projections for next month — but then you can also use those behavioral events in downstream platforms to fire off personalized emails with product recommendations or Spotify Wrapped-style experiences. 9. How do you see the role of customer engagement data evolving in shaping business strategies over the next five years? What we’re excited about is the concept of AI Decisioning — having AI agents actually using customer data to train their own models and decision-making to create personalized experiences. We’re sitting on top of all this behavioral data, engagement data, and user attributes, and our system is learning from all of that to make the best decisions across downstream systems. Whether that’s as simple as driving a loyalty program and figuring out what emails to send or what on-site experiences to show, or exposing insights that might lead you to completely change your business strategy, we see engagement data as the fuel to the engine of reinforcement learning, machine learning, AI agents, this whole next wave of Martech that’s just now coming. But it all starts with having the data to train those systems. I think that behavioral data is the fuel of modern Martech, and that only holds more true as Martech platforms adopt these decisioning and AI capabilities, because they’re only as good as the data that’s training the models.     This interview Q&A was hosted with Alec Haase, Product GTM Lead, Commerce and AI at Hightouch, for Chapter 6 of The Customer Engagement Book: Adapt or Die. Download the PDF or request a physical copy of the book here. The post Alec Haase Q&A: Customer Engagement Book Interview appeared first on MoEngage.
    0 Comments 0 Shares 0 Reviews
  • How AI is reshaping the future of healthcare and medical research

    Transcript       
    PETER LEE: “In ‘The Little Black Bag,’ a classic science fiction story, a high-tech doctor’s kit of the future is accidentally transported back to the 1950s, into the shaky hands of a washed-up, alcoholic doctor. The ultimate medical tool, it redeems the doctor wielding it, allowing him to practice gratifyingly heroic medicine. … The tale ends badly for the doctor and his treacherous assistant, but it offered a picture of how advanced technology could transform medicine—powerful when it was written nearly 75 years ago and still so today. What would be the Al equivalent of that little black bag? At this moment when new capabilities are emerging, how do we imagine them into medicine?”          
    This is The AI Revolution in Medicine, Revisited. I’m your host, Peter Lee.   
    Shortly after OpenAI’s GPT-4 was publicly released, Carey Goldberg, Dr. Zak Kohane, and I published The AI Revolution in Medicine to help educate the world of healthcare and medical research about the transformative impact this new generative AI technology could have. But because we wrote the book when GPT-4 was still a secret, we had to speculate. Now, two years later, what did we get right, and what did we get wrong?    
    In this series, we’ll talk to clinicians, patients, hospital administrators, and others to understand the reality of AI in the field and where we go from here.  The book passage I read at the top is from “Chapter 10: The Big Black Bag.” 
    In imagining AI in medicine, Carey, Zak, and I included in our book two fictional accounts. In the first, a medical resident consults GPT-4 on her personal phone as the patient in front of her crashes. Within seconds, it offers an alternate response based on recent literature. In the second account, a 90-year-old woman with several chronic conditions is living independently and receiving near-constant medical support from an AI aide.   
    In our conversations with the guests we’ve spoken to so far, we’ve caught a glimpse of these predicted futures, seeing how clinicians and patients are actually using AI today and how developers are leveraging the technology in the healthcare products and services they’re creating. In fact, that first fictional account isn’t so fictional after all, as most of the doctors in the real world actually appear to be using AI at least occasionally—and sometimes much more than occasionally—to help in their daily clinical work. And as for the second fictional account, which is more of a science fiction account, it seems we are indeed on the verge of a new way of delivering and receiving healthcare, though the future is still very much open. 
    As we continue to examine the current state of AI in healthcare and its potential to transform the field, I’m pleased to welcome Bill Gates and Sébastien Bubeck.  
    Bill may be best known as the co-founder of Microsoft, having created the company with his childhood friend Paul Allen in 1975. He’s now the founder of Breakthrough Energy, which aims to advance clean energy innovation, and TerraPower, a company developing groundbreaking nuclear energy and science technologies. He also chairs the world’s largest philanthropic organization, the Gates Foundation, and focuses on solving a variety of health challenges around the globe and here at home. 
    Sébastien is a research lead at OpenAI. He was previously a distinguished scientist, vice president of AI, and a colleague of mine here at Microsoft, where his work included spearheading the development of the family of small language models known as Phi. While at Microsoft, he also coauthored the discussion-provoking 2023 paper “Sparks of Artificial General Intelligence,” which presented the results of early experiments with GPT-4 conducted by a small team from Microsoft Research.     
    Here’s my conversation with Bill Gates and Sébastien Bubeck. 
    LEE: Bill, welcome. 
    BILL GATES: Thank you. 
    LEE: Seb … 
    SÉBASTIEN BUBECK: Yeah. Hi, hi, Peter. Nice to be here. 
    LEE: You know, one of the things that I’ve been doing just to get the conversation warmed up is to talk about origin stories, and what I mean about origin stories is, you know, what was the first contact that you had with large language models or the concept of generative AI that convinced you or made you think that something really important was happening? 
    And so, Bill, I think I’ve heard the story about, you know, the time when the OpenAI folks—Sam Altman, Greg Brockman, and others—showed you something, but could we hear from you what those early encounters were like and what was going through your mind?  
    GATES: Well, I’d been visiting OpenAI soon after it was created to see things like GPT-2 and to see the little arm they had that was trying to match human manipulation and, you know, looking at their games like Dota that they were trying to get as good as human play. And honestly, I didn’t think the language model stuff they were doing, even when they got to GPT-3, would show the ability to learn, you know, in the same sense that a human reads a biology book and is able to take that knowledge and access it not only to pass a test but also to create new medicines. 
    And so my challenge to them was that if their LLM could get a five on the advanced placement biology test, then I would say, OK, it took biologic knowledge and encoded it in an accessible way and that I didn’t expect them to do that very quickly but it would be profound.  
    And it was only about six months after I challenged them to do that, that an early version of GPT-4 they brought up to a dinner at my house, and in fact, it answered most of the questions that night very well. The one it got totally wrong, we were … because it was so good, we kept thinking, Oh, we must be wrong. It turned out it was a math weaknessthat, you know, we later understood that that was an area of, weirdly, of incredible weakness of those early models. But, you know, that was when I realized, OK, the age of cheap intelligence was at its beginning. 
    LEE: Yeah. So I guess it seems like you had something similar to me in that my first encounters, I actually harbored some skepticism. Is it fair to say you were skeptical before that? 
    GATES: Well, the idea that we’ve figured out how to encode and access knowledge in this very deep sense without even understanding the nature of the encoding, … 
    LEE: Right.  
    GATES: … that is a bit weird.  
    LEE: Yeah. 
    GATES: We have an algorithm that creates the computation, but even say, OK, where is the president’s birthday stored in there? Where is this fact stored in there? The fact that even now when we’re playing around, getting a little bit more sense of it, it’s opaque to us what the semantic encoding is, it’s, kind of, amazing to me. I thought the invention of knowledge storage would be an explicit way of encoding knowledge, not an implicit statistical training. 
    LEE: Yeah, yeah. All right. So, Seb, you know, on this same topic, you know, I got—as we say at Microsoft—I got pulled into the tent. 
    BUBECK: Yes.  
    LEE: Because this was a very secret project. And then, um, I had the opportunity to select a small number of researchers in MSRto join and start investigating this thing seriously. And the first person I pulled in was you. 
    BUBECK: Yeah. 
    LEE: And so what were your first encounters? Because I actually don’t remember what happened then. 
    BUBECK: Oh, I remember it very well.My first encounter with GPT-4 was in a meeting with the two of you, actually. But my kind of first contact, the first moment where I realized that something was happening with generative AI, was before that. And I agree with Bill that I also wasn’t too impressed by GPT-3. 
    I though that it was kind of, you know, very naturally mimicking the web, sort of parroting what was written there in a nice way. Still in a way which seemed very impressive. But it wasn’t really intelligent in any way. But shortly after GPT-3, there was a model before GPT-4 that really shocked me, and this was the first image generation model, DALL-E 1. 
    So that was in 2021. And I will forever remember the press release of OpenAI where they had this prompt of an avocado chair and then you had this image of the avocado chair.And what really shocked me is that clearly the model kind of “understood” what is a chair, what is an avocado, and was able to merge those concepts. 
    So this was really, to me, the first moment where I saw some understanding in those models.  
    LEE: So this was, just to get the timing right, that was before I pulled you into the tent. 
    BUBECK: That was before. That was like a year before. 
    LEE: Right.  
    BUBECK: And now I will tell you how, you know, we went from that moment to the meeting with the two of you and GPT-4. 
    So once I saw this kind of understanding, I thought, OK, fine. It understands concept, but it’s still not able to reason. It cannot—as, you know, Bill was saying—it cannot learn from your document. It cannot reason.  
    So I set out to try to prove that. You know, this is what I was in the business of at the time, trying to prove things in mathematics. So I was trying to prove that basically autoregressive transformers could never reason. So I was trying to prove this. And after a year of work, I had something reasonable to show. And so I had the meeting with the two of you, and I had this example where I wanted to say, there is no way that an LLM is going to be able to do x. 
    And then as soon as I … I don’t know if you remember, Bill. But as soon as I said that, you said, oh, but wait a second. I had, you know, the OpenAI crew at my house recently, and they showed me a new model. Why don’t we ask this new model this question?  
    LEE: Yeah.
    BUBECK: And we did, and it solved it on the spot. And that really, honestly, just changed my life. Like, you know, I had been working for a year trying to say that this was impossible. And just right there, it was shown to be possible.  
    LEE:One of the very first things I got interested in—because I was really thinking a lot about healthcare—was healthcare and medicine. 
    And I don’t know if the two of you remember, but I ended up doing a lot of tests. I ran through, you know, step one and step two of the US Medical Licensing Exam. Did a whole bunch of other things. I wrote this big report. It was, you know, I can’t remember … a couple hundred pages.  
    And I needed to share this with someone. I didn’t … there weren’t too many people I could share it with. So I sent, I think, a copy to you, Bill. Sent a copy to you, Seb.  
    I hardly slept for about a week putting that report together. And, yeah, and I kept working on it. But I was far from alone. I think everyone who was in the tent, so to speak, in those early days was going through something pretty similar. All right. So I think … of course, a lot of what I put in the report also ended up being examples that made it into the book. 
    But the main purpose of this conversation isn’t to reminisce aboutor indulge in those reminiscences but to talk about what’s happening in healthcare and medicine. And, you know, as I said, we wrote this book. We did it very, very quickly. Seb, you helped. Bill, you know, you provided a review and some endorsements. 
    But, you know, honestly, we didn’t know what we were talking about because no one had access to this thing. And so we just made a bunch of guesses. So really, the whole thing I wanted to probe with the two of you is, now with two years of experience out in the world, what, you know, what do we think is happening today? 
    You know, is AI actually having an impact, positive or negative, on healthcare and medicine? And what do we now think is going to happen in the next two years, five years, or 10 years? And so I realize it’s a little bit too abstract to just ask it that way. So let me just try to narrow the discussion and guide us a little bit.  
    Um, the kind of administrative and clerical work, paperwork, around healthcare—and we made a lot of guesses about that—that appears to be going well, but, you know, Bill, I know we’ve discussed that sometimes that you think there ought to be a lot more going on. Do you have a viewpoint on how AI is actually finding its way into reducing paperwork? 
    GATES: Well, I’m stunned … I don’t think there should be a patient-doctor meeting where the AI is not sitting in and both transcribing, offering to help with the paperwork, and even making suggestions, although the doctor will be the one, you know, who makes the final decision about the diagnosis and whatever prescription gets done.  
    It’s so helpful. You know, when that patient goes home and their, you know, son who wants to understand what happened has some questions, that AI should be available to continue that conversation. And the way you can improve that experience and streamline things and, you know, involve the people who advise you. I don’t understand why that’s not more adopted, because there you still have the human in the loop making that final decision. 
    But even for, like, follow-up calls to make sure the patient did things, to understand if they have concerns and knowing when to escalate back to the doctor, the benefit is incredible. And, you know, that thing is ready for prime time. That paradigm is ready for prime time, in my view. 
    LEE: Yeah, there are some good products, but it seems like the number one use right now—and we kind of got this from some of the previous guests in previous episodes—is the use of AI just to respond to emails from patients.Does that make sense to you? 
    BUBECK: Yeah. So maybe I want to second what Bill was saying but maybe take a step back first. You know, two years ago, like, the concept of clinical scribes, which is one of the things that we’re talking about right now, it would have sounded, in fact, it sounded two years ago, borderline dangerous. Because everybody was worried about hallucinations. What happened if you have this AI listening in and then it transcribes, you know, something wrong? 
    Now, two years later, I think it’s mostly working. And in fact, it is not yet, you know, fully adopted. You’re right. But it is in production. It is used, you know, in many, many places. So this rate of progress is astounding because it wasn’t obvious that we would be able to overcome those obstacles of hallucination. It’s not to say that hallucinations are fully solved. In the case of the closed system, they are.  
    Now, I think more generally what’s going on in the background is that there is something that we, that certainly I, underestimated, which is this management overhead. So I think the reason why this is not adopted everywhere is really a training and teaching aspect. People need to be taught, like, those systems, how to interact with them. 
    And one example that I really like, a study that recently appeared where they tried to use ChatGPT for diagnosis and they were comparing doctors without and with ChatGPT. And the amazing thing … so this was a set of cases where the accuracy of the doctors alone was around 75%. ChatGPT alone was 90%. So that’s already kind of mind blowing. But then the kicker is that doctors with ChatGPT was 80%.  
    Intelligence alone is not enough. It’s also how it’s presented, how you interact with it. And ChatGPT, it’s an amazing tool. Obviously, I absolutely love it. But it’s not … you don’t want a doctor to have to type in, you know, prompts and use it that way. 
    It should be, as Bill was saying, kind of running continuously in the background, sending you notifications. And you have to be really careful of the rate at which those notifications are being sent. Because if they are too frequent, then the doctor will learn to ignore them. So you have to … all of those things matter, in fact, at least as much as the level of intelligence of the machine. 
    LEE: One of the things I think about, Bill, in that scenario that you described, doctors do some thinking about the patient when they write the note. So, you know, I’m always a little uncertain whether it’s actually … you know, you wouldn’t necessarily want to fully automate this, I don’t think. Or at least there needs to be some prompt to the doctor to make sure that the doctor puts some thought into what happened in the encounter with the patient. Does that make sense to you at all? 
    GATES: At this stage, you know, I’d still put the onus on the doctor to write the conclusions and the summary and not delegate that. 
    The tradeoffs you make a little bit are somewhat dependent on the situation you’re in. If you’re in Africa,
    So, yes, the doctor’s still going to have to do a lot of work, but just the quality of letting the patient and the people around them interact and ask questions and have things explained, that alone is such a quality improvement. It’s mind blowing.  
    LEE: So since you mentioned, you know, Africa—and, of course, this touches on the mission and some of the priorities of the Gates Foundation and this idea of democratization of access to expert medical care—what’s the most interesting stuff going on right now? Are there people and organizations or technologies that are impressing you or that you’re tracking? 
    GATES: Yeah. So the Gates Foundation has given out a lot of grants to people in Africa doing education, agriculture but more healthcare examples than anything. And the way these things start off, they often start out either being patient-centric in a narrow situation, like, OK, I’m a pregnant woman; talk to me. Or, I have infectious disease symptoms; talk to me. Or they’re connected to a health worker where they’re helping that worker get their job done. And we have lots of pilots out, you know, in both of those cases.  
    The dream would be eventually to have the thing the patient consults be so broad that it’s like having a doctor available who understands the local things.  
    LEE: Right.  
    GATES: We’re not there yet. But over the next two or three years, you know, particularly given the worsening financial constraints against African health systems, where the withdrawal of money has been dramatic, you know, figuring out how to take this—what I sometimes call “free intelligence”—and build a quality health system around that, we will have to be more radical in low-income countries than any rich country is ever going to be.  
    LEE: Also, there’s maybe a different regulatory environment, so some of those things maybe are easier? Because right now, I think the world hasn’t figured out how to and whether to regulate, let’s say, an AI that might give a medical diagnosis or write a prescription for a medication. 
    BUBECK: Yeah. I think one issue with this, and it’s also slowing down the deployment of AI in healthcare more generally, is a lack of proper benchmark. Because, you know, you were mentioning the USMLE, for example. That’s a great test to test human beings and their knowledge of healthcare and medicine. But it’s not a great test to give to an AI. 
    It’s not asking the right questions. So finding what are the right questions to test whether an AI system is ready to give diagnosis in a constrained setting, that’s a very, very important direction, which to my surprise, is not yet accelerating at the rate that I was hoping for. 
    LEE: OK, so that gives me an excuse to get more now into the core AI tech because something I’ve discussed with both of you is this issue of what are the right tests. And you both know the very first test I give to any new spin of an LLM is I present a patient, the results—a mythical patient—the results of my physical exam, my mythical physical exam. Maybe some results of some initial labs. And then I present or propose a differential diagnosis. And if you’re not in medicine, a differential diagnosis you can just think of as a prioritized list of the possible diagnoses that fit with all that data. And in that proposed differential, I always intentionally make two mistakes. 
    I make a textbook technical error in one of the possible elements of the differential diagnosis, and I have an error of omission. And, you know, I just want to know, does the LLM understand what I’m talking about? And all the good ones out there do now. But then I want to know, can it spot the errors? And then most importantly, is it willing to tell me I’m wrong, that I’ve made a mistake?  
    That last piece seems really hard for AI today. And so let me ask you first, Seb, because at the time of this taping, of course, there was a new spin of GPT-4o last week that became overly sycophantic. In other words, it was actually prone in that test of mine not only to not tell me I’m wrong, but it actually praised me for the creativity of my differential.What’s up with that? 
    BUBECK: Yeah, I guess it’s a testament to the fact that training those models is still more of an art than a science. So it’s a difficult job. Just to be clear with the audience, we have rolled back thatversion of GPT-4o, so now we don’t have the sycophant version out there. 
    Yeah, no, it’s a really difficult question. It has to do … as you said, it’s very technical. It has to do with the post-training and how, like, where do you nudge the model? So, you know, there is this very classical by now technique called RLHF, where you push the model in the direction of a certain reward model. So the reward model is just telling the model, you know, what behavior is good, what behavior is bad. 
    But this reward model is itself an LLM, and, you know, Bill was saying at the very beginning of the conversation that we don’t really understand how those LLMs deal with concepts like, you know, where is the capital of France located? Things like that. It is the same thing for this reward model. We don’t know why it says that it prefers one output to another, and whether this is correlated with some sycophancy is, you know, something that we discovered basically just now. That if you push too hard in optimization on this reward model, you will get a sycophant model. 
    So it’s kind of … what I’m trying to say is we became too good at what we were doing, and we ended up, in fact, in a trap of the reward model. 
    LEE: I mean, you do want … it’s a difficult balance because you do want models to follow your desires and … 
    BUBECK: It’s a very difficult, very difficult balance. 
    LEE: So this brings up then the following question for me, which is the extent to which we think we’ll need to have specially trained models for things. So let me start with you, Bill. Do you have a point of view on whether we will need to, you know, quote-unquote take AI models to med school? Have them specially trained? Like, if you were going to deploy something to give medical care in underserved parts of the world, do we need to do something special to create those models? 
    GATES: We certainly need to teach them the African languages and the unique dialects so that the multimedia interactions are very high quality. We certainly need to teach them the disease prevalence and unique disease patterns like, you know, neglected tropical diseases and malaria. So we need to gather a set of facts that somebody trying to go for a US customer base, you know, wouldn’t necessarily have that in there. 
    Those two things are actually very straightforward because the additional training time is small. I’d say for the next few years, we’ll also need to do reinforcement learning about the context of being a doctor and how important certain behaviors are. Humans learn over the course of their life to some degree that, I’m in a different context and the way I behave in terms of being willing to criticize or be nice, you know, how important is it? Who’s here? What’s my relationship to them?  
    Right now, these machines don’t have that broad social experience. And so if you know it’s going to be used for health things, a lot of reinforcement learning of the very best humans in that context would still be valuable. Eventually, the models will, having read all the literature of the world about good doctors, bad doctors, it’ll understand as soon as you say, “I want you to be a doctor diagnosing somebody.” All of the implicit reinforcement that fits that situation, you know, will be there.
    LEE: Yeah.
    GATES: And so I hope three years from now, we don’t have to do that reinforcement learning. But today, for any medical context, you would want a lot of data to reinforce tone, willingness to say things when, you know, there might be something significant at stake. 
    LEE: Yeah. So, you know, something Bill said, kind of, reminds me of another thing that I think we missed, which is, the context also … and the specialization also pertains to different, I guess, what we still call “modes,” although I don’t know if the idea of multimodal is the same as it was two years ago. But, you know, what do you make of all of the hubbub around—in fact, within Microsoft Research, this is a big deal, but I think we’re far from alone—you know, medical images and vision, video, proteins and molecules, cell, you know, cellular data and so on. 
    BUBECK: Yeah. OK. So there is a lot to say to everything … to the last, you know, couple of minutes. Maybe on the specialization aspect, you know, I think there is, hiding behind this, a really fundamental scientific question of whether eventually we have a singular AGIthat kind of knows everything and you can just put, you know, explain your own context and it will just get it and understand everything. 
    That’s one vision. I have to say, I don’t particularly believe in this vision. In fact, we humans are not like that at all. I think, hopefully, we are general intelligences, yet we have to specialize a lot. And, you know, I did myself a lot of RL, reinforcement learning, on mathematics. Like, that’s what I did, you know, spent a lot of time doing that. And I didn’t improve on other aspects. You know, in fact, I probably degraded in other aspects.So it’s … I think it’s an important example to have in mind. 
    LEE: I think I might disagree with you on that, though, because, like, doesn’t a model have to see both good science and bad science in order to be able to gain the ability to discern between the two? 
    BUBECK: Yeah, no, that absolutely. I think there is value in seeing the generality, in having a very broad base. But then you, kind of, specialize on verticals. And this is where also, you know, open-weights model, which we haven’t talked about yet, are really important because they allow you to provide this broad base to everyone. And then you can specialize on top of it. 
    LEE: So we have about three hours of stuff to talk about, but our time is actually running low.
    BUBECK: Yes, yes, yes.  
    LEE: So I think I want … there’s a more provocative question. It’s almost a silly question, but I need to ask it of the two of you, which is, is there a future, you know, where AI replaces doctors or replaces, you know, medical specialties that we have today? So what does the world look like, say, five years from now? 
    GATES: Well, it’s important to distinguish healthcare discovery activity from healthcare delivery activity. We focused mostly on delivery. I think it’s very much within the realm of possibility that the AI is not only accelerating healthcare discovery but substituting for a lot of the roles of, you know, I’m an organic chemist, or I run various types of assays. I can see those, which are, you know, testable-output-type jobs but with still very high value, I can see, you know, some replacement in those areas before the doctor.  
    The doctor, still understanding the human condition and long-term dialogues, you know, they’ve had a lifetime of reinforcement of that, particularly when you get into areas like mental health. So I wouldn’t say in five years, either people will choose to adopt it, but it will be profound that there’ll be this nearly free intelligence that can do follow-up, that can help you, you know, make sure you went through different possibilities. 
    And so I’d say, yes, we’ll have doctors, but I’d say healthcare will be massively transformed in its quality and in efficiency by AI in that time period. 
    LEE: Is there a comparison, useful comparison, say, between doctors and, say, programmers, computer programmers, or doctors and, I don’t know, lawyers? 
    GATES: Programming is another one that has, kind of, a mathematical correctness to it, you know, and so the objective function that you’re trying to reinforce to, as soon as you can understand the state machines, you can have something that’s “checkable”; that’s correct. So I think programming, you know, which is weird to say, that the machine will beat us at most programming tasks before we let it take over roles that have deep empathy, you know, physical presence and social understanding in them. 
    LEE: Yeah. By the way, you know, I fully expect in five years that AI will produce mathematical proofs that are checkable for validity, easily checkable, because they’ll be written in a proof-checking language like Lean or something but will be so complex that no human mathematician can understand them. I expect that to happen.  
    I can imagine in some fields, like cellular biology, we could have the same situation in the future because the molecular pathways, the chemistry, biochemistry of human cells or living cells is as complex as any mathematics, and so it seems possible that we may be in a state where in wet lab, we see, Oh yeah, this actually works, but no one can understand why. 
    BUBECK: Yeah, absolutely. I mean, I think I really agree with Bill’s distinction of the discovery and the delivery, and indeed, the discovery’s when you can check things, and at the end, there is an artifact that you can verify. You know, you can run the protocol in the wet lab and seeproduced what you wanted. So I absolutely agree with that.  
    And in fact, you know, we don’t have to talk five years from now. I don’t know if you know, but just recently, there was a paper that was published on a scientific discovery using o3- mini. So this is really amazing. And, you know, just very quickly, just so people know, it was about this statistical physics model, the frustrated Potts model, which has to do with coloring, and basically, the case of three colors, like, more than two colors was open for a long time, and o3 was able to reduce the case of three colors to two colors.  
    LEE: Yeah. 
    BUBECK: Which is just, like, astounding. And this is not … this is now. This is happening right now. So this is something that I personally didn’t expect it would happen so quickly, and it’s due to those reasoning models.  
    Now, on the delivery side, I would add something more to it for the reason why doctors and, in fact, lawyers and coders will remain for a long time, and it’s because we still don’t understand how those models generalize. Like, at the end of the day, we are not able to tell you when they are confronted with a really new, novel situation, whether they will work or not. 
    Nobody is able to give you that guarantee. And I think until we understand this generalization better, we’re not going to be willing to just let the system in the wild without human supervision. 
    LEE: But don’t human doctors, human specialists … so, for example, a cardiologist sees a patient in a certain way that a nephrologist … 
    BUBECK: Yeah.
    LEE: … or an endocrinologist might not.
    BUBECK: That’s right. But another cardiologist will understand and, kind of, expect a certain level of generalization from their peer. And this, we just don’t have it with AI models. Now, of course, you’re exactly right. That generalization is also hard for humans. Like, if you have a human trained for one task and you put them into another task, then you don’t … you often don’t know.
    LEE: OK. You know, the podcast is focused on what’s happened over the last two years. But now, I’d like one provocative prediction about what you think the world of AI and medicine is going to be at some point in the future. You pick your timeframe. I don’t care if it’s two years or 20 years from now, but, you know, what do you think will be different about AI in medicine in that future than today? 
    BUBECK: Yeah, I think the deployment is going to accelerate soon. Like, we’re really not missing very much. There is this enormous capability overhang. Like, even if progress completely stopped, with current systems, we can do a lot more than what we’re doing right now. So I think this will … this has to be realized, you know, sooner rather than later. 
    And I think it’s probably dependent on these benchmarks and proper evaluation and tying this with regulation. So these are things that take time in human society and for good reason. But now we already are at two years; you know, give it another two years and it should be really …  
    LEE: Will AI prescribe your medicines? Write your prescriptions? 
    BUBECK: I think yes. I think yes. 
    LEE: OK. Bill? 
    GATES: Well, I think the next two years, we’ll have massive pilots, and so the amount of use of the AI, still in a copilot-type mode, you know, we should get millions of patient visits, you know, both in general medicine and in the mental health side, as well. And I think that’s going to build up both the data and the confidence to give the AI some additional autonomy. You know, are you going to let it talk to you at night when you’re panicked about your mental health with some ability to escalate?
    And, you know, I’ve gone so far as to tell politicians with national health systems that if they deploy AI appropriately, that the quality of care, the overload of the doctors, the improvement in the economics will be enough that their voters will be stunned because they just don’t expect this, and, you know, they could be reelectedjust on this one thing of fixing what is a very overloaded and economically challenged health system in these rich countries. 
    You know, my personal role is going to be to make sure that in the poorer countries, there isn’t some lag; in fact, in many cases, that we’ll be more aggressive because, you know, we’re comparing to having no access to doctors at all. And, you know, so I think whether it’s India or Africa, there’ll be lessons that are globally valuable because we need medical intelligence. And, you know, thank god AI is going to provide a lot of that. 
    LEE: Well, on that optimistic note, I think that’s a good way to end. Bill, Seb, really appreciate all of this.  
    I think the most fundamental prediction we made in the book is that AI would actually find its way into the practice of medicine, and I think that that at least has come true, maybe in different ways than we expected, but it’s come true, and I think it’ll only accelerate from here. So thanks again, both of you.  
    GATES: Yeah. Thanks, you guys. 
    BUBECK: Thank you, Peter. Thanks, Bill. 
    LEE: I just always feel such a sense of privilege to have a chance to interact and actually work with people like Bill and Sébastien.   
    With Bill, I’m always amazed at how practically minded he is. He’s really thinking about the nuts and bolts of what AI might be able to do for people, and his thoughts about underserved parts of the world, the idea that we might actually be able to empower people with access to expert medical knowledge, I think is both inspiring and amazing.  
    And then, Seb, Sébastien Bubeck, he’s just absolutely a brilliant mind. He has a really firm grip on the deep mathematics of artificial intelligence and brings that to bear in his research and development work. And where that mathematics takes him isn’t just into the nuts and bolts of algorithms but into philosophical questions about the nature of intelligence.  
    One of the things that Sébastien brought up was the state of evaluation of AI systems. And indeed, he was fairly critical in our conversation. But of course, the world of AI research and development is just moving so fast, and indeed, since we recorded our conversation, OpenAI, in fact, released a new evaluation metric that is directly relevant to medical applications, and that is something called HealthBench. And Microsoft Research also released a new evaluation approach or process called ADeLe.  
    HealthBench and ADeLe are examples of new approaches to evaluating AI models that are less about testing their knowledge and ability to pass multiple-choice exams and instead are evaluation approaches designed to assess how well AI models are able to complete tasks that actually arise every day in typical healthcare or biomedical research settings. These are examples of really important good work that speak to how well AI models work in the real world of healthcare and biomedical research and how well they can collaborate with human beings in those settings. 
    You know, I asked Bill and Seb to make some predictions about the future. You know, my own answer, I expect that we’re going to be able to use AI to change how we diagnose patients, change how we decide treatment options.  
    If you’re a doctor or a nurse and you encounter a patient, you’ll ask questions, do a physical exam, you know, call out for labs just like you do today, but then you’ll be able to engage with AI based on all of that data and just ask, you know, based on all the other people who have gone through the same experience, who have similar data, how were they diagnosed? How were they treated? What were their outcomes? And what does that mean for the patient I have right now? Some people call it the “patients like me” paradigm. And I think that’s going to become real because of AI within our lifetimes. That idea of really grounding the delivery in healthcare and medical practice through data and intelligence, I actually now don’t see any barriers to that future becoming real.  
    I’d like to extend another big thank you to Bill and Sébastien for their time. And to our listeners, as always, it’s a pleasure to have you along for the ride. I hope you’ll join us for our remaining conversations, as well as a second coauthor roundtable with Carey and Zak.  
    Until next time.  
    #how #reshaping #future #healthcare #medical
    How AI is reshaping the future of healthcare and medical research
    Transcript        PETER LEE: “In ‘The Little Black Bag,’ a classic science fiction story, a high-tech doctor’s kit of the future is accidentally transported back to the 1950s, into the shaky hands of a washed-up, alcoholic doctor. The ultimate medical tool, it redeems the doctor wielding it, allowing him to practice gratifyingly heroic medicine. … The tale ends badly for the doctor and his treacherous assistant, but it offered a picture of how advanced technology could transform medicine—powerful when it was written nearly 75 years ago and still so today. What would be the Al equivalent of that little black bag? At this moment when new capabilities are emerging, how do we imagine them into medicine?”           This is The AI Revolution in Medicine, Revisited. I’m your host, Peter Lee.    Shortly after OpenAI’s GPT-4 was publicly released, Carey Goldberg, Dr. Zak Kohane, and I published The AI Revolution in Medicine to help educate the world of healthcare and medical research about the transformative impact this new generative AI technology could have. But because we wrote the book when GPT-4 was still a secret, we had to speculate. Now, two years later, what did we get right, and what did we get wrong?     In this series, we’ll talk to clinicians, patients, hospital administrators, and others to understand the reality of AI in the field and where we go from here.  The book passage I read at the top is from “Chapter 10: The Big Black Bag.”  In imagining AI in medicine, Carey, Zak, and I included in our book two fictional accounts. In the first, a medical resident consults GPT-4 on her personal phone as the patient in front of her crashes. Within seconds, it offers an alternate response based on recent literature. In the second account, a 90-year-old woman with several chronic conditions is living independently and receiving near-constant medical support from an AI aide.    In our conversations with the guests we’ve spoken to so far, we’ve caught a glimpse of these predicted futures, seeing how clinicians and patients are actually using AI today and how developers are leveraging the technology in the healthcare products and services they’re creating. In fact, that first fictional account isn’t so fictional after all, as most of the doctors in the real world actually appear to be using AI at least occasionally—and sometimes much more than occasionally—to help in their daily clinical work. And as for the second fictional account, which is more of a science fiction account, it seems we are indeed on the verge of a new way of delivering and receiving healthcare, though the future is still very much open.  As we continue to examine the current state of AI in healthcare and its potential to transform the field, I’m pleased to welcome Bill Gates and Sébastien Bubeck.   Bill may be best known as the co-founder of Microsoft, having created the company with his childhood friend Paul Allen in 1975. He’s now the founder of Breakthrough Energy, which aims to advance clean energy innovation, and TerraPower, a company developing groundbreaking nuclear energy and science technologies. He also chairs the world’s largest philanthropic organization, the Gates Foundation, and focuses on solving a variety of health challenges around the globe and here at home.  Sébastien is a research lead at OpenAI. He was previously a distinguished scientist, vice president of AI, and a colleague of mine here at Microsoft, where his work included spearheading the development of the family of small language models known as Phi. While at Microsoft, he also coauthored the discussion-provoking 2023 paper “Sparks of Artificial General Intelligence,” which presented the results of early experiments with GPT-4 conducted by a small team from Microsoft Research.      Here’s my conversation with Bill Gates and Sébastien Bubeck.  LEE: Bill, welcome.  BILL GATES: Thank you.  LEE: Seb …  SÉBASTIEN BUBECK: Yeah. Hi, hi, Peter. Nice to be here.  LEE: You know, one of the things that I’ve been doing just to get the conversation warmed up is to talk about origin stories, and what I mean about origin stories is, you know, what was the first contact that you had with large language models or the concept of generative AI that convinced you or made you think that something really important was happening?  And so, Bill, I think I’ve heard the story about, you know, the time when the OpenAI folks—Sam Altman, Greg Brockman, and others—showed you something, but could we hear from you what those early encounters were like and what was going through your mind?   GATES: Well, I’d been visiting OpenAI soon after it was created to see things like GPT-2 and to see the little arm they had that was trying to match human manipulation and, you know, looking at their games like Dota that they were trying to get as good as human play. And honestly, I didn’t think the language model stuff they were doing, even when they got to GPT-3, would show the ability to learn, you know, in the same sense that a human reads a biology book and is able to take that knowledge and access it not only to pass a test but also to create new medicines.  And so my challenge to them was that if their LLM could get a five on the advanced placement biology test, then I would say, OK, it took biologic knowledge and encoded it in an accessible way and that I didn’t expect them to do that very quickly but it would be profound.   And it was only about six months after I challenged them to do that, that an early version of GPT-4 they brought up to a dinner at my house, and in fact, it answered most of the questions that night very well. The one it got totally wrong, we were … because it was so good, we kept thinking, Oh, we must be wrong. It turned out it was a math weaknessthat, you know, we later understood that that was an area of, weirdly, of incredible weakness of those early models. But, you know, that was when I realized, OK, the age of cheap intelligence was at its beginning.  LEE: Yeah. So I guess it seems like you had something similar to me in that my first encounters, I actually harbored some skepticism. Is it fair to say you were skeptical before that?  GATES: Well, the idea that we’ve figured out how to encode and access knowledge in this very deep sense without even understanding the nature of the encoding, …  LEE: Right.   GATES: … that is a bit weird.   LEE: Yeah.  GATES: We have an algorithm that creates the computation, but even say, OK, where is the president’s birthday stored in there? Where is this fact stored in there? The fact that even now when we’re playing around, getting a little bit more sense of it, it’s opaque to us what the semantic encoding is, it’s, kind of, amazing to me. I thought the invention of knowledge storage would be an explicit way of encoding knowledge, not an implicit statistical training.  LEE: Yeah, yeah. All right. So, Seb, you know, on this same topic, you know, I got—as we say at Microsoft—I got pulled into the tent.  BUBECK: Yes.   LEE: Because this was a very secret project. And then, um, I had the opportunity to select a small number of researchers in MSRto join and start investigating this thing seriously. And the first person I pulled in was you.  BUBECK: Yeah.  LEE: And so what were your first encounters? Because I actually don’t remember what happened then.  BUBECK: Oh, I remember it very well.My first encounter with GPT-4 was in a meeting with the two of you, actually. But my kind of first contact, the first moment where I realized that something was happening with generative AI, was before that. And I agree with Bill that I also wasn’t too impressed by GPT-3.  I though that it was kind of, you know, very naturally mimicking the web, sort of parroting what was written there in a nice way. Still in a way which seemed very impressive. But it wasn’t really intelligent in any way. But shortly after GPT-3, there was a model before GPT-4 that really shocked me, and this was the first image generation model, DALL-E 1.  So that was in 2021. And I will forever remember the press release of OpenAI where they had this prompt of an avocado chair and then you had this image of the avocado chair.And what really shocked me is that clearly the model kind of “understood” what is a chair, what is an avocado, and was able to merge those concepts.  So this was really, to me, the first moment where I saw some understanding in those models.   LEE: So this was, just to get the timing right, that was before I pulled you into the tent.  BUBECK: That was before. That was like a year before.  LEE: Right.   BUBECK: And now I will tell you how, you know, we went from that moment to the meeting with the two of you and GPT-4.  So once I saw this kind of understanding, I thought, OK, fine. It understands concept, but it’s still not able to reason. It cannot—as, you know, Bill was saying—it cannot learn from your document. It cannot reason.   So I set out to try to prove that. You know, this is what I was in the business of at the time, trying to prove things in mathematics. So I was trying to prove that basically autoregressive transformers could never reason. So I was trying to prove this. And after a year of work, I had something reasonable to show. And so I had the meeting with the two of you, and I had this example where I wanted to say, there is no way that an LLM is going to be able to do x.  And then as soon as I … I don’t know if you remember, Bill. But as soon as I said that, you said, oh, but wait a second. I had, you know, the OpenAI crew at my house recently, and they showed me a new model. Why don’t we ask this new model this question?   LEE: Yeah. BUBECK: And we did, and it solved it on the spot. And that really, honestly, just changed my life. Like, you know, I had been working for a year trying to say that this was impossible. And just right there, it was shown to be possible.   LEE:One of the very first things I got interested in—because I was really thinking a lot about healthcare—was healthcare and medicine.  And I don’t know if the two of you remember, but I ended up doing a lot of tests. I ran through, you know, step one and step two of the US Medical Licensing Exam. Did a whole bunch of other things. I wrote this big report. It was, you know, I can’t remember … a couple hundred pages.   And I needed to share this with someone. I didn’t … there weren’t too many people I could share it with. So I sent, I think, a copy to you, Bill. Sent a copy to you, Seb.   I hardly slept for about a week putting that report together. And, yeah, and I kept working on it. But I was far from alone. I think everyone who was in the tent, so to speak, in those early days was going through something pretty similar. All right. So I think … of course, a lot of what I put in the report also ended up being examples that made it into the book.  But the main purpose of this conversation isn’t to reminisce aboutor indulge in those reminiscences but to talk about what’s happening in healthcare and medicine. And, you know, as I said, we wrote this book. We did it very, very quickly. Seb, you helped. Bill, you know, you provided a review and some endorsements.  But, you know, honestly, we didn’t know what we were talking about because no one had access to this thing. And so we just made a bunch of guesses. So really, the whole thing I wanted to probe with the two of you is, now with two years of experience out in the world, what, you know, what do we think is happening today?  You know, is AI actually having an impact, positive or negative, on healthcare and medicine? And what do we now think is going to happen in the next two years, five years, or 10 years? And so I realize it’s a little bit too abstract to just ask it that way. So let me just try to narrow the discussion and guide us a little bit.   Um, the kind of administrative and clerical work, paperwork, around healthcare—and we made a lot of guesses about that—that appears to be going well, but, you know, Bill, I know we’ve discussed that sometimes that you think there ought to be a lot more going on. Do you have a viewpoint on how AI is actually finding its way into reducing paperwork?  GATES: Well, I’m stunned … I don’t think there should be a patient-doctor meeting where the AI is not sitting in and both transcribing, offering to help with the paperwork, and even making suggestions, although the doctor will be the one, you know, who makes the final decision about the diagnosis and whatever prescription gets done.   It’s so helpful. You know, when that patient goes home and their, you know, son who wants to understand what happened has some questions, that AI should be available to continue that conversation. And the way you can improve that experience and streamline things and, you know, involve the people who advise you. I don’t understand why that’s not more adopted, because there you still have the human in the loop making that final decision.  But even for, like, follow-up calls to make sure the patient did things, to understand if they have concerns and knowing when to escalate back to the doctor, the benefit is incredible. And, you know, that thing is ready for prime time. That paradigm is ready for prime time, in my view.  LEE: Yeah, there are some good products, but it seems like the number one use right now—and we kind of got this from some of the previous guests in previous episodes—is the use of AI just to respond to emails from patients.Does that make sense to you?  BUBECK: Yeah. So maybe I want to second what Bill was saying but maybe take a step back first. You know, two years ago, like, the concept of clinical scribes, which is one of the things that we’re talking about right now, it would have sounded, in fact, it sounded two years ago, borderline dangerous. Because everybody was worried about hallucinations. What happened if you have this AI listening in and then it transcribes, you know, something wrong?  Now, two years later, I think it’s mostly working. And in fact, it is not yet, you know, fully adopted. You’re right. But it is in production. It is used, you know, in many, many places. So this rate of progress is astounding because it wasn’t obvious that we would be able to overcome those obstacles of hallucination. It’s not to say that hallucinations are fully solved. In the case of the closed system, they are.   Now, I think more generally what’s going on in the background is that there is something that we, that certainly I, underestimated, which is this management overhead. So I think the reason why this is not adopted everywhere is really a training and teaching aspect. People need to be taught, like, those systems, how to interact with them.  And one example that I really like, a study that recently appeared where they tried to use ChatGPT for diagnosis and they were comparing doctors without and with ChatGPT. And the amazing thing … so this was a set of cases where the accuracy of the doctors alone was around 75%. ChatGPT alone was 90%. So that’s already kind of mind blowing. But then the kicker is that doctors with ChatGPT was 80%.   Intelligence alone is not enough. It’s also how it’s presented, how you interact with it. And ChatGPT, it’s an amazing tool. Obviously, I absolutely love it. But it’s not … you don’t want a doctor to have to type in, you know, prompts and use it that way.  It should be, as Bill was saying, kind of running continuously in the background, sending you notifications. And you have to be really careful of the rate at which those notifications are being sent. Because if they are too frequent, then the doctor will learn to ignore them. So you have to … all of those things matter, in fact, at least as much as the level of intelligence of the machine.  LEE: One of the things I think about, Bill, in that scenario that you described, doctors do some thinking about the patient when they write the note. So, you know, I’m always a little uncertain whether it’s actually … you know, you wouldn’t necessarily want to fully automate this, I don’t think. Or at least there needs to be some prompt to the doctor to make sure that the doctor puts some thought into what happened in the encounter with the patient. Does that make sense to you at all?  GATES: At this stage, you know, I’d still put the onus on the doctor to write the conclusions and the summary and not delegate that.  The tradeoffs you make a little bit are somewhat dependent on the situation you’re in. If you’re in Africa, So, yes, the doctor’s still going to have to do a lot of work, but just the quality of letting the patient and the people around them interact and ask questions and have things explained, that alone is such a quality improvement. It’s mind blowing.   LEE: So since you mentioned, you know, Africa—and, of course, this touches on the mission and some of the priorities of the Gates Foundation and this idea of democratization of access to expert medical care—what’s the most interesting stuff going on right now? Are there people and organizations or technologies that are impressing you or that you’re tracking?  GATES: Yeah. So the Gates Foundation has given out a lot of grants to people in Africa doing education, agriculture but more healthcare examples than anything. And the way these things start off, they often start out either being patient-centric in a narrow situation, like, OK, I’m a pregnant woman; talk to me. Or, I have infectious disease symptoms; talk to me. Or they’re connected to a health worker where they’re helping that worker get their job done. And we have lots of pilots out, you know, in both of those cases.   The dream would be eventually to have the thing the patient consults be so broad that it’s like having a doctor available who understands the local things.   LEE: Right.   GATES: We’re not there yet. But over the next two or three years, you know, particularly given the worsening financial constraints against African health systems, where the withdrawal of money has been dramatic, you know, figuring out how to take this—what I sometimes call “free intelligence”—and build a quality health system around that, we will have to be more radical in low-income countries than any rich country is ever going to be.   LEE: Also, there’s maybe a different regulatory environment, so some of those things maybe are easier? Because right now, I think the world hasn’t figured out how to and whether to regulate, let’s say, an AI that might give a medical diagnosis or write a prescription for a medication.  BUBECK: Yeah. I think one issue with this, and it’s also slowing down the deployment of AI in healthcare more generally, is a lack of proper benchmark. Because, you know, you were mentioning the USMLE, for example. That’s a great test to test human beings and their knowledge of healthcare and medicine. But it’s not a great test to give to an AI.  It’s not asking the right questions. So finding what are the right questions to test whether an AI system is ready to give diagnosis in a constrained setting, that’s a very, very important direction, which to my surprise, is not yet accelerating at the rate that I was hoping for.  LEE: OK, so that gives me an excuse to get more now into the core AI tech because something I’ve discussed with both of you is this issue of what are the right tests. And you both know the very first test I give to any new spin of an LLM is I present a patient, the results—a mythical patient—the results of my physical exam, my mythical physical exam. Maybe some results of some initial labs. And then I present or propose a differential diagnosis. And if you’re not in medicine, a differential diagnosis you can just think of as a prioritized list of the possible diagnoses that fit with all that data. And in that proposed differential, I always intentionally make two mistakes.  I make a textbook technical error in one of the possible elements of the differential diagnosis, and I have an error of omission. And, you know, I just want to know, does the LLM understand what I’m talking about? And all the good ones out there do now. But then I want to know, can it spot the errors? And then most importantly, is it willing to tell me I’m wrong, that I’ve made a mistake?   That last piece seems really hard for AI today. And so let me ask you first, Seb, because at the time of this taping, of course, there was a new spin of GPT-4o last week that became overly sycophantic. In other words, it was actually prone in that test of mine not only to not tell me I’m wrong, but it actually praised me for the creativity of my differential.What’s up with that?  BUBECK: Yeah, I guess it’s a testament to the fact that training those models is still more of an art than a science. So it’s a difficult job. Just to be clear with the audience, we have rolled back thatversion of GPT-4o, so now we don’t have the sycophant version out there.  Yeah, no, it’s a really difficult question. It has to do … as you said, it’s very technical. It has to do with the post-training and how, like, where do you nudge the model? So, you know, there is this very classical by now technique called RLHF, where you push the model in the direction of a certain reward model. So the reward model is just telling the model, you know, what behavior is good, what behavior is bad.  But this reward model is itself an LLM, and, you know, Bill was saying at the very beginning of the conversation that we don’t really understand how those LLMs deal with concepts like, you know, where is the capital of France located? Things like that. It is the same thing for this reward model. We don’t know why it says that it prefers one output to another, and whether this is correlated with some sycophancy is, you know, something that we discovered basically just now. That if you push too hard in optimization on this reward model, you will get a sycophant model.  So it’s kind of … what I’m trying to say is we became too good at what we were doing, and we ended up, in fact, in a trap of the reward model.  LEE: I mean, you do want … it’s a difficult balance because you do want models to follow your desires and …  BUBECK: It’s a very difficult, very difficult balance.  LEE: So this brings up then the following question for me, which is the extent to which we think we’ll need to have specially trained models for things. So let me start with you, Bill. Do you have a point of view on whether we will need to, you know, quote-unquote take AI models to med school? Have them specially trained? Like, if you were going to deploy something to give medical care in underserved parts of the world, do we need to do something special to create those models?  GATES: We certainly need to teach them the African languages and the unique dialects so that the multimedia interactions are very high quality. We certainly need to teach them the disease prevalence and unique disease patterns like, you know, neglected tropical diseases and malaria. So we need to gather a set of facts that somebody trying to go for a US customer base, you know, wouldn’t necessarily have that in there.  Those two things are actually very straightforward because the additional training time is small. I’d say for the next few years, we’ll also need to do reinforcement learning about the context of being a doctor and how important certain behaviors are. Humans learn over the course of their life to some degree that, I’m in a different context and the way I behave in terms of being willing to criticize or be nice, you know, how important is it? Who’s here? What’s my relationship to them?   Right now, these machines don’t have that broad social experience. And so if you know it’s going to be used for health things, a lot of reinforcement learning of the very best humans in that context would still be valuable. Eventually, the models will, having read all the literature of the world about good doctors, bad doctors, it’ll understand as soon as you say, “I want you to be a doctor diagnosing somebody.” All of the implicit reinforcement that fits that situation, you know, will be there. LEE: Yeah. GATES: And so I hope three years from now, we don’t have to do that reinforcement learning. But today, for any medical context, you would want a lot of data to reinforce tone, willingness to say things when, you know, there might be something significant at stake.  LEE: Yeah. So, you know, something Bill said, kind of, reminds me of another thing that I think we missed, which is, the context also … and the specialization also pertains to different, I guess, what we still call “modes,” although I don’t know if the idea of multimodal is the same as it was two years ago. But, you know, what do you make of all of the hubbub around—in fact, within Microsoft Research, this is a big deal, but I think we’re far from alone—you know, medical images and vision, video, proteins and molecules, cell, you know, cellular data and so on.  BUBECK: Yeah. OK. So there is a lot to say to everything … to the last, you know, couple of minutes. Maybe on the specialization aspect, you know, I think there is, hiding behind this, a really fundamental scientific question of whether eventually we have a singular AGIthat kind of knows everything and you can just put, you know, explain your own context and it will just get it and understand everything.  That’s one vision. I have to say, I don’t particularly believe in this vision. In fact, we humans are not like that at all. I think, hopefully, we are general intelligences, yet we have to specialize a lot. And, you know, I did myself a lot of RL, reinforcement learning, on mathematics. Like, that’s what I did, you know, spent a lot of time doing that. And I didn’t improve on other aspects. You know, in fact, I probably degraded in other aspects.So it’s … I think it’s an important example to have in mind.  LEE: I think I might disagree with you on that, though, because, like, doesn’t a model have to see both good science and bad science in order to be able to gain the ability to discern between the two?  BUBECK: Yeah, no, that absolutely. I think there is value in seeing the generality, in having a very broad base. But then you, kind of, specialize on verticals. And this is where also, you know, open-weights model, which we haven’t talked about yet, are really important because they allow you to provide this broad base to everyone. And then you can specialize on top of it.  LEE: So we have about three hours of stuff to talk about, but our time is actually running low. BUBECK: Yes, yes, yes.   LEE: So I think I want … there’s a more provocative question. It’s almost a silly question, but I need to ask it of the two of you, which is, is there a future, you know, where AI replaces doctors or replaces, you know, medical specialties that we have today? So what does the world look like, say, five years from now?  GATES: Well, it’s important to distinguish healthcare discovery activity from healthcare delivery activity. We focused mostly on delivery. I think it’s very much within the realm of possibility that the AI is not only accelerating healthcare discovery but substituting for a lot of the roles of, you know, I’m an organic chemist, or I run various types of assays. I can see those, which are, you know, testable-output-type jobs but with still very high value, I can see, you know, some replacement in those areas before the doctor.   The doctor, still understanding the human condition and long-term dialogues, you know, they’ve had a lifetime of reinforcement of that, particularly when you get into areas like mental health. So I wouldn’t say in five years, either people will choose to adopt it, but it will be profound that there’ll be this nearly free intelligence that can do follow-up, that can help you, you know, make sure you went through different possibilities.  And so I’d say, yes, we’ll have doctors, but I’d say healthcare will be massively transformed in its quality and in efficiency by AI in that time period.  LEE: Is there a comparison, useful comparison, say, between doctors and, say, programmers, computer programmers, or doctors and, I don’t know, lawyers?  GATES: Programming is another one that has, kind of, a mathematical correctness to it, you know, and so the objective function that you’re trying to reinforce to, as soon as you can understand the state machines, you can have something that’s “checkable”; that’s correct. So I think programming, you know, which is weird to say, that the machine will beat us at most programming tasks before we let it take over roles that have deep empathy, you know, physical presence and social understanding in them.  LEE: Yeah. By the way, you know, I fully expect in five years that AI will produce mathematical proofs that are checkable for validity, easily checkable, because they’ll be written in a proof-checking language like Lean or something but will be so complex that no human mathematician can understand them. I expect that to happen.   I can imagine in some fields, like cellular biology, we could have the same situation in the future because the molecular pathways, the chemistry, biochemistry of human cells or living cells is as complex as any mathematics, and so it seems possible that we may be in a state where in wet lab, we see, Oh yeah, this actually works, but no one can understand why.  BUBECK: Yeah, absolutely. I mean, I think I really agree with Bill’s distinction of the discovery and the delivery, and indeed, the discovery’s when you can check things, and at the end, there is an artifact that you can verify. You know, you can run the protocol in the wet lab and seeproduced what you wanted. So I absolutely agree with that.   And in fact, you know, we don’t have to talk five years from now. I don’t know if you know, but just recently, there was a paper that was published on a scientific discovery using o3- mini. So this is really amazing. And, you know, just very quickly, just so people know, it was about this statistical physics model, the frustrated Potts model, which has to do with coloring, and basically, the case of three colors, like, more than two colors was open for a long time, and o3 was able to reduce the case of three colors to two colors.   LEE: Yeah.  BUBECK: Which is just, like, astounding. And this is not … this is now. This is happening right now. So this is something that I personally didn’t expect it would happen so quickly, and it’s due to those reasoning models.   Now, on the delivery side, I would add something more to it for the reason why doctors and, in fact, lawyers and coders will remain for a long time, and it’s because we still don’t understand how those models generalize. Like, at the end of the day, we are not able to tell you when they are confronted with a really new, novel situation, whether they will work or not.  Nobody is able to give you that guarantee. And I think until we understand this generalization better, we’re not going to be willing to just let the system in the wild without human supervision.  LEE: But don’t human doctors, human specialists … so, for example, a cardiologist sees a patient in a certain way that a nephrologist …  BUBECK: Yeah. LEE: … or an endocrinologist might not. BUBECK: That’s right. But another cardiologist will understand and, kind of, expect a certain level of generalization from their peer. And this, we just don’t have it with AI models. Now, of course, you’re exactly right. That generalization is also hard for humans. Like, if you have a human trained for one task and you put them into another task, then you don’t … you often don’t know. LEE: OK. You know, the podcast is focused on what’s happened over the last two years. But now, I’d like one provocative prediction about what you think the world of AI and medicine is going to be at some point in the future. You pick your timeframe. I don’t care if it’s two years or 20 years from now, but, you know, what do you think will be different about AI in medicine in that future than today?  BUBECK: Yeah, I think the deployment is going to accelerate soon. Like, we’re really not missing very much. There is this enormous capability overhang. Like, even if progress completely stopped, with current systems, we can do a lot more than what we’re doing right now. So I think this will … this has to be realized, you know, sooner rather than later.  And I think it’s probably dependent on these benchmarks and proper evaluation and tying this with regulation. So these are things that take time in human society and for good reason. But now we already are at two years; you know, give it another two years and it should be really …   LEE: Will AI prescribe your medicines? Write your prescriptions?  BUBECK: I think yes. I think yes.  LEE: OK. Bill?  GATES: Well, I think the next two years, we’ll have massive pilots, and so the amount of use of the AI, still in a copilot-type mode, you know, we should get millions of patient visits, you know, both in general medicine and in the mental health side, as well. And I think that’s going to build up both the data and the confidence to give the AI some additional autonomy. You know, are you going to let it talk to you at night when you’re panicked about your mental health with some ability to escalate? And, you know, I’ve gone so far as to tell politicians with national health systems that if they deploy AI appropriately, that the quality of care, the overload of the doctors, the improvement in the economics will be enough that their voters will be stunned because they just don’t expect this, and, you know, they could be reelectedjust on this one thing of fixing what is a very overloaded and economically challenged health system in these rich countries.  You know, my personal role is going to be to make sure that in the poorer countries, there isn’t some lag; in fact, in many cases, that we’ll be more aggressive because, you know, we’re comparing to having no access to doctors at all. And, you know, so I think whether it’s India or Africa, there’ll be lessons that are globally valuable because we need medical intelligence. And, you know, thank god AI is going to provide a lot of that.  LEE: Well, on that optimistic note, I think that’s a good way to end. Bill, Seb, really appreciate all of this.   I think the most fundamental prediction we made in the book is that AI would actually find its way into the practice of medicine, and I think that that at least has come true, maybe in different ways than we expected, but it’s come true, and I think it’ll only accelerate from here. So thanks again, both of you.   GATES: Yeah. Thanks, you guys.  BUBECK: Thank you, Peter. Thanks, Bill.  LEE: I just always feel such a sense of privilege to have a chance to interact and actually work with people like Bill and Sébastien.    With Bill, I’m always amazed at how practically minded he is. He’s really thinking about the nuts and bolts of what AI might be able to do for people, and his thoughts about underserved parts of the world, the idea that we might actually be able to empower people with access to expert medical knowledge, I think is both inspiring and amazing.   And then, Seb, Sébastien Bubeck, he’s just absolutely a brilliant mind. He has a really firm grip on the deep mathematics of artificial intelligence and brings that to bear in his research and development work. And where that mathematics takes him isn’t just into the nuts and bolts of algorithms but into philosophical questions about the nature of intelligence.   One of the things that Sébastien brought up was the state of evaluation of AI systems. And indeed, he was fairly critical in our conversation. But of course, the world of AI research and development is just moving so fast, and indeed, since we recorded our conversation, OpenAI, in fact, released a new evaluation metric that is directly relevant to medical applications, and that is something called HealthBench. And Microsoft Research also released a new evaluation approach or process called ADeLe.   HealthBench and ADeLe are examples of new approaches to evaluating AI models that are less about testing their knowledge and ability to pass multiple-choice exams and instead are evaluation approaches designed to assess how well AI models are able to complete tasks that actually arise every day in typical healthcare or biomedical research settings. These are examples of really important good work that speak to how well AI models work in the real world of healthcare and biomedical research and how well they can collaborate with human beings in those settings.  You know, I asked Bill and Seb to make some predictions about the future. You know, my own answer, I expect that we’re going to be able to use AI to change how we diagnose patients, change how we decide treatment options.   If you’re a doctor or a nurse and you encounter a patient, you’ll ask questions, do a physical exam, you know, call out for labs just like you do today, but then you’ll be able to engage with AI based on all of that data and just ask, you know, based on all the other people who have gone through the same experience, who have similar data, how were they diagnosed? How were they treated? What were their outcomes? And what does that mean for the patient I have right now? Some people call it the “patients like me” paradigm. And I think that’s going to become real because of AI within our lifetimes. That idea of really grounding the delivery in healthcare and medical practice through data and intelligence, I actually now don’t see any barriers to that future becoming real.   I’d like to extend another big thank you to Bill and Sébastien for their time. And to our listeners, as always, it’s a pleasure to have you along for the ride. I hope you’ll join us for our remaining conversations, as well as a second coauthor roundtable with Carey and Zak.   Until next time.   #how #reshaping #future #healthcare #medical
    WWW.MICROSOFT.COM
    How AI is reshaping the future of healthcare and medical research
    Transcript [MUSIC]      [BOOK PASSAGE]   PETER LEE: “In ‘The Little Black Bag,’ a classic science fiction story, a high-tech doctor’s kit of the future is accidentally transported back to the 1950s, into the shaky hands of a washed-up, alcoholic doctor. The ultimate medical tool, it redeems the doctor wielding it, allowing him to practice gratifyingly heroic medicine. … The tale ends badly for the doctor and his treacherous assistant, but it offered a picture of how advanced technology could transform medicine—powerful when it was written nearly 75 years ago and still so today. What would be the Al equivalent of that little black bag? At this moment when new capabilities are emerging, how do we imagine them into medicine?”   [END OF BOOK PASSAGE]     [THEME MUSIC]     This is The AI Revolution in Medicine, Revisited. I’m your host, Peter Lee.    Shortly after OpenAI’s GPT-4 was publicly released, Carey Goldberg, Dr. Zak Kohane, and I published The AI Revolution in Medicine to help educate the world of healthcare and medical research about the transformative impact this new generative AI technology could have. But because we wrote the book when GPT-4 was still a secret, we had to speculate. Now, two years later, what did we get right, and what did we get wrong?     In this series, we’ll talk to clinicians, patients, hospital administrators, and others to understand the reality of AI in the field and where we go from here.   [THEME MUSIC FADES] The book passage I read at the top is from “Chapter 10: The Big Black Bag.”  In imagining AI in medicine, Carey, Zak, and I included in our book two fictional accounts. In the first, a medical resident consults GPT-4 on her personal phone as the patient in front of her crashes. Within seconds, it offers an alternate response based on recent literature. In the second account, a 90-year-old woman with several chronic conditions is living independently and receiving near-constant medical support from an AI aide.    In our conversations with the guests we’ve spoken to so far, we’ve caught a glimpse of these predicted futures, seeing how clinicians and patients are actually using AI today and how developers are leveraging the technology in the healthcare products and services they’re creating. In fact, that first fictional account isn’t so fictional after all, as most of the doctors in the real world actually appear to be using AI at least occasionally—and sometimes much more than occasionally—to help in their daily clinical work. And as for the second fictional account, which is more of a science fiction account, it seems we are indeed on the verge of a new way of delivering and receiving healthcare, though the future is still very much open.  As we continue to examine the current state of AI in healthcare and its potential to transform the field, I’m pleased to welcome Bill Gates and Sébastien Bubeck.   Bill may be best known as the co-founder of Microsoft, having created the company with his childhood friend Paul Allen in 1975. He’s now the founder of Breakthrough Energy, which aims to advance clean energy innovation, and TerraPower, a company developing groundbreaking nuclear energy and science technologies. He also chairs the world’s largest philanthropic organization, the Gates Foundation, and focuses on solving a variety of health challenges around the globe and here at home.  Sébastien is a research lead at OpenAI. He was previously a distinguished scientist, vice president of AI, and a colleague of mine here at Microsoft, where his work included spearheading the development of the family of small language models known as Phi. While at Microsoft, he also coauthored the discussion-provoking 2023 paper “Sparks of Artificial General Intelligence,” which presented the results of early experiments with GPT-4 conducted by a small team from Microsoft Research.    [TRANSITION MUSIC]   Here’s my conversation with Bill Gates and Sébastien Bubeck.  LEE: Bill, welcome.  BILL GATES: Thank you.  LEE: Seb …  SÉBASTIEN BUBECK: Yeah. Hi, hi, Peter. Nice to be here.  LEE: You know, one of the things that I’ve been doing just to get the conversation warmed up is to talk about origin stories, and what I mean about origin stories is, you know, what was the first contact that you had with large language models or the concept of generative AI that convinced you or made you think that something really important was happening?  And so, Bill, I think I’ve heard the story about, you know, the time when the OpenAI folks—Sam Altman, Greg Brockman, and others—showed you something, but could we hear from you what those early encounters were like and what was going through your mind?   GATES: Well, I’d been visiting OpenAI soon after it was created to see things like GPT-2 and to see the little arm they had that was trying to match human manipulation and, you know, looking at their games like Dota that they were trying to get as good as human play. And honestly, I didn’t think the language model stuff they were doing, even when they got to GPT-3, would show the ability to learn, you know, in the same sense that a human reads a biology book and is able to take that knowledge and access it not only to pass a test but also to create new medicines.  And so my challenge to them was that if their LLM could get a five on the advanced placement biology test, then I would say, OK, it took biologic knowledge and encoded it in an accessible way and that I didn’t expect them to do that very quickly but it would be profound.   And it was only about six months after I challenged them to do that, that an early version of GPT-4 they brought up to a dinner at my house, and in fact, it answered most of the questions that night very well. The one it got totally wrong, we were … because it was so good, we kept thinking, Oh, we must be wrong. It turned out it was a math weakness [LAUGHTER] that, you know, we later understood that that was an area of, weirdly, of incredible weakness of those early models. But, you know, that was when I realized, OK, the age of cheap intelligence was at its beginning.  LEE: Yeah. So I guess it seems like you had something similar to me in that my first encounters, I actually harbored some skepticism. Is it fair to say you were skeptical before that?  GATES: Well, the idea that we’ve figured out how to encode and access knowledge in this very deep sense without even understanding the nature of the encoding, …  LEE: Right.   GATES: … that is a bit weird.   LEE: Yeah.  GATES: We have an algorithm that creates the computation, but even say, OK, where is the president’s birthday stored in there? Where is this fact stored in there? The fact that even now when we’re playing around, getting a little bit more sense of it, it’s opaque to us what the semantic encoding is, it’s, kind of, amazing to me. I thought the invention of knowledge storage would be an explicit way of encoding knowledge, not an implicit statistical training.  LEE: Yeah, yeah. All right. So, Seb, you know, on this same topic, you know, I got—as we say at Microsoft—I got pulled into the tent. [LAUGHS]  BUBECK: Yes.   LEE: Because this was a very secret project. And then, um, I had the opportunity to select a small number of researchers in MSR [Microsoft Research] to join and start investigating this thing seriously. And the first person I pulled in was you.  BUBECK: Yeah.  LEE: And so what were your first encounters? Because I actually don’t remember what happened then.  BUBECK: Oh, I remember it very well. [LAUGHS] My first encounter with GPT-4 was in a meeting with the two of you, actually. But my kind of first contact, the first moment where I realized that something was happening with generative AI, was before that. And I agree with Bill that I also wasn’t too impressed by GPT-3.  I though that it was kind of, you know, very naturally mimicking the web, sort of parroting what was written there in a nice way. Still in a way which seemed very impressive. But it wasn’t really intelligent in any way. But shortly after GPT-3, there was a model before GPT-4 that really shocked me, and this was the first image generation model, DALL-E 1.  So that was in 2021. And I will forever remember the press release of OpenAI where they had this prompt of an avocado chair and then you had this image of the avocado chair. [LAUGHTER] And what really shocked me is that clearly the model kind of “understood” what is a chair, what is an avocado, and was able to merge those concepts.  So this was really, to me, the first moment where I saw some understanding in those models.   LEE: So this was, just to get the timing right, that was before I pulled you into the tent.  BUBECK: That was before. That was like a year before.  LEE: Right.   BUBECK: And now I will tell you how, you know, we went from that moment to the meeting with the two of you and GPT-4.  So once I saw this kind of understanding, I thought, OK, fine. It understands concept, but it’s still not able to reason. It cannot—as, you know, Bill was saying—it cannot learn from your document. It cannot reason.   So I set out to try to prove that. You know, this is what I was in the business of at the time, trying to prove things in mathematics. So I was trying to prove that basically autoregressive transformers could never reason. So I was trying to prove this. And after a year of work, I had something reasonable to show. And so I had the meeting with the two of you, and I had this example where I wanted to say, there is no way that an LLM is going to be able to do x.  And then as soon as I … I don’t know if you remember, Bill. But as soon as I said that, you said, oh, but wait a second. I had, you know, the OpenAI crew at my house recently, and they showed me a new model. Why don’t we ask this new model this question?   LEE: Yeah. BUBECK: And we did, and it solved it on the spot. And that really, honestly, just changed my life. Like, you know, I had been working for a year trying to say that this was impossible. And just right there, it was shown to be possible.   LEE: [LAUGHS] One of the very first things I got interested in—because I was really thinking a lot about healthcare—was healthcare and medicine.  And I don’t know if the two of you remember, but I ended up doing a lot of tests. I ran through, you know, step one and step two of the US Medical Licensing Exam. Did a whole bunch of other things. I wrote this big report. It was, you know, I can’t remember … a couple hundred pages.   And I needed to share this with someone. I didn’t … there weren’t too many people I could share it with. So I sent, I think, a copy to you, Bill. Sent a copy to you, Seb.   I hardly slept for about a week putting that report together. And, yeah, and I kept working on it. But I was far from alone. I think everyone who was in the tent, so to speak, in those early days was going through something pretty similar. All right. So I think … of course, a lot of what I put in the report also ended up being examples that made it into the book.  But the main purpose of this conversation isn’t to reminisce about [LAUGHS] or indulge in those reminiscences but to talk about what’s happening in healthcare and medicine. And, you know, as I said, we wrote this book. We did it very, very quickly. Seb, you helped. Bill, you know, you provided a review and some endorsements.  But, you know, honestly, we didn’t know what we were talking about because no one had access to this thing. And so we just made a bunch of guesses. So really, the whole thing I wanted to probe with the two of you is, now with two years of experience out in the world, what, you know, what do we think is happening today?  You know, is AI actually having an impact, positive or negative, on healthcare and medicine? And what do we now think is going to happen in the next two years, five years, or 10 years? And so I realize it’s a little bit too abstract to just ask it that way. So let me just try to narrow the discussion and guide us a little bit.   Um, the kind of administrative and clerical work, paperwork, around healthcare—and we made a lot of guesses about that—that appears to be going well, but, you know, Bill, I know we’ve discussed that sometimes that you think there ought to be a lot more going on. Do you have a viewpoint on how AI is actually finding its way into reducing paperwork?  GATES: Well, I’m stunned … I don’t think there should be a patient-doctor meeting where the AI is not sitting in and both transcribing, offering to help with the paperwork, and even making suggestions, although the doctor will be the one, you know, who makes the final decision about the diagnosis and whatever prescription gets done.   It’s so helpful. You know, when that patient goes home and their, you know, son who wants to understand what happened has some questions, that AI should be available to continue that conversation. And the way you can improve that experience and streamline things and, you know, involve the people who advise you. I don’t understand why that’s not more adopted, because there you still have the human in the loop making that final decision.  But even for, like, follow-up calls to make sure the patient did things, to understand if they have concerns and knowing when to escalate back to the doctor, the benefit is incredible. And, you know, that thing is ready for prime time. That paradigm is ready for prime time, in my view.  LEE: Yeah, there are some good products, but it seems like the number one use right now—and we kind of got this from some of the previous guests in previous episodes—is the use of AI just to respond to emails from patients. [LAUGHTER] Does that make sense to you?  BUBECK: Yeah. So maybe I want to second what Bill was saying but maybe take a step back first. You know, two years ago, like, the concept of clinical scribes, which is one of the things that we’re talking about right now, it would have sounded, in fact, it sounded two years ago, borderline dangerous. Because everybody was worried about hallucinations. What happened if you have this AI listening in and then it transcribes, you know, something wrong?  Now, two years later, I think it’s mostly working. And in fact, it is not yet, you know, fully adopted. You’re right. But it is in production. It is used, you know, in many, many places. So this rate of progress is astounding because it wasn’t obvious that we would be able to overcome those obstacles of hallucination. It’s not to say that hallucinations are fully solved. In the case of the closed system, they are.   Now, I think more generally what’s going on in the background is that there is something that we, that certainly I, underestimated, which is this management overhead. So I think the reason why this is not adopted everywhere is really a training and teaching aspect. People need to be taught, like, those systems, how to interact with them.  And one example that I really like, a study that recently appeared where they tried to use ChatGPT for diagnosis and they were comparing doctors without and with ChatGPT (opens in new tab). And the amazing thing … so this was a set of cases where the accuracy of the doctors alone was around 75%. ChatGPT alone was 90%. So that’s already kind of mind blowing. But then the kicker is that doctors with ChatGPT was 80%.   Intelligence alone is not enough. It’s also how it’s presented, how you interact with it. And ChatGPT, it’s an amazing tool. Obviously, I absolutely love it. But it’s not … you don’t want a doctor to have to type in, you know, prompts and use it that way.  It should be, as Bill was saying, kind of running continuously in the background, sending you notifications. And you have to be really careful of the rate at which those notifications are being sent. Because if they are too frequent, then the doctor will learn to ignore them. So you have to … all of those things matter, in fact, at least as much as the level of intelligence of the machine.  LEE: One of the things I think about, Bill, in that scenario that you described, doctors do some thinking about the patient when they write the note. So, you know, I’m always a little uncertain whether it’s actually … you know, you wouldn’t necessarily want to fully automate this, I don’t think. Or at least there needs to be some prompt to the doctor to make sure that the doctor puts some thought into what happened in the encounter with the patient. Does that make sense to you at all?  GATES: At this stage, you know, I’d still put the onus on the doctor to write the conclusions and the summary and not delegate that.  The tradeoffs you make a little bit are somewhat dependent on the situation you’re in. If you’re in Africa, So, yes, the doctor’s still going to have to do a lot of work, but just the quality of letting the patient and the people around them interact and ask questions and have things explained, that alone is such a quality improvement. It’s mind blowing.   LEE: So since you mentioned, you know, Africa—and, of course, this touches on the mission and some of the priorities of the Gates Foundation and this idea of democratization of access to expert medical care—what’s the most interesting stuff going on right now? Are there people and organizations or technologies that are impressing you or that you’re tracking?  GATES: Yeah. So the Gates Foundation has given out a lot of grants to people in Africa doing education, agriculture but more healthcare examples than anything. And the way these things start off, they often start out either being patient-centric in a narrow situation, like, OK, I’m a pregnant woman; talk to me. Or, I have infectious disease symptoms; talk to me. Or they’re connected to a health worker where they’re helping that worker get their job done. And we have lots of pilots out, you know, in both of those cases.   The dream would be eventually to have the thing the patient consults be so broad that it’s like having a doctor available who understands the local things.   LEE: Right.   GATES: We’re not there yet. But over the next two or three years, you know, particularly given the worsening financial constraints against African health systems, where the withdrawal of money has been dramatic, you know, figuring out how to take this—what I sometimes call “free intelligence”—and build a quality health system around that, we will have to be more radical in low-income countries than any rich country is ever going to be.   LEE: Also, there’s maybe a different regulatory environment, so some of those things maybe are easier? Because right now, I think the world hasn’t figured out how to and whether to regulate, let’s say, an AI that might give a medical diagnosis or write a prescription for a medication.  BUBECK: Yeah. I think one issue with this, and it’s also slowing down the deployment of AI in healthcare more generally, is a lack of proper benchmark. Because, you know, you were mentioning the USMLE [United States Medical Licensing Examination], for example. That’s a great test to test human beings and their knowledge of healthcare and medicine. But it’s not a great test to give to an AI.  It’s not asking the right questions. So finding what are the right questions to test whether an AI system is ready to give diagnosis in a constrained setting, that’s a very, very important direction, which to my surprise, is not yet accelerating at the rate that I was hoping for.  LEE: OK, so that gives me an excuse to get more now into the core AI tech because something I’ve discussed with both of you is this issue of what are the right tests. And you both know the very first test I give to any new spin of an LLM is I present a patient, the results—a mythical patient—the results of my physical exam, my mythical physical exam. Maybe some results of some initial labs. And then I present or propose a differential diagnosis. And if you’re not in medicine, a differential diagnosis you can just think of as a prioritized list of the possible diagnoses that fit with all that data. And in that proposed differential, I always intentionally make two mistakes.  I make a textbook technical error in one of the possible elements of the differential diagnosis, and I have an error of omission. And, you know, I just want to know, does the LLM understand what I’m talking about? And all the good ones out there do now. But then I want to know, can it spot the errors? And then most importantly, is it willing to tell me I’m wrong, that I’ve made a mistake?   That last piece seems really hard for AI today. And so let me ask you first, Seb, because at the time of this taping, of course, there was a new spin of GPT-4o last week that became overly sycophantic. In other words, it was actually prone in that test of mine not only to not tell me I’m wrong, but it actually praised me for the creativity of my differential. [LAUGHTER] What’s up with that?  BUBECK: Yeah, I guess it’s a testament to the fact that training those models is still more of an art than a science. So it’s a difficult job. Just to be clear with the audience, we have rolled back that [LAUGHS] version of GPT-4o, so now we don’t have the sycophant version out there.  Yeah, no, it’s a really difficult question. It has to do … as you said, it’s very technical. It has to do with the post-training and how, like, where do you nudge the model? So, you know, there is this very classical by now technique called RLHF [reinforcement learning from human feedback], where you push the model in the direction of a certain reward model. So the reward model is just telling the model, you know, what behavior is good, what behavior is bad.  But this reward model is itself an LLM, and, you know, Bill was saying at the very beginning of the conversation that we don’t really understand how those LLMs deal with concepts like, you know, where is the capital of France located? Things like that. It is the same thing for this reward model. We don’t know why it says that it prefers one output to another, and whether this is correlated with some sycophancy is, you know, something that we discovered basically just now. That if you push too hard in optimization on this reward model, you will get a sycophant model.  So it’s kind of … what I’m trying to say is we became too good at what we were doing, and we ended up, in fact, in a trap of the reward model.  LEE: I mean, you do want … it’s a difficult balance because you do want models to follow your desires and …  BUBECK: It’s a very difficult, very difficult balance.  LEE: So this brings up then the following question for me, which is the extent to which we think we’ll need to have specially trained models for things. So let me start with you, Bill. Do you have a point of view on whether we will need to, you know, quote-unquote take AI models to med school? Have them specially trained? Like, if you were going to deploy something to give medical care in underserved parts of the world, do we need to do something special to create those models?  GATES: We certainly need to teach them the African languages and the unique dialects so that the multimedia interactions are very high quality. We certainly need to teach them the disease prevalence and unique disease patterns like, you know, neglected tropical diseases and malaria. So we need to gather a set of facts that somebody trying to go for a US customer base, you know, wouldn’t necessarily have that in there.  Those two things are actually very straightforward because the additional training time is small. I’d say for the next few years, we’ll also need to do reinforcement learning about the context of being a doctor and how important certain behaviors are. Humans learn over the course of their life to some degree that, I’m in a different context and the way I behave in terms of being willing to criticize or be nice, you know, how important is it? Who’s here? What’s my relationship to them?   Right now, these machines don’t have that broad social experience. And so if you know it’s going to be used for health things, a lot of reinforcement learning of the very best humans in that context would still be valuable. Eventually, the models will, having read all the literature of the world about good doctors, bad doctors, it’ll understand as soon as you say, “I want you to be a doctor diagnosing somebody.” All of the implicit reinforcement that fits that situation, you know, will be there. LEE: Yeah. GATES: And so I hope three years from now, we don’t have to do that reinforcement learning. But today, for any medical context, you would want a lot of data to reinforce tone, willingness to say things when, you know, there might be something significant at stake.  LEE: Yeah. So, you know, something Bill said, kind of, reminds me of another thing that I think we missed, which is, the context also … and the specialization also pertains to different, I guess, what we still call “modes,” although I don’t know if the idea of multimodal is the same as it was two years ago. But, you know, what do you make of all of the hubbub around—in fact, within Microsoft Research, this is a big deal, but I think we’re far from alone—you know, medical images and vision, video, proteins and molecules, cell, you know, cellular data and so on.  BUBECK: Yeah. OK. So there is a lot to say to everything … to the last, you know, couple of minutes. Maybe on the specialization aspect, you know, I think there is, hiding behind this, a really fundamental scientific question of whether eventually we have a singular AGI [artificial general intelligence] that kind of knows everything and you can just put, you know, explain your own context and it will just get it and understand everything.  That’s one vision. I have to say, I don’t particularly believe in this vision. In fact, we humans are not like that at all. I think, hopefully, we are general intelligences, yet we have to specialize a lot. And, you know, I did myself a lot of RL, reinforcement learning, on mathematics. Like, that’s what I did, you know, spent a lot of time doing that. And I didn’t improve on other aspects. You know, in fact, I probably degraded in other aspects. [LAUGHTER] So it’s … I think it’s an important example to have in mind.  LEE: I think I might disagree with you on that, though, because, like, doesn’t a model have to see both good science and bad science in order to be able to gain the ability to discern between the two?  BUBECK: Yeah, no, that absolutely. I think there is value in seeing the generality, in having a very broad base. But then you, kind of, specialize on verticals. And this is where also, you know, open-weights model, which we haven’t talked about yet, are really important because they allow you to provide this broad base to everyone. And then you can specialize on top of it.  LEE: So we have about three hours of stuff to talk about, but our time is actually running low. BUBECK: Yes, yes, yes.   LEE: So I think I want … there’s a more provocative question. It’s almost a silly question, but I need to ask it of the two of you, which is, is there a future, you know, where AI replaces doctors or replaces, you know, medical specialties that we have today? So what does the world look like, say, five years from now?  GATES: Well, it’s important to distinguish healthcare discovery activity from healthcare delivery activity. We focused mostly on delivery. I think it’s very much within the realm of possibility that the AI is not only accelerating healthcare discovery but substituting for a lot of the roles of, you know, I’m an organic chemist, or I run various types of assays. I can see those, which are, you know, testable-output-type jobs but with still very high value, I can see, you know, some replacement in those areas before the doctor.   The doctor, still understanding the human condition and long-term dialogues, you know, they’ve had a lifetime of reinforcement of that, particularly when you get into areas like mental health. So I wouldn’t say in five years, either people will choose to adopt it, but it will be profound that there’ll be this nearly free intelligence that can do follow-up, that can help you, you know, make sure you went through different possibilities.  And so I’d say, yes, we’ll have doctors, but I’d say healthcare will be massively transformed in its quality and in efficiency by AI in that time period.  LEE: Is there a comparison, useful comparison, say, between doctors and, say, programmers, computer programmers, or doctors and, I don’t know, lawyers?  GATES: Programming is another one that has, kind of, a mathematical correctness to it, you know, and so the objective function that you’re trying to reinforce to, as soon as you can understand the state machines, you can have something that’s “checkable”; that’s correct. So I think programming, you know, which is weird to say, that the machine will beat us at most programming tasks before we let it take over roles that have deep empathy, you know, physical presence and social understanding in them.  LEE: Yeah. By the way, you know, I fully expect in five years that AI will produce mathematical proofs that are checkable for validity, easily checkable, because they’ll be written in a proof-checking language like Lean or something but will be so complex that no human mathematician can understand them. I expect that to happen.   I can imagine in some fields, like cellular biology, we could have the same situation in the future because the molecular pathways, the chemistry, biochemistry of human cells or living cells is as complex as any mathematics, and so it seems possible that we may be in a state where in wet lab, we see, Oh yeah, this actually works, but no one can understand why.  BUBECK: Yeah, absolutely. I mean, I think I really agree with Bill’s distinction of the discovery and the delivery, and indeed, the discovery’s when you can check things, and at the end, there is an artifact that you can verify. You know, you can run the protocol in the wet lab and see [if you have] produced what you wanted. So I absolutely agree with that.   And in fact, you know, we don’t have to talk five years from now. I don’t know if you know, but just recently, there was a paper that was published on a scientific discovery using o3- mini (opens in new tab). So this is really amazing. And, you know, just very quickly, just so people know, it was about this statistical physics model, the frustrated Potts model, which has to do with coloring, and basically, the case of three colors, like, more than two colors was open for a long time, and o3 was able to reduce the case of three colors to two colors.   LEE: Yeah.  BUBECK: Which is just, like, astounding. And this is not … this is now. This is happening right now. So this is something that I personally didn’t expect it would happen so quickly, and it’s due to those reasoning models.   Now, on the delivery side, I would add something more to it for the reason why doctors and, in fact, lawyers and coders will remain for a long time, and it’s because we still don’t understand how those models generalize. Like, at the end of the day, we are not able to tell you when they are confronted with a really new, novel situation, whether they will work or not.  Nobody is able to give you that guarantee. And I think until we understand this generalization better, we’re not going to be willing to just let the system in the wild without human supervision.  LEE: But don’t human doctors, human specialists … so, for example, a cardiologist sees a patient in a certain way that a nephrologist …  BUBECK: Yeah. LEE: … or an endocrinologist might not. BUBECK: That’s right. But another cardiologist will understand and, kind of, expect a certain level of generalization from their peer. And this, we just don’t have it with AI models. Now, of course, you’re exactly right. That generalization is also hard for humans. Like, if you have a human trained for one task and you put them into another task, then you don’t … you often don’t know. LEE: OK. You know, the podcast is focused on what’s happened over the last two years. But now, I’d like one provocative prediction about what you think the world of AI and medicine is going to be at some point in the future. You pick your timeframe. I don’t care if it’s two years or 20 years from now, but, you know, what do you think will be different about AI in medicine in that future than today?  BUBECK: Yeah, I think the deployment is going to accelerate soon. Like, we’re really not missing very much. There is this enormous capability overhang. Like, even if progress completely stopped, with current systems, we can do a lot more than what we’re doing right now. So I think this will … this has to be realized, you know, sooner rather than later.  And I think it’s probably dependent on these benchmarks and proper evaluation and tying this with regulation. So these are things that take time in human society and for good reason. But now we already are at two years; you know, give it another two years and it should be really …   LEE: Will AI prescribe your medicines? Write your prescriptions?  BUBECK: I think yes. I think yes.  LEE: OK. Bill?  GATES: Well, I think the next two years, we’ll have massive pilots, and so the amount of use of the AI, still in a copilot-type mode, you know, we should get millions of patient visits, you know, both in general medicine and in the mental health side, as well. And I think that’s going to build up both the data and the confidence to give the AI some additional autonomy. You know, are you going to let it talk to you at night when you’re panicked about your mental health with some ability to escalate? And, you know, I’ve gone so far as to tell politicians with national health systems that if they deploy AI appropriately, that the quality of care, the overload of the doctors, the improvement in the economics will be enough that their voters will be stunned because they just don’t expect this, and, you know, they could be reelected [LAUGHTER] just on this one thing of fixing what is a very overloaded and economically challenged health system in these rich countries.  You know, my personal role is going to be to make sure that in the poorer countries, there isn’t some lag; in fact, in many cases, that we’ll be more aggressive because, you know, we’re comparing to having no access to doctors at all. And, you know, so I think whether it’s India or Africa, there’ll be lessons that are globally valuable because we need medical intelligence. And, you know, thank god AI is going to provide a lot of that.  LEE: Well, on that optimistic note, I think that’s a good way to end. Bill, Seb, really appreciate all of this.   I think the most fundamental prediction we made in the book is that AI would actually find its way into the practice of medicine, and I think that that at least has come true, maybe in different ways than we expected, but it’s come true, and I think it’ll only accelerate from here. So thanks again, both of you.  [TRANSITION MUSIC]  GATES: Yeah. Thanks, you guys.  BUBECK: Thank you, Peter. Thanks, Bill.  LEE: I just always feel such a sense of privilege to have a chance to interact and actually work with people like Bill and Sébastien.    With Bill, I’m always amazed at how practically minded he is. He’s really thinking about the nuts and bolts of what AI might be able to do for people, and his thoughts about underserved parts of the world, the idea that we might actually be able to empower people with access to expert medical knowledge, I think is both inspiring and amazing.   And then, Seb, Sébastien Bubeck, he’s just absolutely a brilliant mind. He has a really firm grip on the deep mathematics of artificial intelligence and brings that to bear in his research and development work. And where that mathematics takes him isn’t just into the nuts and bolts of algorithms but into philosophical questions about the nature of intelligence.   One of the things that Sébastien brought up was the state of evaluation of AI systems. And indeed, he was fairly critical in our conversation. But of course, the world of AI research and development is just moving so fast, and indeed, since we recorded our conversation, OpenAI, in fact, released a new evaluation metric that is directly relevant to medical applications, and that is something called HealthBench. And Microsoft Research also released a new evaluation approach or process called ADeLe.   HealthBench and ADeLe are examples of new approaches to evaluating AI models that are less about testing their knowledge and ability to pass multiple-choice exams and instead are evaluation approaches designed to assess how well AI models are able to complete tasks that actually arise every day in typical healthcare or biomedical research settings. These are examples of really important good work that speak to how well AI models work in the real world of healthcare and biomedical research and how well they can collaborate with human beings in those settings.  You know, I asked Bill and Seb to make some predictions about the future. You know, my own answer, I expect that we’re going to be able to use AI to change how we diagnose patients, change how we decide treatment options.   If you’re a doctor or a nurse and you encounter a patient, you’ll ask questions, do a physical exam, you know, call out for labs just like you do today, but then you’ll be able to engage with AI based on all of that data and just ask, you know, based on all the other people who have gone through the same experience, who have similar data, how were they diagnosed? How were they treated? What were their outcomes? And what does that mean for the patient I have right now? Some people call it the “patients like me” paradigm. And I think that’s going to become real because of AI within our lifetimes. That idea of really grounding the delivery in healthcare and medical practice through data and intelligence, I actually now don’t see any barriers to that future becoming real.  [THEME MUSIC]  I’d like to extend another big thank you to Bill and Sébastien for their time. And to our listeners, as always, it’s a pleasure to have you along for the ride. I hope you’ll join us for our remaining conversations, as well as a second coauthor roundtable with Carey and Zak.   Until next time.   [MUSIC FADES]
    0 Comments 0 Shares 0 Reviews
  • Pixar Slate Reveal: What We Learned About Toy Story 5, Hoppers, And More

    Pixar has been delighting audiences with its house animation style and world-building for three decades, and the Disney-owned animation studio is showing no signs of slowing down. And unlike Andy, they haven’t aged out of playing with their toys. 
    At the Annecy’s International Animation Film Festival, Pixar dropped a series of announcements, teasers, and special previews of their upcoming slate, including the much-anticipated first-look at Toy Story 5. 

    Den of Geek attended a private screening, with remarks from Pixar’s Chief Creative Officer, Pete Docter, in early June ahead of the festival. During the presentation to the press, Docter hinted at the company putting its focus and energy to its theatrical slate, a notable change after recent releases like Dream Productions, set in the Inside Out universe, and the original Win or Lose debuted in early 2025. It’s a telling sign for Disney’s shifting approach to Disney+. The studio’s latest film, Elio, hit theaters on June 20th.
    “Our hope is that we can somehow tap into the things that people remember about the communal experience of seeing things together,” Docter said. “It’s different than sitting at home on your computer watching somethingwhen you sit with other human beings in the dark and watch the flickering light on the screen. There’s something kind of magic about that.” 

    Pixar is aiming to be back on a timeline of three films every two years, with Toy Story 5 and an original story titled Hoppers releasing in 2026, and another original, Gatto, hitting theaters in 2027. 
    Docter boldly stated that Pixar is “standing on one of the strongest slates we’ve ever had.” While bullish for a studio that has had an unprecedented run of success in the world of animated features, the early footage we saw leaves plenty of room for optimism.
    Is Pixar so back? Here’s what we learned from the presentation and footage… 
    Toy Story 5 – June 19, 2026 
    Woody, Buzz, Jesse and the gang will all be returning for the fifth feature film in one of Pixar’s most beloved franchises. Docter confirmed Tom Hanks, Tim Allen and Joan Cusack will reprise their respective roles.
    Written and directed by Andrew Stanton, who has worked on all of the films, and co-directed by McKenna Harris, Toy Story 5 catches up to our modern, tech-oriented world, and how that affects children’s interests. Bonnie, now eight, is given a brand new, shiny tablet, called a Lily Pad. The new tech allows Bonnie to stay connected and chat with all of her friends, slowly detaching her from her old toys. But just like all the other toys, Lily can talk, and she’s quite sneaky. Lily believes Bonnie needs to get rid of her old, childish toys completely. Feeling Bonnie slipping away, the toys call Woody for back up, but after not seeing Buzz for some time, the two go back to their old ways of constantly butting heads. 
    “With some films, you’ll struggle to find new things to talk about. And you know, this is. We still are finding new aspects of what it is to be a toy… There’s more of a spotlight on Jesse, so there’s that’s a whole nother facet to it as well. And she’s just such a rich, wonderful character to see on screen,” Docter says.

    Pixar screened the opening scene for press, which saw a fresh pallet of new Buzz Lightyear figures washed up in a shipping container on a remote island. Think Toy Story meets Cast Away as the Lightyears band together to concoct a way to get home, wherever that might be, in an unexpectedly gripping start to the fifth installment.

    Join our mailing list
    Get the best of Den of Geek delivered right to your inbox!

    HOPPERS – © 2025 Disney/Pixar. All Rights Reserved.
    Hoppers – March 6, 2026 
    Preceding Toy Story 5 and kicking off 2026 for Pixar will be an all-new story, Hoppers. 
    The film follows Mabel, a college student and nature enthusiast as she fights to save a beloved glade near her childhood home from a highway project that will bulldoze through it– brought forth by the greedy mayor voiced by Jon Hamm. With little support from those around her, Mabel enlists the help of “hoppers,” a clever group of scientists who’ve found a way to “hop” their minds into robots. When Mabel hops into the body of a beaver, she sets off to get other animals to return to the glade, hopefully halting construction. The animals take her to meet their rather conflict-avoidant leader, King George, and she soon learns that the animal world is a lot more complex than she had thought. 
    The footage screened saw Jon Hamm’s mayor abducted by beavers in a slapstick scene that corroborated Docter’s excitement for the project. Like Pixar’s highest highs, Hoppers appears to be charming and big-hearted, and it certainly won’t hurt merchandise sales at the Disney parks with the adorably designed animals in this film. Docter compared Hoppers to Mission Impossible meets Planet Earth. We’re locked in. 
    GATTO – © 2025 Disney/Pixar. All Rights Reserved.
    Gatto – Summer 2027 
    In maybe the most creatively intriguing announcement, a new film titled Gatto is in production from the team behind Luca. Gatto will employ the same classic Pixar animation-style, but with a painterly twist to match the artistic vibe of Venice. The art direction shown in short clips was stunning and unique spin on Pixar’s house style.
    The film is set in Venice, Italy, a destination popular for its stunning architecture and romantic ambience, that some only dream of visiting one day. It’s not so ideal, however, for Nero, the protagonist of the upcoming Pixar-original film, Gato. Nero is a black cat, who people turn the other way from because they fear he’s bad luck. With no other options, Nero turns to the seedier side of the stray cat scene in Venice, where he soon finds himself in hot water with Rocco, a cat mob boss. The heart of the film is Nero’s love for music, and his budding friendship with a street musician named Maya, who is also an outsider.
    #pixar #slate #reveal #what #learned
    Pixar Slate Reveal: What We Learned About Toy Story 5, Hoppers, And More
    Pixar has been delighting audiences with its house animation style and world-building for three decades, and the Disney-owned animation studio is showing no signs of slowing down. And unlike Andy, they haven’t aged out of playing with their toys.  At the Annecy’s International Animation Film Festival, Pixar dropped a series of announcements, teasers, and special previews of their upcoming slate, including the much-anticipated first-look at Toy Story 5.  Den of Geek attended a private screening, with remarks from Pixar’s Chief Creative Officer, Pete Docter, in early June ahead of the festival. During the presentation to the press, Docter hinted at the company putting its focus and energy to its theatrical slate, a notable change after recent releases like Dream Productions, set in the Inside Out universe, and the original Win or Lose debuted in early 2025. It’s a telling sign for Disney’s shifting approach to Disney+. The studio’s latest film, Elio, hit theaters on June 20th. “Our hope is that we can somehow tap into the things that people remember about the communal experience of seeing things together,” Docter said. “It’s different than sitting at home on your computer watching somethingwhen you sit with other human beings in the dark and watch the flickering light on the screen. There’s something kind of magic about that.”  Pixar is aiming to be back on a timeline of three films every two years, with Toy Story 5 and an original story titled Hoppers releasing in 2026, and another original, Gatto, hitting theaters in 2027.  Docter boldly stated that Pixar is “standing on one of the strongest slates we’ve ever had.” While bullish for a studio that has had an unprecedented run of success in the world of animated features, the early footage we saw leaves plenty of room for optimism. Is Pixar so back? Here’s what we learned from the presentation and footage…  Toy Story 5 – June 19, 2026  Woody, Buzz, Jesse and the gang will all be returning for the fifth feature film in one of Pixar’s most beloved franchises. Docter confirmed Tom Hanks, Tim Allen and Joan Cusack will reprise their respective roles. Written and directed by Andrew Stanton, who has worked on all of the films, and co-directed by McKenna Harris, Toy Story 5 catches up to our modern, tech-oriented world, and how that affects children’s interests. Bonnie, now eight, is given a brand new, shiny tablet, called a Lily Pad. The new tech allows Bonnie to stay connected and chat with all of her friends, slowly detaching her from her old toys. But just like all the other toys, Lily can talk, and she’s quite sneaky. Lily believes Bonnie needs to get rid of her old, childish toys completely. Feeling Bonnie slipping away, the toys call Woody for back up, but after not seeing Buzz for some time, the two go back to their old ways of constantly butting heads.  “With some films, you’ll struggle to find new things to talk about. And you know, this is. We still are finding new aspects of what it is to be a toy… There’s more of a spotlight on Jesse, so there’s that’s a whole nother facet to it as well. And she’s just such a rich, wonderful character to see on screen,” Docter says. Pixar screened the opening scene for press, which saw a fresh pallet of new Buzz Lightyear figures washed up in a shipping container on a remote island. Think Toy Story meets Cast Away as the Lightyears band together to concoct a way to get home, wherever that might be, in an unexpectedly gripping start to the fifth installment. Join our mailing list Get the best of Den of Geek delivered right to your inbox! HOPPERS – © 2025 Disney/Pixar. All Rights Reserved. Hoppers – March 6, 2026  Preceding Toy Story 5 and kicking off 2026 for Pixar will be an all-new story, Hoppers.  The film follows Mabel, a college student and nature enthusiast as she fights to save a beloved glade near her childhood home from a highway project that will bulldoze through it– brought forth by the greedy mayor voiced by Jon Hamm. With little support from those around her, Mabel enlists the help of “hoppers,” a clever group of scientists who’ve found a way to “hop” their minds into robots. When Mabel hops into the body of a beaver, she sets off to get other animals to return to the glade, hopefully halting construction. The animals take her to meet their rather conflict-avoidant leader, King George, and she soon learns that the animal world is a lot more complex than she had thought.  The footage screened saw Jon Hamm’s mayor abducted by beavers in a slapstick scene that corroborated Docter’s excitement for the project. Like Pixar’s highest highs, Hoppers appears to be charming and big-hearted, and it certainly won’t hurt merchandise sales at the Disney parks with the adorably designed animals in this film. Docter compared Hoppers to Mission Impossible meets Planet Earth. We’re locked in.  GATTO – © 2025 Disney/Pixar. All Rights Reserved. Gatto – Summer 2027  In maybe the most creatively intriguing announcement, a new film titled Gatto is in production from the team behind Luca. Gatto will employ the same classic Pixar animation-style, but with a painterly twist to match the artistic vibe of Venice. The art direction shown in short clips was stunning and unique spin on Pixar’s house style. The film is set in Venice, Italy, a destination popular for its stunning architecture and romantic ambience, that some only dream of visiting one day. It’s not so ideal, however, for Nero, the protagonist of the upcoming Pixar-original film, Gato. Nero is a black cat, who people turn the other way from because they fear he’s bad luck. With no other options, Nero turns to the seedier side of the stray cat scene in Venice, where he soon finds himself in hot water with Rocco, a cat mob boss. The heart of the film is Nero’s love for music, and his budding friendship with a street musician named Maya, who is also an outsider. #pixar #slate #reveal #what #learned
    WWW.DENOFGEEK.COM
    Pixar Slate Reveal: What We Learned About Toy Story 5, Hoppers, And More
    Pixar has been delighting audiences with its house animation style and world-building for three decades, and the Disney-owned animation studio is showing no signs of slowing down. And unlike Andy, they haven’t aged out of playing with their toys.  At the Annecy’s International Animation Film Festival, Pixar dropped a series of announcements, teasers, and special previews of their upcoming slate, including the much-anticipated first-look at Toy Story 5.  Den of Geek attended a private screening, with remarks from Pixar’s Chief Creative Officer, Pete Docter, in early June ahead of the festival. During the presentation to the press, Docter hinted at the company putting its focus and energy to its theatrical slate, a notable change after recent releases like Dream Productions, set in the Inside Out universe, and the original Win or Lose debuted in early 2025. It’s a telling sign for Disney’s shifting approach to Disney+. The studio’s latest film, Elio, hit theaters on June 20th. “Our hope is that we can somehow tap into the things that people remember about the communal experience of seeing things together,” Docter said. “It’s different than sitting at home on your computer watching something [compared to] when you sit with other human beings in the dark and watch the flickering light on the screen. There’s something kind of magic about that.”  Pixar is aiming to be back on a timeline of three films every two years, with Toy Story 5 and an original story titled Hoppers releasing in 2026, and another original, Gatto, hitting theaters in 2027.  Docter boldly stated that Pixar is “standing on one of the strongest slates we’ve ever had.” While bullish for a studio that has had an unprecedented run of success in the world of animated features, the early footage we saw leaves plenty of room for optimism. Is Pixar so back? Here’s what we learned from the presentation and footage…  Toy Story 5 – June 19, 2026  Woody, Buzz, Jesse and the gang will all be returning for the fifth feature film in one of Pixar’s most beloved franchises. Docter confirmed Tom Hanks, Tim Allen and Joan Cusack will reprise their respective roles. Written and directed by Andrew Stanton, who has worked on all of the films, and co-directed by McKenna Harris, Toy Story 5 catches up to our modern, tech-oriented world, and how that affects children’s interests. Bonnie, now eight, is given a brand new, shiny tablet, called a Lily Pad. The new tech allows Bonnie to stay connected and chat with all of her friends, slowly detaching her from her old toys. But just like all the other toys, Lily can talk, and she’s quite sneaky. Lily believes Bonnie needs to get rid of her old, childish toys completely. Feeling Bonnie slipping away, the toys call Woody for back up, but after not seeing Buzz for some time, the two go back to their old ways of constantly butting heads.  “With some films, you’ll struggle to find new things to talk about. And you know, this is [Toy Story 5]. We still are finding new aspects of what it is to be a toy… There’s more of a spotlight on Jesse, so there’s that’s a whole nother facet to it as well. And she’s just such a rich, wonderful character to see on screen,” Docter says. Pixar screened the opening scene for press, which saw a fresh pallet of new Buzz Lightyear figures washed up in a shipping container on a remote island. Think Toy Story meets Cast Away as the Lightyears band together to concoct a way to get home, wherever that might be, in an unexpectedly gripping start to the fifth installment. Join our mailing list Get the best of Den of Geek delivered right to your inbox! HOPPERS – © 2025 Disney/Pixar. All Rights Reserved. Hoppers – March 6, 2026  Preceding Toy Story 5 and kicking off 2026 for Pixar will be an all-new story, Hoppers.  The film follows Mabel (Piper Curda), a college student and nature enthusiast as she fights to save a beloved glade near her childhood home from a highway project that will bulldoze through it– brought forth by the greedy mayor voiced by Jon Hamm. With little support from those around her, Mabel enlists the help of “hoppers,” a clever group of scientists who’ve found a way to “hop” their minds into robots. When Mabel hops into the body of a beaver, she sets off to get other animals to return to the glade, hopefully halting construction. The animals take her to meet their rather conflict-avoidant leader, King George (Bobby Moynihan), and she soon learns that the animal world is a lot more complex than she had thought.  The footage screened saw Jon Hamm’s mayor abducted by beavers in a slapstick scene that corroborated Docter’s excitement for the project. Like Pixar’s highest highs, Hoppers appears to be charming and big-hearted, and it certainly won’t hurt merchandise sales at the Disney parks with the adorably designed animals in this film. Docter compared Hoppers to Mission Impossible meets Planet Earth. We’re locked in.  GATTO – © 2025 Disney/Pixar. All Rights Reserved. Gatto – Summer 2027  In maybe the most creatively intriguing announcement, a new film titled Gatto is in production from the team behind Luca. Gatto will employ the same classic Pixar animation-style, but with a painterly twist to match the artistic vibe of Venice. The art direction shown in short clips was stunning and unique spin on Pixar’s house style. The film is set in Venice, Italy, a destination popular for its stunning architecture and romantic ambience, that some only dream of visiting one day. It’s not so ideal, however, for Nero, the protagonist of the upcoming Pixar-original film, Gato. Nero is a black cat, who people turn the other way from because they fear he’s bad luck. With no other options, Nero turns to the seedier side of the stray cat scene in Venice, where he soon finds himself in hot water with Rocco, a cat mob boss. The heart of the film is Nero’s love for music, and his budding friendship with a street musician named Maya, who is also an outsider.
    0 Comments 0 Shares 0 Reviews
  • Looking Back at Two Classics: ILM Deploys the Fleet in ‘Star Trek: First Contact’ and ‘Rogue One: A Star Wars Story’

    Guided by visual effects supervisor John Knoll, ILM embraced continually evolving methodologies to craft breathtaking visual effects for the iconic space battles in First Contact and Rogue One.
    By Jay Stobie
    Visual effects supervisor John Knollconfers with modelmakers Kim Smith and John Goodson with the miniature of the U.S.S. Enterprise-E during production of Star Trek: First Contact.
    Bolstered by visual effects from Industrial Light & Magic, Star Trek: First Contactand Rogue One: A Star Wars Storypropelled their respective franchises to new heights. While Star Trek Generationswelcomed Captain Jean-Luc Picard’screw to the big screen, First Contact stood as the first Star Trek feature that did not focus on its original captain, the legendary James T. Kirk. Similarly, though Rogue One immediately preceded the events of Star Wars: A New Hope, it was set apart from the episodic Star Wars films and launched an era of storytelling outside of the main Skywalker saga that has gone on to include Solo: A Star Wars Story, The Mandalorian, Andor, Ahsoka, The Acolyte, and more.
    The two films also shared a key ILM contributor, John Knoll, who served as visual effects supervisor on both projects, as well as an executive producer on Rogue One. Currently, ILM’s executive creative director and senior visual effects supervisor, Knoll – who also conceived the initial framework for Rogue One’s story – guided ILM as it brought its talents to bear on these sci-fi and fantasy epics. The work involved crafting two spectacular starship-packed space clashes – First Contact’s Battle of Sector 001 and Rogue One’s Battle of Scarif. Although these iconic installments were released roughly two decades apart, they represent a captivating case study of how ILM’s approach to visual effects has evolved over time. With this in mind, let’s examine the films’ unforgettable space battles through the lens of fascinating in-universe parallels and the ILM-produced fleets that face off near Earth and Scarif.
    A final frame from the Battle of Scarif in Rogue One: A Star Wars Story.
    A Context for Conflict
    In First Contact, the United Federation of Planets – a 200-year-old interstellar government consisting of more than 150 member worlds – braces itself for an invasion by the Borg – an overwhelmingly powerful collective composed of cybernetic beings who devastate entire planets by assimilating their biological populations and technological innovations. The Borg only send a single vessel, a massive cube containing thousands of hive-minded drones and their queen, pushing the Federation’s Starfleet defenders to Earth’s doorstep. Conversely, in Rogue One, the Rebel Alliance – a fledgling coalition of freedom fighters – seeks to undermine and overthrow the stalwart Galactic Empire – a totalitarian regime preparing to tighten its grip on the galaxy by revealing a horrifying superweapon. A rebel team infiltrates a top-secret vault on Scarif in a bid to steal plans to that battle station, the dreaded Death Star, with hopes of exploiting a vulnerability in its design.
    On the surface, the situations could not seem to be more disparate, particularly in terms of the Federation’s well-established prestige and the Rebel Alliance’s haphazardly organized factions. Yet, upon closer inspection, the spaceborne conflicts at Earth and Scarif are linked by a vital commonality. The threat posed by the Borg is well-known to the Federation, but the sudden intrusion upon their space takes its defenses by surprise. Starfleet assembles any vessel within range – including antiquated Oberth-class science ships – to intercept the Borg cube in the Typhon Sector, only to be forced back to Earth on the edge of defeat. The unsanctioned mission to Scarif with Jyn Ersoand Cassian Andorand the sudden need to take down the planet’s shield gate propels the Rebel Alliance fleet into rushing to their rescue with everything from their flagship Profundity to GR-75 medium transports. Whether Federation or Rebel Alliance, these fleets gather in last-ditch efforts to oppose enemies who would embrace their eradication – the Battles of Sector 001 and Scarif are fights for survival.
    From Physical to Digital
    By the time Jonathan Frakes was selected to direct First Contact, Star Trek’s reliance on constructing traditional physical modelsfor its features was gradually giving way to innovative computer graphicsmodels, resulting in the film’s use of both techniques. “If one of the ships was to be seen full-screen and at length,” associate visual effects supervisor George Murphy told Cinefex’s Kevin H. Martin, “we knew it would be done as a stage model. Ships that would be doing a lot of elaborate maneuvers in space battle scenes would be created digitally.” In fact, physical and CG versions of the U.S.S. Enterprise-E appear in the film, with the latter being harnessed in shots involving the vessel’s entry into a temporal vortex at the conclusion of the Battle of Sector 001.
    Despite the technological leaps that ILM pioneered in the decades between First Contact and Rogue One, they considered filming physical miniatures for certain ship-related shots in the latter film. ILM considered filming physical miniatures for certain ship-related shots in Rogue One. The feature’s fleets were ultimately created digitally to allow for changes throughout post-production. “If it’s a photographed miniature element, it’s not possible to go back and make adjustments. So it’s the additional flexibility that comes with the computer graphics models that’s very attractive to many people,” John Knoll relayed to writer Jon Witmer at American Cinematographer’s TheASC.com.
    However, Knoll aimed to develop computer graphics that retained the same high-quality details as their physical counterparts, leading ILM to employ a modern approach to a time-honored modelmaking tactic. “I also wanted to emulate the kit-bashing aesthetic that had been part of Star Wars from the very beginning, where a lot of mechanical detail had been added onto the ships by using little pieces from plastic model kits,” explained Knoll in his chat with TheASC.com. For Rogue One, ILM replicated the process by obtaining such kits, scanning their parts, building a computer graphics library, and applying the CG parts to digitally modeled ships. “I’m very happy to say it was super-successful,” concluded Knoll. “I think a lot of our digital models look like they are motion-control models.”
    John Knollconfers with Kim Smith and John Goodson with the miniature of the U.S.S. Enterprise-E during production of Star Trek: First Contact.
    Legendary Lineages
    In First Contact, Captain Picard commanded a brand-new vessel, the Sovereign-class U.S.S. Enterprise-E, continuing the celebrated starship’s legacy in terms of its famous name and design aesthetic. Designed by John Eaves and developed into blueprints by Rick Sternbach, the Enterprise-E was built into a 10-foot physical model by ILM model project supervisor John Goodson and his shop’s talented team. ILM infused the ship with extraordinary detail, including viewports equipped with backlit set images from the craft’s predecessor, the U.S.S. Enterprise-D. For the vessel’s larger windows, namely those associated with the observation lounge and arboretum, ILM took a painstakingly practical approach to match the interiors shown with the real-world set pieces. “We filled that area of the model with tiny, micro-scale furniture,” Goodson informed Cinefex, “including tables and chairs.”
    Rogue One’s rebel team initially traversed the galaxy in a U-wing transport/gunship, which, much like the Enterprise-E, was a unique vessel that nonetheless channeled a certain degree of inspiration from a classic design. Lucasfilm’s Doug Chiang, a co-production designer for Rogue One, referred to the U-wing as the film’s “Huey helicopter version of an X-wing” in the Designing Rogue One bonus featurette on Disney+ before revealing that, “Towards the end of the design cycle, we actually decided that maybe we should put in more X-wing features. And so we took the X-wing engines and literally mounted them onto the configuration that we had going.” Modeled by ILM digital artist Colie Wertz, the U-wing’s final computer graphics design subtly incorporated these X-wing influences to give the transport a distinctive feel without making the craft seem out of place within the rebel fleet.
    While ILM’s work on the Enterprise-E’s viewports offered a compelling view toward the ship’s interior, a breakthrough LED setup for Rogue One permitted ILM to obtain realistic lighting on actors as they looked out from their ships and into the space around them. “All of our major spaceship cockpit scenes were done that way, with the gimbal in this giant horseshoe of LED panels we got fromVER, and we prepared graphics that went on the screens,” John Knoll shared with American Cinematographer’s Benjamin B and Jon D. Witmer. Furthermore, in Disney+’s Rogue One: Digital Storytelling bonus featurette, visual effects producer Janet Lewin noted, “For the actors, I think, in the space battle cockpits, for them to be able to see what was happening in the battle brought a higher level of accuracy to their performance.”
    The U.S.S. Enterprise-E in Star Trek: First Contact.
    Familiar Foes
    To transport First Contact’s Borg invaders, John Goodson’s team at ILM resurrected the Borg cube design previously seen in Star Trek: The Next Generationand Star Trek: Deep Space Nine, creating a nearly three-foot physical model to replace the one from the series. Art consultant and ILM veteran Bill George proposed that the cube’s seemingly straightforward layout be augmented with a complex network of photo-etched brass, a suggestion which produced a jagged surface and offered a visual that was both intricate and menacing. ILM also developed a two-foot motion-control model for a Borg sphere, a brand-new auxiliary vessel that emerged from the cube. “We vacuformed about 15 different patterns that conformed to this spherical curve and covered those with a lot of molded and cast pieces. Then we added tons of acid-etched brass over it, just like we had on the cube,” Goodson outlined to Cinefex’s Kevin H. Martin.
    As for Rogue One’s villainous fleet, reproducing the original trilogy’s Death Star and Imperial Star Destroyers centered upon translating physical models into digital assets. Although ILM no longer possessed A New Hope’s three-foot Death Star shooting model, John Knoll recreated the station’s surface paneling by gathering archival images, and as he spelled out to writer Joe Fordham in Cinefex, “I pieced all the images together. I unwrapped them into texture space and projected them onto a sphere with a trench. By doing that with enough pictures, I got pretty complete coverage of the original model, and that became a template upon which to redraw very high-resolution texture maps. Every panel, every vertical striped line, I matched from a photograph. It was as accurate as it was possible to be as a reproduction of the original model.”
    Knoll’s investigative eye continued to pay dividends when analyzing the three-foot and eight-foot Star Destroyer motion-control models, which had been built for A New Hope and Star Wars: The Empire Strikes Back, respectively. “Our general mantra was, ‘Match your memory of it more than the reality,’ because sometimes you go look at the actual prop in the archive building or you look back at the actual shot from the movie, and you go, ‘Oh, I remember it being a little better than that,’” Knoll conveyed to TheASC.com. This philosophy motivated ILM to combine elements from those two physical models into a single digital design. “Generally, we copied the three-footer for details like the superstructure on the top of the bridge, but then we copied the internal lighting plan from the eight-footer,” Knoll explained. “And then the upper surface of the three-footer was relatively undetailed because there were no shots that saw it closely, so we took a lot of the high-detail upper surface from the eight-footer. So it’s this amalgam of the two models, but the goal was to try to make it look like you remember it from A New Hope.”
    A final frame from Rogue One: A Star Wars Story.
    Forming Up the Fleets
    In addition to the U.S.S. Enterprise-E, the Battle of Sector 001 debuted numerous vessels representing four new Starfleet ship classes – the Akira, Steamrunner, Saber, and Norway – all designed by ILM visual effects art director Alex Jaeger. “Since we figured a lot of the background action in the space battle would be done with computer graphics ships that needed to be built from scratch anyway, I realized that there was no reason not to do some new designs,” John Knoll told American Cinematographer writer Ron Magid. Used in previous Star Trek projects, older physical models for the Oberth and Nebula classes were mixed into the fleet for good measure, though the vast majority of the armada originated as computer graphics.
    Over at Scarif, ILM portrayed the Rebel Alliance forces with computer graphics models of fresh designs, live-action versions of Star Wars Rebels’ VCX-100 light freighter Ghost and Hammerhead corvettes, and Star Wars staples. These ships face off against two Imperial Star Destroyers and squadrons of TIE fighters, and – upon their late arrival to the battle – Darth Vader’s Star Destroyer and the Death Star. The Tantive IV, a CR90 corvette more popularly referred to as a blockade runner, made its own special cameo at the tail end of the fight. As Princess Leia Organa’spersonal ship, the Tantive IV received the Death Star plans and fled the scene, destined to be captured by Vader’s Star Destroyer at the beginning of A New Hope. And, while we’re on the subject of intricate starship maneuvers and space-based choreography…
    Although the First Contact team could plan visual effects shots with animated storyboards, ILM supplied Gareth Edwards with a next-level virtual viewfinder that allowed the director to select his shots by immersing himself among Rogue One’s ships in real time. “What we wanted to do is give Gareth the opportunity to shoot his space battles and other all-digital scenes the same way he shoots his live-action. Then he could go in with this sort of virtual viewfinder and view the space battle going on, and figure out what the best angle was to shoot those ships from,” senior animation supervisor Hal Hickel described in the Rogue One: Digital Storytelling featurette. Hickel divulged that the sequence involving the dish array docking with the Death Star was an example of the “spontaneous discovery of great angles,” as the scene was never storyboarded or previsualized.
    Visual effects supervisor John Knoll with director Gareth Edwards during production of Rogue One: A Star Wars Story.
    Tough Little Ships
    The Federation and Rebel Alliance each deployed “tough little ships”in their respective conflicts, namely the U.S.S. Defiant from Deep Space Nine and the Tantive IV from A New Hope. VisionArt had already built a CG Defiant for the Deep Space Nine series, but ILM upgraded the model with images gathered from the ship’s three-foot physical model. A similar tactic was taken to bring the Tantive IV into the digital realm for Rogue One. “This was the Blockade Runner. This was the most accurate 1:1 reproduction we could possibly have made,” model supervisor Russell Paul declared to Cinefex’s Joe Fordham. “We did an extensive photo reference shoot and photogrammetry re-creation of the miniature. From there, we built it out as accurately as possible.” Speaking of sturdy ships, if you look very closely, you can spot a model of the Millennium Falcon flashing across the background as the U.S.S. Defiant makes an attack run on the Borg cube at the Battle of Sector 001!
    Exploration and Hope
    The in-universe ramifications that materialize from the Battles of Sector 001 and Scarif are monumental. The destruction of the Borg cube compels the Borg Queen to travel back in time in an attempt to vanquish Earth before the Federation can even be formed, but Captain Picard and the Enterprise-E foil the plot and end up helping their 21st century ancestors make “first contact” with another species, the logic-revering Vulcans. The post-Scarif benefits take longer to play out for the Rebel Alliance, but the theft of the Death Star plans eventually leads to the superweapon’s destruction. The Galactic Civil War is far from over, but Scarif is a significant step in the Alliance’s effort to overthrow the Empire.
    The visual effects ILM provided for First Contact and Rogue One contributed significantly to the critical and commercial acclaim both pictures enjoyed, a victory reflecting the relentless dedication, tireless work ethic, and innovative spirit embodied by visual effects supervisor John Knoll and ILM’s entire staff. While being interviewed for The Making of Star Trek: First Contact, actor Patrick Stewart praised ILM’s invaluable influence, emphasizing, “ILM was with us, on this movie, almost every day on set. There is so much that they are involved in.” And, regardless of your personal preferences – phasers or lasers, photon torpedoes or proton torpedoes, warp speed or hyperspace – perhaps Industrial Light & Magic’s ability to infuse excitement into both franchises demonstrates that Star Trek and Star Wars encompass themes that are not competitive, but compatible. After all, what goes together better than exploration and hope?

    Jay Stobieis a writer, author, and consultant who has contributed articles to ILM.com, Skysound.com, Star Wars Insider, StarWars.com, Star Trek Explorer, Star Trek Magazine, and StarTrek.com. Jay loves sci-fi, fantasy, and film, and you can learn more about him by visiting JayStobie.com or finding him on Twitter, Instagram, and other social media platforms at @StobiesGalaxy.
    #looking #back #two #classics #ilm
    Looking Back at Two Classics: ILM Deploys the Fleet in ‘Star Trek: First Contact’ and ‘Rogue One: A Star Wars Story’
    Guided by visual effects supervisor John Knoll, ILM embraced continually evolving methodologies to craft breathtaking visual effects for the iconic space battles in First Contact and Rogue One. By Jay Stobie Visual effects supervisor John Knollconfers with modelmakers Kim Smith and John Goodson with the miniature of the U.S.S. Enterprise-E during production of Star Trek: First Contact. Bolstered by visual effects from Industrial Light & Magic, Star Trek: First Contactand Rogue One: A Star Wars Storypropelled their respective franchises to new heights. While Star Trek Generationswelcomed Captain Jean-Luc Picard’screw to the big screen, First Contact stood as the first Star Trek feature that did not focus on its original captain, the legendary James T. Kirk. Similarly, though Rogue One immediately preceded the events of Star Wars: A New Hope, it was set apart from the episodic Star Wars films and launched an era of storytelling outside of the main Skywalker saga that has gone on to include Solo: A Star Wars Story, The Mandalorian, Andor, Ahsoka, The Acolyte, and more. The two films also shared a key ILM contributor, John Knoll, who served as visual effects supervisor on both projects, as well as an executive producer on Rogue One. Currently, ILM’s executive creative director and senior visual effects supervisor, Knoll – who also conceived the initial framework for Rogue One’s story – guided ILM as it brought its talents to bear on these sci-fi and fantasy epics. The work involved crafting two spectacular starship-packed space clashes – First Contact’s Battle of Sector 001 and Rogue One’s Battle of Scarif. Although these iconic installments were released roughly two decades apart, they represent a captivating case study of how ILM’s approach to visual effects has evolved over time. With this in mind, let’s examine the films’ unforgettable space battles through the lens of fascinating in-universe parallels and the ILM-produced fleets that face off near Earth and Scarif. A final frame from the Battle of Scarif in Rogue One: A Star Wars Story. A Context for Conflict In First Contact, the United Federation of Planets – a 200-year-old interstellar government consisting of more than 150 member worlds – braces itself for an invasion by the Borg – an overwhelmingly powerful collective composed of cybernetic beings who devastate entire planets by assimilating their biological populations and technological innovations. The Borg only send a single vessel, a massive cube containing thousands of hive-minded drones and their queen, pushing the Federation’s Starfleet defenders to Earth’s doorstep. Conversely, in Rogue One, the Rebel Alliance – a fledgling coalition of freedom fighters – seeks to undermine and overthrow the stalwart Galactic Empire – a totalitarian regime preparing to tighten its grip on the galaxy by revealing a horrifying superweapon. A rebel team infiltrates a top-secret vault on Scarif in a bid to steal plans to that battle station, the dreaded Death Star, with hopes of exploiting a vulnerability in its design. On the surface, the situations could not seem to be more disparate, particularly in terms of the Federation’s well-established prestige and the Rebel Alliance’s haphazardly organized factions. Yet, upon closer inspection, the spaceborne conflicts at Earth and Scarif are linked by a vital commonality. The threat posed by the Borg is well-known to the Federation, but the sudden intrusion upon their space takes its defenses by surprise. Starfleet assembles any vessel within range – including antiquated Oberth-class science ships – to intercept the Borg cube in the Typhon Sector, only to be forced back to Earth on the edge of defeat. The unsanctioned mission to Scarif with Jyn Ersoand Cassian Andorand the sudden need to take down the planet’s shield gate propels the Rebel Alliance fleet into rushing to their rescue with everything from their flagship Profundity to GR-75 medium transports. Whether Federation or Rebel Alliance, these fleets gather in last-ditch efforts to oppose enemies who would embrace their eradication – the Battles of Sector 001 and Scarif are fights for survival. From Physical to Digital By the time Jonathan Frakes was selected to direct First Contact, Star Trek’s reliance on constructing traditional physical modelsfor its features was gradually giving way to innovative computer graphicsmodels, resulting in the film’s use of both techniques. “If one of the ships was to be seen full-screen and at length,” associate visual effects supervisor George Murphy told Cinefex’s Kevin H. Martin, “we knew it would be done as a stage model. Ships that would be doing a lot of elaborate maneuvers in space battle scenes would be created digitally.” In fact, physical and CG versions of the U.S.S. Enterprise-E appear in the film, with the latter being harnessed in shots involving the vessel’s entry into a temporal vortex at the conclusion of the Battle of Sector 001. Despite the technological leaps that ILM pioneered in the decades between First Contact and Rogue One, they considered filming physical miniatures for certain ship-related shots in the latter film. ILM considered filming physical miniatures for certain ship-related shots in Rogue One. The feature’s fleets were ultimately created digitally to allow for changes throughout post-production. “If it’s a photographed miniature element, it’s not possible to go back and make adjustments. So it’s the additional flexibility that comes with the computer graphics models that’s very attractive to many people,” John Knoll relayed to writer Jon Witmer at American Cinematographer’s TheASC.com. However, Knoll aimed to develop computer graphics that retained the same high-quality details as their physical counterparts, leading ILM to employ a modern approach to a time-honored modelmaking tactic. “I also wanted to emulate the kit-bashing aesthetic that had been part of Star Wars from the very beginning, where a lot of mechanical detail had been added onto the ships by using little pieces from plastic model kits,” explained Knoll in his chat with TheASC.com. For Rogue One, ILM replicated the process by obtaining such kits, scanning their parts, building a computer graphics library, and applying the CG parts to digitally modeled ships. “I’m very happy to say it was super-successful,” concluded Knoll. “I think a lot of our digital models look like they are motion-control models.” John Knollconfers with Kim Smith and John Goodson with the miniature of the U.S.S. Enterprise-E during production of Star Trek: First Contact. Legendary Lineages In First Contact, Captain Picard commanded a brand-new vessel, the Sovereign-class U.S.S. Enterprise-E, continuing the celebrated starship’s legacy in terms of its famous name and design aesthetic. Designed by John Eaves and developed into blueprints by Rick Sternbach, the Enterprise-E was built into a 10-foot physical model by ILM model project supervisor John Goodson and his shop’s talented team. ILM infused the ship with extraordinary detail, including viewports equipped with backlit set images from the craft’s predecessor, the U.S.S. Enterprise-D. For the vessel’s larger windows, namely those associated with the observation lounge and arboretum, ILM took a painstakingly practical approach to match the interiors shown with the real-world set pieces. “We filled that area of the model with tiny, micro-scale furniture,” Goodson informed Cinefex, “including tables and chairs.” Rogue One’s rebel team initially traversed the galaxy in a U-wing transport/gunship, which, much like the Enterprise-E, was a unique vessel that nonetheless channeled a certain degree of inspiration from a classic design. Lucasfilm’s Doug Chiang, a co-production designer for Rogue One, referred to the U-wing as the film’s “Huey helicopter version of an X-wing” in the Designing Rogue One bonus featurette on Disney+ before revealing that, “Towards the end of the design cycle, we actually decided that maybe we should put in more X-wing features. And so we took the X-wing engines and literally mounted them onto the configuration that we had going.” Modeled by ILM digital artist Colie Wertz, the U-wing’s final computer graphics design subtly incorporated these X-wing influences to give the transport a distinctive feel without making the craft seem out of place within the rebel fleet. While ILM’s work on the Enterprise-E’s viewports offered a compelling view toward the ship’s interior, a breakthrough LED setup for Rogue One permitted ILM to obtain realistic lighting on actors as they looked out from their ships and into the space around them. “All of our major spaceship cockpit scenes were done that way, with the gimbal in this giant horseshoe of LED panels we got fromVER, and we prepared graphics that went on the screens,” John Knoll shared with American Cinematographer’s Benjamin B and Jon D. Witmer. Furthermore, in Disney+’s Rogue One: Digital Storytelling bonus featurette, visual effects producer Janet Lewin noted, “For the actors, I think, in the space battle cockpits, for them to be able to see what was happening in the battle brought a higher level of accuracy to their performance.” The U.S.S. Enterprise-E in Star Trek: First Contact. Familiar Foes To transport First Contact’s Borg invaders, John Goodson’s team at ILM resurrected the Borg cube design previously seen in Star Trek: The Next Generationand Star Trek: Deep Space Nine, creating a nearly three-foot physical model to replace the one from the series. Art consultant and ILM veteran Bill George proposed that the cube’s seemingly straightforward layout be augmented with a complex network of photo-etched brass, a suggestion which produced a jagged surface and offered a visual that was both intricate and menacing. ILM also developed a two-foot motion-control model for a Borg sphere, a brand-new auxiliary vessel that emerged from the cube. “We vacuformed about 15 different patterns that conformed to this spherical curve and covered those with a lot of molded and cast pieces. Then we added tons of acid-etched brass over it, just like we had on the cube,” Goodson outlined to Cinefex’s Kevin H. Martin. As for Rogue One’s villainous fleet, reproducing the original trilogy’s Death Star and Imperial Star Destroyers centered upon translating physical models into digital assets. Although ILM no longer possessed A New Hope’s three-foot Death Star shooting model, John Knoll recreated the station’s surface paneling by gathering archival images, and as he spelled out to writer Joe Fordham in Cinefex, “I pieced all the images together. I unwrapped them into texture space and projected them onto a sphere with a trench. By doing that with enough pictures, I got pretty complete coverage of the original model, and that became a template upon which to redraw very high-resolution texture maps. Every panel, every vertical striped line, I matched from a photograph. It was as accurate as it was possible to be as a reproduction of the original model.” Knoll’s investigative eye continued to pay dividends when analyzing the three-foot and eight-foot Star Destroyer motion-control models, which had been built for A New Hope and Star Wars: The Empire Strikes Back, respectively. “Our general mantra was, ‘Match your memory of it more than the reality,’ because sometimes you go look at the actual prop in the archive building or you look back at the actual shot from the movie, and you go, ‘Oh, I remember it being a little better than that,’” Knoll conveyed to TheASC.com. This philosophy motivated ILM to combine elements from those two physical models into a single digital design. “Generally, we copied the three-footer for details like the superstructure on the top of the bridge, but then we copied the internal lighting plan from the eight-footer,” Knoll explained. “And then the upper surface of the three-footer was relatively undetailed because there were no shots that saw it closely, so we took a lot of the high-detail upper surface from the eight-footer. So it’s this amalgam of the two models, but the goal was to try to make it look like you remember it from A New Hope.” A final frame from Rogue One: A Star Wars Story. Forming Up the Fleets In addition to the U.S.S. Enterprise-E, the Battle of Sector 001 debuted numerous vessels representing four new Starfleet ship classes – the Akira, Steamrunner, Saber, and Norway – all designed by ILM visual effects art director Alex Jaeger. “Since we figured a lot of the background action in the space battle would be done with computer graphics ships that needed to be built from scratch anyway, I realized that there was no reason not to do some new designs,” John Knoll told American Cinematographer writer Ron Magid. Used in previous Star Trek projects, older physical models for the Oberth and Nebula classes were mixed into the fleet for good measure, though the vast majority of the armada originated as computer graphics. Over at Scarif, ILM portrayed the Rebel Alliance forces with computer graphics models of fresh designs, live-action versions of Star Wars Rebels’ VCX-100 light freighter Ghost and Hammerhead corvettes, and Star Wars staples. These ships face off against two Imperial Star Destroyers and squadrons of TIE fighters, and – upon their late arrival to the battle – Darth Vader’s Star Destroyer and the Death Star. The Tantive IV, a CR90 corvette more popularly referred to as a blockade runner, made its own special cameo at the tail end of the fight. As Princess Leia Organa’spersonal ship, the Tantive IV received the Death Star plans and fled the scene, destined to be captured by Vader’s Star Destroyer at the beginning of A New Hope. And, while we’re on the subject of intricate starship maneuvers and space-based choreography… Although the First Contact team could plan visual effects shots with animated storyboards, ILM supplied Gareth Edwards with a next-level virtual viewfinder that allowed the director to select his shots by immersing himself among Rogue One’s ships in real time. “What we wanted to do is give Gareth the opportunity to shoot his space battles and other all-digital scenes the same way he shoots his live-action. Then he could go in with this sort of virtual viewfinder and view the space battle going on, and figure out what the best angle was to shoot those ships from,” senior animation supervisor Hal Hickel described in the Rogue One: Digital Storytelling featurette. Hickel divulged that the sequence involving the dish array docking with the Death Star was an example of the “spontaneous discovery of great angles,” as the scene was never storyboarded or previsualized. Visual effects supervisor John Knoll with director Gareth Edwards during production of Rogue One: A Star Wars Story. Tough Little Ships The Federation and Rebel Alliance each deployed “tough little ships”in their respective conflicts, namely the U.S.S. Defiant from Deep Space Nine and the Tantive IV from A New Hope. VisionArt had already built a CG Defiant for the Deep Space Nine series, but ILM upgraded the model with images gathered from the ship’s three-foot physical model. A similar tactic was taken to bring the Tantive IV into the digital realm for Rogue One. “This was the Blockade Runner. This was the most accurate 1:1 reproduction we could possibly have made,” model supervisor Russell Paul declared to Cinefex’s Joe Fordham. “We did an extensive photo reference shoot and photogrammetry re-creation of the miniature. From there, we built it out as accurately as possible.” Speaking of sturdy ships, if you look very closely, you can spot a model of the Millennium Falcon flashing across the background as the U.S.S. Defiant makes an attack run on the Borg cube at the Battle of Sector 001! Exploration and Hope The in-universe ramifications that materialize from the Battles of Sector 001 and Scarif are monumental. The destruction of the Borg cube compels the Borg Queen to travel back in time in an attempt to vanquish Earth before the Federation can even be formed, but Captain Picard and the Enterprise-E foil the plot and end up helping their 21st century ancestors make “first contact” with another species, the logic-revering Vulcans. The post-Scarif benefits take longer to play out for the Rebel Alliance, but the theft of the Death Star plans eventually leads to the superweapon’s destruction. The Galactic Civil War is far from over, but Scarif is a significant step in the Alliance’s effort to overthrow the Empire. The visual effects ILM provided for First Contact and Rogue One contributed significantly to the critical and commercial acclaim both pictures enjoyed, a victory reflecting the relentless dedication, tireless work ethic, and innovative spirit embodied by visual effects supervisor John Knoll and ILM’s entire staff. While being interviewed for The Making of Star Trek: First Contact, actor Patrick Stewart praised ILM’s invaluable influence, emphasizing, “ILM was with us, on this movie, almost every day on set. There is so much that they are involved in.” And, regardless of your personal preferences – phasers or lasers, photon torpedoes or proton torpedoes, warp speed or hyperspace – perhaps Industrial Light & Magic’s ability to infuse excitement into both franchises demonstrates that Star Trek and Star Wars encompass themes that are not competitive, but compatible. After all, what goes together better than exploration and hope? – Jay Stobieis a writer, author, and consultant who has contributed articles to ILM.com, Skysound.com, Star Wars Insider, StarWars.com, Star Trek Explorer, Star Trek Magazine, and StarTrek.com. Jay loves sci-fi, fantasy, and film, and you can learn more about him by visiting JayStobie.com or finding him on Twitter, Instagram, and other social media platforms at @StobiesGalaxy. #looking #back #two #classics #ilm
    WWW.ILM.COM
    Looking Back at Two Classics: ILM Deploys the Fleet in ‘Star Trek: First Contact’ and ‘Rogue One: A Star Wars Story’
    Guided by visual effects supervisor John Knoll, ILM embraced continually evolving methodologies to craft breathtaking visual effects for the iconic space battles in First Contact and Rogue One. By Jay Stobie Visual effects supervisor John Knoll (right) confers with modelmakers Kim Smith and John Goodson with the miniature of the U.S.S. Enterprise-E during production of Star Trek: First Contact (Credit: ILM). Bolstered by visual effects from Industrial Light & Magic, Star Trek: First Contact (1996) and Rogue One: A Star Wars Story (2016) propelled their respective franchises to new heights. While Star Trek Generations (1994) welcomed Captain Jean-Luc Picard’s (Patrick Stewart) crew to the big screen, First Contact stood as the first Star Trek feature that did not focus on its original captain, the legendary James T. Kirk (William Shatner). Similarly, though Rogue One immediately preceded the events of Star Wars: A New Hope (1977), it was set apart from the episodic Star Wars films and launched an era of storytelling outside of the main Skywalker saga that has gone on to include Solo: A Star Wars Story (2018), The Mandalorian (2019-23), Andor (2022-25), Ahsoka (2023), The Acolyte (2024), and more. The two films also shared a key ILM contributor, John Knoll, who served as visual effects supervisor on both projects, as well as an executive producer on Rogue One. Currently, ILM’s executive creative director and senior visual effects supervisor, Knoll – who also conceived the initial framework for Rogue One’s story – guided ILM as it brought its talents to bear on these sci-fi and fantasy epics. The work involved crafting two spectacular starship-packed space clashes – First Contact’s Battle of Sector 001 and Rogue One’s Battle of Scarif. Although these iconic installments were released roughly two decades apart, they represent a captivating case study of how ILM’s approach to visual effects has evolved over time. With this in mind, let’s examine the films’ unforgettable space battles through the lens of fascinating in-universe parallels and the ILM-produced fleets that face off near Earth and Scarif. A final frame from the Battle of Scarif in Rogue One: A Star Wars Story (Credit: ILM & Lucasfilm). A Context for Conflict In First Contact, the United Federation of Planets – a 200-year-old interstellar government consisting of more than 150 member worlds – braces itself for an invasion by the Borg – an overwhelmingly powerful collective composed of cybernetic beings who devastate entire planets by assimilating their biological populations and technological innovations. The Borg only send a single vessel, a massive cube containing thousands of hive-minded drones and their queen, pushing the Federation’s Starfleet defenders to Earth’s doorstep. Conversely, in Rogue One, the Rebel Alliance – a fledgling coalition of freedom fighters – seeks to undermine and overthrow the stalwart Galactic Empire – a totalitarian regime preparing to tighten its grip on the galaxy by revealing a horrifying superweapon. A rebel team infiltrates a top-secret vault on Scarif in a bid to steal plans to that battle station, the dreaded Death Star, with hopes of exploiting a vulnerability in its design. On the surface, the situations could not seem to be more disparate, particularly in terms of the Federation’s well-established prestige and the Rebel Alliance’s haphazardly organized factions. Yet, upon closer inspection, the spaceborne conflicts at Earth and Scarif are linked by a vital commonality. The threat posed by the Borg is well-known to the Federation, but the sudden intrusion upon their space takes its defenses by surprise. Starfleet assembles any vessel within range – including antiquated Oberth-class science ships – to intercept the Borg cube in the Typhon Sector, only to be forced back to Earth on the edge of defeat. The unsanctioned mission to Scarif with Jyn Erso (Felicity Jones) and Cassian Andor (Diego Luna) and the sudden need to take down the planet’s shield gate propels the Rebel Alliance fleet into rushing to their rescue with everything from their flagship Profundity to GR-75 medium transports. Whether Federation or Rebel Alliance, these fleets gather in last-ditch efforts to oppose enemies who would embrace their eradication – the Battles of Sector 001 and Scarif are fights for survival. From Physical to Digital By the time Jonathan Frakes was selected to direct First Contact, Star Trek’s reliance on constructing traditional physical models (many of which were built by ILM) for its features was gradually giving way to innovative computer graphics (CG) models, resulting in the film’s use of both techniques. “If one of the ships was to be seen full-screen and at length,” associate visual effects supervisor George Murphy told Cinefex’s Kevin H. Martin, “we knew it would be done as a stage model. Ships that would be doing a lot of elaborate maneuvers in space battle scenes would be created digitally.” In fact, physical and CG versions of the U.S.S. Enterprise-E appear in the film, with the latter being harnessed in shots involving the vessel’s entry into a temporal vortex at the conclusion of the Battle of Sector 001. Despite the technological leaps that ILM pioneered in the decades between First Contact and Rogue One, they considered filming physical miniatures for certain ship-related shots in the latter film. ILM considered filming physical miniatures for certain ship-related shots in Rogue One. The feature’s fleets were ultimately created digitally to allow for changes throughout post-production. “If it’s a photographed miniature element, it’s not possible to go back and make adjustments. So it’s the additional flexibility that comes with the computer graphics models that’s very attractive to many people,” John Knoll relayed to writer Jon Witmer at American Cinematographer’s TheASC.com. However, Knoll aimed to develop computer graphics that retained the same high-quality details as their physical counterparts, leading ILM to employ a modern approach to a time-honored modelmaking tactic. “I also wanted to emulate the kit-bashing aesthetic that had been part of Star Wars from the very beginning, where a lot of mechanical detail had been added onto the ships by using little pieces from plastic model kits,” explained Knoll in his chat with TheASC.com. For Rogue One, ILM replicated the process by obtaining such kits, scanning their parts, building a computer graphics library, and applying the CG parts to digitally modeled ships. “I’m very happy to say it was super-successful,” concluded Knoll. “I think a lot of our digital models look like they are motion-control models.” John Knoll (second from left) confers with Kim Smith and John Goodson with the miniature of the U.S.S. Enterprise-E during production of Star Trek: First Contact (Credit: ILM). Legendary Lineages In First Contact, Captain Picard commanded a brand-new vessel, the Sovereign-class U.S.S. Enterprise-E, continuing the celebrated starship’s legacy in terms of its famous name and design aesthetic. Designed by John Eaves and developed into blueprints by Rick Sternbach, the Enterprise-E was built into a 10-foot physical model by ILM model project supervisor John Goodson and his shop’s talented team. ILM infused the ship with extraordinary detail, including viewports equipped with backlit set images from the craft’s predecessor, the U.S.S. Enterprise-D. For the vessel’s larger windows, namely those associated with the observation lounge and arboretum, ILM took a painstakingly practical approach to match the interiors shown with the real-world set pieces. “We filled that area of the model with tiny, micro-scale furniture,” Goodson informed Cinefex, “including tables and chairs.” Rogue One’s rebel team initially traversed the galaxy in a U-wing transport/gunship, which, much like the Enterprise-E, was a unique vessel that nonetheless channeled a certain degree of inspiration from a classic design. Lucasfilm’s Doug Chiang, a co-production designer for Rogue One, referred to the U-wing as the film’s “Huey helicopter version of an X-wing” in the Designing Rogue One bonus featurette on Disney+ before revealing that, “Towards the end of the design cycle, we actually decided that maybe we should put in more X-wing features. And so we took the X-wing engines and literally mounted them onto the configuration that we had going.” Modeled by ILM digital artist Colie Wertz, the U-wing’s final computer graphics design subtly incorporated these X-wing influences to give the transport a distinctive feel without making the craft seem out of place within the rebel fleet. While ILM’s work on the Enterprise-E’s viewports offered a compelling view toward the ship’s interior, a breakthrough LED setup for Rogue One permitted ILM to obtain realistic lighting on actors as they looked out from their ships and into the space around them. “All of our major spaceship cockpit scenes were done that way, with the gimbal in this giant horseshoe of LED panels we got from [equipment vendor] VER, and we prepared graphics that went on the screens,” John Knoll shared with American Cinematographer’s Benjamin B and Jon D. Witmer. Furthermore, in Disney+’s Rogue One: Digital Storytelling bonus featurette, visual effects producer Janet Lewin noted, “For the actors, I think, in the space battle cockpits, for them to be able to see what was happening in the battle brought a higher level of accuracy to their performance.” The U.S.S. Enterprise-E in Star Trek: First Contact (Credit: Paramount). Familiar Foes To transport First Contact’s Borg invaders, John Goodson’s team at ILM resurrected the Borg cube design previously seen in Star Trek: The Next Generation (1987) and Star Trek: Deep Space Nine (1993), creating a nearly three-foot physical model to replace the one from the series. Art consultant and ILM veteran Bill George proposed that the cube’s seemingly straightforward layout be augmented with a complex network of photo-etched brass, a suggestion which produced a jagged surface and offered a visual that was both intricate and menacing. ILM also developed a two-foot motion-control model for a Borg sphere, a brand-new auxiliary vessel that emerged from the cube. “We vacuformed about 15 different patterns that conformed to this spherical curve and covered those with a lot of molded and cast pieces. Then we added tons of acid-etched brass over it, just like we had on the cube,” Goodson outlined to Cinefex’s Kevin H. Martin. As for Rogue One’s villainous fleet, reproducing the original trilogy’s Death Star and Imperial Star Destroyers centered upon translating physical models into digital assets. Although ILM no longer possessed A New Hope’s three-foot Death Star shooting model, John Knoll recreated the station’s surface paneling by gathering archival images, and as he spelled out to writer Joe Fordham in Cinefex, “I pieced all the images together. I unwrapped them into texture space and projected them onto a sphere with a trench. By doing that with enough pictures, I got pretty complete coverage of the original model, and that became a template upon which to redraw very high-resolution texture maps. Every panel, every vertical striped line, I matched from a photograph. It was as accurate as it was possible to be as a reproduction of the original model.” Knoll’s investigative eye continued to pay dividends when analyzing the three-foot and eight-foot Star Destroyer motion-control models, which had been built for A New Hope and Star Wars: The Empire Strikes Back (1980), respectively. “Our general mantra was, ‘Match your memory of it more than the reality,’ because sometimes you go look at the actual prop in the archive building or you look back at the actual shot from the movie, and you go, ‘Oh, I remember it being a little better than that,’” Knoll conveyed to TheASC.com. This philosophy motivated ILM to combine elements from those two physical models into a single digital design. “Generally, we copied the three-footer for details like the superstructure on the top of the bridge, but then we copied the internal lighting plan from the eight-footer,” Knoll explained. “And then the upper surface of the three-footer was relatively undetailed because there were no shots that saw it closely, so we took a lot of the high-detail upper surface from the eight-footer. So it’s this amalgam of the two models, but the goal was to try to make it look like you remember it from A New Hope.” A final frame from Rogue One: A Star Wars Story (Credit: ILM & Lucasfilm). Forming Up the Fleets In addition to the U.S.S. Enterprise-E, the Battle of Sector 001 debuted numerous vessels representing four new Starfleet ship classes – the Akira, Steamrunner, Saber, and Norway – all designed by ILM visual effects art director Alex Jaeger. “Since we figured a lot of the background action in the space battle would be done with computer graphics ships that needed to be built from scratch anyway, I realized that there was no reason not to do some new designs,” John Knoll told American Cinematographer writer Ron Magid. Used in previous Star Trek projects, older physical models for the Oberth and Nebula classes were mixed into the fleet for good measure, though the vast majority of the armada originated as computer graphics. Over at Scarif, ILM portrayed the Rebel Alliance forces with computer graphics models of fresh designs (the MC75 cruiser Profundity and U-wings), live-action versions of Star Wars Rebels’ VCX-100 light freighter Ghost and Hammerhead corvettes, and Star Wars staples (Nebulon-B frigates, X-wings, Y-wings, and more). These ships face off against two Imperial Star Destroyers and squadrons of TIE fighters, and – upon their late arrival to the battle – Darth Vader’s Star Destroyer and the Death Star. The Tantive IV, a CR90 corvette more popularly referred to as a blockade runner, made its own special cameo at the tail end of the fight. As Princess Leia Organa’s (Carrie Fisher and Ingvild Deila) personal ship, the Tantive IV received the Death Star plans and fled the scene, destined to be captured by Vader’s Star Destroyer at the beginning of A New Hope. And, while we’re on the subject of intricate starship maneuvers and space-based choreography… Although the First Contact team could plan visual effects shots with animated storyboards, ILM supplied Gareth Edwards with a next-level virtual viewfinder that allowed the director to select his shots by immersing himself among Rogue One’s ships in real time. “What we wanted to do is give Gareth the opportunity to shoot his space battles and other all-digital scenes the same way he shoots his live-action. Then he could go in with this sort of virtual viewfinder and view the space battle going on, and figure out what the best angle was to shoot those ships from,” senior animation supervisor Hal Hickel described in the Rogue One: Digital Storytelling featurette. Hickel divulged that the sequence involving the dish array docking with the Death Star was an example of the “spontaneous discovery of great angles,” as the scene was never storyboarded or previsualized. Visual effects supervisor John Knoll with director Gareth Edwards during production of Rogue One: A Star Wars Story (Credit: ILM & Lucasfilm). Tough Little Ships The Federation and Rebel Alliance each deployed “tough little ships” (an endearing description Commander William T. Riker [Jonathan Frakes] bestowed upon the U.S.S. Defiant in First Contact) in their respective conflicts, namely the U.S.S. Defiant from Deep Space Nine and the Tantive IV from A New Hope. VisionArt had already built a CG Defiant for the Deep Space Nine series, but ILM upgraded the model with images gathered from the ship’s three-foot physical model. A similar tactic was taken to bring the Tantive IV into the digital realm for Rogue One. “This was the Blockade Runner. This was the most accurate 1:1 reproduction we could possibly have made,” model supervisor Russell Paul declared to Cinefex’s Joe Fordham. “We did an extensive photo reference shoot and photogrammetry re-creation of the miniature. From there, we built it out as accurately as possible.” Speaking of sturdy ships, if you look very closely, you can spot a model of the Millennium Falcon flashing across the background as the U.S.S. Defiant makes an attack run on the Borg cube at the Battle of Sector 001! Exploration and Hope The in-universe ramifications that materialize from the Battles of Sector 001 and Scarif are monumental. The destruction of the Borg cube compels the Borg Queen to travel back in time in an attempt to vanquish Earth before the Federation can even be formed, but Captain Picard and the Enterprise-E foil the plot and end up helping their 21st century ancestors make “first contact” with another species, the logic-revering Vulcans. The post-Scarif benefits take longer to play out for the Rebel Alliance, but the theft of the Death Star plans eventually leads to the superweapon’s destruction. The Galactic Civil War is far from over, but Scarif is a significant step in the Alliance’s effort to overthrow the Empire. The visual effects ILM provided for First Contact and Rogue One contributed significantly to the critical and commercial acclaim both pictures enjoyed, a victory reflecting the relentless dedication, tireless work ethic, and innovative spirit embodied by visual effects supervisor John Knoll and ILM’s entire staff. While being interviewed for The Making of Star Trek: First Contact, actor Patrick Stewart praised ILM’s invaluable influence, emphasizing, “ILM was with us, on this movie, almost every day on set. There is so much that they are involved in.” And, regardless of your personal preferences – phasers or lasers, photon torpedoes or proton torpedoes, warp speed or hyperspace – perhaps Industrial Light & Magic’s ability to infuse excitement into both franchises demonstrates that Star Trek and Star Wars encompass themes that are not competitive, but compatible. After all, what goes together better than exploration and hope? – Jay Stobie (he/him) is a writer, author, and consultant who has contributed articles to ILM.com, Skysound.com, Star Wars Insider, StarWars.com, Star Trek Explorer, Star Trek Magazine, and StarTrek.com. Jay loves sci-fi, fantasy, and film, and you can learn more about him by visiting JayStobie.com or finding him on Twitter, Instagram, and other social media platforms at @StobiesGalaxy.
    0 Comments 0 Shares 0 Reviews
CGShares https://cgshares.com