• In a world where hackers are the modern-day ninjas, lurking in the shadows of our screens, it’s fascinating to watch the dance of their tactics unfold. Enter the realm of ESD diodes—yes, those little components that seem to be the unsung heroes of electronic protection. You’d think any self-respecting hacker would treat them with the reverence they deserve. But alas, as the saying goes, not all heroes wear capes—some just forget to wear their ESD protection.

    Let’s take a moment to appreciate the artistry of neglecting ESD protection. You have your novice hackers, who, in their quest for glory, overlook the importance of these diodes, thinking, “What’s the worst that could happen? A little static never hurt anyone!” Ah, the blissful ignorance! It’s like going into battle without armor, convinced that sheer bravado will carry the day. Spoiler alert: it won’t. Their circuits will fry faster than you can say “short circuit,” leaving them wondering why their master plan turned into a crispy failure.

    Then, we have the seasoned veterans—the ones who should know better but still scoff at the idea of ESD protection. Perhaps they think they’re above such mundane concerns, like some digital demigods who can manipulate the very fabric of electronics without consequence. I mean, who needs ESD diodes when you have years of experience, right? It’s almost adorable, watching them prance into their tech disasters, blissfully unaware that their arrogance is merely a prelude to a spectacular downfall.

    And let’s not forget the “lone wolves,” those hackers who fancy themselves as rebels without a cause. They see ESD protection as a sign of weakness, a crutch for the faint-hearted. In their minds, real hackers thrive on chaos—why bother with protection when you can revel in the thrill of watching your carefully crafted device go up in flames? It’s the equivalent of a toddler throwing a tantrum because they’re told not to touch the hot stove. Spoiler alert number two: the stove doesn’t care about your feelings.

    In this grand tapestry of hacker culture, the neglect of ESD protection is not merely a technical oversight; it’s a statement, a badge of honor for those who believe they can outsmart the very devices they tinker with. But let’s be real: ESD diodes are the unsung protectors of the digital realm, and ignoring them is like inviting disaster to your tech party and hoping it doesn’t show up. Newsflash: it will.

    So, the next time you find yourself in the presence of a hacker who scoffs at ESD protections, take a moment to revel in their bravado. Just remember to pack some marshmallows for when their devices inevitably catch fire. After all, it’s only a matter of time before the sparks start flying.

    #Hackers #ESDDiodes #TechFails #CyberSecurity #DIYDisasters
    In a world where hackers are the modern-day ninjas, lurking in the shadows of our screens, it’s fascinating to watch the dance of their tactics unfold. Enter the realm of ESD diodes—yes, those little components that seem to be the unsung heroes of electronic protection. You’d think any self-respecting hacker would treat them with the reverence they deserve. But alas, as the saying goes, not all heroes wear capes—some just forget to wear their ESD protection. Let’s take a moment to appreciate the artistry of neglecting ESD protection. You have your novice hackers, who, in their quest for glory, overlook the importance of these diodes, thinking, “What’s the worst that could happen? A little static never hurt anyone!” Ah, the blissful ignorance! It’s like going into battle without armor, convinced that sheer bravado will carry the day. Spoiler alert: it won’t. Their circuits will fry faster than you can say “short circuit,” leaving them wondering why their master plan turned into a crispy failure. Then, we have the seasoned veterans—the ones who should know better but still scoff at the idea of ESD protection. Perhaps they think they’re above such mundane concerns, like some digital demigods who can manipulate the very fabric of electronics without consequence. I mean, who needs ESD diodes when you have years of experience, right? It’s almost adorable, watching them prance into their tech disasters, blissfully unaware that their arrogance is merely a prelude to a spectacular downfall. And let’s not forget the “lone wolves,” those hackers who fancy themselves as rebels without a cause. They see ESD protection as a sign of weakness, a crutch for the faint-hearted. In their minds, real hackers thrive on chaos—why bother with protection when you can revel in the thrill of watching your carefully crafted device go up in flames? It’s the equivalent of a toddler throwing a tantrum because they’re told not to touch the hot stove. Spoiler alert number two: the stove doesn’t care about your feelings. In this grand tapestry of hacker culture, the neglect of ESD protection is not merely a technical oversight; it’s a statement, a badge of honor for those who believe they can outsmart the very devices they tinker with. But let’s be real: ESD diodes are the unsung protectors of the digital realm, and ignoring them is like inviting disaster to your tech party and hoping it doesn’t show up. Newsflash: it will. So, the next time you find yourself in the presence of a hacker who scoffs at ESD protections, take a moment to revel in their bravado. Just remember to pack some marshmallows for when their devices inevitably catch fire. After all, it’s only a matter of time before the sparks start flying. #Hackers #ESDDiodes #TechFails #CyberSecurity #DIYDisasters
    Hacker Tactic: ESD Diodes
    A hacker’s view on ESD protection can tell you a lot about them. I’ve seen a good few categories of hackers neglecting ESD protection – there’s the yet-inexperienced ones, ones …read more
    Like
    Love
    Wow
    Sad
    Angry
    206
    1 Comentários 0 Compartilhamentos 0 Anterior
  • The recent announcement of CEAD inaugurating a center dedicated to 3D printing for manufacturing boat hulls is nothing short of infuriating. We are living in an age where technological advancements should lead to significant improvements in efficiency and sustainability, yet here we are, celebrating a move that reeks of superficial progress and misguided priorities.

    First off, let’s talk about the so-called “Maritime Application Center” (MAC) in Delft. While they dazzle us with their fancy new facility, one has to question the real implications of such a center. Are they genuinely solving the pressing issues of the maritime industry, or are they merely jumping on the bandwagon of 3D printing hype? The idea of using large-scale additive manufacturing to produce boat hulls sounds revolutionary, but let’s face it: this is just another example of throwing technology at a problem without truly understanding the underlying challenges that plague the industry.

    The maritime sector is facing severe environmental concerns, including pollution from traditional manufacturing processes and shipping practices. Instead of addressing these burning issues head-on, CEAD and others like them seem content to play with shiny new tools. 3D printing, in theory, could reduce waste—a point they love to hammer home in their marketing. But what about the energy consumption and material sourcing involved? Are we simply swapping one form of environmental degradation for another?

    Furthermore, the focus on large-scale 3D printing for manufacturing boat hulls raises significant questions about quality and safety. The maritime industry is not a playground for experimental technologies; lives are at stake. Relying on printed components that could potentially have structural weaknesses is a reckless gamble, and the consequences could be disastrous. Are we prepared to accept the liability if these hulls fail at sea?

    Let’s not forget the economic implications of this move. Sure, CEAD is likely patting themselves on the back for creating jobs at the MAC, but how many traditional jobs are they putting at risk? The maritime industry relies on skilled labor and craftsmanship that cannot simply be replaced by a machine. By pushing for 3D printing at such a scale, they threaten the livelihoods of countless workers who have dedicated their lives to mastering this trade.

    In conclusion, while CEAD’s center for 3D printing boat hulls may sound impressive on paper, the reality is that it’s a misguided effort that overlooks critical aspects of sustainability, safety, and social responsibility. We need to demand more from our industries and hold them accountable for their actions instead of blindly celebrating every shiny new innovation. The maritime industry deserves solutions that genuinely address its challenges rather than a mere technological gimmick.

    #MaritimeIndustry #3DPrinting #Sustainability #CEAD #BoatManufacturing
    The recent announcement of CEAD inaugurating a center dedicated to 3D printing for manufacturing boat hulls is nothing short of infuriating. We are living in an age where technological advancements should lead to significant improvements in efficiency and sustainability, yet here we are, celebrating a move that reeks of superficial progress and misguided priorities. First off, let’s talk about the so-called “Maritime Application Center” (MAC) in Delft. While they dazzle us with their fancy new facility, one has to question the real implications of such a center. Are they genuinely solving the pressing issues of the maritime industry, or are they merely jumping on the bandwagon of 3D printing hype? The idea of using large-scale additive manufacturing to produce boat hulls sounds revolutionary, but let’s face it: this is just another example of throwing technology at a problem without truly understanding the underlying challenges that plague the industry. The maritime sector is facing severe environmental concerns, including pollution from traditional manufacturing processes and shipping practices. Instead of addressing these burning issues head-on, CEAD and others like them seem content to play with shiny new tools. 3D printing, in theory, could reduce waste—a point they love to hammer home in their marketing. But what about the energy consumption and material sourcing involved? Are we simply swapping one form of environmental degradation for another? Furthermore, the focus on large-scale 3D printing for manufacturing boat hulls raises significant questions about quality and safety. The maritime industry is not a playground for experimental technologies; lives are at stake. Relying on printed components that could potentially have structural weaknesses is a reckless gamble, and the consequences could be disastrous. Are we prepared to accept the liability if these hulls fail at sea? Let’s not forget the economic implications of this move. Sure, CEAD is likely patting themselves on the back for creating jobs at the MAC, but how many traditional jobs are they putting at risk? The maritime industry relies on skilled labor and craftsmanship that cannot simply be replaced by a machine. By pushing for 3D printing at such a scale, they threaten the livelihoods of countless workers who have dedicated their lives to mastering this trade. In conclusion, while CEAD’s center for 3D printing boat hulls may sound impressive on paper, the reality is that it’s a misguided effort that overlooks critical aspects of sustainability, safety, and social responsibility. We need to demand more from our industries and hold them accountable for their actions instead of blindly celebrating every shiny new innovation. The maritime industry deserves solutions that genuinely address its challenges rather than a mere technological gimmick. #MaritimeIndustry #3DPrinting #Sustainability #CEAD #BoatManufacturing
    CEAD inaugura un centro dedicado a la impresión 3D para fabricar cascos de barcos
    La industria marítima está experimentando una transformación importante gracias a la impresión 3D de gran formato. El grupo holandés CEAD, especialista en fabricación aditiva a gran escala, ha inaugurado recientemente su Maritime Application Center (
    Like
    Love
    Wow
    Sad
    Angry
    587
    1 Comentários 0 Compartilhamentos 0 Anterior
  • Malicious PyPI Package Masquerades as Chimera Module to Steal AWS, CI/CD, and macOS Data

    Jun 16, 2025Ravie LakshmananMalware / DevOps

    Cybersecurity researchers have discovered a malicious package on the Python Package Indexrepository that's capable of harvesting sensitive developer-related information, such as credentials, configuration data, and environment variables, among others.
    The package, named chimera-sandbox-extensions, attracted 143 downloads and likely targets users of a service called Chimera Sandbox, which was released by Singaporean tech company Grab last August to facilitate "experimentation and development ofsolutions."
    The package masquerades as a helper module for Chimera Sandbox, but "aims to steal credentials and other sensitive information such as Jamf configuration, CI/CD environment variables, AWS tokens, and more," JFrog security researcher Guy Korolevski said in a report published last week.
    Once installed, it attempts to connect to an external domain whose domain name is generated using a domain generation algorithmin order to download and execute a next-stage payload.
    Specifically, the malware acquires from the domain an authentication token, which is then used to send a request to the same domain and retrieve the Python-based information stealer.

    The stealer malware is equipped to siphon a wide range of data from infected machines. This includes -

    JAMF receipts, which are records of software packages installed by Jamf Pro on managed computers
    Pod sandbox environment authentication tokens and git information
    CI/CD information from environment variables
    Zscaler host configuration
    Amazon Web Services account information and tokens
    Public IP address
    General platform, user, and host information

    The kind of data gathered by the malware shows that it's mainly geared towards corporate and cloud infrastructure. In addition, the extraction of JAMF receipts indicates that it's also capable of targeting Apple macOS systems.
    The collected information is sent via a POST request back to the same domain, after which the server assesses if the machine is a worthy target for further exploitation. However, JFrog said it was unable to obtain the payload at the time of analysis.
    "The targeted approach employed by this malware, along with the complexity of its multi-stage targeted payload, distinguishes it from the more generic open-source malware threats we have encountered thus far, highlighting the advancements that malicious packages have made recently," Jonathan Sar Shalom, director of threat research at JFrog Security Research team, said.

    "This new sophistication of malware underscores why development teams remain vigilant with updates—alongside proactive security research – to defend against emerging threats and maintain software integrity."
    The disclosure comes as SafeDep and Veracode detailed a number of malware-laced npm packages that are designed to execute remote code and download additional payloads. The packages in question are listed below -

    eslint-config-airbnb-compatts-runtime-compat-checksolders@mediawave/libAll the identified npm packages have since been taken down from npm, but not before they were downloaded hundreds of times from the package registry.
    SafeDep's analysis of eslint-config-airbnb-compat found that the JavaScript library has ts-runtime-compat-check listed as a dependency, which, in turn, contacts an external server defined in the former packageto retrieve and execute a Base64-encoded string. The exact nature of the payload is unknown.
    "It implements a multi-stage remote code execution attack using a transitive dependency to hide the malicious code," SafeDep researcher Kunal Singh said.
    Solders, on the other hand, has been found to incorporate a post-install script in its package.json, causing the malicious code to be automatically executed as soon as the package is installed.
    "At first glance, it's hard to believe that this is actually valid JavaScript," the Veracode Threat Research team said. "It looks like a seemingly random collection of Japanese symbols. It turns out that this particular obfuscation scheme uses the Unicode characters as variable names and a sophisticated chain of dynamic code generation to work."
    Decoding the script reveals an extra layer of obfuscation, unpacking which reveals its main function: Check if the compromised machine is Windows, and if so, run a PowerShell command to retrieve a next-stage payload from a remote server.
    This second-stage PowerShell script, also obscured, is designed to fetch a Windows batch script from another domainand configures a Windows Defender Antivirus exclusion list to avoid detection. The batch script then paves the way for the execution of a .NET DLL that reaches out to a PNG image hosted on ImgBB.
    "is grabbing the last two pixels from this image and then looping through some data contained elsewhere in it," Veracode said. "It ultimately builds up in memory YET ANOTHER .NET DLL."

    Furthermore, the DLL is equipped to create task scheduler entries and features the ability to bypass user account controlusing a combination of FodHelper.exe and programmatic identifiersto evade defenses and avoid triggering any security alerts to the user.
    The newly-downloaded DLL is Pulsar RAT, a "free, open-source Remote Administration Tool for Windows" and a variant of the Quasar RAT.
    "From a wall of Japanese characters to a RAT hidden within the pixels of a PNG file, the attacker went to extraordinary lengths to conceal their payload, nesting it a dozen layers deep to evade detection," Veracode said. "While the attacker's ultimate objective for deploying the Pulsar RAT remains unclear, the sheer complexity of this delivery mechanism is a powerful indicator of malicious intent."
    Crypto Malware in the Open-Source Supply Chain
    The findings also coincide with a report from Socket that identified credential stealers, cryptocurrency drainers, cryptojackers, and clippers as the main types of threats targeting the cryptocurrency and blockchain development ecosystem.

    Some of the examples of these packages include -

    express-dompurify and pumptoolforvolumeandcomment, which are capable of harvesting browser credentials and cryptocurrency wallet keys
    bs58js, which drains a victim's wallet and uses multi-hop transfers to obscure theft and frustrate forensic tracing.
    lsjglsjdv, asyncaiosignal, and raydium-sdk-liquidity-init, which functions as a clipper to monitor the system clipboard for cryptocurrency wallet strings and replace them with threat actor‑controlled addresses to reroute transactions to the attackers

    "As Web3 development converges with mainstream software engineering, the attack surface for blockchain-focused projects is expanding in both scale and complexity," Socket security researcher Kirill Boychenko said.
    "Financially motivated threat actors and state-sponsored groups are rapidly evolving their tactics to exploit systemic weaknesses in the software supply chain. These campaigns are iterative, persistent, and increasingly tailored to high-value targets."
    AI and Slopsquatting
    The rise of artificial intelligence-assisted coding, also called vibe coding, has unleashed another novel threat in the form of slopsquatting, where large language modelscan hallucinate non-existent but plausible package names that bad actors can weaponize to conduct supply chain attacks.
    Trend Micro, in a report last week, said it observed an unnamed advanced agent "confidently" cooking up a phantom Python package named starlette-reverse-proxy, only for the build process to crash with the error "module not found." However, should an adversary upload a package with the same name on the repository, it can have serious security consequences.

    Furthermore, the cybersecurity company noted that advanced coding agents and workflows such as Claude Code CLI, OpenAI Codex CLI, and Cursor AI with Model Context Protocol-backed validation can help reduce, but not completely eliminate, the risk of slopsquatting.
    "When agents hallucinate dependencies or install unverified packages, they create an opportunity for slopsquatting attacks, in which malicious actors pre-register those same hallucinated names on public registries," security researcher Sean Park said.
    "While reasoning-enhanced agents can reduce the rate of phantom suggestions by approximately half, they do not eliminate them entirely. Even the vibe-coding workflow augmented with live MCP validations achieves the lowest rates of slip-through, but still misses edge cases."

    Found this article interesting? Follow us on Twitter  and LinkedIn to read more exclusive content we post.

    SHARE




    #malicious #pypi #package #masquerades #chimera
    Malicious PyPI Package Masquerades as Chimera Module to Steal AWS, CI/CD, and macOS Data
    Jun 16, 2025Ravie LakshmananMalware / DevOps Cybersecurity researchers have discovered a malicious package on the Python Package Indexrepository that's capable of harvesting sensitive developer-related information, such as credentials, configuration data, and environment variables, among others. The package, named chimera-sandbox-extensions, attracted 143 downloads and likely targets users of a service called Chimera Sandbox, which was released by Singaporean tech company Grab last August to facilitate "experimentation and development ofsolutions." The package masquerades as a helper module for Chimera Sandbox, but "aims to steal credentials and other sensitive information such as Jamf configuration, CI/CD environment variables, AWS tokens, and more," JFrog security researcher Guy Korolevski said in a report published last week. Once installed, it attempts to connect to an external domain whose domain name is generated using a domain generation algorithmin order to download and execute a next-stage payload. Specifically, the malware acquires from the domain an authentication token, which is then used to send a request to the same domain and retrieve the Python-based information stealer. The stealer malware is equipped to siphon a wide range of data from infected machines. This includes - JAMF receipts, which are records of software packages installed by Jamf Pro on managed computers Pod sandbox environment authentication tokens and git information CI/CD information from environment variables Zscaler host configuration Amazon Web Services account information and tokens Public IP address General platform, user, and host information The kind of data gathered by the malware shows that it's mainly geared towards corporate and cloud infrastructure. In addition, the extraction of JAMF receipts indicates that it's also capable of targeting Apple macOS systems. The collected information is sent via a POST request back to the same domain, after which the server assesses if the machine is a worthy target for further exploitation. However, JFrog said it was unable to obtain the payload at the time of analysis. "The targeted approach employed by this malware, along with the complexity of its multi-stage targeted payload, distinguishes it from the more generic open-source malware threats we have encountered thus far, highlighting the advancements that malicious packages have made recently," Jonathan Sar Shalom, director of threat research at JFrog Security Research team, said. "This new sophistication of malware underscores why development teams remain vigilant with updates—alongside proactive security research – to defend against emerging threats and maintain software integrity." The disclosure comes as SafeDep and Veracode detailed a number of malware-laced npm packages that are designed to execute remote code and download additional payloads. The packages in question are listed below - eslint-config-airbnb-compatts-runtime-compat-checksolders@mediawave/libAll the identified npm packages have since been taken down from npm, but not before they were downloaded hundreds of times from the package registry. SafeDep's analysis of eslint-config-airbnb-compat found that the JavaScript library has ts-runtime-compat-check listed as a dependency, which, in turn, contacts an external server defined in the former packageto retrieve and execute a Base64-encoded string. The exact nature of the payload is unknown. "It implements a multi-stage remote code execution attack using a transitive dependency to hide the malicious code," SafeDep researcher Kunal Singh said. Solders, on the other hand, has been found to incorporate a post-install script in its package.json, causing the malicious code to be automatically executed as soon as the package is installed. "At first glance, it's hard to believe that this is actually valid JavaScript," the Veracode Threat Research team said. "It looks like a seemingly random collection of Japanese symbols. It turns out that this particular obfuscation scheme uses the Unicode characters as variable names and a sophisticated chain of dynamic code generation to work." Decoding the script reveals an extra layer of obfuscation, unpacking which reveals its main function: Check if the compromised machine is Windows, and if so, run a PowerShell command to retrieve a next-stage payload from a remote server. This second-stage PowerShell script, also obscured, is designed to fetch a Windows batch script from another domainand configures a Windows Defender Antivirus exclusion list to avoid detection. The batch script then paves the way for the execution of a .NET DLL that reaches out to a PNG image hosted on ImgBB. "is grabbing the last two pixels from this image and then looping through some data contained elsewhere in it," Veracode said. "It ultimately builds up in memory YET ANOTHER .NET DLL." Furthermore, the DLL is equipped to create task scheduler entries and features the ability to bypass user account controlusing a combination of FodHelper.exe and programmatic identifiersto evade defenses and avoid triggering any security alerts to the user. The newly-downloaded DLL is Pulsar RAT, a "free, open-source Remote Administration Tool for Windows" and a variant of the Quasar RAT. "From a wall of Japanese characters to a RAT hidden within the pixels of a PNG file, the attacker went to extraordinary lengths to conceal their payload, nesting it a dozen layers deep to evade detection," Veracode said. "While the attacker's ultimate objective for deploying the Pulsar RAT remains unclear, the sheer complexity of this delivery mechanism is a powerful indicator of malicious intent." Crypto Malware in the Open-Source Supply Chain The findings also coincide with a report from Socket that identified credential stealers, cryptocurrency drainers, cryptojackers, and clippers as the main types of threats targeting the cryptocurrency and blockchain development ecosystem. Some of the examples of these packages include - express-dompurify and pumptoolforvolumeandcomment, which are capable of harvesting browser credentials and cryptocurrency wallet keys bs58js, which drains a victim's wallet and uses multi-hop transfers to obscure theft and frustrate forensic tracing. lsjglsjdv, asyncaiosignal, and raydium-sdk-liquidity-init, which functions as a clipper to monitor the system clipboard for cryptocurrency wallet strings and replace them with threat actor‑controlled addresses to reroute transactions to the attackers "As Web3 development converges with mainstream software engineering, the attack surface for blockchain-focused projects is expanding in both scale and complexity," Socket security researcher Kirill Boychenko said. "Financially motivated threat actors and state-sponsored groups are rapidly evolving their tactics to exploit systemic weaknesses in the software supply chain. These campaigns are iterative, persistent, and increasingly tailored to high-value targets." AI and Slopsquatting The rise of artificial intelligence-assisted coding, also called vibe coding, has unleashed another novel threat in the form of slopsquatting, where large language modelscan hallucinate non-existent but plausible package names that bad actors can weaponize to conduct supply chain attacks. Trend Micro, in a report last week, said it observed an unnamed advanced agent "confidently" cooking up a phantom Python package named starlette-reverse-proxy, only for the build process to crash with the error "module not found." However, should an adversary upload a package with the same name on the repository, it can have serious security consequences. Furthermore, the cybersecurity company noted that advanced coding agents and workflows such as Claude Code CLI, OpenAI Codex CLI, and Cursor AI with Model Context Protocol-backed validation can help reduce, but not completely eliminate, the risk of slopsquatting. "When agents hallucinate dependencies or install unverified packages, they create an opportunity for slopsquatting attacks, in which malicious actors pre-register those same hallucinated names on public registries," security researcher Sean Park said. "While reasoning-enhanced agents can reduce the rate of phantom suggestions by approximately half, they do not eliminate them entirely. Even the vibe-coding workflow augmented with live MCP validations achieves the lowest rates of slip-through, but still misses edge cases." Found this article interesting? Follow us on Twitter  and LinkedIn to read more exclusive content we post. SHARE     #malicious #pypi #package #masquerades #chimera
    THEHACKERNEWS.COM
    Malicious PyPI Package Masquerades as Chimera Module to Steal AWS, CI/CD, and macOS Data
    Jun 16, 2025Ravie LakshmananMalware / DevOps Cybersecurity researchers have discovered a malicious package on the Python Package Index (PyPI) repository that's capable of harvesting sensitive developer-related information, such as credentials, configuration data, and environment variables, among others. The package, named chimera-sandbox-extensions, attracted 143 downloads and likely targets users of a service called Chimera Sandbox, which was released by Singaporean tech company Grab last August to facilitate "experimentation and development of [machine learning] solutions." The package masquerades as a helper module for Chimera Sandbox, but "aims to steal credentials and other sensitive information such as Jamf configuration, CI/CD environment variables, AWS tokens, and more," JFrog security researcher Guy Korolevski said in a report published last week. Once installed, it attempts to connect to an external domain whose domain name is generated using a domain generation algorithm (DGA) in order to download and execute a next-stage payload. Specifically, the malware acquires from the domain an authentication token, which is then used to send a request to the same domain and retrieve the Python-based information stealer. The stealer malware is equipped to siphon a wide range of data from infected machines. This includes - JAMF receipts, which are records of software packages installed by Jamf Pro on managed computers Pod sandbox environment authentication tokens and git information CI/CD information from environment variables Zscaler host configuration Amazon Web Services account information and tokens Public IP address General platform, user, and host information The kind of data gathered by the malware shows that it's mainly geared towards corporate and cloud infrastructure. In addition, the extraction of JAMF receipts indicates that it's also capable of targeting Apple macOS systems. The collected information is sent via a POST request back to the same domain, after which the server assesses if the machine is a worthy target for further exploitation. However, JFrog said it was unable to obtain the payload at the time of analysis. "The targeted approach employed by this malware, along with the complexity of its multi-stage targeted payload, distinguishes it from the more generic open-source malware threats we have encountered thus far, highlighting the advancements that malicious packages have made recently," Jonathan Sar Shalom, director of threat research at JFrog Security Research team, said. "This new sophistication of malware underscores why development teams remain vigilant with updates—alongside proactive security research – to defend against emerging threats and maintain software integrity." The disclosure comes as SafeDep and Veracode detailed a number of malware-laced npm packages that are designed to execute remote code and download additional payloads. The packages in question are listed below - eslint-config-airbnb-compat (676 Downloads) ts-runtime-compat-check (1,588 Downloads) solders (983 Downloads) @mediawave/lib (386 Downloads) All the identified npm packages have since been taken down from npm, but not before they were downloaded hundreds of times from the package registry. SafeDep's analysis of eslint-config-airbnb-compat found that the JavaScript library has ts-runtime-compat-check listed as a dependency, which, in turn, contacts an external server defined in the former package ("proxy.eslint-proxy[.]site") to retrieve and execute a Base64-encoded string. The exact nature of the payload is unknown. "It implements a multi-stage remote code execution attack using a transitive dependency to hide the malicious code," SafeDep researcher Kunal Singh said. Solders, on the other hand, has been found to incorporate a post-install script in its package.json, causing the malicious code to be automatically executed as soon as the package is installed. "At first glance, it's hard to believe that this is actually valid JavaScript," the Veracode Threat Research team said. "It looks like a seemingly random collection of Japanese symbols. It turns out that this particular obfuscation scheme uses the Unicode characters as variable names and a sophisticated chain of dynamic code generation to work." Decoding the script reveals an extra layer of obfuscation, unpacking which reveals its main function: Check if the compromised machine is Windows, and if so, run a PowerShell command to retrieve a next-stage payload from a remote server ("firewall[.]tel"). This second-stage PowerShell script, also obscured, is designed to fetch a Windows batch script from another domain ("cdn.audiowave[.]org") and configures a Windows Defender Antivirus exclusion list to avoid detection. The batch script then paves the way for the execution of a .NET DLL that reaches out to a PNG image hosted on ImgBB ("i.ibb[.]co"). "[The DLL] is grabbing the last two pixels from this image and then looping through some data contained elsewhere in it," Veracode said. "It ultimately builds up in memory YET ANOTHER .NET DLL." Furthermore, the DLL is equipped to create task scheduler entries and features the ability to bypass user account control (UAC) using a combination of FodHelper.exe and programmatic identifiers (ProgIDs) to evade defenses and avoid triggering any security alerts to the user. The newly-downloaded DLL is Pulsar RAT, a "free, open-source Remote Administration Tool for Windows" and a variant of the Quasar RAT. "From a wall of Japanese characters to a RAT hidden within the pixels of a PNG file, the attacker went to extraordinary lengths to conceal their payload, nesting it a dozen layers deep to evade detection," Veracode said. "While the attacker's ultimate objective for deploying the Pulsar RAT remains unclear, the sheer complexity of this delivery mechanism is a powerful indicator of malicious intent." Crypto Malware in the Open-Source Supply Chain The findings also coincide with a report from Socket that identified credential stealers, cryptocurrency drainers, cryptojackers, and clippers as the main types of threats targeting the cryptocurrency and blockchain development ecosystem. Some of the examples of these packages include - express-dompurify and pumptoolforvolumeandcomment, which are capable of harvesting browser credentials and cryptocurrency wallet keys bs58js, which drains a victim's wallet and uses multi-hop transfers to obscure theft and frustrate forensic tracing. lsjglsjdv, asyncaiosignal, and raydium-sdk-liquidity-init, which functions as a clipper to monitor the system clipboard for cryptocurrency wallet strings and replace them with threat actor‑controlled addresses to reroute transactions to the attackers "As Web3 development converges with mainstream software engineering, the attack surface for blockchain-focused projects is expanding in both scale and complexity," Socket security researcher Kirill Boychenko said. "Financially motivated threat actors and state-sponsored groups are rapidly evolving their tactics to exploit systemic weaknesses in the software supply chain. These campaigns are iterative, persistent, and increasingly tailored to high-value targets." AI and Slopsquatting The rise of artificial intelligence (AI)-assisted coding, also called vibe coding, has unleashed another novel threat in the form of slopsquatting, where large language models (LLMs) can hallucinate non-existent but plausible package names that bad actors can weaponize to conduct supply chain attacks. Trend Micro, in a report last week, said it observed an unnamed advanced agent "confidently" cooking up a phantom Python package named starlette-reverse-proxy, only for the build process to crash with the error "module not found." However, should an adversary upload a package with the same name on the repository, it can have serious security consequences. Furthermore, the cybersecurity company noted that advanced coding agents and workflows such as Claude Code CLI, OpenAI Codex CLI, and Cursor AI with Model Context Protocol (MCP)-backed validation can help reduce, but not completely eliminate, the risk of slopsquatting. "When agents hallucinate dependencies or install unverified packages, they create an opportunity for slopsquatting attacks, in which malicious actors pre-register those same hallucinated names on public registries," security researcher Sean Park said. "While reasoning-enhanced agents can reduce the rate of phantom suggestions by approximately half, they do not eliminate them entirely. Even the vibe-coding workflow augmented with live MCP validations achieves the lowest rates of slip-through, but still misses edge cases." Found this article interesting? Follow us on Twitter  and LinkedIn to read more exclusive content we post. SHARE    
    Like
    Love
    Wow
    Sad
    Angry
    514
    2 Comentários 0 Compartilhamentos 0 Anterior
  • How AI is reshaping the future of healthcare and medical research

    Transcript       
    PETER LEE: “In ‘The Little Black Bag,’ a classic science fiction story, a high-tech doctor’s kit of the future is accidentally transported back to the 1950s, into the shaky hands of a washed-up, alcoholic doctor. The ultimate medical tool, it redeems the doctor wielding it, allowing him to practice gratifyingly heroic medicine. … The tale ends badly for the doctor and his treacherous assistant, but it offered a picture of how advanced technology could transform medicine—powerful when it was written nearly 75 years ago and still so today. What would be the Al equivalent of that little black bag? At this moment when new capabilities are emerging, how do we imagine them into medicine?”          
    This is The AI Revolution in Medicine, Revisited. I’m your host, Peter Lee.   
    Shortly after OpenAI’s GPT-4 was publicly released, Carey Goldberg, Dr. Zak Kohane, and I published The AI Revolution in Medicine to help educate the world of healthcare and medical research about the transformative impact this new generative AI technology could have. But because we wrote the book when GPT-4 was still a secret, we had to speculate. Now, two years later, what did we get right, and what did we get wrong?    
    In this series, we’ll talk to clinicians, patients, hospital administrators, and others to understand the reality of AI in the field and where we go from here.  The book passage I read at the top is from “Chapter 10: The Big Black Bag.” 
    In imagining AI in medicine, Carey, Zak, and I included in our book two fictional accounts. In the first, a medical resident consults GPT-4 on her personal phone as the patient in front of her crashes. Within seconds, it offers an alternate response based on recent literature. In the second account, a 90-year-old woman with several chronic conditions is living independently and receiving near-constant medical support from an AI aide.   
    In our conversations with the guests we’ve spoken to so far, we’ve caught a glimpse of these predicted futures, seeing how clinicians and patients are actually using AI today and how developers are leveraging the technology in the healthcare products and services they’re creating. In fact, that first fictional account isn’t so fictional after all, as most of the doctors in the real world actually appear to be using AI at least occasionally—and sometimes much more than occasionally—to help in their daily clinical work. And as for the second fictional account, which is more of a science fiction account, it seems we are indeed on the verge of a new way of delivering and receiving healthcare, though the future is still very much open. 
    As we continue to examine the current state of AI in healthcare and its potential to transform the field, I’m pleased to welcome Bill Gates and Sébastien Bubeck.  
    Bill may be best known as the co-founder of Microsoft, having created the company with his childhood friend Paul Allen in 1975. He’s now the founder of Breakthrough Energy, which aims to advance clean energy innovation, and TerraPower, a company developing groundbreaking nuclear energy and science technologies. He also chairs the world’s largest philanthropic organization, the Gates Foundation, and focuses on solving a variety of health challenges around the globe and here at home. 
    Sébastien is a research lead at OpenAI. He was previously a distinguished scientist, vice president of AI, and a colleague of mine here at Microsoft, where his work included spearheading the development of the family of small language models known as Phi. While at Microsoft, he also coauthored the discussion-provoking 2023 paper “Sparks of Artificial General Intelligence,” which presented the results of early experiments with GPT-4 conducted by a small team from Microsoft Research.     
    Here’s my conversation with Bill Gates and Sébastien Bubeck. 
    LEE: Bill, welcome. 
    BILL GATES: Thank you. 
    LEE: Seb … 
    SÉBASTIEN BUBECK: Yeah. Hi, hi, Peter. Nice to be here. 
    LEE: You know, one of the things that I’ve been doing just to get the conversation warmed up is to talk about origin stories, and what I mean about origin stories is, you know, what was the first contact that you had with large language models or the concept of generative AI that convinced you or made you think that something really important was happening? 
    And so, Bill, I think I’ve heard the story about, you know, the time when the OpenAI folks—Sam Altman, Greg Brockman, and others—showed you something, but could we hear from you what those early encounters were like and what was going through your mind?  
    GATES: Well, I’d been visiting OpenAI soon after it was created to see things like GPT-2 and to see the little arm they had that was trying to match human manipulation and, you know, looking at their games like Dota that they were trying to get as good as human play. And honestly, I didn’t think the language model stuff they were doing, even when they got to GPT-3, would show the ability to learn, you know, in the same sense that a human reads a biology book and is able to take that knowledge and access it not only to pass a test but also to create new medicines. 
    And so my challenge to them was that if their LLM could get a five on the advanced placement biology test, then I would say, OK, it took biologic knowledge and encoded it in an accessible way and that I didn’t expect them to do that very quickly but it would be profound.  
    And it was only about six months after I challenged them to do that, that an early version of GPT-4 they brought up to a dinner at my house, and in fact, it answered most of the questions that night very well. The one it got totally wrong, we were … because it was so good, we kept thinking, Oh, we must be wrong. It turned out it was a math weaknessthat, you know, we later understood that that was an area of, weirdly, of incredible weakness of those early models. But, you know, that was when I realized, OK, the age of cheap intelligence was at its beginning. 
    LEE: Yeah. So I guess it seems like you had something similar to me in that my first encounters, I actually harbored some skepticism. Is it fair to say you were skeptical before that? 
    GATES: Well, the idea that we’ve figured out how to encode and access knowledge in this very deep sense without even understanding the nature of the encoding, … 
    LEE: Right.  
    GATES: … that is a bit weird.  
    LEE: Yeah. 
    GATES: We have an algorithm that creates the computation, but even say, OK, where is the president’s birthday stored in there? Where is this fact stored in there? The fact that even now when we’re playing around, getting a little bit more sense of it, it’s opaque to us what the semantic encoding is, it’s, kind of, amazing to me. I thought the invention of knowledge storage would be an explicit way of encoding knowledge, not an implicit statistical training. 
    LEE: Yeah, yeah. All right. So, Seb, you know, on this same topic, you know, I got—as we say at Microsoft—I got pulled into the tent. 
    BUBECK: Yes.  
    LEE: Because this was a very secret project. And then, um, I had the opportunity to select a small number of researchers in MSRto join and start investigating this thing seriously. And the first person I pulled in was you. 
    BUBECK: Yeah. 
    LEE: And so what were your first encounters? Because I actually don’t remember what happened then. 
    BUBECK: Oh, I remember it very well.My first encounter with GPT-4 was in a meeting with the two of you, actually. But my kind of first contact, the first moment where I realized that something was happening with generative AI, was before that. And I agree with Bill that I also wasn’t too impressed by GPT-3. 
    I though that it was kind of, you know, very naturally mimicking the web, sort of parroting what was written there in a nice way. Still in a way which seemed very impressive. But it wasn’t really intelligent in any way. But shortly after GPT-3, there was a model before GPT-4 that really shocked me, and this was the first image generation model, DALL-E 1. 
    So that was in 2021. And I will forever remember the press release of OpenAI where they had this prompt of an avocado chair and then you had this image of the avocado chair.And what really shocked me is that clearly the model kind of “understood” what is a chair, what is an avocado, and was able to merge those concepts. 
    So this was really, to me, the first moment where I saw some understanding in those models.  
    LEE: So this was, just to get the timing right, that was before I pulled you into the tent. 
    BUBECK: That was before. That was like a year before. 
    LEE: Right.  
    BUBECK: And now I will tell you how, you know, we went from that moment to the meeting with the two of you and GPT-4. 
    So once I saw this kind of understanding, I thought, OK, fine. It understands concept, but it’s still not able to reason. It cannot—as, you know, Bill was saying—it cannot learn from your document. It cannot reason.  
    So I set out to try to prove that. You know, this is what I was in the business of at the time, trying to prove things in mathematics. So I was trying to prove that basically autoregressive transformers could never reason. So I was trying to prove this. And after a year of work, I had something reasonable to show. And so I had the meeting with the two of you, and I had this example where I wanted to say, there is no way that an LLM is going to be able to do x. 
    And then as soon as I … I don’t know if you remember, Bill. But as soon as I said that, you said, oh, but wait a second. I had, you know, the OpenAI crew at my house recently, and they showed me a new model. Why don’t we ask this new model this question?  
    LEE: Yeah.
    BUBECK: And we did, and it solved it on the spot. And that really, honestly, just changed my life. Like, you know, I had been working for a year trying to say that this was impossible. And just right there, it was shown to be possible.  
    LEE:One of the very first things I got interested in—because I was really thinking a lot about healthcare—was healthcare and medicine. 
    And I don’t know if the two of you remember, but I ended up doing a lot of tests. I ran through, you know, step one and step two of the US Medical Licensing Exam. Did a whole bunch of other things. I wrote this big report. It was, you know, I can’t remember … a couple hundred pages.  
    And I needed to share this with someone. I didn’t … there weren’t too many people I could share it with. So I sent, I think, a copy to you, Bill. Sent a copy to you, Seb.  
    I hardly slept for about a week putting that report together. And, yeah, and I kept working on it. But I was far from alone. I think everyone who was in the tent, so to speak, in those early days was going through something pretty similar. All right. So I think … of course, a lot of what I put in the report also ended up being examples that made it into the book. 
    But the main purpose of this conversation isn’t to reminisce aboutor indulge in those reminiscences but to talk about what’s happening in healthcare and medicine. And, you know, as I said, we wrote this book. We did it very, very quickly. Seb, you helped. Bill, you know, you provided a review and some endorsements. 
    But, you know, honestly, we didn’t know what we were talking about because no one had access to this thing. And so we just made a bunch of guesses. So really, the whole thing I wanted to probe with the two of you is, now with two years of experience out in the world, what, you know, what do we think is happening today? 
    You know, is AI actually having an impact, positive or negative, on healthcare and medicine? And what do we now think is going to happen in the next two years, five years, or 10 years? And so I realize it’s a little bit too abstract to just ask it that way. So let me just try to narrow the discussion and guide us a little bit.  
    Um, the kind of administrative and clerical work, paperwork, around healthcare—and we made a lot of guesses about that—that appears to be going well, but, you know, Bill, I know we’ve discussed that sometimes that you think there ought to be a lot more going on. Do you have a viewpoint on how AI is actually finding its way into reducing paperwork? 
    GATES: Well, I’m stunned … I don’t think there should be a patient-doctor meeting where the AI is not sitting in and both transcribing, offering to help with the paperwork, and even making suggestions, although the doctor will be the one, you know, who makes the final decision about the diagnosis and whatever prescription gets done.  
    It’s so helpful. You know, when that patient goes home and their, you know, son who wants to understand what happened has some questions, that AI should be available to continue that conversation. And the way you can improve that experience and streamline things and, you know, involve the people who advise you. I don’t understand why that’s not more adopted, because there you still have the human in the loop making that final decision. 
    But even for, like, follow-up calls to make sure the patient did things, to understand if they have concerns and knowing when to escalate back to the doctor, the benefit is incredible. And, you know, that thing is ready for prime time. That paradigm is ready for prime time, in my view. 
    LEE: Yeah, there are some good products, but it seems like the number one use right now—and we kind of got this from some of the previous guests in previous episodes—is the use of AI just to respond to emails from patients.Does that make sense to you? 
    BUBECK: Yeah. So maybe I want to second what Bill was saying but maybe take a step back first. You know, two years ago, like, the concept of clinical scribes, which is one of the things that we’re talking about right now, it would have sounded, in fact, it sounded two years ago, borderline dangerous. Because everybody was worried about hallucinations. What happened if you have this AI listening in and then it transcribes, you know, something wrong? 
    Now, two years later, I think it’s mostly working. And in fact, it is not yet, you know, fully adopted. You’re right. But it is in production. It is used, you know, in many, many places. So this rate of progress is astounding because it wasn’t obvious that we would be able to overcome those obstacles of hallucination. It’s not to say that hallucinations are fully solved. In the case of the closed system, they are.  
    Now, I think more generally what’s going on in the background is that there is something that we, that certainly I, underestimated, which is this management overhead. So I think the reason why this is not adopted everywhere is really a training and teaching aspect. People need to be taught, like, those systems, how to interact with them. 
    And one example that I really like, a study that recently appeared where they tried to use ChatGPT for diagnosis and they were comparing doctors without and with ChatGPT. And the amazing thing … so this was a set of cases where the accuracy of the doctors alone was around 75%. ChatGPT alone was 90%. So that’s already kind of mind blowing. But then the kicker is that doctors with ChatGPT was 80%.  
    Intelligence alone is not enough. It’s also how it’s presented, how you interact with it. And ChatGPT, it’s an amazing tool. Obviously, I absolutely love it. But it’s not … you don’t want a doctor to have to type in, you know, prompts and use it that way. 
    It should be, as Bill was saying, kind of running continuously in the background, sending you notifications. And you have to be really careful of the rate at which those notifications are being sent. Because if they are too frequent, then the doctor will learn to ignore them. So you have to … all of those things matter, in fact, at least as much as the level of intelligence of the machine. 
    LEE: One of the things I think about, Bill, in that scenario that you described, doctors do some thinking about the patient when they write the note. So, you know, I’m always a little uncertain whether it’s actually … you know, you wouldn’t necessarily want to fully automate this, I don’t think. Or at least there needs to be some prompt to the doctor to make sure that the doctor puts some thought into what happened in the encounter with the patient. Does that make sense to you at all? 
    GATES: At this stage, you know, I’d still put the onus on the doctor to write the conclusions and the summary and not delegate that. 
    The tradeoffs you make a little bit are somewhat dependent on the situation you’re in. If you’re in Africa,
    So, yes, the doctor’s still going to have to do a lot of work, but just the quality of letting the patient and the people around them interact and ask questions and have things explained, that alone is such a quality improvement. It’s mind blowing.  
    LEE: So since you mentioned, you know, Africa—and, of course, this touches on the mission and some of the priorities of the Gates Foundation and this idea of democratization of access to expert medical care—what’s the most interesting stuff going on right now? Are there people and organizations or technologies that are impressing you or that you’re tracking? 
    GATES: Yeah. So the Gates Foundation has given out a lot of grants to people in Africa doing education, agriculture but more healthcare examples than anything. And the way these things start off, they often start out either being patient-centric in a narrow situation, like, OK, I’m a pregnant woman; talk to me. Or, I have infectious disease symptoms; talk to me. Or they’re connected to a health worker where they’re helping that worker get their job done. And we have lots of pilots out, you know, in both of those cases.  
    The dream would be eventually to have the thing the patient consults be so broad that it’s like having a doctor available who understands the local things.  
    LEE: Right.  
    GATES: We’re not there yet. But over the next two or three years, you know, particularly given the worsening financial constraints against African health systems, where the withdrawal of money has been dramatic, you know, figuring out how to take this—what I sometimes call “free intelligence”—and build a quality health system around that, we will have to be more radical in low-income countries than any rich country is ever going to be.  
    LEE: Also, there’s maybe a different regulatory environment, so some of those things maybe are easier? Because right now, I think the world hasn’t figured out how to and whether to regulate, let’s say, an AI that might give a medical diagnosis or write a prescription for a medication. 
    BUBECK: Yeah. I think one issue with this, and it’s also slowing down the deployment of AI in healthcare more generally, is a lack of proper benchmark. Because, you know, you were mentioning the USMLE, for example. That’s a great test to test human beings and their knowledge of healthcare and medicine. But it’s not a great test to give to an AI. 
    It’s not asking the right questions. So finding what are the right questions to test whether an AI system is ready to give diagnosis in a constrained setting, that’s a very, very important direction, which to my surprise, is not yet accelerating at the rate that I was hoping for. 
    LEE: OK, so that gives me an excuse to get more now into the core AI tech because something I’ve discussed with both of you is this issue of what are the right tests. And you both know the very first test I give to any new spin of an LLM is I present a patient, the results—a mythical patient—the results of my physical exam, my mythical physical exam. Maybe some results of some initial labs. And then I present or propose a differential diagnosis. And if you’re not in medicine, a differential diagnosis you can just think of as a prioritized list of the possible diagnoses that fit with all that data. And in that proposed differential, I always intentionally make two mistakes. 
    I make a textbook technical error in one of the possible elements of the differential diagnosis, and I have an error of omission. And, you know, I just want to know, does the LLM understand what I’m talking about? And all the good ones out there do now. But then I want to know, can it spot the errors? And then most importantly, is it willing to tell me I’m wrong, that I’ve made a mistake?  
    That last piece seems really hard for AI today. And so let me ask you first, Seb, because at the time of this taping, of course, there was a new spin of GPT-4o last week that became overly sycophantic. In other words, it was actually prone in that test of mine not only to not tell me I’m wrong, but it actually praised me for the creativity of my differential.What’s up with that? 
    BUBECK: Yeah, I guess it’s a testament to the fact that training those models is still more of an art than a science. So it’s a difficult job. Just to be clear with the audience, we have rolled back thatversion of GPT-4o, so now we don’t have the sycophant version out there. 
    Yeah, no, it’s a really difficult question. It has to do … as you said, it’s very technical. It has to do with the post-training and how, like, where do you nudge the model? So, you know, there is this very classical by now technique called RLHF, where you push the model in the direction of a certain reward model. So the reward model is just telling the model, you know, what behavior is good, what behavior is bad. 
    But this reward model is itself an LLM, and, you know, Bill was saying at the very beginning of the conversation that we don’t really understand how those LLMs deal with concepts like, you know, where is the capital of France located? Things like that. It is the same thing for this reward model. We don’t know why it says that it prefers one output to another, and whether this is correlated with some sycophancy is, you know, something that we discovered basically just now. That if you push too hard in optimization on this reward model, you will get a sycophant model. 
    So it’s kind of … what I’m trying to say is we became too good at what we were doing, and we ended up, in fact, in a trap of the reward model. 
    LEE: I mean, you do want … it’s a difficult balance because you do want models to follow your desires and … 
    BUBECK: It’s a very difficult, very difficult balance. 
    LEE: So this brings up then the following question for me, which is the extent to which we think we’ll need to have specially trained models for things. So let me start with you, Bill. Do you have a point of view on whether we will need to, you know, quote-unquote take AI models to med school? Have them specially trained? Like, if you were going to deploy something to give medical care in underserved parts of the world, do we need to do something special to create those models? 
    GATES: We certainly need to teach them the African languages and the unique dialects so that the multimedia interactions are very high quality. We certainly need to teach them the disease prevalence and unique disease patterns like, you know, neglected tropical diseases and malaria. So we need to gather a set of facts that somebody trying to go for a US customer base, you know, wouldn’t necessarily have that in there. 
    Those two things are actually very straightforward because the additional training time is small. I’d say for the next few years, we’ll also need to do reinforcement learning about the context of being a doctor and how important certain behaviors are. Humans learn over the course of their life to some degree that, I’m in a different context and the way I behave in terms of being willing to criticize or be nice, you know, how important is it? Who’s here? What’s my relationship to them?  
    Right now, these machines don’t have that broad social experience. And so if you know it’s going to be used for health things, a lot of reinforcement learning of the very best humans in that context would still be valuable. Eventually, the models will, having read all the literature of the world about good doctors, bad doctors, it’ll understand as soon as you say, “I want you to be a doctor diagnosing somebody.” All of the implicit reinforcement that fits that situation, you know, will be there.
    LEE: Yeah.
    GATES: And so I hope three years from now, we don’t have to do that reinforcement learning. But today, for any medical context, you would want a lot of data to reinforce tone, willingness to say things when, you know, there might be something significant at stake. 
    LEE: Yeah. So, you know, something Bill said, kind of, reminds me of another thing that I think we missed, which is, the context also … and the specialization also pertains to different, I guess, what we still call “modes,” although I don’t know if the idea of multimodal is the same as it was two years ago. But, you know, what do you make of all of the hubbub around—in fact, within Microsoft Research, this is a big deal, but I think we’re far from alone—you know, medical images and vision, video, proteins and molecules, cell, you know, cellular data and so on. 
    BUBECK: Yeah. OK. So there is a lot to say to everything … to the last, you know, couple of minutes. Maybe on the specialization aspect, you know, I think there is, hiding behind this, a really fundamental scientific question of whether eventually we have a singular AGIthat kind of knows everything and you can just put, you know, explain your own context and it will just get it and understand everything. 
    That’s one vision. I have to say, I don’t particularly believe in this vision. In fact, we humans are not like that at all. I think, hopefully, we are general intelligences, yet we have to specialize a lot. And, you know, I did myself a lot of RL, reinforcement learning, on mathematics. Like, that’s what I did, you know, spent a lot of time doing that. And I didn’t improve on other aspects. You know, in fact, I probably degraded in other aspects.So it’s … I think it’s an important example to have in mind. 
    LEE: I think I might disagree with you on that, though, because, like, doesn’t a model have to see both good science and bad science in order to be able to gain the ability to discern between the two? 
    BUBECK: Yeah, no, that absolutely. I think there is value in seeing the generality, in having a very broad base. But then you, kind of, specialize on verticals. And this is where also, you know, open-weights model, which we haven’t talked about yet, are really important because they allow you to provide this broad base to everyone. And then you can specialize on top of it. 
    LEE: So we have about three hours of stuff to talk about, but our time is actually running low.
    BUBECK: Yes, yes, yes.  
    LEE: So I think I want … there’s a more provocative question. It’s almost a silly question, but I need to ask it of the two of you, which is, is there a future, you know, where AI replaces doctors or replaces, you know, medical specialties that we have today? So what does the world look like, say, five years from now? 
    GATES: Well, it’s important to distinguish healthcare discovery activity from healthcare delivery activity. We focused mostly on delivery. I think it’s very much within the realm of possibility that the AI is not only accelerating healthcare discovery but substituting for a lot of the roles of, you know, I’m an organic chemist, or I run various types of assays. I can see those, which are, you know, testable-output-type jobs but with still very high value, I can see, you know, some replacement in those areas before the doctor.  
    The doctor, still understanding the human condition and long-term dialogues, you know, they’ve had a lifetime of reinforcement of that, particularly when you get into areas like mental health. So I wouldn’t say in five years, either people will choose to adopt it, but it will be profound that there’ll be this nearly free intelligence that can do follow-up, that can help you, you know, make sure you went through different possibilities. 
    And so I’d say, yes, we’ll have doctors, but I’d say healthcare will be massively transformed in its quality and in efficiency by AI in that time period. 
    LEE: Is there a comparison, useful comparison, say, between doctors and, say, programmers, computer programmers, or doctors and, I don’t know, lawyers? 
    GATES: Programming is another one that has, kind of, a mathematical correctness to it, you know, and so the objective function that you’re trying to reinforce to, as soon as you can understand the state machines, you can have something that’s “checkable”; that’s correct. So I think programming, you know, which is weird to say, that the machine will beat us at most programming tasks before we let it take over roles that have deep empathy, you know, physical presence and social understanding in them. 
    LEE: Yeah. By the way, you know, I fully expect in five years that AI will produce mathematical proofs that are checkable for validity, easily checkable, because they’ll be written in a proof-checking language like Lean or something but will be so complex that no human mathematician can understand them. I expect that to happen.  
    I can imagine in some fields, like cellular biology, we could have the same situation in the future because the molecular pathways, the chemistry, biochemistry of human cells or living cells is as complex as any mathematics, and so it seems possible that we may be in a state where in wet lab, we see, Oh yeah, this actually works, but no one can understand why. 
    BUBECK: Yeah, absolutely. I mean, I think I really agree with Bill’s distinction of the discovery and the delivery, and indeed, the discovery’s when you can check things, and at the end, there is an artifact that you can verify. You know, you can run the protocol in the wet lab and seeproduced what you wanted. So I absolutely agree with that.  
    And in fact, you know, we don’t have to talk five years from now. I don’t know if you know, but just recently, there was a paper that was published on a scientific discovery using o3- mini. So this is really amazing. And, you know, just very quickly, just so people know, it was about this statistical physics model, the frustrated Potts model, which has to do with coloring, and basically, the case of three colors, like, more than two colors was open for a long time, and o3 was able to reduce the case of three colors to two colors.  
    LEE: Yeah. 
    BUBECK: Which is just, like, astounding. And this is not … this is now. This is happening right now. So this is something that I personally didn’t expect it would happen so quickly, and it’s due to those reasoning models.  
    Now, on the delivery side, I would add something more to it for the reason why doctors and, in fact, lawyers and coders will remain for a long time, and it’s because we still don’t understand how those models generalize. Like, at the end of the day, we are not able to tell you when they are confronted with a really new, novel situation, whether they will work or not. 
    Nobody is able to give you that guarantee. And I think until we understand this generalization better, we’re not going to be willing to just let the system in the wild without human supervision. 
    LEE: But don’t human doctors, human specialists … so, for example, a cardiologist sees a patient in a certain way that a nephrologist … 
    BUBECK: Yeah.
    LEE: … or an endocrinologist might not.
    BUBECK: That’s right. But another cardiologist will understand and, kind of, expect a certain level of generalization from their peer. And this, we just don’t have it with AI models. Now, of course, you’re exactly right. That generalization is also hard for humans. Like, if you have a human trained for one task and you put them into another task, then you don’t … you often don’t know.
    LEE: OK. You know, the podcast is focused on what’s happened over the last two years. But now, I’d like one provocative prediction about what you think the world of AI and medicine is going to be at some point in the future. You pick your timeframe. I don’t care if it’s two years or 20 years from now, but, you know, what do you think will be different about AI in medicine in that future than today? 
    BUBECK: Yeah, I think the deployment is going to accelerate soon. Like, we’re really not missing very much. There is this enormous capability overhang. Like, even if progress completely stopped, with current systems, we can do a lot more than what we’re doing right now. So I think this will … this has to be realized, you know, sooner rather than later. 
    And I think it’s probably dependent on these benchmarks and proper evaluation and tying this with regulation. So these are things that take time in human society and for good reason. But now we already are at two years; you know, give it another two years and it should be really …  
    LEE: Will AI prescribe your medicines? Write your prescriptions? 
    BUBECK: I think yes. I think yes. 
    LEE: OK. Bill? 
    GATES: Well, I think the next two years, we’ll have massive pilots, and so the amount of use of the AI, still in a copilot-type mode, you know, we should get millions of patient visits, you know, both in general medicine and in the mental health side, as well. And I think that’s going to build up both the data and the confidence to give the AI some additional autonomy. You know, are you going to let it talk to you at night when you’re panicked about your mental health with some ability to escalate?
    And, you know, I’ve gone so far as to tell politicians with national health systems that if they deploy AI appropriately, that the quality of care, the overload of the doctors, the improvement in the economics will be enough that their voters will be stunned because they just don’t expect this, and, you know, they could be reelectedjust on this one thing of fixing what is a very overloaded and economically challenged health system in these rich countries. 
    You know, my personal role is going to be to make sure that in the poorer countries, there isn’t some lag; in fact, in many cases, that we’ll be more aggressive because, you know, we’re comparing to having no access to doctors at all. And, you know, so I think whether it’s India or Africa, there’ll be lessons that are globally valuable because we need medical intelligence. And, you know, thank god AI is going to provide a lot of that. 
    LEE: Well, on that optimistic note, I think that’s a good way to end. Bill, Seb, really appreciate all of this.  
    I think the most fundamental prediction we made in the book is that AI would actually find its way into the practice of medicine, and I think that that at least has come true, maybe in different ways than we expected, but it’s come true, and I think it’ll only accelerate from here. So thanks again, both of you.  
    GATES: Yeah. Thanks, you guys. 
    BUBECK: Thank you, Peter. Thanks, Bill. 
    LEE: I just always feel such a sense of privilege to have a chance to interact and actually work with people like Bill and Sébastien.   
    With Bill, I’m always amazed at how practically minded he is. He’s really thinking about the nuts and bolts of what AI might be able to do for people, and his thoughts about underserved parts of the world, the idea that we might actually be able to empower people with access to expert medical knowledge, I think is both inspiring and amazing.  
    And then, Seb, Sébastien Bubeck, he’s just absolutely a brilliant mind. He has a really firm grip on the deep mathematics of artificial intelligence and brings that to bear in his research and development work. And where that mathematics takes him isn’t just into the nuts and bolts of algorithms but into philosophical questions about the nature of intelligence.  
    One of the things that Sébastien brought up was the state of evaluation of AI systems. And indeed, he was fairly critical in our conversation. But of course, the world of AI research and development is just moving so fast, and indeed, since we recorded our conversation, OpenAI, in fact, released a new evaluation metric that is directly relevant to medical applications, and that is something called HealthBench. And Microsoft Research also released a new evaluation approach or process called ADeLe.  
    HealthBench and ADeLe are examples of new approaches to evaluating AI models that are less about testing their knowledge and ability to pass multiple-choice exams and instead are evaluation approaches designed to assess how well AI models are able to complete tasks that actually arise every day in typical healthcare or biomedical research settings. These are examples of really important good work that speak to how well AI models work in the real world of healthcare and biomedical research and how well they can collaborate with human beings in those settings. 
    You know, I asked Bill and Seb to make some predictions about the future. You know, my own answer, I expect that we’re going to be able to use AI to change how we diagnose patients, change how we decide treatment options.  
    If you’re a doctor or a nurse and you encounter a patient, you’ll ask questions, do a physical exam, you know, call out for labs just like you do today, but then you’ll be able to engage with AI based on all of that data and just ask, you know, based on all the other people who have gone through the same experience, who have similar data, how were they diagnosed? How were they treated? What were their outcomes? And what does that mean for the patient I have right now? Some people call it the “patients like me” paradigm. And I think that’s going to become real because of AI within our lifetimes. That idea of really grounding the delivery in healthcare and medical practice through data and intelligence, I actually now don’t see any barriers to that future becoming real.  
    I’d like to extend another big thank you to Bill and Sébastien for their time. And to our listeners, as always, it’s a pleasure to have you along for the ride. I hope you’ll join us for our remaining conversations, as well as a second coauthor roundtable with Carey and Zak.  
    Until next time.  
    #how #reshaping #future #healthcare #medical
    How AI is reshaping the future of healthcare and medical research
    Transcript        PETER LEE: “In ‘The Little Black Bag,’ a classic science fiction story, a high-tech doctor’s kit of the future is accidentally transported back to the 1950s, into the shaky hands of a washed-up, alcoholic doctor. The ultimate medical tool, it redeems the doctor wielding it, allowing him to practice gratifyingly heroic medicine. … The tale ends badly for the doctor and his treacherous assistant, but it offered a picture of how advanced technology could transform medicine—powerful when it was written nearly 75 years ago and still so today. What would be the Al equivalent of that little black bag? At this moment when new capabilities are emerging, how do we imagine them into medicine?”           This is The AI Revolution in Medicine, Revisited. I’m your host, Peter Lee.    Shortly after OpenAI’s GPT-4 was publicly released, Carey Goldberg, Dr. Zak Kohane, and I published The AI Revolution in Medicine to help educate the world of healthcare and medical research about the transformative impact this new generative AI technology could have. But because we wrote the book when GPT-4 was still a secret, we had to speculate. Now, two years later, what did we get right, and what did we get wrong?     In this series, we’ll talk to clinicians, patients, hospital administrators, and others to understand the reality of AI in the field and where we go from here.  The book passage I read at the top is from “Chapter 10: The Big Black Bag.”  In imagining AI in medicine, Carey, Zak, and I included in our book two fictional accounts. In the first, a medical resident consults GPT-4 on her personal phone as the patient in front of her crashes. Within seconds, it offers an alternate response based on recent literature. In the second account, a 90-year-old woman with several chronic conditions is living independently and receiving near-constant medical support from an AI aide.    In our conversations with the guests we’ve spoken to so far, we’ve caught a glimpse of these predicted futures, seeing how clinicians and patients are actually using AI today and how developers are leveraging the technology in the healthcare products and services they’re creating. In fact, that first fictional account isn’t so fictional after all, as most of the doctors in the real world actually appear to be using AI at least occasionally—and sometimes much more than occasionally—to help in their daily clinical work. And as for the second fictional account, which is more of a science fiction account, it seems we are indeed on the verge of a new way of delivering and receiving healthcare, though the future is still very much open.  As we continue to examine the current state of AI in healthcare and its potential to transform the field, I’m pleased to welcome Bill Gates and Sébastien Bubeck.   Bill may be best known as the co-founder of Microsoft, having created the company with his childhood friend Paul Allen in 1975. He’s now the founder of Breakthrough Energy, which aims to advance clean energy innovation, and TerraPower, a company developing groundbreaking nuclear energy and science technologies. He also chairs the world’s largest philanthropic organization, the Gates Foundation, and focuses on solving a variety of health challenges around the globe and here at home.  Sébastien is a research lead at OpenAI. He was previously a distinguished scientist, vice president of AI, and a colleague of mine here at Microsoft, where his work included spearheading the development of the family of small language models known as Phi. While at Microsoft, he also coauthored the discussion-provoking 2023 paper “Sparks of Artificial General Intelligence,” which presented the results of early experiments with GPT-4 conducted by a small team from Microsoft Research.      Here’s my conversation with Bill Gates and Sébastien Bubeck.  LEE: Bill, welcome.  BILL GATES: Thank you.  LEE: Seb …  SÉBASTIEN BUBECK: Yeah. Hi, hi, Peter. Nice to be here.  LEE: You know, one of the things that I’ve been doing just to get the conversation warmed up is to talk about origin stories, and what I mean about origin stories is, you know, what was the first contact that you had with large language models or the concept of generative AI that convinced you or made you think that something really important was happening?  And so, Bill, I think I’ve heard the story about, you know, the time when the OpenAI folks—Sam Altman, Greg Brockman, and others—showed you something, but could we hear from you what those early encounters were like and what was going through your mind?   GATES: Well, I’d been visiting OpenAI soon after it was created to see things like GPT-2 and to see the little arm they had that was trying to match human manipulation and, you know, looking at their games like Dota that they were trying to get as good as human play. And honestly, I didn’t think the language model stuff they were doing, even when they got to GPT-3, would show the ability to learn, you know, in the same sense that a human reads a biology book and is able to take that knowledge and access it not only to pass a test but also to create new medicines.  And so my challenge to them was that if their LLM could get a five on the advanced placement biology test, then I would say, OK, it took biologic knowledge and encoded it in an accessible way and that I didn’t expect them to do that very quickly but it would be profound.   And it was only about six months after I challenged them to do that, that an early version of GPT-4 they brought up to a dinner at my house, and in fact, it answered most of the questions that night very well. The one it got totally wrong, we were … because it was so good, we kept thinking, Oh, we must be wrong. It turned out it was a math weaknessthat, you know, we later understood that that was an area of, weirdly, of incredible weakness of those early models. But, you know, that was when I realized, OK, the age of cheap intelligence was at its beginning.  LEE: Yeah. So I guess it seems like you had something similar to me in that my first encounters, I actually harbored some skepticism. Is it fair to say you were skeptical before that?  GATES: Well, the idea that we’ve figured out how to encode and access knowledge in this very deep sense without even understanding the nature of the encoding, …  LEE: Right.   GATES: … that is a bit weird.   LEE: Yeah.  GATES: We have an algorithm that creates the computation, but even say, OK, where is the president’s birthday stored in there? Where is this fact stored in there? The fact that even now when we’re playing around, getting a little bit more sense of it, it’s opaque to us what the semantic encoding is, it’s, kind of, amazing to me. I thought the invention of knowledge storage would be an explicit way of encoding knowledge, not an implicit statistical training.  LEE: Yeah, yeah. All right. So, Seb, you know, on this same topic, you know, I got—as we say at Microsoft—I got pulled into the tent.  BUBECK: Yes.   LEE: Because this was a very secret project. And then, um, I had the opportunity to select a small number of researchers in MSRto join and start investigating this thing seriously. And the first person I pulled in was you.  BUBECK: Yeah.  LEE: And so what were your first encounters? Because I actually don’t remember what happened then.  BUBECK: Oh, I remember it very well.My first encounter with GPT-4 was in a meeting with the two of you, actually. But my kind of first contact, the first moment where I realized that something was happening with generative AI, was before that. And I agree with Bill that I also wasn’t too impressed by GPT-3.  I though that it was kind of, you know, very naturally mimicking the web, sort of parroting what was written there in a nice way. Still in a way which seemed very impressive. But it wasn’t really intelligent in any way. But shortly after GPT-3, there was a model before GPT-4 that really shocked me, and this was the first image generation model, DALL-E 1.  So that was in 2021. And I will forever remember the press release of OpenAI where they had this prompt of an avocado chair and then you had this image of the avocado chair.And what really shocked me is that clearly the model kind of “understood” what is a chair, what is an avocado, and was able to merge those concepts.  So this was really, to me, the first moment where I saw some understanding in those models.   LEE: So this was, just to get the timing right, that was before I pulled you into the tent.  BUBECK: That was before. That was like a year before.  LEE: Right.   BUBECK: And now I will tell you how, you know, we went from that moment to the meeting with the two of you and GPT-4.  So once I saw this kind of understanding, I thought, OK, fine. It understands concept, but it’s still not able to reason. It cannot—as, you know, Bill was saying—it cannot learn from your document. It cannot reason.   So I set out to try to prove that. You know, this is what I was in the business of at the time, trying to prove things in mathematics. So I was trying to prove that basically autoregressive transformers could never reason. So I was trying to prove this. And after a year of work, I had something reasonable to show. And so I had the meeting with the two of you, and I had this example where I wanted to say, there is no way that an LLM is going to be able to do x.  And then as soon as I … I don’t know if you remember, Bill. But as soon as I said that, you said, oh, but wait a second. I had, you know, the OpenAI crew at my house recently, and they showed me a new model. Why don’t we ask this new model this question?   LEE: Yeah. BUBECK: And we did, and it solved it on the spot. And that really, honestly, just changed my life. Like, you know, I had been working for a year trying to say that this was impossible. And just right there, it was shown to be possible.   LEE:One of the very first things I got interested in—because I was really thinking a lot about healthcare—was healthcare and medicine.  And I don’t know if the two of you remember, but I ended up doing a lot of tests. I ran through, you know, step one and step two of the US Medical Licensing Exam. Did a whole bunch of other things. I wrote this big report. It was, you know, I can’t remember … a couple hundred pages.   And I needed to share this with someone. I didn’t … there weren’t too many people I could share it with. So I sent, I think, a copy to you, Bill. Sent a copy to you, Seb.   I hardly slept for about a week putting that report together. And, yeah, and I kept working on it. But I was far from alone. I think everyone who was in the tent, so to speak, in those early days was going through something pretty similar. All right. So I think … of course, a lot of what I put in the report also ended up being examples that made it into the book.  But the main purpose of this conversation isn’t to reminisce aboutor indulge in those reminiscences but to talk about what’s happening in healthcare and medicine. And, you know, as I said, we wrote this book. We did it very, very quickly. Seb, you helped. Bill, you know, you provided a review and some endorsements.  But, you know, honestly, we didn’t know what we were talking about because no one had access to this thing. And so we just made a bunch of guesses. So really, the whole thing I wanted to probe with the two of you is, now with two years of experience out in the world, what, you know, what do we think is happening today?  You know, is AI actually having an impact, positive or negative, on healthcare and medicine? And what do we now think is going to happen in the next two years, five years, or 10 years? And so I realize it’s a little bit too abstract to just ask it that way. So let me just try to narrow the discussion and guide us a little bit.   Um, the kind of administrative and clerical work, paperwork, around healthcare—and we made a lot of guesses about that—that appears to be going well, but, you know, Bill, I know we’ve discussed that sometimes that you think there ought to be a lot more going on. Do you have a viewpoint on how AI is actually finding its way into reducing paperwork?  GATES: Well, I’m stunned … I don’t think there should be a patient-doctor meeting where the AI is not sitting in and both transcribing, offering to help with the paperwork, and even making suggestions, although the doctor will be the one, you know, who makes the final decision about the diagnosis and whatever prescription gets done.   It’s so helpful. You know, when that patient goes home and their, you know, son who wants to understand what happened has some questions, that AI should be available to continue that conversation. And the way you can improve that experience and streamline things and, you know, involve the people who advise you. I don’t understand why that’s not more adopted, because there you still have the human in the loop making that final decision.  But even for, like, follow-up calls to make sure the patient did things, to understand if they have concerns and knowing when to escalate back to the doctor, the benefit is incredible. And, you know, that thing is ready for prime time. That paradigm is ready for prime time, in my view.  LEE: Yeah, there are some good products, but it seems like the number one use right now—and we kind of got this from some of the previous guests in previous episodes—is the use of AI just to respond to emails from patients.Does that make sense to you?  BUBECK: Yeah. So maybe I want to second what Bill was saying but maybe take a step back first. You know, two years ago, like, the concept of clinical scribes, which is one of the things that we’re talking about right now, it would have sounded, in fact, it sounded two years ago, borderline dangerous. Because everybody was worried about hallucinations. What happened if you have this AI listening in and then it transcribes, you know, something wrong?  Now, two years later, I think it’s mostly working. And in fact, it is not yet, you know, fully adopted. You’re right. But it is in production. It is used, you know, in many, many places. So this rate of progress is astounding because it wasn’t obvious that we would be able to overcome those obstacles of hallucination. It’s not to say that hallucinations are fully solved. In the case of the closed system, they are.   Now, I think more generally what’s going on in the background is that there is something that we, that certainly I, underestimated, which is this management overhead. So I think the reason why this is not adopted everywhere is really a training and teaching aspect. People need to be taught, like, those systems, how to interact with them.  And one example that I really like, a study that recently appeared where they tried to use ChatGPT for diagnosis and they were comparing doctors without and with ChatGPT. And the amazing thing … so this was a set of cases where the accuracy of the doctors alone was around 75%. ChatGPT alone was 90%. So that’s already kind of mind blowing. But then the kicker is that doctors with ChatGPT was 80%.   Intelligence alone is not enough. It’s also how it’s presented, how you interact with it. And ChatGPT, it’s an amazing tool. Obviously, I absolutely love it. But it’s not … you don’t want a doctor to have to type in, you know, prompts and use it that way.  It should be, as Bill was saying, kind of running continuously in the background, sending you notifications. And you have to be really careful of the rate at which those notifications are being sent. Because if they are too frequent, then the doctor will learn to ignore them. So you have to … all of those things matter, in fact, at least as much as the level of intelligence of the machine.  LEE: One of the things I think about, Bill, in that scenario that you described, doctors do some thinking about the patient when they write the note. So, you know, I’m always a little uncertain whether it’s actually … you know, you wouldn’t necessarily want to fully automate this, I don’t think. Or at least there needs to be some prompt to the doctor to make sure that the doctor puts some thought into what happened in the encounter with the patient. Does that make sense to you at all?  GATES: At this stage, you know, I’d still put the onus on the doctor to write the conclusions and the summary and not delegate that.  The tradeoffs you make a little bit are somewhat dependent on the situation you’re in. If you’re in Africa, So, yes, the doctor’s still going to have to do a lot of work, but just the quality of letting the patient and the people around them interact and ask questions and have things explained, that alone is such a quality improvement. It’s mind blowing.   LEE: So since you mentioned, you know, Africa—and, of course, this touches on the mission and some of the priorities of the Gates Foundation and this idea of democratization of access to expert medical care—what’s the most interesting stuff going on right now? Are there people and organizations or technologies that are impressing you or that you’re tracking?  GATES: Yeah. So the Gates Foundation has given out a lot of grants to people in Africa doing education, agriculture but more healthcare examples than anything. And the way these things start off, they often start out either being patient-centric in a narrow situation, like, OK, I’m a pregnant woman; talk to me. Or, I have infectious disease symptoms; talk to me. Or they’re connected to a health worker where they’re helping that worker get their job done. And we have lots of pilots out, you know, in both of those cases.   The dream would be eventually to have the thing the patient consults be so broad that it’s like having a doctor available who understands the local things.   LEE: Right.   GATES: We’re not there yet. But over the next two or three years, you know, particularly given the worsening financial constraints against African health systems, where the withdrawal of money has been dramatic, you know, figuring out how to take this—what I sometimes call “free intelligence”—and build a quality health system around that, we will have to be more radical in low-income countries than any rich country is ever going to be.   LEE: Also, there’s maybe a different regulatory environment, so some of those things maybe are easier? Because right now, I think the world hasn’t figured out how to and whether to regulate, let’s say, an AI that might give a medical diagnosis or write a prescription for a medication.  BUBECK: Yeah. I think one issue with this, and it’s also slowing down the deployment of AI in healthcare more generally, is a lack of proper benchmark. Because, you know, you were mentioning the USMLE, for example. That’s a great test to test human beings and their knowledge of healthcare and medicine. But it’s not a great test to give to an AI.  It’s not asking the right questions. So finding what are the right questions to test whether an AI system is ready to give diagnosis in a constrained setting, that’s a very, very important direction, which to my surprise, is not yet accelerating at the rate that I was hoping for.  LEE: OK, so that gives me an excuse to get more now into the core AI tech because something I’ve discussed with both of you is this issue of what are the right tests. And you both know the very first test I give to any new spin of an LLM is I present a patient, the results—a mythical patient—the results of my physical exam, my mythical physical exam. Maybe some results of some initial labs. And then I present or propose a differential diagnosis. And if you’re not in medicine, a differential diagnosis you can just think of as a prioritized list of the possible diagnoses that fit with all that data. And in that proposed differential, I always intentionally make two mistakes.  I make a textbook technical error in one of the possible elements of the differential diagnosis, and I have an error of omission. And, you know, I just want to know, does the LLM understand what I’m talking about? And all the good ones out there do now. But then I want to know, can it spot the errors? And then most importantly, is it willing to tell me I’m wrong, that I’ve made a mistake?   That last piece seems really hard for AI today. And so let me ask you first, Seb, because at the time of this taping, of course, there was a new spin of GPT-4o last week that became overly sycophantic. In other words, it was actually prone in that test of mine not only to not tell me I’m wrong, but it actually praised me for the creativity of my differential.What’s up with that?  BUBECK: Yeah, I guess it’s a testament to the fact that training those models is still more of an art than a science. So it’s a difficult job. Just to be clear with the audience, we have rolled back thatversion of GPT-4o, so now we don’t have the sycophant version out there.  Yeah, no, it’s a really difficult question. It has to do … as you said, it’s very technical. It has to do with the post-training and how, like, where do you nudge the model? So, you know, there is this very classical by now technique called RLHF, where you push the model in the direction of a certain reward model. So the reward model is just telling the model, you know, what behavior is good, what behavior is bad.  But this reward model is itself an LLM, and, you know, Bill was saying at the very beginning of the conversation that we don’t really understand how those LLMs deal with concepts like, you know, where is the capital of France located? Things like that. It is the same thing for this reward model. We don’t know why it says that it prefers one output to another, and whether this is correlated with some sycophancy is, you know, something that we discovered basically just now. That if you push too hard in optimization on this reward model, you will get a sycophant model.  So it’s kind of … what I’m trying to say is we became too good at what we were doing, and we ended up, in fact, in a trap of the reward model.  LEE: I mean, you do want … it’s a difficult balance because you do want models to follow your desires and …  BUBECK: It’s a very difficult, very difficult balance.  LEE: So this brings up then the following question for me, which is the extent to which we think we’ll need to have specially trained models for things. So let me start with you, Bill. Do you have a point of view on whether we will need to, you know, quote-unquote take AI models to med school? Have them specially trained? Like, if you were going to deploy something to give medical care in underserved parts of the world, do we need to do something special to create those models?  GATES: We certainly need to teach them the African languages and the unique dialects so that the multimedia interactions are very high quality. We certainly need to teach them the disease prevalence and unique disease patterns like, you know, neglected tropical diseases and malaria. So we need to gather a set of facts that somebody trying to go for a US customer base, you know, wouldn’t necessarily have that in there.  Those two things are actually very straightforward because the additional training time is small. I’d say for the next few years, we’ll also need to do reinforcement learning about the context of being a doctor and how important certain behaviors are. Humans learn over the course of their life to some degree that, I’m in a different context and the way I behave in terms of being willing to criticize or be nice, you know, how important is it? Who’s here? What’s my relationship to them?   Right now, these machines don’t have that broad social experience. And so if you know it’s going to be used for health things, a lot of reinforcement learning of the very best humans in that context would still be valuable. Eventually, the models will, having read all the literature of the world about good doctors, bad doctors, it’ll understand as soon as you say, “I want you to be a doctor diagnosing somebody.” All of the implicit reinforcement that fits that situation, you know, will be there. LEE: Yeah. GATES: And so I hope three years from now, we don’t have to do that reinforcement learning. But today, for any medical context, you would want a lot of data to reinforce tone, willingness to say things when, you know, there might be something significant at stake.  LEE: Yeah. So, you know, something Bill said, kind of, reminds me of another thing that I think we missed, which is, the context also … and the specialization also pertains to different, I guess, what we still call “modes,” although I don’t know if the idea of multimodal is the same as it was two years ago. But, you know, what do you make of all of the hubbub around—in fact, within Microsoft Research, this is a big deal, but I think we’re far from alone—you know, medical images and vision, video, proteins and molecules, cell, you know, cellular data and so on.  BUBECK: Yeah. OK. So there is a lot to say to everything … to the last, you know, couple of minutes. Maybe on the specialization aspect, you know, I think there is, hiding behind this, a really fundamental scientific question of whether eventually we have a singular AGIthat kind of knows everything and you can just put, you know, explain your own context and it will just get it and understand everything.  That’s one vision. I have to say, I don’t particularly believe in this vision. In fact, we humans are not like that at all. I think, hopefully, we are general intelligences, yet we have to specialize a lot. And, you know, I did myself a lot of RL, reinforcement learning, on mathematics. Like, that’s what I did, you know, spent a lot of time doing that. And I didn’t improve on other aspects. You know, in fact, I probably degraded in other aspects.So it’s … I think it’s an important example to have in mind.  LEE: I think I might disagree with you on that, though, because, like, doesn’t a model have to see both good science and bad science in order to be able to gain the ability to discern between the two?  BUBECK: Yeah, no, that absolutely. I think there is value in seeing the generality, in having a very broad base. But then you, kind of, specialize on verticals. And this is where also, you know, open-weights model, which we haven’t talked about yet, are really important because they allow you to provide this broad base to everyone. And then you can specialize on top of it.  LEE: So we have about three hours of stuff to talk about, but our time is actually running low. BUBECK: Yes, yes, yes.   LEE: So I think I want … there’s a more provocative question. It’s almost a silly question, but I need to ask it of the two of you, which is, is there a future, you know, where AI replaces doctors or replaces, you know, medical specialties that we have today? So what does the world look like, say, five years from now?  GATES: Well, it’s important to distinguish healthcare discovery activity from healthcare delivery activity. We focused mostly on delivery. I think it’s very much within the realm of possibility that the AI is not only accelerating healthcare discovery but substituting for a lot of the roles of, you know, I’m an organic chemist, or I run various types of assays. I can see those, which are, you know, testable-output-type jobs but with still very high value, I can see, you know, some replacement in those areas before the doctor.   The doctor, still understanding the human condition and long-term dialogues, you know, they’ve had a lifetime of reinforcement of that, particularly when you get into areas like mental health. So I wouldn’t say in five years, either people will choose to adopt it, but it will be profound that there’ll be this nearly free intelligence that can do follow-up, that can help you, you know, make sure you went through different possibilities.  And so I’d say, yes, we’ll have doctors, but I’d say healthcare will be massively transformed in its quality and in efficiency by AI in that time period.  LEE: Is there a comparison, useful comparison, say, between doctors and, say, programmers, computer programmers, or doctors and, I don’t know, lawyers?  GATES: Programming is another one that has, kind of, a mathematical correctness to it, you know, and so the objective function that you’re trying to reinforce to, as soon as you can understand the state machines, you can have something that’s “checkable”; that’s correct. So I think programming, you know, which is weird to say, that the machine will beat us at most programming tasks before we let it take over roles that have deep empathy, you know, physical presence and social understanding in them.  LEE: Yeah. By the way, you know, I fully expect in five years that AI will produce mathematical proofs that are checkable for validity, easily checkable, because they’ll be written in a proof-checking language like Lean or something but will be so complex that no human mathematician can understand them. I expect that to happen.   I can imagine in some fields, like cellular biology, we could have the same situation in the future because the molecular pathways, the chemistry, biochemistry of human cells or living cells is as complex as any mathematics, and so it seems possible that we may be in a state where in wet lab, we see, Oh yeah, this actually works, but no one can understand why.  BUBECK: Yeah, absolutely. I mean, I think I really agree with Bill’s distinction of the discovery and the delivery, and indeed, the discovery’s when you can check things, and at the end, there is an artifact that you can verify. You know, you can run the protocol in the wet lab and seeproduced what you wanted. So I absolutely agree with that.   And in fact, you know, we don’t have to talk five years from now. I don’t know if you know, but just recently, there was a paper that was published on a scientific discovery using o3- mini. So this is really amazing. And, you know, just very quickly, just so people know, it was about this statistical physics model, the frustrated Potts model, which has to do with coloring, and basically, the case of three colors, like, more than two colors was open for a long time, and o3 was able to reduce the case of three colors to two colors.   LEE: Yeah.  BUBECK: Which is just, like, astounding. And this is not … this is now. This is happening right now. So this is something that I personally didn’t expect it would happen so quickly, and it’s due to those reasoning models.   Now, on the delivery side, I would add something more to it for the reason why doctors and, in fact, lawyers and coders will remain for a long time, and it’s because we still don’t understand how those models generalize. Like, at the end of the day, we are not able to tell you when they are confronted with a really new, novel situation, whether they will work or not.  Nobody is able to give you that guarantee. And I think until we understand this generalization better, we’re not going to be willing to just let the system in the wild without human supervision.  LEE: But don’t human doctors, human specialists … so, for example, a cardiologist sees a patient in a certain way that a nephrologist …  BUBECK: Yeah. LEE: … or an endocrinologist might not. BUBECK: That’s right. But another cardiologist will understand and, kind of, expect a certain level of generalization from their peer. And this, we just don’t have it with AI models. Now, of course, you’re exactly right. That generalization is also hard for humans. Like, if you have a human trained for one task and you put them into another task, then you don’t … you often don’t know. LEE: OK. You know, the podcast is focused on what’s happened over the last two years. But now, I’d like one provocative prediction about what you think the world of AI and medicine is going to be at some point in the future. You pick your timeframe. I don’t care if it’s two years or 20 years from now, but, you know, what do you think will be different about AI in medicine in that future than today?  BUBECK: Yeah, I think the deployment is going to accelerate soon. Like, we’re really not missing very much. There is this enormous capability overhang. Like, even if progress completely stopped, with current systems, we can do a lot more than what we’re doing right now. So I think this will … this has to be realized, you know, sooner rather than later.  And I think it’s probably dependent on these benchmarks and proper evaluation and tying this with regulation. So these are things that take time in human society and for good reason. But now we already are at two years; you know, give it another two years and it should be really …   LEE: Will AI prescribe your medicines? Write your prescriptions?  BUBECK: I think yes. I think yes.  LEE: OK. Bill?  GATES: Well, I think the next two years, we’ll have massive pilots, and so the amount of use of the AI, still in a copilot-type mode, you know, we should get millions of patient visits, you know, both in general medicine and in the mental health side, as well. And I think that’s going to build up both the data and the confidence to give the AI some additional autonomy. You know, are you going to let it talk to you at night when you’re panicked about your mental health with some ability to escalate? And, you know, I’ve gone so far as to tell politicians with national health systems that if they deploy AI appropriately, that the quality of care, the overload of the doctors, the improvement in the economics will be enough that their voters will be stunned because they just don’t expect this, and, you know, they could be reelectedjust on this one thing of fixing what is a very overloaded and economically challenged health system in these rich countries.  You know, my personal role is going to be to make sure that in the poorer countries, there isn’t some lag; in fact, in many cases, that we’ll be more aggressive because, you know, we’re comparing to having no access to doctors at all. And, you know, so I think whether it’s India or Africa, there’ll be lessons that are globally valuable because we need medical intelligence. And, you know, thank god AI is going to provide a lot of that.  LEE: Well, on that optimistic note, I think that’s a good way to end. Bill, Seb, really appreciate all of this.   I think the most fundamental prediction we made in the book is that AI would actually find its way into the practice of medicine, and I think that that at least has come true, maybe in different ways than we expected, but it’s come true, and I think it’ll only accelerate from here. So thanks again, both of you.   GATES: Yeah. Thanks, you guys.  BUBECK: Thank you, Peter. Thanks, Bill.  LEE: I just always feel such a sense of privilege to have a chance to interact and actually work with people like Bill and Sébastien.    With Bill, I’m always amazed at how practically minded he is. He’s really thinking about the nuts and bolts of what AI might be able to do for people, and his thoughts about underserved parts of the world, the idea that we might actually be able to empower people with access to expert medical knowledge, I think is both inspiring and amazing.   And then, Seb, Sébastien Bubeck, he’s just absolutely a brilliant mind. He has a really firm grip on the deep mathematics of artificial intelligence and brings that to bear in his research and development work. And where that mathematics takes him isn’t just into the nuts and bolts of algorithms but into philosophical questions about the nature of intelligence.   One of the things that Sébastien brought up was the state of evaluation of AI systems. And indeed, he was fairly critical in our conversation. But of course, the world of AI research and development is just moving so fast, and indeed, since we recorded our conversation, OpenAI, in fact, released a new evaluation metric that is directly relevant to medical applications, and that is something called HealthBench. And Microsoft Research also released a new evaluation approach or process called ADeLe.   HealthBench and ADeLe are examples of new approaches to evaluating AI models that are less about testing their knowledge and ability to pass multiple-choice exams and instead are evaluation approaches designed to assess how well AI models are able to complete tasks that actually arise every day in typical healthcare or biomedical research settings. These are examples of really important good work that speak to how well AI models work in the real world of healthcare and biomedical research and how well they can collaborate with human beings in those settings.  You know, I asked Bill and Seb to make some predictions about the future. You know, my own answer, I expect that we’re going to be able to use AI to change how we diagnose patients, change how we decide treatment options.   If you’re a doctor or a nurse and you encounter a patient, you’ll ask questions, do a physical exam, you know, call out for labs just like you do today, but then you’ll be able to engage with AI based on all of that data and just ask, you know, based on all the other people who have gone through the same experience, who have similar data, how were they diagnosed? How were they treated? What were their outcomes? And what does that mean for the patient I have right now? Some people call it the “patients like me” paradigm. And I think that’s going to become real because of AI within our lifetimes. That idea of really grounding the delivery in healthcare and medical practice through data and intelligence, I actually now don’t see any barriers to that future becoming real.   I’d like to extend another big thank you to Bill and Sébastien for their time. And to our listeners, as always, it’s a pleasure to have you along for the ride. I hope you’ll join us for our remaining conversations, as well as a second coauthor roundtable with Carey and Zak.   Until next time.   #how #reshaping #future #healthcare #medical
    WWW.MICROSOFT.COM
    How AI is reshaping the future of healthcare and medical research
    Transcript [MUSIC]      [BOOK PASSAGE]   PETER LEE: “In ‘The Little Black Bag,’ a classic science fiction story, a high-tech doctor’s kit of the future is accidentally transported back to the 1950s, into the shaky hands of a washed-up, alcoholic doctor. The ultimate medical tool, it redeems the doctor wielding it, allowing him to practice gratifyingly heroic medicine. … The tale ends badly for the doctor and his treacherous assistant, but it offered a picture of how advanced technology could transform medicine—powerful when it was written nearly 75 years ago and still so today. What would be the Al equivalent of that little black bag? At this moment when new capabilities are emerging, how do we imagine them into medicine?”   [END OF BOOK PASSAGE]     [THEME MUSIC]     This is The AI Revolution in Medicine, Revisited. I’m your host, Peter Lee.    Shortly after OpenAI’s GPT-4 was publicly released, Carey Goldberg, Dr. Zak Kohane, and I published The AI Revolution in Medicine to help educate the world of healthcare and medical research about the transformative impact this new generative AI technology could have. But because we wrote the book when GPT-4 was still a secret, we had to speculate. Now, two years later, what did we get right, and what did we get wrong?     In this series, we’ll talk to clinicians, patients, hospital administrators, and others to understand the reality of AI in the field and where we go from here.   [THEME MUSIC FADES] The book passage I read at the top is from “Chapter 10: The Big Black Bag.”  In imagining AI in medicine, Carey, Zak, and I included in our book two fictional accounts. In the first, a medical resident consults GPT-4 on her personal phone as the patient in front of her crashes. Within seconds, it offers an alternate response based on recent literature. In the second account, a 90-year-old woman with several chronic conditions is living independently and receiving near-constant medical support from an AI aide.    In our conversations with the guests we’ve spoken to so far, we’ve caught a glimpse of these predicted futures, seeing how clinicians and patients are actually using AI today and how developers are leveraging the technology in the healthcare products and services they’re creating. In fact, that first fictional account isn’t so fictional after all, as most of the doctors in the real world actually appear to be using AI at least occasionally—and sometimes much more than occasionally—to help in their daily clinical work. And as for the second fictional account, which is more of a science fiction account, it seems we are indeed on the verge of a new way of delivering and receiving healthcare, though the future is still very much open.  As we continue to examine the current state of AI in healthcare and its potential to transform the field, I’m pleased to welcome Bill Gates and Sébastien Bubeck.   Bill may be best known as the co-founder of Microsoft, having created the company with his childhood friend Paul Allen in 1975. He’s now the founder of Breakthrough Energy, which aims to advance clean energy innovation, and TerraPower, a company developing groundbreaking nuclear energy and science technologies. He also chairs the world’s largest philanthropic organization, the Gates Foundation, and focuses on solving a variety of health challenges around the globe and here at home.  Sébastien is a research lead at OpenAI. He was previously a distinguished scientist, vice president of AI, and a colleague of mine here at Microsoft, where his work included spearheading the development of the family of small language models known as Phi. While at Microsoft, he also coauthored the discussion-provoking 2023 paper “Sparks of Artificial General Intelligence,” which presented the results of early experiments with GPT-4 conducted by a small team from Microsoft Research.    [TRANSITION MUSIC]   Here’s my conversation with Bill Gates and Sébastien Bubeck.  LEE: Bill, welcome.  BILL GATES: Thank you.  LEE: Seb …  SÉBASTIEN BUBECK: Yeah. Hi, hi, Peter. Nice to be here.  LEE: You know, one of the things that I’ve been doing just to get the conversation warmed up is to talk about origin stories, and what I mean about origin stories is, you know, what was the first contact that you had with large language models or the concept of generative AI that convinced you or made you think that something really important was happening?  And so, Bill, I think I’ve heard the story about, you know, the time when the OpenAI folks—Sam Altman, Greg Brockman, and others—showed you something, but could we hear from you what those early encounters were like and what was going through your mind?   GATES: Well, I’d been visiting OpenAI soon after it was created to see things like GPT-2 and to see the little arm they had that was trying to match human manipulation and, you know, looking at their games like Dota that they were trying to get as good as human play. And honestly, I didn’t think the language model stuff they were doing, even when they got to GPT-3, would show the ability to learn, you know, in the same sense that a human reads a biology book and is able to take that knowledge and access it not only to pass a test but also to create new medicines.  And so my challenge to them was that if their LLM could get a five on the advanced placement biology test, then I would say, OK, it took biologic knowledge and encoded it in an accessible way and that I didn’t expect them to do that very quickly but it would be profound.   And it was only about six months after I challenged them to do that, that an early version of GPT-4 they brought up to a dinner at my house, and in fact, it answered most of the questions that night very well. The one it got totally wrong, we were … because it was so good, we kept thinking, Oh, we must be wrong. It turned out it was a math weakness [LAUGHTER] that, you know, we later understood that that was an area of, weirdly, of incredible weakness of those early models. But, you know, that was when I realized, OK, the age of cheap intelligence was at its beginning.  LEE: Yeah. So I guess it seems like you had something similar to me in that my first encounters, I actually harbored some skepticism. Is it fair to say you were skeptical before that?  GATES: Well, the idea that we’ve figured out how to encode and access knowledge in this very deep sense without even understanding the nature of the encoding, …  LEE: Right.   GATES: … that is a bit weird.   LEE: Yeah.  GATES: We have an algorithm that creates the computation, but even say, OK, where is the president’s birthday stored in there? Where is this fact stored in there? The fact that even now when we’re playing around, getting a little bit more sense of it, it’s opaque to us what the semantic encoding is, it’s, kind of, amazing to me. I thought the invention of knowledge storage would be an explicit way of encoding knowledge, not an implicit statistical training.  LEE: Yeah, yeah. All right. So, Seb, you know, on this same topic, you know, I got—as we say at Microsoft—I got pulled into the tent. [LAUGHS]  BUBECK: Yes.   LEE: Because this was a very secret project. And then, um, I had the opportunity to select a small number of researchers in MSR [Microsoft Research] to join and start investigating this thing seriously. And the first person I pulled in was you.  BUBECK: Yeah.  LEE: And so what were your first encounters? Because I actually don’t remember what happened then.  BUBECK: Oh, I remember it very well. [LAUGHS] My first encounter with GPT-4 was in a meeting with the two of you, actually. But my kind of first contact, the first moment where I realized that something was happening with generative AI, was before that. And I agree with Bill that I also wasn’t too impressed by GPT-3.  I though that it was kind of, you know, very naturally mimicking the web, sort of parroting what was written there in a nice way. Still in a way which seemed very impressive. But it wasn’t really intelligent in any way. But shortly after GPT-3, there was a model before GPT-4 that really shocked me, and this was the first image generation model, DALL-E 1.  So that was in 2021. And I will forever remember the press release of OpenAI where they had this prompt of an avocado chair and then you had this image of the avocado chair. [LAUGHTER] And what really shocked me is that clearly the model kind of “understood” what is a chair, what is an avocado, and was able to merge those concepts.  So this was really, to me, the first moment where I saw some understanding in those models.   LEE: So this was, just to get the timing right, that was before I pulled you into the tent.  BUBECK: That was before. That was like a year before.  LEE: Right.   BUBECK: And now I will tell you how, you know, we went from that moment to the meeting with the two of you and GPT-4.  So once I saw this kind of understanding, I thought, OK, fine. It understands concept, but it’s still not able to reason. It cannot—as, you know, Bill was saying—it cannot learn from your document. It cannot reason.   So I set out to try to prove that. You know, this is what I was in the business of at the time, trying to prove things in mathematics. So I was trying to prove that basically autoregressive transformers could never reason. So I was trying to prove this. And after a year of work, I had something reasonable to show. And so I had the meeting with the two of you, and I had this example where I wanted to say, there is no way that an LLM is going to be able to do x.  And then as soon as I … I don’t know if you remember, Bill. But as soon as I said that, you said, oh, but wait a second. I had, you know, the OpenAI crew at my house recently, and they showed me a new model. Why don’t we ask this new model this question?   LEE: Yeah. BUBECK: And we did, and it solved it on the spot. And that really, honestly, just changed my life. Like, you know, I had been working for a year trying to say that this was impossible. And just right there, it was shown to be possible.   LEE: [LAUGHS] One of the very first things I got interested in—because I was really thinking a lot about healthcare—was healthcare and medicine.  And I don’t know if the two of you remember, but I ended up doing a lot of tests. I ran through, you know, step one and step two of the US Medical Licensing Exam. Did a whole bunch of other things. I wrote this big report. It was, you know, I can’t remember … a couple hundred pages.   And I needed to share this with someone. I didn’t … there weren’t too many people I could share it with. So I sent, I think, a copy to you, Bill. Sent a copy to you, Seb.   I hardly slept for about a week putting that report together. And, yeah, and I kept working on it. But I was far from alone. I think everyone who was in the tent, so to speak, in those early days was going through something pretty similar. All right. So I think … of course, a lot of what I put in the report also ended up being examples that made it into the book.  But the main purpose of this conversation isn’t to reminisce about [LAUGHS] or indulge in those reminiscences but to talk about what’s happening in healthcare and medicine. And, you know, as I said, we wrote this book. We did it very, very quickly. Seb, you helped. Bill, you know, you provided a review and some endorsements.  But, you know, honestly, we didn’t know what we were talking about because no one had access to this thing. And so we just made a bunch of guesses. So really, the whole thing I wanted to probe with the two of you is, now with two years of experience out in the world, what, you know, what do we think is happening today?  You know, is AI actually having an impact, positive or negative, on healthcare and medicine? And what do we now think is going to happen in the next two years, five years, or 10 years? And so I realize it’s a little bit too abstract to just ask it that way. So let me just try to narrow the discussion and guide us a little bit.   Um, the kind of administrative and clerical work, paperwork, around healthcare—and we made a lot of guesses about that—that appears to be going well, but, you know, Bill, I know we’ve discussed that sometimes that you think there ought to be a lot more going on. Do you have a viewpoint on how AI is actually finding its way into reducing paperwork?  GATES: Well, I’m stunned … I don’t think there should be a patient-doctor meeting where the AI is not sitting in and both transcribing, offering to help with the paperwork, and even making suggestions, although the doctor will be the one, you know, who makes the final decision about the diagnosis and whatever prescription gets done.   It’s so helpful. You know, when that patient goes home and their, you know, son who wants to understand what happened has some questions, that AI should be available to continue that conversation. And the way you can improve that experience and streamline things and, you know, involve the people who advise you. I don’t understand why that’s not more adopted, because there you still have the human in the loop making that final decision.  But even for, like, follow-up calls to make sure the patient did things, to understand if they have concerns and knowing when to escalate back to the doctor, the benefit is incredible. And, you know, that thing is ready for prime time. That paradigm is ready for prime time, in my view.  LEE: Yeah, there are some good products, but it seems like the number one use right now—and we kind of got this from some of the previous guests in previous episodes—is the use of AI just to respond to emails from patients. [LAUGHTER] Does that make sense to you?  BUBECK: Yeah. So maybe I want to second what Bill was saying but maybe take a step back first. You know, two years ago, like, the concept of clinical scribes, which is one of the things that we’re talking about right now, it would have sounded, in fact, it sounded two years ago, borderline dangerous. Because everybody was worried about hallucinations. What happened if you have this AI listening in and then it transcribes, you know, something wrong?  Now, two years later, I think it’s mostly working. And in fact, it is not yet, you know, fully adopted. You’re right. But it is in production. It is used, you know, in many, many places. So this rate of progress is astounding because it wasn’t obvious that we would be able to overcome those obstacles of hallucination. It’s not to say that hallucinations are fully solved. In the case of the closed system, they are.   Now, I think more generally what’s going on in the background is that there is something that we, that certainly I, underestimated, which is this management overhead. So I think the reason why this is not adopted everywhere is really a training and teaching aspect. People need to be taught, like, those systems, how to interact with them.  And one example that I really like, a study that recently appeared where they tried to use ChatGPT for diagnosis and they were comparing doctors without and with ChatGPT (opens in new tab). And the amazing thing … so this was a set of cases where the accuracy of the doctors alone was around 75%. ChatGPT alone was 90%. So that’s already kind of mind blowing. But then the kicker is that doctors with ChatGPT was 80%.   Intelligence alone is not enough. It’s also how it’s presented, how you interact with it. And ChatGPT, it’s an amazing tool. Obviously, I absolutely love it. But it’s not … you don’t want a doctor to have to type in, you know, prompts and use it that way.  It should be, as Bill was saying, kind of running continuously in the background, sending you notifications. And you have to be really careful of the rate at which those notifications are being sent. Because if they are too frequent, then the doctor will learn to ignore them. So you have to … all of those things matter, in fact, at least as much as the level of intelligence of the machine.  LEE: One of the things I think about, Bill, in that scenario that you described, doctors do some thinking about the patient when they write the note. So, you know, I’m always a little uncertain whether it’s actually … you know, you wouldn’t necessarily want to fully automate this, I don’t think. Or at least there needs to be some prompt to the doctor to make sure that the doctor puts some thought into what happened in the encounter with the patient. Does that make sense to you at all?  GATES: At this stage, you know, I’d still put the onus on the doctor to write the conclusions and the summary and not delegate that.  The tradeoffs you make a little bit are somewhat dependent on the situation you’re in. If you’re in Africa, So, yes, the doctor’s still going to have to do a lot of work, but just the quality of letting the patient and the people around them interact and ask questions and have things explained, that alone is such a quality improvement. It’s mind blowing.   LEE: So since you mentioned, you know, Africa—and, of course, this touches on the mission and some of the priorities of the Gates Foundation and this idea of democratization of access to expert medical care—what’s the most interesting stuff going on right now? Are there people and organizations or technologies that are impressing you or that you’re tracking?  GATES: Yeah. So the Gates Foundation has given out a lot of grants to people in Africa doing education, agriculture but more healthcare examples than anything. And the way these things start off, they often start out either being patient-centric in a narrow situation, like, OK, I’m a pregnant woman; talk to me. Or, I have infectious disease symptoms; talk to me. Or they’re connected to a health worker where they’re helping that worker get their job done. And we have lots of pilots out, you know, in both of those cases.   The dream would be eventually to have the thing the patient consults be so broad that it’s like having a doctor available who understands the local things.   LEE: Right.   GATES: We’re not there yet. But over the next two or three years, you know, particularly given the worsening financial constraints against African health systems, where the withdrawal of money has been dramatic, you know, figuring out how to take this—what I sometimes call “free intelligence”—and build a quality health system around that, we will have to be more radical in low-income countries than any rich country is ever going to be.   LEE: Also, there’s maybe a different regulatory environment, so some of those things maybe are easier? Because right now, I think the world hasn’t figured out how to and whether to regulate, let’s say, an AI that might give a medical diagnosis or write a prescription for a medication.  BUBECK: Yeah. I think one issue with this, and it’s also slowing down the deployment of AI in healthcare more generally, is a lack of proper benchmark. Because, you know, you were mentioning the USMLE [United States Medical Licensing Examination], for example. That’s a great test to test human beings and their knowledge of healthcare and medicine. But it’s not a great test to give to an AI.  It’s not asking the right questions. So finding what are the right questions to test whether an AI system is ready to give diagnosis in a constrained setting, that’s a very, very important direction, which to my surprise, is not yet accelerating at the rate that I was hoping for.  LEE: OK, so that gives me an excuse to get more now into the core AI tech because something I’ve discussed with both of you is this issue of what are the right tests. And you both know the very first test I give to any new spin of an LLM is I present a patient, the results—a mythical patient—the results of my physical exam, my mythical physical exam. Maybe some results of some initial labs. And then I present or propose a differential diagnosis. And if you’re not in medicine, a differential diagnosis you can just think of as a prioritized list of the possible diagnoses that fit with all that data. And in that proposed differential, I always intentionally make two mistakes.  I make a textbook technical error in one of the possible elements of the differential diagnosis, and I have an error of omission. And, you know, I just want to know, does the LLM understand what I’m talking about? And all the good ones out there do now. But then I want to know, can it spot the errors? And then most importantly, is it willing to tell me I’m wrong, that I’ve made a mistake?   That last piece seems really hard for AI today. And so let me ask you first, Seb, because at the time of this taping, of course, there was a new spin of GPT-4o last week that became overly sycophantic. In other words, it was actually prone in that test of mine not only to not tell me I’m wrong, but it actually praised me for the creativity of my differential. [LAUGHTER] What’s up with that?  BUBECK: Yeah, I guess it’s a testament to the fact that training those models is still more of an art than a science. So it’s a difficult job. Just to be clear with the audience, we have rolled back that [LAUGHS] version of GPT-4o, so now we don’t have the sycophant version out there.  Yeah, no, it’s a really difficult question. It has to do … as you said, it’s very technical. It has to do with the post-training and how, like, where do you nudge the model? So, you know, there is this very classical by now technique called RLHF [reinforcement learning from human feedback], where you push the model in the direction of a certain reward model. So the reward model is just telling the model, you know, what behavior is good, what behavior is bad.  But this reward model is itself an LLM, and, you know, Bill was saying at the very beginning of the conversation that we don’t really understand how those LLMs deal with concepts like, you know, where is the capital of France located? Things like that. It is the same thing for this reward model. We don’t know why it says that it prefers one output to another, and whether this is correlated with some sycophancy is, you know, something that we discovered basically just now. That if you push too hard in optimization on this reward model, you will get a sycophant model.  So it’s kind of … what I’m trying to say is we became too good at what we were doing, and we ended up, in fact, in a trap of the reward model.  LEE: I mean, you do want … it’s a difficult balance because you do want models to follow your desires and …  BUBECK: It’s a very difficult, very difficult balance.  LEE: So this brings up then the following question for me, which is the extent to which we think we’ll need to have specially trained models for things. So let me start with you, Bill. Do you have a point of view on whether we will need to, you know, quote-unquote take AI models to med school? Have them specially trained? Like, if you were going to deploy something to give medical care in underserved parts of the world, do we need to do something special to create those models?  GATES: We certainly need to teach them the African languages and the unique dialects so that the multimedia interactions are very high quality. We certainly need to teach them the disease prevalence and unique disease patterns like, you know, neglected tropical diseases and malaria. So we need to gather a set of facts that somebody trying to go for a US customer base, you know, wouldn’t necessarily have that in there.  Those two things are actually very straightforward because the additional training time is small. I’d say for the next few years, we’ll also need to do reinforcement learning about the context of being a doctor and how important certain behaviors are. Humans learn over the course of their life to some degree that, I’m in a different context and the way I behave in terms of being willing to criticize or be nice, you know, how important is it? Who’s here? What’s my relationship to them?   Right now, these machines don’t have that broad social experience. And so if you know it’s going to be used for health things, a lot of reinforcement learning of the very best humans in that context would still be valuable. Eventually, the models will, having read all the literature of the world about good doctors, bad doctors, it’ll understand as soon as you say, “I want you to be a doctor diagnosing somebody.” All of the implicit reinforcement that fits that situation, you know, will be there. LEE: Yeah. GATES: And so I hope three years from now, we don’t have to do that reinforcement learning. But today, for any medical context, you would want a lot of data to reinforce tone, willingness to say things when, you know, there might be something significant at stake.  LEE: Yeah. So, you know, something Bill said, kind of, reminds me of another thing that I think we missed, which is, the context also … and the specialization also pertains to different, I guess, what we still call “modes,” although I don’t know if the idea of multimodal is the same as it was two years ago. But, you know, what do you make of all of the hubbub around—in fact, within Microsoft Research, this is a big deal, but I think we’re far from alone—you know, medical images and vision, video, proteins and molecules, cell, you know, cellular data and so on.  BUBECK: Yeah. OK. So there is a lot to say to everything … to the last, you know, couple of minutes. Maybe on the specialization aspect, you know, I think there is, hiding behind this, a really fundamental scientific question of whether eventually we have a singular AGI [artificial general intelligence] that kind of knows everything and you can just put, you know, explain your own context and it will just get it and understand everything.  That’s one vision. I have to say, I don’t particularly believe in this vision. In fact, we humans are not like that at all. I think, hopefully, we are general intelligences, yet we have to specialize a lot. And, you know, I did myself a lot of RL, reinforcement learning, on mathematics. Like, that’s what I did, you know, spent a lot of time doing that. And I didn’t improve on other aspects. You know, in fact, I probably degraded in other aspects. [LAUGHTER] So it’s … I think it’s an important example to have in mind.  LEE: I think I might disagree with you on that, though, because, like, doesn’t a model have to see both good science and bad science in order to be able to gain the ability to discern between the two?  BUBECK: Yeah, no, that absolutely. I think there is value in seeing the generality, in having a very broad base. But then you, kind of, specialize on verticals. And this is where also, you know, open-weights model, which we haven’t talked about yet, are really important because they allow you to provide this broad base to everyone. And then you can specialize on top of it.  LEE: So we have about three hours of stuff to talk about, but our time is actually running low. BUBECK: Yes, yes, yes.   LEE: So I think I want … there’s a more provocative question. It’s almost a silly question, but I need to ask it of the two of you, which is, is there a future, you know, where AI replaces doctors or replaces, you know, medical specialties that we have today? So what does the world look like, say, five years from now?  GATES: Well, it’s important to distinguish healthcare discovery activity from healthcare delivery activity. We focused mostly on delivery. I think it’s very much within the realm of possibility that the AI is not only accelerating healthcare discovery but substituting for a lot of the roles of, you know, I’m an organic chemist, or I run various types of assays. I can see those, which are, you know, testable-output-type jobs but with still very high value, I can see, you know, some replacement in those areas before the doctor.   The doctor, still understanding the human condition and long-term dialogues, you know, they’ve had a lifetime of reinforcement of that, particularly when you get into areas like mental health. So I wouldn’t say in five years, either people will choose to adopt it, but it will be profound that there’ll be this nearly free intelligence that can do follow-up, that can help you, you know, make sure you went through different possibilities.  And so I’d say, yes, we’ll have doctors, but I’d say healthcare will be massively transformed in its quality and in efficiency by AI in that time period.  LEE: Is there a comparison, useful comparison, say, between doctors and, say, programmers, computer programmers, or doctors and, I don’t know, lawyers?  GATES: Programming is another one that has, kind of, a mathematical correctness to it, you know, and so the objective function that you’re trying to reinforce to, as soon as you can understand the state machines, you can have something that’s “checkable”; that’s correct. So I think programming, you know, which is weird to say, that the machine will beat us at most programming tasks before we let it take over roles that have deep empathy, you know, physical presence and social understanding in them.  LEE: Yeah. By the way, you know, I fully expect in five years that AI will produce mathematical proofs that are checkable for validity, easily checkable, because they’ll be written in a proof-checking language like Lean or something but will be so complex that no human mathematician can understand them. I expect that to happen.   I can imagine in some fields, like cellular biology, we could have the same situation in the future because the molecular pathways, the chemistry, biochemistry of human cells or living cells is as complex as any mathematics, and so it seems possible that we may be in a state where in wet lab, we see, Oh yeah, this actually works, but no one can understand why.  BUBECK: Yeah, absolutely. I mean, I think I really agree with Bill’s distinction of the discovery and the delivery, and indeed, the discovery’s when you can check things, and at the end, there is an artifact that you can verify. You know, you can run the protocol in the wet lab and see [if you have] produced what you wanted. So I absolutely agree with that.   And in fact, you know, we don’t have to talk five years from now. I don’t know if you know, but just recently, there was a paper that was published on a scientific discovery using o3- mini (opens in new tab). So this is really amazing. And, you know, just very quickly, just so people know, it was about this statistical physics model, the frustrated Potts model, which has to do with coloring, and basically, the case of three colors, like, more than two colors was open for a long time, and o3 was able to reduce the case of three colors to two colors.   LEE: Yeah.  BUBECK: Which is just, like, astounding. And this is not … this is now. This is happening right now. So this is something that I personally didn’t expect it would happen so quickly, and it’s due to those reasoning models.   Now, on the delivery side, I would add something more to it for the reason why doctors and, in fact, lawyers and coders will remain for a long time, and it’s because we still don’t understand how those models generalize. Like, at the end of the day, we are not able to tell you when they are confronted with a really new, novel situation, whether they will work or not.  Nobody is able to give you that guarantee. And I think until we understand this generalization better, we’re not going to be willing to just let the system in the wild without human supervision.  LEE: But don’t human doctors, human specialists … so, for example, a cardiologist sees a patient in a certain way that a nephrologist …  BUBECK: Yeah. LEE: … or an endocrinologist might not. BUBECK: That’s right. But another cardiologist will understand and, kind of, expect a certain level of generalization from their peer. And this, we just don’t have it with AI models. Now, of course, you’re exactly right. That generalization is also hard for humans. Like, if you have a human trained for one task and you put them into another task, then you don’t … you often don’t know. LEE: OK. You know, the podcast is focused on what’s happened over the last two years. But now, I’d like one provocative prediction about what you think the world of AI and medicine is going to be at some point in the future. You pick your timeframe. I don’t care if it’s two years or 20 years from now, but, you know, what do you think will be different about AI in medicine in that future than today?  BUBECK: Yeah, I think the deployment is going to accelerate soon. Like, we’re really not missing very much. There is this enormous capability overhang. Like, even if progress completely stopped, with current systems, we can do a lot more than what we’re doing right now. So I think this will … this has to be realized, you know, sooner rather than later.  And I think it’s probably dependent on these benchmarks and proper evaluation and tying this with regulation. So these are things that take time in human society and for good reason. But now we already are at two years; you know, give it another two years and it should be really …   LEE: Will AI prescribe your medicines? Write your prescriptions?  BUBECK: I think yes. I think yes.  LEE: OK. Bill?  GATES: Well, I think the next two years, we’ll have massive pilots, and so the amount of use of the AI, still in a copilot-type mode, you know, we should get millions of patient visits, you know, both in general medicine and in the mental health side, as well. And I think that’s going to build up both the data and the confidence to give the AI some additional autonomy. You know, are you going to let it talk to you at night when you’re panicked about your mental health with some ability to escalate? And, you know, I’ve gone so far as to tell politicians with national health systems that if they deploy AI appropriately, that the quality of care, the overload of the doctors, the improvement in the economics will be enough that their voters will be stunned because they just don’t expect this, and, you know, they could be reelected [LAUGHTER] just on this one thing of fixing what is a very overloaded and economically challenged health system in these rich countries.  You know, my personal role is going to be to make sure that in the poorer countries, there isn’t some lag; in fact, in many cases, that we’ll be more aggressive because, you know, we’re comparing to having no access to doctors at all. And, you know, so I think whether it’s India or Africa, there’ll be lessons that are globally valuable because we need medical intelligence. And, you know, thank god AI is going to provide a lot of that.  LEE: Well, on that optimistic note, I think that’s a good way to end. Bill, Seb, really appreciate all of this.   I think the most fundamental prediction we made in the book is that AI would actually find its way into the practice of medicine, and I think that that at least has come true, maybe in different ways than we expected, but it’s come true, and I think it’ll only accelerate from here. So thanks again, both of you.  [TRANSITION MUSIC]  GATES: Yeah. Thanks, you guys.  BUBECK: Thank you, Peter. Thanks, Bill.  LEE: I just always feel such a sense of privilege to have a chance to interact and actually work with people like Bill and Sébastien.    With Bill, I’m always amazed at how practically minded he is. He’s really thinking about the nuts and bolts of what AI might be able to do for people, and his thoughts about underserved parts of the world, the idea that we might actually be able to empower people with access to expert medical knowledge, I think is both inspiring and amazing.   And then, Seb, Sébastien Bubeck, he’s just absolutely a brilliant mind. He has a really firm grip on the deep mathematics of artificial intelligence and brings that to bear in his research and development work. And where that mathematics takes him isn’t just into the nuts and bolts of algorithms but into philosophical questions about the nature of intelligence.   One of the things that Sébastien brought up was the state of evaluation of AI systems. And indeed, he was fairly critical in our conversation. But of course, the world of AI research and development is just moving so fast, and indeed, since we recorded our conversation, OpenAI, in fact, released a new evaluation metric that is directly relevant to medical applications, and that is something called HealthBench. And Microsoft Research also released a new evaluation approach or process called ADeLe.   HealthBench and ADeLe are examples of new approaches to evaluating AI models that are less about testing their knowledge and ability to pass multiple-choice exams and instead are evaluation approaches designed to assess how well AI models are able to complete tasks that actually arise every day in typical healthcare or biomedical research settings. These are examples of really important good work that speak to how well AI models work in the real world of healthcare and biomedical research and how well they can collaborate with human beings in those settings.  You know, I asked Bill and Seb to make some predictions about the future. You know, my own answer, I expect that we’re going to be able to use AI to change how we diagnose patients, change how we decide treatment options.   If you’re a doctor or a nurse and you encounter a patient, you’ll ask questions, do a physical exam, you know, call out for labs just like you do today, but then you’ll be able to engage with AI based on all of that data and just ask, you know, based on all the other people who have gone through the same experience, who have similar data, how were they diagnosed? How were they treated? What were their outcomes? And what does that mean for the patient I have right now? Some people call it the “patients like me” paradigm. And I think that’s going to become real because of AI within our lifetimes. That idea of really grounding the delivery in healthcare and medical practice through data and intelligence, I actually now don’t see any barriers to that future becoming real.  [THEME MUSIC]  I’d like to extend another big thank you to Bill and Sébastien for their time. And to our listeners, as always, it’s a pleasure to have you along for the ride. I hope you’ll join us for our remaining conversations, as well as a second coauthor roundtable with Carey and Zak.   Until next time.   [MUSIC FADES]
    0 Comentários 0 Compartilhamentos 0 Anterior
  • An excerpt from a new book by Sérgio Ferro, published by MACK Books, showcases the architect’s moment of disenchantment

    Last year, MACK Books published Architecture from Below, which anthologized writings by the French Brazilian architect, theorist, and painter Sérgio Ferro.Now, MACK follows with Design and the Building Site and Complementary Essays, the second in the trilogy of books dedicated to Ferro’s scholarship. The following excerpt of the author’s 2023 preface to the English edition, which preserves its British phrasing, captures Ferro’s realization about the working conditions of construction sites in Brasília. The sentiment is likely relatable even today for young architects as they discover how drawings become buildings. Design and the Building Site and Complementary Essays will be released on May 22.

    If I remember correctly, it was in 1958 or 1959, when Rodrigo and I were second- or third year architecture students at FAUUSP, that my father, the real estate developer Armando Simone Pereira, commissioned us to design two large office buildings and eleven shops in Brasilia, which was then under construction. Of course, we were not adequately prepared for such an undertaking. Fortunately, Oscar Niemeyer and his team, who were responsible for overseeing the construction of the capital, had drawn up a detailed document determining the essential characteristics of all the private sector buildings. We followed these prescriptions to the letter, which saved us from disaster.
    Nowadays, it is hard to imagine the degree to which the construction of Brasilia inspired enthusiasm and professional pride in the country’s architects. And in the national imagination, the city’s establishment in the supposedly unpopulated hinterland evoked a re-founding of Brazil. Up until that point, the occupation of our immense territory had been reduced to a collection of arborescent communication routes, generally converging upon some river, following it up to the Atlantic Ocean. Through its ports, agricultural or extractive commodities produced by enslaved peoples or their substitutes passed towards the metropolises; goods were exchanged in the metropolises for more elaborate products, which took the opposite route. Our national identity was summed up in a few symbols, such as the anthem or the flag, and this scattering of paths pointing overseas. Brasilia would radically change this situation, or so we believed. It would create a central hub where the internal communication routes could converge, linking together hithertoseparate junctions, stimulating trade and economic progress in the country’s interior. It was as if, for the first time, we were taking care of ourselves. At the nucleus of this centripetal movement, architecture would embody the renaissance. And at the naval of the nucleus, the symbolic mandala of this utopia: the cathedral.
    Rodrigo and I got caught up in the euphoria. And perhaps more so than our colleagues, because we were taking part in the adventure with ‘our’ designs. The reality was very different — but we did not know that yet.

    At that time, architects in Brazil were responsible for verifying that the construction was in line with the design. We had already monitored some of our first building sites. But the construction company in charge of them, Osmar Souza e Silva’s CENPLA, specialized in the building sites of modernist architects from the so-called Escola Paulista led by Vilanova Artigas. Osmar was very attentive to his clients and his workers, who formed a supportive and helpful team. He was even more careful with us, because he knew how inexperienced we were. I believe that the CENPLA was particularly important in São Paulo modernism: with its congeniality, it facilitated experimentation, but for the same reason, it deceived novices like us about the reality of other building sites.
    Consequently, Rodrigo and I travelled to Brasilia several times to check that the constructions followed ‘our’ designs and to resolve any issues. From the very first trip, our little bubble burst. Our building sites, like all the others in the future capital, bore no relation to Osmar’s. They were more like a branch of hell. A huge, muddy wasteland, in which a few cranes, pile drivers, tractors, and excavators dotted the mound of scaffolding occupied by thousands of skinny, seemingly exhausted wretches, who were nevertheless driven on by the shouts of master builders and foremen, in turn pressured by the imminence of the fateful inauguration date. Surrounding or huddled underneath the marquees of buildings under construction, entire families, equally skeletal and ragged, were waiting for some accident or death to open up a vacancy. In contact only with the master builders, and under close surveillance so we would not speak to the workers, we were not allowed to see what comrades who had worked on these sites later told us in prison: suicide abounded; escape was known to be futile in the unpopulated surroundings with no viable roads; fatal accidents were often caused by weakness due to chronic diarrhoea, brought on by rotten food that came from far away; outright theft took place in the calculation of wages and expenses in the contractor’s grocery store; camps were surrounded by law enforcement.
    I repeat this anecdote yet again not to invoke the benevolence of potential readers, but rather to point out the conditions that, in my opinion, allowed two studentsstill in their professional infancy to quickly adopt positions that were contrary to the usual stance of architects. As the project was more Oscar Niemeyer’s than it was our own, we did not have the same emotional attachment that is understandably engendered between real authors and their designs. We had not yet been imbued with the charm and aura of the métier. And the only building sites we had visited thus far, Osmar’s, were incomparable to those we discovered in Brasilia. In short, our youthfulness and unpreparedness up against an unbearable situation made us react almost immediately to the profession’s satisfied doxa.

    Unprepared and young perhaps, but already with Marx by our side. Rodrigo and I joined the student cell of the Brazilian Communist Party during our first year at university. In itself, this did not help us much: the Party’s Marxism, revised in the interests of the USSR, was pitiful. Even high-level leaders rarely went beyond the first chapter of Capital. But at the end of the 1950s, the effervescence of the years to come was already nascent: this extraordinary revivalthe rediscovery of Marxism and the great dialectical texts and traditions in the 1960s: an excitement that identifies a forgotten or repressed moment of the past as the new and subversive, and learns the dialectical grammar of a Hegel or an Adorno, a Marx or a Lukács, like a foreign language that has resources unavailable in our own.
    And what is more: the Chinese and Cuban revolutions, the war in Vietnam, guerrilla warfare of all kinds, national liberation movements, and a rare libertarian disposition in contemporary history, totally averse to fanaticism and respect for ideological apparatuses ofstate or institution. Going against the grain was almost the norm. We were of course no more than contemporaries of our time. We were soon able to position ourselves from chapters 13, 14, and 15 of Capital, but only because we could constantly cross-reference Marx with our observations from well-contrasted building sites and do our own experimenting. As soon as we identified construction as manufacture, for example, thanks to the willingness and even encouragement of two friends and clients, Boris Fausto and Bernardo Issler, I was able to test both types of manufacture — organic and heterogeneous — on similar-sized projects taking place simultaneously, in order to find out which would be most convenient for the situation in Brazil, particularly in São Paulo. Despite the scientific shortcomings of these tests, they sufficed for us to select organic manufacture. Arquitetura Nova had defined its line of practice, studies, and research.
    There were other sources that were central to our theory and practice. Flávio Império was one of the founders of the Teatro de Arena, undoubtedly the vanguard of popular, militant theatre in Brazil. He won practically every set design award. He brought us his marvelous findings in spatial condensation and malleability, and in the creative diversion of techniques and material—appropriate devices for an underdeveloped country. This is what helped us pave the way to reformulating the reigning design paradigms. 

    We had to do what Flávio had done in the theatre: thoroughly rethink how to be an architect. Upend the perspective. The way we were taught was to start from a desired result; then others would take care of getting there, no matter how. We, on the other hand, set out to go down to the building site and accompany those carrying out the labor itself, those who actually build, the formally subsumed workers in manufacture who are increasingly deprived of the knowledge and know-how presupposed by this kind of subsumption. We should have been fostering the reconstitution of this knowledge and know-how—not so as to fulfil this assumption, but in order to reinvigorate the other side of this assumption according to Marx: the historical rebellion of the manufacture worker, especially the construction worker. We had to rekindle the demand that fueled this rebellion: total self-determination, and not just that of the manual operation as such. Our aim was above all political and ethical. Aesthetics only mattered by way of what it included—ethics. Instead of estética, we wrote est ética. We wanted to make building sites into nests for the return of revolutionary syndicalism, which we ourselves had yet to discover.
    Sérgio Ferro, born in Brazil in 1938, studied architecture at FAUUSP, São Paulo. In the 1960s, he joined the Brazilian communist party and started, along with Rodrigo Lefevre and Flávio Império, the collective known as Arquitetura Nova. After being arrested by the military dictatorship that took power in Brazil in 1964, he moved to France as an exile. As a painter and a professor at the École Nationale Supérieure d’Architecture de Grenoble, where he founded the Dessin/Chantier laboratory, he engaged in extensive research which resulted in several publications, exhibitions, and awards in Brazil and in France, including the title of Chevalier des Arts et des Lettres in 1992. Following his retirement from teaching, Ferro continues to research, write, and paint.
    #excerpt #new #book #sérgio #ferro
    An excerpt from a new book by Sérgio Ferro, published by MACK Books, showcases the architect’s moment of disenchantment
    Last year, MACK Books published Architecture from Below, which anthologized writings by the French Brazilian architect, theorist, and painter Sérgio Ferro.Now, MACK follows with Design and the Building Site and Complementary Essays, the second in the trilogy of books dedicated to Ferro’s scholarship. The following excerpt of the author’s 2023 preface to the English edition, which preserves its British phrasing, captures Ferro’s realization about the working conditions of construction sites in Brasília. The sentiment is likely relatable even today for young architects as they discover how drawings become buildings. Design and the Building Site and Complementary Essays will be released on May 22. If I remember correctly, it was in 1958 or 1959, when Rodrigo and I were second- or third year architecture students at FAUUSP, that my father, the real estate developer Armando Simone Pereira, commissioned us to design two large office buildings and eleven shops in Brasilia, which was then under construction. Of course, we were not adequately prepared for such an undertaking. Fortunately, Oscar Niemeyer and his team, who were responsible for overseeing the construction of the capital, had drawn up a detailed document determining the essential characteristics of all the private sector buildings. We followed these prescriptions to the letter, which saved us from disaster. Nowadays, it is hard to imagine the degree to which the construction of Brasilia inspired enthusiasm and professional pride in the country’s architects. And in the national imagination, the city’s establishment in the supposedly unpopulated hinterland evoked a re-founding of Brazil. Up until that point, the occupation of our immense territory had been reduced to a collection of arborescent communication routes, generally converging upon some river, following it up to the Atlantic Ocean. Through its ports, agricultural or extractive commodities produced by enslaved peoples or their substitutes passed towards the metropolises; goods were exchanged in the metropolises for more elaborate products, which took the opposite route. Our national identity was summed up in a few symbols, such as the anthem or the flag, and this scattering of paths pointing overseas. Brasilia would radically change this situation, or so we believed. It would create a central hub where the internal communication routes could converge, linking together hithertoseparate junctions, stimulating trade and economic progress in the country’s interior. It was as if, for the first time, we were taking care of ourselves. At the nucleus of this centripetal movement, architecture would embody the renaissance. And at the naval of the nucleus, the symbolic mandala of this utopia: the cathedral. Rodrigo and I got caught up in the euphoria. And perhaps more so than our colleagues, because we were taking part in the adventure with ‘our’ designs. The reality was very different — but we did not know that yet. At that time, architects in Brazil were responsible for verifying that the construction was in line with the design. We had already monitored some of our first building sites. But the construction company in charge of them, Osmar Souza e Silva’s CENPLA, specialized in the building sites of modernist architects from the so-called Escola Paulista led by Vilanova Artigas. Osmar was very attentive to his clients and his workers, who formed a supportive and helpful team. He was even more careful with us, because he knew how inexperienced we were. I believe that the CENPLA was particularly important in São Paulo modernism: with its congeniality, it facilitated experimentation, but for the same reason, it deceived novices like us about the reality of other building sites. Consequently, Rodrigo and I travelled to Brasilia several times to check that the constructions followed ‘our’ designs and to resolve any issues. From the very first trip, our little bubble burst. Our building sites, like all the others in the future capital, bore no relation to Osmar’s. They were more like a branch of hell. A huge, muddy wasteland, in which a few cranes, pile drivers, tractors, and excavators dotted the mound of scaffolding occupied by thousands of skinny, seemingly exhausted wretches, who were nevertheless driven on by the shouts of master builders and foremen, in turn pressured by the imminence of the fateful inauguration date. Surrounding or huddled underneath the marquees of buildings under construction, entire families, equally skeletal and ragged, were waiting for some accident or death to open up a vacancy. In contact only with the master builders, and under close surveillance so we would not speak to the workers, we were not allowed to see what comrades who had worked on these sites later told us in prison: suicide abounded; escape was known to be futile in the unpopulated surroundings with no viable roads; fatal accidents were often caused by weakness due to chronic diarrhoea, brought on by rotten food that came from far away; outright theft took place in the calculation of wages and expenses in the contractor’s grocery store; camps were surrounded by law enforcement. I repeat this anecdote yet again not to invoke the benevolence of potential readers, but rather to point out the conditions that, in my opinion, allowed two studentsstill in their professional infancy to quickly adopt positions that were contrary to the usual stance of architects. As the project was more Oscar Niemeyer’s than it was our own, we did not have the same emotional attachment that is understandably engendered between real authors and their designs. We had not yet been imbued with the charm and aura of the métier. And the only building sites we had visited thus far, Osmar’s, were incomparable to those we discovered in Brasilia. In short, our youthfulness and unpreparedness up against an unbearable situation made us react almost immediately to the profession’s satisfied doxa. Unprepared and young perhaps, but already with Marx by our side. Rodrigo and I joined the student cell of the Brazilian Communist Party during our first year at university. In itself, this did not help us much: the Party’s Marxism, revised in the interests of the USSR, was pitiful. Even high-level leaders rarely went beyond the first chapter of Capital. But at the end of the 1950s, the effervescence of the years to come was already nascent: this extraordinary revivalthe rediscovery of Marxism and the great dialectical texts and traditions in the 1960s: an excitement that identifies a forgotten or repressed moment of the past as the new and subversive, and learns the dialectical grammar of a Hegel or an Adorno, a Marx or a Lukács, like a foreign language that has resources unavailable in our own. And what is more: the Chinese and Cuban revolutions, the war in Vietnam, guerrilla warfare of all kinds, national liberation movements, and a rare libertarian disposition in contemporary history, totally averse to fanaticism and respect for ideological apparatuses ofstate or institution. Going against the grain was almost the norm. We were of course no more than contemporaries of our time. We were soon able to position ourselves from chapters 13, 14, and 15 of Capital, but only because we could constantly cross-reference Marx with our observations from well-contrasted building sites and do our own experimenting. As soon as we identified construction as manufacture, for example, thanks to the willingness and even encouragement of two friends and clients, Boris Fausto and Bernardo Issler, I was able to test both types of manufacture — organic and heterogeneous — on similar-sized projects taking place simultaneously, in order to find out which would be most convenient for the situation in Brazil, particularly in São Paulo. Despite the scientific shortcomings of these tests, they sufficed for us to select organic manufacture. Arquitetura Nova had defined its line of practice, studies, and research. There were other sources that were central to our theory and practice. Flávio Império was one of the founders of the Teatro de Arena, undoubtedly the vanguard of popular, militant theatre in Brazil. He won practically every set design award. He brought us his marvelous findings in spatial condensation and malleability, and in the creative diversion of techniques and material—appropriate devices for an underdeveloped country. This is what helped us pave the way to reformulating the reigning design paradigms.  We had to do what Flávio had done in the theatre: thoroughly rethink how to be an architect. Upend the perspective. The way we were taught was to start from a desired result; then others would take care of getting there, no matter how. We, on the other hand, set out to go down to the building site and accompany those carrying out the labor itself, those who actually build, the formally subsumed workers in manufacture who are increasingly deprived of the knowledge and know-how presupposed by this kind of subsumption. We should have been fostering the reconstitution of this knowledge and know-how—not so as to fulfil this assumption, but in order to reinvigorate the other side of this assumption according to Marx: the historical rebellion of the manufacture worker, especially the construction worker. We had to rekindle the demand that fueled this rebellion: total self-determination, and not just that of the manual operation as such. Our aim was above all political and ethical. Aesthetics only mattered by way of what it included—ethics. Instead of estética, we wrote est ética. We wanted to make building sites into nests for the return of revolutionary syndicalism, which we ourselves had yet to discover. Sérgio Ferro, born in Brazil in 1938, studied architecture at FAUUSP, São Paulo. In the 1960s, he joined the Brazilian communist party and started, along with Rodrigo Lefevre and Flávio Império, the collective known as Arquitetura Nova. After being arrested by the military dictatorship that took power in Brazil in 1964, he moved to France as an exile. As a painter and a professor at the École Nationale Supérieure d’Architecture de Grenoble, where he founded the Dessin/Chantier laboratory, he engaged in extensive research which resulted in several publications, exhibitions, and awards in Brazil and in France, including the title of Chevalier des Arts et des Lettres in 1992. Following his retirement from teaching, Ferro continues to research, write, and paint. #excerpt #new #book #sérgio #ferro
    An excerpt from a new book by Sérgio Ferro, published by MACK Books, showcases the architect’s moment of disenchantment
    Last year, MACK Books published Architecture from Below, which anthologized writings by the French Brazilian architect, theorist, and painter Sérgio Ferro. (Douglas Spencer reviewed it for AN.) Now, MACK follows with Design and the Building Site and Complementary Essays, the second in the trilogy of books dedicated to Ferro’s scholarship. The following excerpt of the author’s 2023 preface to the English edition, which preserves its British phrasing, captures Ferro’s realization about the working conditions of construction sites in Brasília. The sentiment is likely relatable even today for young architects as they discover how drawings become buildings. Design and the Building Site and Complementary Essays will be released on May 22. If I remember correctly, it was in 1958 or 1959, when Rodrigo and I were second- or third year architecture students at FAUUSP, that my father, the real estate developer Armando Simone Pereira, commissioned us to design two large office buildings and eleven shops in Brasilia, which was then under construction. Of course, we were not adequately prepared for such an undertaking. Fortunately, Oscar Niemeyer and his team, who were responsible for overseeing the construction of the capital, had drawn up a detailed document determining the essential characteristics of all the private sector buildings. We followed these prescriptions to the letter, which saved us from disaster. Nowadays, it is hard to imagine the degree to which the construction of Brasilia inspired enthusiasm and professional pride in the country’s architects. And in the national imagination, the city’s establishment in the supposedly unpopulated hinterland evoked a re-founding of Brazil. Up until that point, the occupation of our immense territory had been reduced to a collection of arborescent communication routes, generally converging upon some river, following it up to the Atlantic Ocean. Through its ports, agricultural or extractive commodities produced by enslaved peoples or their substitutes passed towards the metropolises; goods were exchanged in the metropolises for more elaborate products, which took the opposite route. Our national identity was summed up in a few symbols, such as the anthem or the flag, and this scattering of paths pointing overseas. Brasilia would radically change this situation, or so we believed. It would create a central hub where the internal communication routes could converge, linking together hithertoseparate junctions, stimulating trade and economic progress in the country’s interior. It was as if, for the first time, we were taking care of ourselves. At the nucleus of this centripetal movement, architecture would embody the renaissance. And at the naval of the nucleus, the symbolic mandala of this utopia: the cathedral. Rodrigo and I got caught up in the euphoria. And perhaps more so than our colleagues, because we were taking part in the adventure with ‘our’ designs. The reality was very different — but we did not know that yet. At that time, architects in Brazil were responsible for verifying that the construction was in line with the design. We had already monitored some of our first building sites. But the construction company in charge of them, Osmar Souza e Silva’s CENPLA, specialized in the building sites of modernist architects from the so-called Escola Paulista led by Vilanova Artigas (which we aspired to be a part of, like the pretentious students we were). Osmar was very attentive to his clients and his workers, who formed a supportive and helpful team. He was even more careful with us, because he knew how inexperienced we were. I believe that the CENPLA was particularly important in São Paulo modernism: with its congeniality, it facilitated experimentation, but for the same reason, it deceived novices like us about the reality of other building sites. Consequently, Rodrigo and I travelled to Brasilia several times to check that the constructions followed ‘our’ designs and to resolve any issues. From the very first trip, our little bubble burst. Our building sites, like all the others in the future capital, bore no relation to Osmar’s. They were more like a branch of hell. A huge, muddy wasteland, in which a few cranes, pile drivers, tractors, and excavators dotted the mound of scaffolding occupied by thousands of skinny, seemingly exhausted wretches, who were nevertheless driven on by the shouts of master builders and foremen, in turn pressured by the imminence of the fateful inauguration date. Surrounding or huddled underneath the marquees of buildings under construction, entire families, equally skeletal and ragged, were waiting for some accident or death to open up a vacancy. In contact only with the master builders, and under close surveillance so we would not speak to the workers, we were not allowed to see what comrades who had worked on these sites later told us in prison: suicide abounded; escape was known to be futile in the unpopulated surroundings with no viable roads; fatal accidents were often caused by weakness due to chronic diarrhoea, brought on by rotten food that came from far away; outright theft took place in the calculation of wages and expenses in the contractor’s grocery store; camps were surrounded by law enforcement. I repeat this anecdote yet again not to invoke the benevolence of potential readers, but rather to point out the conditions that, in my opinion, allowed two students (Flávio Império joined us a little later) still in their professional infancy to quickly adopt positions that were contrary to the usual stance of architects. As the project was more Oscar Niemeyer’s than it was our own, we did not have the same emotional attachment that is understandably engendered between real authors and their designs. We had not yet been imbued with the charm and aura of the métier. And the only building sites we had visited thus far, Osmar’s, were incomparable to those we discovered in Brasilia. In short, our youthfulness and unpreparedness up against an unbearable situation made us react almost immediately to the profession’s satisfied doxa. Unprepared and young perhaps, but already with Marx by our side. Rodrigo and I joined the student cell of the Brazilian Communist Party during our first year at university. In itself, this did not help us much: the Party’s Marxism, revised in the interests of the USSR, was pitiful. Even high-level leaders rarely went beyond the first chapter of Capital. But at the end of the 1950s, the effervescence of the years to come was already nascent:  […] this extraordinary revival […] the rediscovery of Marxism and the great dialectical texts and traditions in the 1960s: an excitement that identifies a forgotten or repressed moment of the past as the new and subversive, and learns the dialectical grammar of a Hegel or an Adorno, a Marx or a Lukács, like a foreign language that has resources unavailable in our own. And what is more: the Chinese and Cuban revolutions, the war in Vietnam, guerrilla warfare of all kinds, national liberation movements, and a rare libertarian disposition in contemporary history, totally averse to fanaticism and respect for ideological apparatuses of (any) state or institution. Going against the grain was almost the norm. We were of course no more than contemporaries of our time. We were soon able to position ourselves from chapters 13, 14, and 15 of Capital, but only because we could constantly cross-reference Marx with our observations from well-contrasted building sites and do our own experimenting. As soon as we identified construction as manufacture, for example, thanks to the willingness and even encouragement of two friends and clients, Boris Fausto and Bernardo Issler, I was able to test both types of manufacture — organic and heterogeneous — on similar-sized projects taking place simultaneously, in order to find out which would be most convenient for the situation in Brazil, particularly in São Paulo. Despite the scientific shortcomings of these tests, they sufficed for us to select organic manufacture. Arquitetura Nova had defined its line of practice, studies, and research. There were other sources that were central to our theory and practice. Flávio Império was one of the founders of the Teatro de Arena, undoubtedly the vanguard of popular, militant theatre in Brazil. He won practically every set design award. He brought us his marvelous findings in spatial condensation and malleability, and in the creative diversion of techniques and material—appropriate devices for an underdeveloped country. This is what helped us pave the way to reformulating the reigning design paradigms.  We had to do what Flávio had done in the theatre: thoroughly rethink how to be an architect. Upend the perspective. The way we were taught was to start from a desired result; then others would take care of getting there, no matter how. We, on the other hand, set out to go down to the building site and accompany those carrying out the labor itself, those who actually build, the formally subsumed workers in manufacture who are increasingly deprived of the knowledge and know-how presupposed by this kind of subsumption. We should have been fostering the reconstitution of this knowledge and know-how—not so as to fulfil this assumption, but in order to reinvigorate the other side of this assumption according to Marx: the historical rebellion of the manufacture worker, especially the construction worker. We had to rekindle the demand that fueled this rebellion: total self-determination, and not just that of the manual operation as such. Our aim was above all political and ethical. Aesthetics only mattered by way of what it included—ethics. Instead of estética, we wrote est ética [this is ethics]. We wanted to make building sites into nests for the return of revolutionary syndicalism, which we ourselves had yet to discover. Sérgio Ferro, born in Brazil in 1938, studied architecture at FAUUSP, São Paulo. In the 1960s, he joined the Brazilian communist party and started, along with Rodrigo Lefevre and Flávio Império, the collective known as Arquitetura Nova. After being arrested by the military dictatorship that took power in Brazil in 1964, he moved to France as an exile. As a painter and a professor at the École Nationale Supérieure d’Architecture de Grenoble, where he founded the Dessin/Chantier laboratory, he engaged in extensive research which resulted in several publications, exhibitions, and awards in Brazil and in France, including the title of Chevalier des Arts et des Lettres in 1992. Following his retirement from teaching, Ferro continues to research, write, and paint.
    0 Comentários 0 Compartilhamentos 0 Anterior
  • CERT Director Greg Touhill: To Lead Is to Serve

    Greg Touhill, director of the Software Engineering’s Institute’sComputer Emergency Response Teamdivision is an atypical technology leader. For one thing, he’s been in tech and other leadership positions that span the US Air Force, the US government, the private sector and now SEI’s CERT. More importantly, he’s been a major force in the cybersecurity realm, making the world a safer place and even saving lives. Touhill earned a bachelor’s degree from the Pennsylvania State University, a master’s degree from the University of Southern California, a master’s degree from the Air War College, was a senior executive fellow at the Harvard University Kennedy School of Government and completed executive education studies at the University of North Carolina. “I was a student intern at Carnegie Mellon, but I was going to college at Penn State and studying chemical engineering. As an Air Force ROTC scholarship recipient, I knew I was going to become an Air Force officer but soon realized that I didn’t necessarily want to be a chemical engineer in the Air Force,” says Touhill. “Because I passed all the mathematics, physics, and engineering courses, I ended up becoming a communications, electronics, and computer systems officer in the Air Force. I spent 30 years, one month and three days on active duty in the United States Air Force, eventually retiring as a brigadier general and having done many different types of jobs that were available to me within and even beyond my career field.” Related:Specifically, he was an operational commander at the squadron, group, and wing levels. For example, as a colonel, Touhill served as director of command, control, communications and computersfor the United States Central Command Forces, then he was appointed chief information officer and director, communications and information at Air Mobility Command. Later, he served as commander, 81st Training Wing at Kessler Air Force Base where he was promoted to brigadier general and commanded over 12,500 personnel. After that, he served as the senior defense officer and US defense attaché at the US Embassy in Kuwait, before concluding his military career as the chief information officer and director, C4 systems at the US Transportation Command, one of 10 US combatant commands, where he and his team were awarded the NSA Rowlett Award for the best cybersecurity program in the government. While in the Air Force, Touhill received numerous awards and decorations including the Bronze Star medal and the Air Force Science and Engineering Award. He is the only three-time recipient of the USAF C4 Professionalism Award. Related:Greg Touhill“I got to serve at major combatant commands, work with coalition partners from many different countries and represented the US as part of a diplomatic mission to Kuwait for two years as the senior defense official at a time when America was withdrawing forces out of Iraq. I also led the negotiation of a new bilateral defense agreement with the Kuwaitis,” says Touhill. “Then I was recruited to continue my service and was asked to serve as the deputy assistant secretary of cybersecurity and communications at the Department of Homeland Security, where I ran the operations of what is now known as the Cybersecurity and Infrastructure Security Agency. I was there at a pivotal moment because we were building up the capacity of that organization and setting the stage for it to become its own agency.” While at DHS, there were many noteworthy breaches including the infamous US Office of People Managementbreach. Those events led to Obama’s visit to the National Cybersecurity and Communications Integration Center.  “I got to brief the president on the state of cybersecurity, what we had seen with the OPM breach and some other deficiencies,” says Touhill. “I was on the federal CIO council as the cybersecurity advisor to that since I’d been a federal CIO before and I got to conclude my federal career by being the first United States government chief information security officer. From there, I pivoted to industry, but I also got to return to Carnegie Mellon as a faculty member at Carnegie Mellon’s Heinz College, where I've been teaching since January 2017.” Related:Touhill has been involved in three startups, two of which were successfully acquired. He also served on three Fortune 100 advisory boards and on the Information Systems Audit and Control Association board, eventually becoming its chair for a term during the seven years he served there. Touhill just celebrated his fourth year at CERT, which he considers the pinnacle of the cybersecurity profession and everything he’s done to date. “Over my career I've led teams that have done major software builds in the national security space. I've also been the guy who's pulled cables and set up routers, hubs and switches, and I've been a system administrator. I've done everything that I could do from the keyboard up all the way up to the White House,” says Touhill. “For 40 years, the Software Engineering Institute has been leading the world in secure by design, cybersecurity, software engineering, artificial intelligence and engineering, pioneering best practices, and figuring out how to make the world a safer more secure and trustworthy place. I’ve had a hand in the making of today’s modern military and government information technology environment, beginning as a 22-year-old lieutenant, and hope to inspire the next generation to do even better.” What ‘Success’ Means Many people would be satisfied with their careers as a brigadier general, a tech leader, the White House’s first anything, or working at CERT, let alone running it. Touhill has spent his entire career making the world a safer place, so it’s not surprising that he considers his greatest achievement saving lives. “In the Middle East and Iraq, convoys were being attacked with improvised explosive devices. There were also ‘direct fire’ attacks where people are firing weapons at you and indirect fire attacks where you could be in the line of fire,” says Touhill. “The convoys were using SINCGARS line-of-site walkie-talkies for communications that are most effective when the ground is flat, and Iraq is not flat. As a result, our troops were at risk of not having reliable communications while under attack. As my team brainstormed options to remedy the situation, one of my guys found some technology, about the size of an iPhone, that could covert a radio signal, which is basically a waveform, into a digital pulse I could put on a dedicated network to support the convoy missions.” For million, Touhill and his team quickly architected, tested, and fielded the Radio over IP networkthat had a 99% reliability rate anywhere in Iraq. Better still, convoys could communicate over the network using any radios. That solution saved a minimum of six lives. In one case, the hospital doctor said if the patient had arrived five minutes later, he would have died. Sage Advice Anyone who has ever spent time in the military or in a military family knows that soldiers are very well disciplined, or they wash out. Other traits include being physically fit, mentally fit, and achieving balance in life, though that’s difficult to achieve in combat. Still, it’s a necessity. “I served three and a half years down range in combat operations. My experience taught me you could be doing 20-hour days for a year or two on end. If you haven’t built a good foundation of being disciplined and fit, it impacts your ability to maintain presence in times of stress, and CISOs work in stressful situations,” says Touhill. “Staying fit also fortifies you for the long haul, so you don’t get burned out as fast.” Another necessary skill is the ability to work well with others.  “Cybersecurity is an interdisciplinary practice. One of the great joys I have as CERT director is the wide range of experts in many different fields that include software engineers, computer engineers, computer scientists, data scientists, mathematicians and physicists,” says Touhill. “I have folks who have business degrees and others who have philosophy degrees. It's really a rich community of interests all coming together towards that common goal of making the world a safer, more secure and more trusted place in the cyber domain. We’re are kind of like the cyber neighborhood watch for the whole world.” He also says that money isn’t everything, having taken a pay cut to go from being an Air Force brigadier general to the deputy assistant secretary of the Department of Homeland Security . “You’ll always do well if you pick the job that matters most. That’s what I did, and I’ve been rewarded every step,” says Touhill.  The biggest challenge he sees is the complexity of cyber systems and software, which can have second, third, and fourth order effects.  “Complexity raises the cost of the attack surface, increases the attack surface, raises the number of vulnerabilities and exploits human weaknesses,” says Touhill. “The No. 1 thing we need to be paying attention to is privacy when it comes to AI because AI can unearth and discover knowledge from data we already have. While it gives us greater insights at greater velocities, we need to be careful that we take precautions to better protect our privacy, civil rights and civil liberties.” 
    #cert #director #greg #touhill #lead
    CERT Director Greg Touhill: To Lead Is to Serve
    Greg Touhill, director of the Software Engineering’s Institute’sComputer Emergency Response Teamdivision is an atypical technology leader. For one thing, he’s been in tech and other leadership positions that span the US Air Force, the US government, the private sector and now SEI’s CERT. More importantly, he’s been a major force in the cybersecurity realm, making the world a safer place and even saving lives. Touhill earned a bachelor’s degree from the Pennsylvania State University, a master’s degree from the University of Southern California, a master’s degree from the Air War College, was a senior executive fellow at the Harvard University Kennedy School of Government and completed executive education studies at the University of North Carolina. “I was a student intern at Carnegie Mellon, but I was going to college at Penn State and studying chemical engineering. As an Air Force ROTC scholarship recipient, I knew I was going to become an Air Force officer but soon realized that I didn’t necessarily want to be a chemical engineer in the Air Force,” says Touhill. “Because I passed all the mathematics, physics, and engineering courses, I ended up becoming a communications, electronics, and computer systems officer in the Air Force. I spent 30 years, one month and three days on active duty in the United States Air Force, eventually retiring as a brigadier general and having done many different types of jobs that were available to me within and even beyond my career field.” Related:Specifically, he was an operational commander at the squadron, group, and wing levels. For example, as a colonel, Touhill served as director of command, control, communications and computersfor the United States Central Command Forces, then he was appointed chief information officer and director, communications and information at Air Mobility Command. Later, he served as commander, 81st Training Wing at Kessler Air Force Base where he was promoted to brigadier general and commanded over 12,500 personnel. After that, he served as the senior defense officer and US defense attaché at the US Embassy in Kuwait, before concluding his military career as the chief information officer and director, C4 systems at the US Transportation Command, one of 10 US combatant commands, where he and his team were awarded the NSA Rowlett Award for the best cybersecurity program in the government. While in the Air Force, Touhill received numerous awards and decorations including the Bronze Star medal and the Air Force Science and Engineering Award. He is the only three-time recipient of the USAF C4 Professionalism Award. Related:Greg Touhill“I got to serve at major combatant commands, work with coalition partners from many different countries and represented the US as part of a diplomatic mission to Kuwait for two years as the senior defense official at a time when America was withdrawing forces out of Iraq. I also led the negotiation of a new bilateral defense agreement with the Kuwaitis,” says Touhill. “Then I was recruited to continue my service and was asked to serve as the deputy assistant secretary of cybersecurity and communications at the Department of Homeland Security, where I ran the operations of what is now known as the Cybersecurity and Infrastructure Security Agency. I was there at a pivotal moment because we were building up the capacity of that organization and setting the stage for it to become its own agency.” While at DHS, there were many noteworthy breaches including the infamous US Office of People Managementbreach. Those events led to Obama’s visit to the National Cybersecurity and Communications Integration Center.  “I got to brief the president on the state of cybersecurity, what we had seen with the OPM breach and some other deficiencies,” says Touhill. “I was on the federal CIO council as the cybersecurity advisor to that since I’d been a federal CIO before and I got to conclude my federal career by being the first United States government chief information security officer. From there, I pivoted to industry, but I also got to return to Carnegie Mellon as a faculty member at Carnegie Mellon’s Heinz College, where I've been teaching since January 2017.” Related:Touhill has been involved in three startups, two of which were successfully acquired. He also served on three Fortune 100 advisory boards and on the Information Systems Audit and Control Association board, eventually becoming its chair for a term during the seven years he served there. Touhill just celebrated his fourth year at CERT, which he considers the pinnacle of the cybersecurity profession and everything he’s done to date. “Over my career I've led teams that have done major software builds in the national security space. I've also been the guy who's pulled cables and set up routers, hubs and switches, and I've been a system administrator. I've done everything that I could do from the keyboard up all the way up to the White House,” says Touhill. “For 40 years, the Software Engineering Institute has been leading the world in secure by design, cybersecurity, software engineering, artificial intelligence and engineering, pioneering best practices, and figuring out how to make the world a safer more secure and trustworthy place. I’ve had a hand in the making of today’s modern military and government information technology environment, beginning as a 22-year-old lieutenant, and hope to inspire the next generation to do even better.” What ‘Success’ Means Many people would be satisfied with their careers as a brigadier general, a tech leader, the White House’s first anything, or working at CERT, let alone running it. Touhill has spent his entire career making the world a safer place, so it’s not surprising that he considers his greatest achievement saving lives. “In the Middle East and Iraq, convoys were being attacked with improvised explosive devices. There were also ‘direct fire’ attacks where people are firing weapons at you and indirect fire attacks where you could be in the line of fire,” says Touhill. “The convoys were using SINCGARS line-of-site walkie-talkies for communications that are most effective when the ground is flat, and Iraq is not flat. As a result, our troops were at risk of not having reliable communications while under attack. As my team brainstormed options to remedy the situation, one of my guys found some technology, about the size of an iPhone, that could covert a radio signal, which is basically a waveform, into a digital pulse I could put on a dedicated network to support the convoy missions.” For million, Touhill and his team quickly architected, tested, and fielded the Radio over IP networkthat had a 99% reliability rate anywhere in Iraq. Better still, convoys could communicate over the network using any radios. That solution saved a minimum of six lives. In one case, the hospital doctor said if the patient had arrived five minutes later, he would have died. Sage Advice Anyone who has ever spent time in the military or in a military family knows that soldiers are very well disciplined, or they wash out. Other traits include being physically fit, mentally fit, and achieving balance in life, though that’s difficult to achieve in combat. Still, it’s a necessity. “I served three and a half years down range in combat operations. My experience taught me you could be doing 20-hour days for a year or two on end. If you haven’t built a good foundation of being disciplined and fit, it impacts your ability to maintain presence in times of stress, and CISOs work in stressful situations,” says Touhill. “Staying fit also fortifies you for the long haul, so you don’t get burned out as fast.” Another necessary skill is the ability to work well with others.  “Cybersecurity is an interdisciplinary practice. One of the great joys I have as CERT director is the wide range of experts in many different fields that include software engineers, computer engineers, computer scientists, data scientists, mathematicians and physicists,” says Touhill. “I have folks who have business degrees and others who have philosophy degrees. It's really a rich community of interests all coming together towards that common goal of making the world a safer, more secure and more trusted place in the cyber domain. We’re are kind of like the cyber neighborhood watch for the whole world.” He also says that money isn’t everything, having taken a pay cut to go from being an Air Force brigadier general to the deputy assistant secretary of the Department of Homeland Security . “You’ll always do well if you pick the job that matters most. That’s what I did, and I’ve been rewarded every step,” says Touhill.  The biggest challenge he sees is the complexity of cyber systems and software, which can have second, third, and fourth order effects.  “Complexity raises the cost of the attack surface, increases the attack surface, raises the number of vulnerabilities and exploits human weaknesses,” says Touhill. “The No. 1 thing we need to be paying attention to is privacy when it comes to AI because AI can unearth and discover knowledge from data we already have. While it gives us greater insights at greater velocities, we need to be careful that we take precautions to better protect our privacy, civil rights and civil liberties.”  #cert #director #greg #touhill #lead
    WWW.INFORMATIONWEEK.COM
    CERT Director Greg Touhill: To Lead Is to Serve
    Greg Touhill, director of the Software Engineering’s Institute’s (SEI’s) Computer Emergency Response Team (CERT) division is an atypical technology leader. For one thing, he’s been in tech and other leadership positions that span the US Air Force, the US government, the private sector and now SEI’s CERT. More importantly, he’s been a major force in the cybersecurity realm, making the world a safer place and even saving lives. Touhill earned a bachelor’s degree from the Pennsylvania State University, a master’s degree from the University of Southern California, a master’s degree from the Air War College, was a senior executive fellow at the Harvard University Kennedy School of Government and completed executive education studies at the University of North Carolina. “I was a student intern at Carnegie Mellon, but I was going to college at Penn State and studying chemical engineering. As an Air Force ROTC scholarship recipient, I knew I was going to become an Air Force officer but soon realized that I didn’t necessarily want to be a chemical engineer in the Air Force,” says Touhill. “Because I passed all the mathematics, physics, and engineering courses, I ended up becoming a communications, electronics, and computer systems officer in the Air Force. I spent 30 years, one month and three days on active duty in the United States Air Force, eventually retiring as a brigadier general and having done many different types of jobs that were available to me within and even beyond my career field.” Related:Specifically, he was an operational commander at the squadron, group, and wing levels. For example, as a colonel, Touhill served as director of command, control, communications and computers (C4) for the United States Central Command Forces, then he was appointed chief information officer and director, communications and information at Air Mobility Command. Later, he served as commander, 81st Training Wing at Kessler Air Force Base where he was promoted to brigadier general and commanded over 12,500 personnel. After that, he served as the senior defense officer and US defense attaché at the US Embassy in Kuwait, before concluding his military career as the chief information officer and director, C4 systems at the US Transportation Command, one of 10 US combatant commands, where he and his team were awarded the NSA Rowlett Award for the best cybersecurity program in the government. While in the Air Force, Touhill received numerous awards and decorations including the Bronze Star medal and the Air Force Science and Engineering Award. He is the only three-time recipient of the USAF C4 Professionalism Award. Related:Greg Touhill“I got to serve at major combatant commands, work with coalition partners from many different countries and represented the US as part of a diplomatic mission to Kuwait for two years as the senior defense official at a time when America was withdrawing forces out of Iraq. I also led the negotiation of a new bilateral defense agreement with the Kuwaitis,” says Touhill. “Then I was recruited to continue my service and was asked to serve as the deputy assistant secretary of cybersecurity and communications at the Department of Homeland Security, where I ran the operations of what is now known as the Cybersecurity and Infrastructure Security Agency. I was there at a pivotal moment because we were building up the capacity of that organization and setting the stage for it to become its own agency.” While at DHS, there were many noteworthy breaches including the infamous US Office of People Management (OPM) breach. Those events led to Obama’s visit to the National Cybersecurity and Communications Integration Center.  “I got to brief the president on the state of cybersecurity, what we had seen with the OPM breach and some other deficiencies,” says Touhill. “I was on the federal CIO council as the cybersecurity advisor to that since I’d been a federal CIO before and I got to conclude my federal career by being the first United States government chief information security officer. From there, I pivoted to industry, but I also got to return to Carnegie Mellon as a faculty member at Carnegie Mellon’s Heinz College, where I've been teaching since January 2017.” Related:Touhill has been involved in three startups, two of which were successfully acquired. He also served on three Fortune 100 advisory boards and on the Information Systems Audit and Control Association board, eventually becoming its chair for a term during the seven years he served there. Touhill just celebrated his fourth year at CERT, which he considers the pinnacle of the cybersecurity profession and everything he’s done to date. “Over my career I've led teams that have done major software builds in the national security space. I've also been the guy who's pulled cables and set up routers, hubs and switches, and I've been a system administrator. I've done everything that I could do from the keyboard up all the way up to the White House,” says Touhill. “For 40 years, the Software Engineering Institute has been leading the world in secure by design, cybersecurity, software engineering, artificial intelligence and engineering, pioneering best practices, and figuring out how to make the world a safer more secure and trustworthy place. I’ve had a hand in the making of today’s modern military and government information technology environment, beginning as a 22-year-old lieutenant, and hope to inspire the next generation to do even better.” What ‘Success’ Means Many people would be satisfied with their careers as a brigadier general, a tech leader, the White House’s first anything, or working at CERT, let alone running it. Touhill has spent his entire career making the world a safer place, so it’s not surprising that he considers his greatest achievement saving lives. “In the Middle East and Iraq, convoys were being attacked with improvised explosive devices. There were also ‘direct fire’ attacks where people are firing weapons at you and indirect fire attacks where you could be in the line of fire,” says Touhill. “The convoys were using SINCGARS line-of-site walkie-talkies for communications that are most effective when the ground is flat, and Iraq is not flat. As a result, our troops were at risk of not having reliable communications while under attack. As my team brainstormed options to remedy the situation, one of my guys found some technology, about the size of an iPhone, that could covert a radio signal, which is basically a waveform, into a digital pulse I could put on a dedicated network to support the convoy missions.” For $11 million, Touhill and his team quickly architected, tested, and fielded the Radio over IP network (aka “Ripper Net”) that had a 99% reliability rate anywhere in Iraq. Better still, convoys could communicate over the network using any radios. That solution saved a minimum of six lives. In one case, the hospital doctor said if the patient had arrived five minutes later, he would have died. Sage Advice Anyone who has ever spent time in the military or in a military family knows that soldiers are very well disciplined, or they wash out. Other traits include being physically fit, mentally fit, and achieving balance in life, though that’s difficult to achieve in combat. Still, it’s a necessity. “I served three and a half years down range in combat operations. My experience taught me you could be doing 20-hour days for a year or two on end. If you haven’t built a good foundation of being disciplined and fit, it impacts your ability to maintain presence in times of stress, and CISOs work in stressful situations,” says Touhill. “Staying fit also fortifies you for the long haul, so you don’t get burned out as fast.” Another necessary skill is the ability to work well with others.  “Cybersecurity is an interdisciplinary practice. One of the great joys I have as CERT director is the wide range of experts in many different fields that include software engineers, computer engineers, computer scientists, data scientists, mathematicians and physicists,” says Touhill. “I have folks who have business degrees and others who have philosophy degrees. It's really a rich community of interests all coming together towards that common goal of making the world a safer, more secure and more trusted place in the cyber domain. We’re are kind of like the cyber neighborhood watch for the whole world.” He also says that money isn’t everything, having taken a pay cut to go from being an Air Force brigadier general to the deputy assistant secretary of the Department of Homeland Security . “You’ll always do well if you pick the job that matters most. That’s what I did, and I’ve been rewarded every step,” says Touhill.  The biggest challenge he sees is the complexity of cyber systems and software, which can have second, third, and fourth order effects.  “Complexity raises the cost of the attack surface, increases the attack surface, raises the number of vulnerabilities and exploits human weaknesses,” says Touhill. “The No. 1 thing we need to be paying attention to is privacy when it comes to AI because AI can unearth and discover knowledge from data we already have. While it gives us greater insights at greater velocities, we need to be careful that we take precautions to better protect our privacy, civil rights and civil liberties.” 
    0 Comentários 0 Compartilhamentos 0 Anterior
  • Discord Invite Link Hijacking Delivers AsyncRAT and Skuld Stealer Targeting Crypto Wallets

    Jun 14, 2025Ravie LakshmananMalware / Threat Intelligence

    A new malware campaign is exploiting a weakness in Discord's invitation system to deliver an information stealer called Skuld and the AsyncRAT remote access trojan.
    "Attackers hijacked the links through vanity link registration, allowing them to silently redirect users from trusted sources to malicious servers," Check Point said in a technical report. "The attackers combined the ClickFix phishing technique, multi-stage loaders, and time-based evasions to stealthily deliver AsyncRAT, and a customized Skuld Stealer targeting crypto wallets."
    The issue with Discord's invite mechanism is that it allows attackers to hijack expired or deleted invite links and secretly redirect unsuspecting users to malicious servers under their control. This also means that a Discord invite link that was once trusted and shared on forums or social media platforms could unwittingly lead users to malicious sites.

    Details of the campaign come a little over a month after the cybersecurity company revealed another sophisticated phishing campaign that hijacked expired vanity invite links to entice users into joining a Discord server and instruct them to visit a phishing site to verify ownership, only to have their digital assets drained upon connecting their wallets.
    While users can create temporary, permanent, or custominvite links on Discord, the platform prevents other legitimate servers from reclaiming a previously expired or deleted invite. However, Check Point found that creating custom invite links allows the reuse of expired invite codes and even deleted permanent invite codes in some cases.

    This ability to reuse Discord expired or deleted codes when creating custom vanity invite links opens the door to abuse, allowing attackers to claim it for their malicious server.
    "This creates a serious risk: Users who follow previously trusted invite linkscan unknowingly be redirected to fake Discord servers created by threat actors," Check Point said.
    The Discord invite-link hijacking, in a nutshell, involves taking control of invite links originally shared by legitimate communities and then using them to redirect users to the malicious server. Users who fall prey to the scheme and join the server are asked to complete a verification step in order to gain full server access by authorizing a bot, which then leads them to a fake website with a prominent "Verify" button.
    This is where the attackers take the attack to the next level by incorporating the infamous ClickFix social engineering tactic to trick users into infecting their systems under the pretext of verification.

    Specifically, clicking the "Verify" button surreptitiously executes JavaScript that copies a PowerShell command to the machine's clipboard, after which the users are urged to launch the Windows Run dialog, paste the already copied "verification string", and press Enter to authenticate their accounts.
    But in reality, performing these steps triggers the download of a PowerShell script hosted on Pastebin that subsequently retrieves and executes a first-stage downloader, which is ultimately used to drop AsyncRAT and Skuld Stealer from a remote server and execute them.
    At the heart of this attack lies a meticulously engineered, multi-stage infection process designed for both precision and stealth, while also taking steps to subvert security protections through sandbox security checks.
    AsyncRAT, which offers comprehensive remote control capabilities over infected systems, has been found to employ a technique called dead drop resolver to access the actual command-and-controlserver by reading a Pastebin file.
    The other payload is a Golang information stealer that's downloaded from Bitbucket. It's equipped to steal sensitive user data from Discord, various browsers, crypto wallets, and gaming platforms.
    Skuld is also capable of harvesting crypto wallet seed phrases and passwords from the Exodus and Atomic crypto wallets. It accomplishes this using an approach called wallet injection that replaces legitimate application files with trojanized versions downloaded from GitHub. It's worth noting that a similar technique was recently put to use by a rogue npm package named pdf-to-office.
    The attack also employs a custom version of an open-source tool known as ChromeKatz to bypass Chrome's app-bound encryption protections. The collected data is exfiltrated to the miscreants via a Discord webhook.
    The fact that payload delivery and data exfiltration occur via trusted cloud services such as GitHub, Bitbucket, Pastebin, and Discord allows the threat actors to blend in with normal traffic and fly under the radar. Discord has since disabled the malicious bot, effectively breaking the attack chain.

    Check Point said it also identified another campaign mounted by the same threat actor that distributes the loader as a modified version of a hacktool for unlocking pirated games. The malicious program, also hosted on Bitbucket, has been downloaded 350 times.
    It has been assessed that the victims of these campaigns are primarily located in the United States, Vietnam, France, Germany, Slovakia, Austria, the Netherlands, and the United Kingdom.
    The findings represent the latest example of how cybercriminals are targeting the popular social platform, which has had its content delivery networkabused to host malware in the past.
    "This campaign illustrates how a subtle feature of Discord's invite system, the ability to reuse expired or deleted invite codes in vanity invite links, can be exploited as a powerful attack vector," the researchers said. "By hijacking legitimate invite links, threat actors silently redirect unsuspecting users to malicious Discord servers."
    "The choice of payloads, including a powerful stealer specifically targeting cryptocurrency wallets, suggests that the attackers are primarily focused on crypto users and motivated by financial gain."

    Found this article interesting? Follow us on Twitter  and LinkedIn to read more exclusive content we post.

    SHARE




    #discord #invite #link #hijacking #delivers
    Discord Invite Link Hijacking Delivers AsyncRAT and Skuld Stealer Targeting Crypto Wallets
    Jun 14, 2025Ravie LakshmananMalware / Threat Intelligence A new malware campaign is exploiting a weakness in Discord's invitation system to deliver an information stealer called Skuld and the AsyncRAT remote access trojan. "Attackers hijacked the links through vanity link registration, allowing them to silently redirect users from trusted sources to malicious servers," Check Point said in a technical report. "The attackers combined the ClickFix phishing technique, multi-stage loaders, and time-based evasions to stealthily deliver AsyncRAT, and a customized Skuld Stealer targeting crypto wallets." The issue with Discord's invite mechanism is that it allows attackers to hijack expired or deleted invite links and secretly redirect unsuspecting users to malicious servers under their control. This also means that a Discord invite link that was once trusted and shared on forums or social media platforms could unwittingly lead users to malicious sites. Details of the campaign come a little over a month after the cybersecurity company revealed another sophisticated phishing campaign that hijacked expired vanity invite links to entice users into joining a Discord server and instruct them to visit a phishing site to verify ownership, only to have their digital assets drained upon connecting their wallets. While users can create temporary, permanent, or custominvite links on Discord, the platform prevents other legitimate servers from reclaiming a previously expired or deleted invite. However, Check Point found that creating custom invite links allows the reuse of expired invite codes and even deleted permanent invite codes in some cases. This ability to reuse Discord expired or deleted codes when creating custom vanity invite links opens the door to abuse, allowing attackers to claim it for their malicious server. "This creates a serious risk: Users who follow previously trusted invite linkscan unknowingly be redirected to fake Discord servers created by threat actors," Check Point said. The Discord invite-link hijacking, in a nutshell, involves taking control of invite links originally shared by legitimate communities and then using them to redirect users to the malicious server. Users who fall prey to the scheme and join the server are asked to complete a verification step in order to gain full server access by authorizing a bot, which then leads them to a fake website with a prominent "Verify" button. This is where the attackers take the attack to the next level by incorporating the infamous ClickFix social engineering tactic to trick users into infecting their systems under the pretext of verification. Specifically, clicking the "Verify" button surreptitiously executes JavaScript that copies a PowerShell command to the machine's clipboard, after which the users are urged to launch the Windows Run dialog, paste the already copied "verification string", and press Enter to authenticate their accounts. But in reality, performing these steps triggers the download of a PowerShell script hosted on Pastebin that subsequently retrieves and executes a first-stage downloader, which is ultimately used to drop AsyncRAT and Skuld Stealer from a remote server and execute them. At the heart of this attack lies a meticulously engineered, multi-stage infection process designed for both precision and stealth, while also taking steps to subvert security protections through sandbox security checks. AsyncRAT, which offers comprehensive remote control capabilities over infected systems, has been found to employ a technique called dead drop resolver to access the actual command-and-controlserver by reading a Pastebin file. The other payload is a Golang information stealer that's downloaded from Bitbucket. It's equipped to steal sensitive user data from Discord, various browsers, crypto wallets, and gaming platforms. Skuld is also capable of harvesting crypto wallet seed phrases and passwords from the Exodus and Atomic crypto wallets. It accomplishes this using an approach called wallet injection that replaces legitimate application files with trojanized versions downloaded from GitHub. It's worth noting that a similar technique was recently put to use by a rogue npm package named pdf-to-office. The attack also employs a custom version of an open-source tool known as ChromeKatz to bypass Chrome's app-bound encryption protections. The collected data is exfiltrated to the miscreants via a Discord webhook. The fact that payload delivery and data exfiltration occur via trusted cloud services such as GitHub, Bitbucket, Pastebin, and Discord allows the threat actors to blend in with normal traffic and fly under the radar. Discord has since disabled the malicious bot, effectively breaking the attack chain. Check Point said it also identified another campaign mounted by the same threat actor that distributes the loader as a modified version of a hacktool for unlocking pirated games. The malicious program, also hosted on Bitbucket, has been downloaded 350 times. It has been assessed that the victims of these campaigns are primarily located in the United States, Vietnam, France, Germany, Slovakia, Austria, the Netherlands, and the United Kingdom. The findings represent the latest example of how cybercriminals are targeting the popular social platform, which has had its content delivery networkabused to host malware in the past. "This campaign illustrates how a subtle feature of Discord's invite system, the ability to reuse expired or deleted invite codes in vanity invite links, can be exploited as a powerful attack vector," the researchers said. "By hijacking legitimate invite links, threat actors silently redirect unsuspecting users to malicious Discord servers." "The choice of payloads, including a powerful stealer specifically targeting cryptocurrency wallets, suggests that the attackers are primarily focused on crypto users and motivated by financial gain." Found this article interesting? Follow us on Twitter  and LinkedIn to read more exclusive content we post. SHARE     #discord #invite #link #hijacking #delivers
    THEHACKERNEWS.COM
    Discord Invite Link Hijacking Delivers AsyncRAT and Skuld Stealer Targeting Crypto Wallets
    Jun 14, 2025Ravie LakshmananMalware / Threat Intelligence A new malware campaign is exploiting a weakness in Discord's invitation system to deliver an information stealer called Skuld and the AsyncRAT remote access trojan. "Attackers hijacked the links through vanity link registration, allowing them to silently redirect users from trusted sources to malicious servers," Check Point said in a technical report. "The attackers combined the ClickFix phishing technique, multi-stage loaders, and time-based evasions to stealthily deliver AsyncRAT, and a customized Skuld Stealer targeting crypto wallets." The issue with Discord's invite mechanism is that it allows attackers to hijack expired or deleted invite links and secretly redirect unsuspecting users to malicious servers under their control. This also means that a Discord invite link that was once trusted and shared on forums or social media platforms could unwittingly lead users to malicious sites. Details of the campaign come a little over a month after the cybersecurity company revealed another sophisticated phishing campaign that hijacked expired vanity invite links to entice users into joining a Discord server and instruct them to visit a phishing site to verify ownership, only to have their digital assets drained upon connecting their wallets. While users can create temporary, permanent, or custom (vanity) invite links on Discord, the platform prevents other legitimate servers from reclaiming a previously expired or deleted invite. However, Check Point found that creating custom invite links allows the reuse of expired invite codes and even deleted permanent invite codes in some cases. This ability to reuse Discord expired or deleted codes when creating custom vanity invite links opens the door to abuse, allowing attackers to claim it for their malicious server. "This creates a serious risk: Users who follow previously trusted invite links (e.g., on websites, blogs, or forums) can unknowingly be redirected to fake Discord servers created by threat actors," Check Point said. The Discord invite-link hijacking, in a nutshell, involves taking control of invite links originally shared by legitimate communities and then using them to redirect users to the malicious server. Users who fall prey to the scheme and join the server are asked to complete a verification step in order to gain full server access by authorizing a bot, which then leads them to a fake website with a prominent "Verify" button. This is where the attackers take the attack to the next level by incorporating the infamous ClickFix social engineering tactic to trick users into infecting their systems under the pretext of verification. Specifically, clicking the "Verify" button surreptitiously executes JavaScript that copies a PowerShell command to the machine's clipboard, after which the users are urged to launch the Windows Run dialog, paste the already copied "verification string" (i.e., the PowerShell command), and press Enter to authenticate their accounts. But in reality, performing these steps triggers the download of a PowerShell script hosted on Pastebin that subsequently retrieves and executes a first-stage downloader, which is ultimately used to drop AsyncRAT and Skuld Stealer from a remote server and execute them. At the heart of this attack lies a meticulously engineered, multi-stage infection process designed for both precision and stealth, while also taking steps to subvert security protections through sandbox security checks. AsyncRAT, which offers comprehensive remote control capabilities over infected systems, has been found to employ a technique called dead drop resolver to access the actual command-and-control (C2) server by reading a Pastebin file. The other payload is a Golang information stealer that's downloaded from Bitbucket. It's equipped to steal sensitive user data from Discord, various browsers, crypto wallets, and gaming platforms. Skuld is also capable of harvesting crypto wallet seed phrases and passwords from the Exodus and Atomic crypto wallets. It accomplishes this using an approach called wallet injection that replaces legitimate application files with trojanized versions downloaded from GitHub. It's worth noting that a similar technique was recently put to use by a rogue npm package named pdf-to-office. The attack also employs a custom version of an open-source tool known as ChromeKatz to bypass Chrome's app-bound encryption protections. The collected data is exfiltrated to the miscreants via a Discord webhook. The fact that payload delivery and data exfiltration occur via trusted cloud services such as GitHub, Bitbucket, Pastebin, and Discord allows the threat actors to blend in with normal traffic and fly under the radar. Discord has since disabled the malicious bot, effectively breaking the attack chain. Check Point said it also identified another campaign mounted by the same threat actor that distributes the loader as a modified version of a hacktool for unlocking pirated games. The malicious program, also hosted on Bitbucket, has been downloaded 350 times. It has been assessed that the victims of these campaigns are primarily located in the United States, Vietnam, France, Germany, Slovakia, Austria, the Netherlands, and the United Kingdom. The findings represent the latest example of how cybercriminals are targeting the popular social platform, which has had its content delivery network (CDN) abused to host malware in the past. "This campaign illustrates how a subtle feature of Discord's invite system, the ability to reuse expired or deleted invite codes in vanity invite links, can be exploited as a powerful attack vector," the researchers said. "By hijacking legitimate invite links, threat actors silently redirect unsuspecting users to malicious Discord servers." "The choice of payloads, including a powerful stealer specifically targeting cryptocurrency wallets, suggests that the attackers are primarily focused on crypto users and motivated by financial gain." Found this article interesting? Follow us on Twitter  and LinkedIn to read more exclusive content we post. SHARE    
    0 Comentários 0 Compartilhamentos 0 Anterior
  • Tanks, guns and face-painting

    Of all the jarring things I’ve witnessed on the National Mall, nothing will beat the image of the first thing I saw after I cleared security at the Army festival: a child, sitting at the controls of an M119A3 Howitzer, being instructed by a soldier on how to aim it, as his red-hatted parents took a photo with the Washington Monument in the background. The primary stated reason for the Grand Military Parade is to celebrate the US Army’s 250th birthday. The second stated reason is to use the event for recruiting purposes. Like other military branches, the Army has struggled to meet its enlistment quotas for over the past decade. And according to very defensive Army spokespeople trying to convince skeptics that the parade was not for Donald Trump’s birthday, there had always been a festival planned on the National Mall that day, and it had been in the works for over two years, and the parade, tacked on just two months ago, was purely incidental. Assuming that their statement was true, I wasn’t quite sure if they had anticipated so many people in blatant MAGA swag in attendance — or how eager they were to bring their children and hand them assault rifles. WASHINGTON, DC - JUNE 14: An Army festival attendee holds a M3 Carl Gustav Recoilless Rifle on June 14, 2025 in Washington, DC. Photo by Anna Moneymaker / Getty ImagesThere had been kid-friendly events planned: an NFL Kids Zone with a photo op with the Washington Commanders’ mascot, a few face-painting booths, several rock-climbing walls. But they were dwarfed, literally, by dozens of war machines parked along the jogging paths: massive tanks, trucks with gun-mounted turrets, assault helicopters, many of them currently used in combat, all with helpful signs explaining the history of each vehicle, as well as the guns and ammo it could carry. And the families — wearing everything from J6 shirts to Vineyard Vines — were drawn more to the military vehicles, all-too-ready to place their kids in the cockpit of an AH-1F Cobra 998 helicopter as they pretended to aim the nose-mounted 3-barrelled Gatling Cannon. Parents told their children to smile as they poked their little heads out of the hatch of an M1135 Stryker armored vehicle; reminded them to be patient as they waited in line to sit inside an M109A7 self-propelled Howitzer with a 155MM rifled cannon.Attendees look at a military vehicle on display. Bloomberg via Getty ImagesBut seeing a kid’s happiness of being inside a big thing that goes boom was nothing compared to the grownups’ faces when they got the chance to hold genuine military assault rifles — especially the grownups who had made sure to wear Trump merch during the Army’s birthday party.It seemed that not even a free Army-branded Bluetooth speaker could compare to how fucking sick the modded AR-15 was. Attendees were in raptures over the Boston Dynamics robot dog gun, the quadcopter drone gun, or really any of the other guns available.RelatedHowever many protesters made it out to DC, they were dwarfed by thousands of people winding down Constitution Avenue to enter the parade viewing grounds: lots of MAGA heads, lots of foreign tourists, all people who really just like to see big, big tanks. “Angry LOSERS!” they jeered at the protesters.and after walking past them, crossing the bridge, winding through hundreds of yards of metal fencing, Funneling through security, crossing a choked pedestrian bridge over Constitution Ave, I was finally dumped onto the parade viewing section: slightly muggy and surprisingly navigable. But whatever sluggishness the crowd was feeling, it would immediately dissipate the moment a tank turned the corner — and the music started blasting.Americans have a critical weakness for 70s and 80s rock, and this crowd seemed more than willing to look past the questionable origins of the parade so long as the soundtrack had a sick guitar solo. An M1 Abrams tank driving past you while Barracuda blasts on a tower of speakers? Badass. Black Hawk helicopters circling the Washington Monument and disappearing behind the African-American history museum, thrashing your head to “separate ways” by Journey? Fucking badass. ANOTHER M1 ABRAMS TANK?!?!! AND TO FORTUNATE SON??!?!? “They got me fucking hooked,” a young redheaded man said behind me as the crowd screamed for the waving drivers.Members of the U.S. Army drive Bradley Fighting Vehicles in the 250th birthday parade on June 14, 2025 in Washington, DC. Getty ImagesWhen you listen to the hardest fucking rock soundtrack long enough, and learn more about how fucking sick the Bradley Fighting Vehicles streaming by you are, an animalistic hype takes over you — enough to drown out all the nationwide anger about the parade, the enormity of Trump’s power grab, the fact that two Minnesota Democratic lawmakers were shot in their homes just that morning, the riot police roving the streets of LA.It helped that it didn’t rain. It helped that the only people at the parade were the diehards who didn’t care if they were rained out. And by the end of the parade, they didn’t even bother to stay for Trump’s speech, beelining back to the bridge at the first drop of rain.The only thing that mattered to this crowd inside the security perimeter — more than the Army’s honor and history, and barely more than Trump himself — was firepower, strength, hard rock, and America’s unparalleled, world-class ability to kill.See More:
    #tanks #guns #facepainting
    Tanks, guns and face-painting
    Of all the jarring things I’ve witnessed on the National Mall, nothing will beat the image of the first thing I saw after I cleared security at the Army festival: a child, sitting at the controls of an M119A3 Howitzer, being instructed by a soldier on how to aim it, as his red-hatted parents took a photo with the Washington Monument in the background. The primary stated reason for the Grand Military Parade is to celebrate the US Army’s 250th birthday. The second stated reason is to use the event for recruiting purposes. Like other military branches, the Army has struggled to meet its enlistment quotas for over the past decade. And according to very defensive Army spokespeople trying to convince skeptics that the parade was not for Donald Trump’s birthday, there had always been a festival planned on the National Mall that day, and it had been in the works for over two years, and the parade, tacked on just two months ago, was purely incidental. Assuming that their statement was true, I wasn’t quite sure if they had anticipated so many people in blatant MAGA swag in attendance — or how eager they were to bring their children and hand them assault rifles. WASHINGTON, DC - JUNE 14: An Army festival attendee holds a M3 Carl Gustav Recoilless Rifle on June 14, 2025 in Washington, DC. Photo by Anna Moneymaker / Getty ImagesThere had been kid-friendly events planned: an NFL Kids Zone with a photo op with the Washington Commanders’ mascot, a few face-painting booths, several rock-climbing walls. But they were dwarfed, literally, by dozens of war machines parked along the jogging paths: massive tanks, trucks with gun-mounted turrets, assault helicopters, many of them currently used in combat, all with helpful signs explaining the history of each vehicle, as well as the guns and ammo it could carry. And the families — wearing everything from J6 shirts to Vineyard Vines — were drawn more to the military vehicles, all-too-ready to place their kids in the cockpit of an AH-1F Cobra 998 helicopter as they pretended to aim the nose-mounted 3-barrelled Gatling Cannon. Parents told their children to smile as they poked their little heads out of the hatch of an M1135 Stryker armored vehicle; reminded them to be patient as they waited in line to sit inside an M109A7 self-propelled Howitzer with a 155MM rifled cannon.Attendees look at a military vehicle on display. Bloomberg via Getty ImagesBut seeing a kid’s happiness of being inside a big thing that goes boom was nothing compared to the grownups’ faces when they got the chance to hold genuine military assault rifles — especially the grownups who had made sure to wear Trump merch during the Army’s birthday party.It seemed that not even a free Army-branded Bluetooth speaker could compare to how fucking sick the modded AR-15 was. Attendees were in raptures over the Boston Dynamics robot dog gun, the quadcopter drone gun, or really any of the other guns available.RelatedHowever many protesters made it out to DC, they were dwarfed by thousands of people winding down Constitution Avenue to enter the parade viewing grounds: lots of MAGA heads, lots of foreign tourists, all people who really just like to see big, big tanks. “Angry LOSERS!” they jeered at the protesters.and after walking past them, crossing the bridge, winding through hundreds of yards of metal fencing, Funneling through security, crossing a choked pedestrian bridge over Constitution Ave, I was finally dumped onto the parade viewing section: slightly muggy and surprisingly navigable. But whatever sluggishness the crowd was feeling, it would immediately dissipate the moment a tank turned the corner — and the music started blasting.Americans have a critical weakness for 70s and 80s rock, and this crowd seemed more than willing to look past the questionable origins of the parade so long as the soundtrack had a sick guitar solo. An M1 Abrams tank driving past you while Barracuda blasts on a tower of speakers? Badass. Black Hawk helicopters circling the Washington Monument and disappearing behind the African-American history museum, thrashing your head to “separate ways” by Journey? Fucking badass. ANOTHER M1 ABRAMS TANK?!?!! AND TO FORTUNATE SON??!?!? “They got me fucking hooked,” a young redheaded man said behind me as the crowd screamed for the waving drivers.Members of the U.S. Army drive Bradley Fighting Vehicles in the 250th birthday parade on June 14, 2025 in Washington, DC. Getty ImagesWhen you listen to the hardest fucking rock soundtrack long enough, and learn more about how fucking sick the Bradley Fighting Vehicles streaming by you are, an animalistic hype takes over you — enough to drown out all the nationwide anger about the parade, the enormity of Trump’s power grab, the fact that two Minnesota Democratic lawmakers were shot in their homes just that morning, the riot police roving the streets of LA.It helped that it didn’t rain. It helped that the only people at the parade were the diehards who didn’t care if they were rained out. And by the end of the parade, they didn’t even bother to stay for Trump’s speech, beelining back to the bridge at the first drop of rain.The only thing that mattered to this crowd inside the security perimeter — more than the Army’s honor and history, and barely more than Trump himself — was firepower, strength, hard rock, and America’s unparalleled, world-class ability to kill.See More: #tanks #guns #facepainting
    WWW.THEVERGE.COM
    Tanks, guns and face-painting
    Of all the jarring things I’ve witnessed on the National Mall, nothing will beat the image of the first thing I saw after I cleared security at the Army festival: a child, sitting at the controls of an M119A3 Howitzer, being instructed by a soldier on how to aim it, as his red-hatted parents took a photo with the Washington Monument in the background. The primary stated reason for the Grand Military Parade is to celebrate the US Army’s 250th birthday. The second stated reason is to use the event for recruiting purposes. Like other military branches, the Army has struggled to meet its enlistment quotas for over the past decade. And according to very defensive Army spokespeople trying to convince skeptics that the parade was not for Donald Trump’s birthday, there had always been a festival planned on the National Mall that day, and it had been in the works for over two years, and the parade, tacked on just two months ago, was purely incidental. Assuming that their statement was true, I wasn’t quite sure if they had anticipated so many people in blatant MAGA swag in attendance — or how eager they were to bring their children and hand them assault rifles. WASHINGTON, DC - JUNE 14: An Army festival attendee holds a M3 Carl Gustav Recoilless Rifle on June 14, 2025 in Washington, DC. Photo by Anna Moneymaker / Getty ImagesThere had been kid-friendly events planned: an NFL Kids Zone with a photo op with the Washington Commanders’ mascot, a few face-painting booths, several rock-climbing walls. But they were dwarfed, literally, by dozens of war machines parked along the jogging paths: massive tanks, trucks with gun-mounted turrets, assault helicopters, many of them currently used in combat, all with helpful signs explaining the history of each vehicle, as well as the guns and ammo it could carry. And the families — wearing everything from J6 shirts to Vineyard Vines — were drawn more to the military vehicles, all-too-ready to place their kids in the cockpit of an AH-1F Cobra 998 helicopter as they pretended to aim the nose-mounted 3-barrelled Gatling Cannon. Parents told their children to smile as they poked their little heads out of the hatch of an M1135 Stryker armored vehicle; reminded them to be patient as they waited in line to sit inside an M109A7 self-propelled Howitzer with a 155MM rifled cannon.Attendees look at a military vehicle on display. Bloomberg via Getty ImagesBut seeing a kid’s happiness of being inside a big thing that goes boom was nothing compared to the grownups’ faces when they got the chance to hold genuine military assault rifles — especially the grownups who had made sure to wear Trump merch during the Army’s birthday party. (Some even handed the rifles to their children for their own photo ops.) It seemed that not even a free Army-branded Bluetooth speaker could compare to how fucking sick the modded AR-15 was. Attendees were in raptures over the Boston Dynamics robot dog gun, the quadcopter drone gun, or really any of the other guns available (except for those historic guns, those were only maybe cool).RelatedHowever many protesters made it out to DC, they were dwarfed by thousands of people winding down Constitution Avenue to enter the parade viewing grounds: lots of MAGA heads, lots of foreign tourists, all people who really just like to see big, big tanks. “Angry LOSERS!” they jeered at the protesters. (“Don’t worry about them,” said one cop, “they lost anyways.”) and after walking past them, crossing the bridge, winding through hundreds of yards of metal fencing, Funneling through security, crossing a choked pedestrian bridge over Constitution Ave, I was finally dumped onto the parade viewing section: slightly muggy and surprisingly navigable. But whatever sluggishness the crowd was feeling, it would immediately dissipate the moment a tank turned the corner — and the music started blasting.Americans have a critical weakness for 70s and 80s rock, and this crowd seemed more than willing to look past the questionable origins of the parade so long as the soundtrack had a sick guitar solo. An M1 Abrams tank driving past you while Barracuda blasts on a tower of speakers? Badass. Black Hawk helicopters circling the Washington Monument and disappearing behind the African-American history museum, thrashing your head to “separate ways” by Journey? Fucking badass. ANOTHER M1 ABRAMS TANK?!?!! AND TO FORTUNATE SON??!?!? “They got me fucking hooked,” a young redheaded man said behind me as the crowd screamed for the waving drivers. (The tank was so badass that the irony of “Fortunate Son” didn’t matter.)Members of the U.S. Army drive Bradley Fighting Vehicles in the 250th birthday parade on June 14, 2025 in Washington, DC. Getty ImagesWhen you listen to the hardest fucking rock soundtrack long enough, and learn more about how fucking sick the Bradley Fighting Vehicles streaming by you are (either from the parade announcer or the tank enthusiast next to you), an animalistic hype takes over you — enough to drown out all the nationwide anger about the parade, the enormity of Trump’s power grab, the fact that two Minnesota Democratic lawmakers were shot in their homes just that morning, the riot police roving the streets of LA.It helped that it didn’t rain. It helped that the only people at the parade were the diehards who didn’t care if they were rained out. And by the end of the parade, they didn’t even bother to stay for Trump’s speech, beelining back to the bridge at the first drop of rain.The only thing that mattered to this crowd inside the security perimeter — more than the Army’s honor and history, and barely more than Trump himself — was firepower, strength, hard rock, and America’s unparalleled, world-class ability to kill.See More:
    0 Comentários 0 Compartilhamentos 0 Anterior
CGShares https://cgshares.com