WWW.TECHNOLOGYREVIEW.COM
Palmer Luckey on the Pentagons future of mixed reality
Palmer Luckey has, in some ways, come full circle.His first experience with virtual-reality headsets was as a teenage lab technician at a defense research center in Southern California, studying their potential to curb PTSD symptoms in veterans. He then built Oculus, sold it to Facebook for $2 billion, left Facebook after a highly public ousting, and founded Anduril, which focuses on drones, cruise missiles, and other AI-enhanced technologies for the US Department of Defense. The company is now valued at $14 billion.Now Luckey is redirecting his energy again, to headsets for the military. In September, Anduril announced it would partner with Microsoft on the US Armys Integrated Visual Augmentation System (IVAS), arguably the militarys largest effort to develop a headset for use on the battlefield. Luckey says the IVAS project is his top priority at Anduril.There is going to be a heads-up display on every soldier within a pretty short period of time, he told MIT Technology Review in an interview last week on his work with the IVAS goggles. The stuff that were buildingits going to be a big part of that.Though few would bet against Luckeys expertise in the realm of mixed reality, few observers share his optimism for the IVAS program. They view it, thus far, as an avalanche of failures.IVAS was first approved in 2018 as an effort to build state-of-the-art mixed-reality headsets for soldiers. In March 2021, Microsoft was awarded nearly $22 billion over 10 years to lead the project, but it quickly became mired in delays. Just a year later, a Pentagon audit criticized the program for not properly testing the goggles, saying its choices could result in wasting up to $21.88 billion in taxpayer funds to field a system that soldiers may not want to use or use as intended. The first two variants of the gogglesof which the army purchased 10,000 unitsgave soldiers nausea, neck pain, and eye strain, according to internal documents obtained by Bloomberg.Such reports have left IVAS on a short leash with members of the Senate Armed Services Committee, which helps determine how much money should be spent on the program. In a subcommittee meeting in May, Senator Tom Cotton, an Arkansas Republican and ranking member, expressed frustration at IVASs slow pace and high costs, and in July the committee suggested a $200 million cut to the program.Meanwhile, Microsoft has for years been cutting investments into its HoloLens headsetthe hardware on which the IVAS program is basedfor lack of adoption. In June, Microsoft announced layoffs to its HoloLens teams, suggesting the project is now focused solely on serving the Department of Defense. The company received a serious blow in August, when reports revealed that the Army is considering reopening bidding for the contract to oust Microsoft entirely.This is the catastrophe that Luckeys stepped into. Andurils contribution to the project will be Lattice, an AI-powered system that connects everything from drones to radar jammers to surveil, detect objects, and aid in decision-making. Lattice is increasingly becoming Andurils flagship offering. Its a tool that allows soldiers to receive instantaneous information not only from Andurils hardware, but also from radars, vehicles, sensors, and other equipment not made by Anduril. Now it will be built into the IVAS goggles. Its not quite a hive mind, but its certainly a hive eye is how Luckey described it to me.Anvil, seen here held by Luckey in Andurils Costa Mesa Headquarters, integrates with the Lattice OS and can navigate autonomously to intercept hostile drones.PHILIP CHEUNGBoosted by Lattice, the IVAS program aims to produce a headset that can help soldiers rapidly identify potential threats and take decisive action on the battlefield, according to the Army. If designed well, the device will automatically sort through countless pieces of informationdrone locations, vehicles, intelligenceand flag the most important ones to the wearer in real time.Luckey defends the IVAS programs bumps in the road as exactly what one should expect when developing mixed reality for defense. None of these problems are anything that you would consider insurmountable, he says. Its just a matter of if its going to be this year or a few years from now. He adds that delaying a product is far better than releasing an inferior product, quoting Shigeru Miyamoto, the game director of Nintendo: A delayed game is delayed only once, but a bad game is bad forever.Hes increasingly convinced that the military, not consumers, will be the most important testing ground for mixed-reality hardware: Youre going to see an AR headset on every soldier, long before you see it on every civilian, he says. In the consumer world, any headset company is competing with the ubiquity and ease of the smartphone, but he sees entirely different trade-offs in defense.The gains are so different when we talk about life-or-death scenarios. You dont have to worry about things like Oh, this is kind of dorky looking, or Oh, you know, this is slightly heavier than I would prefer, he says. Because the alternatives of, you know, getting killed or failing your mission are a lot less desirable.Those in charge of the IVAS program remain steadfast in the expectation that it will pay off with huge gains for those on the battlefield. If it works, James Rainey, commanding general of the Army Futures Command, told the Armed Services Committee in May, it is a legitimate 10x upgrade to our most important formations. Thats a big if, and one that currently depends on Microsofts ability to deliver. Luckey didnt get specific when I asked if Anduril was positioning itself to bid to become IVASs primary contractor should the opportunity arise.If that happens, US troops may, willingly or not, become the most important test subjects for augmented- and virtual-reality technology as it is developed in the coming decades. The commercial sector doesnt have thousands of individuals within a single institution who can test hardware in physically and mentally demanding situations and provide their feedback on how to improve it.Thats one of the ways selling to the defense sector is very different from selling to consumers, Luckey says: You dont actually have to convince every single soldier that they personally want to use it. You need to convince the people in charge of him, his commanding officer, and the people in charge of him that this is a thing that is worth wearing. The iterations that eventually come from IVASif it keeps its fundingcould signal whats coming next for the commercial market.When I asked Luckey if there were lessons from Oculus he had to unlearn when working with the Department of Defense, he said theres one: worrying about budgets. I prided myself for years, you knowIm the guy whos figured out how to make VR accessible to the masses by being absolutely brutal at every part of the design process, trying to get costs down. That isnt what the DOD wants, he says. They dont want the cheapest headset in a vacuum. They want to save money, and generally, spending a bit more money on a headset that is more durable or that has better visionand therefore allows you to complete a mission fasteris definitely worth the extra few hundred dollars.I asked if hes impressed by the progress thats been made during his eight-year hiatus from mixed reality. Since he left Facebook in 2017, Apple, Magic Leap, Meta, Snap, and a cascade of startups have been racing to move the technology from the fringe to the mainstream. Everything in mixed reality is about trade-offs, he says. Would you like more computing power, or a lighter and more comfortable headset?With more time at Meta, I would have made different trade-offs in a way that I think would have led to greater adoption, he says. But of course, everyone thinks that. While hes impressed with the gains, having been on the inside, I also feel like things could be moving faster.Years after leaving, Luckey remains noticeably annoyed by one specific decision he thinks Meta got wrong: not offloading the battery. Dwelling on technical details is unsurprising from someone who spent his formative years living in a trailer in his parents driveway posting in obscure forums and obsessing over goggle prototypes. He pontificated on the benefits of packing the heavy batteries and chips in removable pucks that the user could put in a pocket, rather than in the headset itself. Doing so makes the headset lighter and more comfortable. He says he was pushing Facebook to go that route before he was ousted, but when he left, it abandoned the idea. Apple chose to have an external battery for its Vision Pro, which Luckey praised.Anyway, he told me. Im still sore about it eight years later.Speaking of soreness, Luckeys most public professional wound, his ouster from Facebook in 2017, was partially healed last month. The storyinvolving countless Twitter threads, doxxing, retractions and corrections to news articles, suppressed statements, and a significant segment in Blake Harriss 2020 book The History of the Futureis difficult to boil down. But heres the short version: A donation by Luckey to a pro-Trump group called Nimble America in late 2016 led to turmoil within Facebook after it was reported by the Daily Beast. That turmoil grew, especially after Ars Technica wrote that his donation was funding racist memes (the founders of Nimble America were involved in the subreddit r/TheDonald, but the organization itself was focused on creating pro-Trump billboards). Luckey left in March 2017, but Meta has never disclosed why.This April, Oculuss former CTO John Carmack posted on X that he regretted not supporting Luckey more. Metas CTO, Andrew Bosworth, argued with Carmack, largely siding with Meta. In response, Luckey said, You publicly told everyone my departure had nothing to do with politics, which is absolutely insane and obviously contradicted by reams of internal communications. The two argued. In the X argument, Bosworth cautioned that there are limits on what can be said here, to which Luckey responded, I am down to throw it all out there. We can make everything public and let people judge for themselves. Just say the word.Six months later, Bosworth apologized to Luckey for the comments. Luckey responded, writing that although he is infamously good at holding grudges, neither Bosworth nor current leadership at Meta was involved in the incident.By now Luckey has spent years mulling over how much of his remaining anger is irrational or misplaced, but one thing is clear. He has a grudge left, but its against people behind the scenesPR agents, lawyers, reporterswho, from his perspective, created a situation that forced him to accept and react to an account he found totally flawed. Hes angry about the steps Facebook took to keep him from communicating his side (Luckey has said he wrote versions of a statement at the time but that Facebook threatened further escalation if he posted it).What am I actually angry at? Am I angry that my life went in that direction? Absolutely, he says.I have a lot more anger for the people who lied in a way that ruined my entire life and that saw my own company ripped out from under me that Id spent my entire adult life building, he says. Ive got plenty of anger left, but its not at Meta, the corporate entity. Its not at Zuck. Its not at Boz. Those are not the people who wronged me.While various subcommittees within the Senate and House deliberate how many millions to spend on IVAS each year, what is not in question is the Pentagon is investing to prepare for a potential conflict in the Pacific between China and Taiwan. The Pentagon requested nearly $10 billion for the Pacific Deterrence Initiative in its latest budget. The prospect of such a conflict is something Luckey considers often.He told the authors of Unit X: How the Pentagon and Silicon Valley Are Transforming the Future of War that Andurils entire internal road map has been organized around the question How do you deter China? Not just in Taiwan, but Taiwan and beyond?At this point, nothing about IVAS is geared specifically toward use in the South Pacific as opposed to Ukraine or anywhere else. The design is in early stages. According to transcripts of a Senate Armed Services Subcommittee meeting in May, the military was scheduled to receive the third iteration of IVAS goggles earlier this summer. If they were on schedule, theyre currently in testing. That version is likely to change dramatically before it approaches Luckeys vision for the future of mixed-reality warfare, in which you have a little bit of an AI guardian angel on your shoulder, helping you out and doing all the stuff that is easy to miss in the midst of battle.Designs for IVAS will have to adapt amid a shifting landscape of global conflict.PHILIP CHEUNGBut will soldiers ever trust such a guardian angel? If the goggles of the future rely on AI-powered software like Lattice to identify threatssay, an enemy drone ahead or an autonomous vehicle racing toward youAnduril is making the promise that it can sort through the false positives, recognize threats with impeccable accuracy, and surface critical information when it counts most.Luckey says the real test is how the technology compares with the current abilities of humans. In a lot of cases, its already better, he says, referring to Lattice, as measured by Andurils internal tests (it has not released these, and they have not been assessed by any independent external experts). People are fallible in ways that machines arent necessarily, he adds.Still, Luckey admits he does worry about the threats Lattice will miss.One of the things that really worries me is theres going to be people who die because Lattice misunderstood something, or missed a threat to a soldier that it should have seen, he says. At the same time, I can recognize that its still doing far better than people are doing today.When Lattice makes a significant mistake, its unlikely the public will know. Asked about the balance between transparency and national security in disclosing these errors, Luckey said that Andurils customer, the Pentagon, will receive complete information about what went wrong. Thats in line with the Pentagons policies on responsible AI adoption, which require that AI-driven systems be developed with methodologies, data sources, design procedures, and documentation that are transparent to and auditable by their relevant defense personnel.However, the policies promise nothing about disclosure to the public, a fact thats led some progressive think tanks, like the Brennan Center for Justice, to call on federal agencies to modernize public transparency efforts for the age of AI.Its easy to say, Well, shouldnt you be honest about this failure of your system to detect something? Luckey says, regarding Andurils obligations. Well, what if the failure was because the Chinese figured out a hole in the system and leveraged that to speed past our defenses of some military base? Id say theres not very much public good served in saying, Attention, everyonethere is a way to get past all of the security on every US military base around the world. I would say that transparency would be the worst thing you could do.
0 Kommentare
0 Anteile
35 Ansichten