Why doesnt Siri know what it doesnt know?
www.macworld.com
MacworldKids ask questions all the time, and I think its important as a parent to admit when you dont know the answer. Naturally, its tempting to bluff, in order to preserve their childlike belief in your omniscience, but youre only storing up trouble for that moment when they gain access to Google and find out for themselves that the president of Cuba is not, in fact, Fray Bentos.Not everyone, however, feels the same way about admitting ignorance. Take Siri, for example, which loves to take a swing at tasks when it isnt even sure what you just said. Ask your HomePod to play a song in a slightly muffled voice and instead of saying, I didnt catch that, would you mind repeating yourself? it just plows ahead and plays something random.Ive been complaining about Siri for a few years now and as far as I can see nothing has improved in that time. The arrival of Apple Intelligence is supposed to bring a new and improved version of Siri, but of course that doesnt apply to the HomePod; and even on devices like the iPhone which do have Apple Intelligence there doesnt seem to be more accuracy, just a greater degree of misplaced confidence about the inaccuracy.Last week the blogger One Foot Tsunami decided to ask Siri, in order, who won each Super Bowla seemingly straightforward taskand received a set of answers that were hilarious and horrifyingly inaccurate. It got just 34 percent correct, and at least one of those was right for the wrong reason. (Asked about Super Bowl X, it talked at length about Super Bowl IX which happened to have the same winner.) At one point it got the answer wrong 15 times in a row, and it gave the Philadelphia Eagles 33 wins rather than the one they actually collected.I dont think its especially important for voice assistants or the AI models beneath them to be able to answer questions about sports trivia, but theres a principle at stake here that is important: they need to admit when they dont know the correct answer. In this case, there was no harm done because the matter at hand was, well, trivial. But if someone asks about, say, the lug nut torque for a 2015 Nissan Frontier, being confidently given a totally wrong answer could have damaging consequences.To admit you dont know the answer, sadly, you first have to know that you dont know the answer, and this is AIs great weakness. AI doesnt know anything in any kind of sense wed understand. It matches patterns without understanding what they signify, which means it doesnt have any way of sanity-checking a fact that a human would instantly see as nonsense. But at the same time, it would be nice if AI developers put more thought into analyzing and signaling to the user a models degree of confidence in a specific output: a rating of the quantity and quality of training data on that topic, for example, or the proportion of similar queries which have received positive feedback.What wont cut it, in my opinion, is a generalized disclaimer. Following complaints about mangled notification summaries, Apple recently agreed to tweak the way that Apple Intelligence handles that function, but the immediate changes all affect not the information itself, but the way the information is presented: theres a clearer label that the feature is a beta and may contain errors, while the summarized text appears in italics to distinguish it from a standard notification. Initially, this may help, because the changed format will stand out. But it will quickly fade into visual furniture, much like the disclaimers on Googles AI results intended to prevent lawsuits from incorrect torque information. We need a specific disclaimer that one result is shaky (this product contains nuts) rather than a blunderbuss disclaimer that all of them are (any products made by this company or its subsidiaries may contain nuts).At the moment AI is in its hype cycle, and all the players are jockeying for position in the public eye. Right now, the main thing theyre aiming for is PR, and it makes sense in that context that they would want to make exaggerated claims for their products capabilities, and dislike the idea of signaling their limitations. But Apple of all people should understand that reputations are built up slowly and lost very quickly. Its better to be honestly ignorant than confidently incorrect. And the phrase I most want to hear from Siri is Im sorry, Dave. Im afraid I cant do that.FoundryWelcome to our weekly Apple Breakfast column, which includes all the Apple news you missed last week in a handy bite-sized roundup. We call it Apple Breakfast because we think it goes great with a Monday morning cup of coffee or tea, but its cool if you want to give it a read during lunch or dinner hours too.Trending: Top storiesIf Apple Intelligence is so great, why doesnt Apple trust us to turn it on?Mahmoud Itani reminds us all about 6 forgotten Apple products getting an update this year.Team up with Apple at your own risk.Its Samsungs turn to copy Apple! Here are over a dozen ways Samsung ripped off Apple at the Galaxy S25 launch.Apples wild new Miami store takes carbon neutral to new heights.U.K. iPhone users will soon be able to store digital licenses and IDs.Podcast of the weekThe rumors are heating up for Apples next major iPhone release. In the latest episode of the Macworld Podcast, we cover whats going on with the iPhone SE and what we could see in the next few weeks.You can catch every episode of the Macworld Podcast onSpotify,Soundcloud, thePodcasts app, orour own site.Reviews cornerMicrosoft Word for Mac review: Whats new in Word 2024?Microsoft Excel review: Whats new in Excel 2024?Apple Passwords review: Free and simple password manager.pCloud review: Back up your Mac to the cloud.The rumor milliPhone 17 Air leak shows off new camera bar design.New leak changes everything we know about the iPhone SE 4.Software updates, bugs, and problemsHate the new Mail app on your iPhone? Heres how to change it back.Starting this week, youll need to turn off Apple Intelligence if you dont want it.Mac users will finally get the smarter Mail app in macOS 15.4.AirPods Pro 2 update in coming weeks to expand hearing aid feature.And with that, were done for this weeks Apple Breakfast. If youd like to get regular roundups, sign up forour newsletters. You can also follow uson Facebook, Threads, or Twitter for discussion of breaking Apple news stories. See you next Monday, and stay Appley.
0 Commentarios ·0 Acciones ·29 Views