Leaked Apple meeting shows how dire the Siri situation is

All Apple needs to do is allow 3rd party apps to integrate with Siri, then use an LLM as a way to convert natural language into the set of allowed API calls. Basically similar what chatGPT does with function calling. And on osx, they should have already had a head start due to applescript integration everywhere in most apps. I have no idea why they're trying to reinvent the wheel with "Shortcuts" which is severely limited.

They already have everything waiting there for their taking, and they're squandering it for no reason. Siri on osx should have been built on top of AppleScript from the get-go, then the switch to LLM would have been easy.

For that matter, I wonder why on osx I haven't seen any 3rd party apps be a siri replacement using applescript to drive applications directly. So much effort is spent on screen scraping and trying to get "agents" to use computers like humans do, but for decades osx has already had a parallel set of APIs built into every application specifically for machine consumption. And most good 3rd party apps even make use of it.

I’m not sure I want / need an LLM for the handful of basic commands I want Siri to do in the car with no complication. Siri is good at some straightforward command patterns. And that’s how users have been trained.

Adding an LLM feels like a solution looking for a problem

Everyone thought LLMs would scale exponentially like everything else in computer science. It's actually linear and they're all stuck holding a bag of mostly worthless goods.

> The models are incredibly adept at providing a proof-of-concept. But it's often a monumental endeavour to cross the gap

That's not just with LLMs. This has been an Achille's heel of demos for decades.

It's usually quite easy to make a demo of something sexy, but making it ship, is a very, very different thing.

This is made exponentially worse, by managers thinking that the demo means "we're almost ready," and even by the engineers, themselves, who are often not experienced enough to understand that demo != ship. Also, it has been my experience, that researchers that come up with amazing ideas, have absolutely no appreciation for what it takes to turn their ideas into ship product. Since Apple is all about ship product, they have to bridge that gulf. They are trying to commoditize something that is mostly buzz and raw research, right now.

I'm really pretty jaded about ML and LLMs in general, right now. I feel as if a whole bunch of really sexy demos were released, creating a huge level of buzz, but that the shipping product is still a ways off.

> But it's often a monumental endeavour to cross the gap between 70% and 90% accurcacy; 90%-95% even more.

That sounds exactly what we went through in the late 90's with voice recognition (remember dragon naturally speaking and via voice?).

I’m getting this sense too, as my own employer is starting to add AI features. The pitches for what is possible are not grounded in what engineers know the system can do, they’re just assumptions about “if we give the LLM these inputs, we’d expect to see these outputs that are accurate enough for meaningful productivity gains.”

I’m not an AI skeptic, but it’ll be interesting to see how we manage the uncertainty in these projects.

> so I switched it back.

Google has helpfully decided that you don't actually want to do that:

https://blog.google/products/gemini/google-assistant-gemini-...

later this year, the classic Google Assistant will no longer be accessible on most mobile devices

Using a fully-fledged flagship model as a personal assistant actually works pretty well. You don't need a task app, just tell the AI to remember and remind you. Granted, now you have to inject prompts to work around context limitations, allow the AI to schedule wake ups for itself or allow it to send notifications, etc. But those are issues with straight forward solutions. The issue is that for now those models are incredibly expensive and resource intensive to run. And a quantized 3.25B model like Gemini Nano just can't perform the same.

Yup, had to stop using it, next step was to get rid of Gemini altogether. It's so weird to claim you give people an advanced AI when actually you are removing a whole bunch of features.

My Nest can still only get “ok google turn on the light” the second time I say it. Good shit!

Their ai summary really eats shit when it comes to slang heavy, short messages.

It’d be genuinely useful if it kicked on in threads with high amounts of unread messages

It's not even to the level of Apple Intelligence. If I tell Siri to set a timer for 5 minutes, it goes off, and I say "Hey Siri, stop alarm," it stops my daily wake-up alarm, not the timer. It feels like Siri hasn't gotten any better in the past decade.

The same year the tech industry got taken by storm by LLMs, Apple announced a brutally expensive, niche VR/AR headset with no particular use case. That product was essentially DOA.

I think this is the first time in decades where Apple took a step from their back foot. The Vision Pro announcement during the initial hype cycle of ChatGPT made it look like they were just totally out of step with the industry. It was a dud product that cost billions in R&D, the market for it was unclear, and the industry zeitgeist (and talent) was going in a totally different direction.

So their response was to rush something out the door to avoid looking like they were caught flat-footed. Now it's abundantly obvious they were unprepared for AI, they've failed to ship features they promised, and they have to commit resources to support an additional platform that no one cares about because they did ship that one. Worst of all worlds.

Had they slept on the Vision Pro, I'm pretty convinced they could have credibly kept their powder dry on LLMs, like they've done with many industry hype cycles in the last. People would've assumed they were doing the Apple thing of not being first to ship something, but the first to ship compelling.

On a related note, I don’t know how Apple has managed to mess up so badly the idea of storing your own information like address, phone number, etc.

Every time I’ve moved, for months afterwards I keep having to fix my address when using Apple Pay or setting directions to home, etc. because it seems like they keep multiple copies of that information all over the place, instead of referring to a single piece of information.

Places I can think of where I’ve had to update my address one by one: each credit card I have associated with Apple Pay; Maps; my own contact that I can share with others; subscription information; shipping address for Apple Pay.

I get it that some people use separate billing and shipping addresses, but for the vast majority of people the use case is “I have a single home address where I want to direct Maps to, be billed at, and ship stuff to, please update it all at once when I update my address.”

FYI, I find that "navigate home" works, if you want to get home. (I agree with your complaint however that Siri is brain dead much of the time)

Enterprise MDM and Apple Configurator can disable voice assistant entirely, which may be needed in some work contexts.

And how, given Apple's Severance-grade secrecy?