Man trapped inside driverless car as it spins in circles
127 comments
·January 9, 2025aetherspawn
Zak
I'd be impatient too if I was in a robot car that suddenly started driving erratically. There's no way to know whether it's going to continue harmlessly driving around a parking lot or suddenly do something dangerous.
I probably would have looked for a stop button in the app before calling a phone number because that's faster. It seems weird there isn't a physical stop button inside the car.
aetherspawn
Well, erratic seems like a bit of an exaggeration. It’s continuously driving the same circular route around a parking lot, but it’s not driving on a kerb or anything like that.
You can see on the map that it goes into the drop off lane, for some reason thinks it can’t drop him off, so drives back into the car park, turns around, and heads back into the drop off lane. It’s not like it’s stuck in full lock doing donuts.
macNchz
With most software I am immediately concerned about what other unforeseen problems might happen as soon as it seems like the system has entered some kind of un- or semi-handled failure state.
oneeyedpigeon
It's not illegal, but definitely erratic — the kind of thing the cops might well pull you over for, no? I understand the urgency since having to make a flight is one of the most expensive scheduling issues you could face.
627467
"hey, you were just stuck in an elevator going erratically up and down never opening the door for you to leave or stop. I don't know why you're freaking out about it"
throwaway290
Today it is driving in circles, tomorrow it is spinning in circles, next month all your doors lock up in 100 degree heat and your phone dies while you wait for support to pick up. Undefined behavior is undefined behavior.
kamaal
C'mon.
Sitting inside a malfunctioning robot taxi, helplessly, with no clarity why its doing that. And being afraid it could do some thing more erratic, like going fast and hit a wall, or do some other life threatening thing.
Bugs like these are not acceptable in machines where humans sit.
carlosjobim
>It seems weird there isn't a physical stop button inside the car.
Think it through once more.
For the down voters: It would be extremely dangerous if somebody pressed such a button when the car is going down the highway. The real world is not a video game where you can just abort or pause at any time. If you try, you will die.
Zak
An emergency stop button for a robot car shouldn't immediately disengage all other automation and apply maximum braking as the parent comment seems to be implying. That would, indeed be very dangerous.
An emergency stop routine should involve a code path designed to bring the car to a stop in a safe location as soon as practical. It should be thoroughly tested on degraded hardware with minimal branching relative to normal operation. It should also activate the hazard lights, contact the operator's emergency department, unlock all the doors, and lower the windows.
willvarfar
In his defence, I am imagining this being a really stressful time for the passenger. Many people - most, even - would be panicking at this point, and people react differently and irrationally in these kind of not-in-control scenarios.
krisoft
> I’m assuming that when he just did the 2-click thing the operator told him to do on his phone, the Waymo probably stopped and let him out
That's not really good though. It is not his job to fix the car. His protestation that the operator should do it, whatever "it" is, is a reasonable one.
ArnoVW
I get the impression he did not want to change apps as that would interrupt the video, and thus ‘proof’
3m
My first thought was that the guy sounded like an asshole too.
matsemann
Are you perfectly rational and calm when trapped in a malfunctioning vehicle that could harm you on a whim? I feel it's unfair to judge this person sitting relaxed in front of a computer.
fallinghawks
I was askance at him refusing the helpdesk request to do something in the app. One of my hats for decades was support and this feels like someone refusing to tell you what's on their screen, or whether a certain light is on, or to push a button.
secondcoming
How would someone force a driverless car to get stuck in an infinite loop?
Also, requiring someone to stop the car via an app is appalling. What if there's no phone reception?
criddell
I wonder if opening the door a bit would force the car to stop?
netsharc
I bet it would. I read the comments before watching the video, then I watched it. All the comments saying it's a stressful and dangerous situation because it could potentially become a dangerous situation make me facepalm. It must be American "(over-)abundance of caution"...
secondcoming
Not sure, but the guy did say he couldn't get out.
oneeyedpigeon
Or if your phone is out of battery. Or if the app fails, etc. etc.
madmulita
"He is holding it wrong"
kragen
These cars need an emergency stop button which brings the car to a halt as fast as could plausibly be safe and then opens the doors. This is standard equipment on all industrial machinery and used to be standard equipment on all trains as well; I got into a lot of trouble once for pulling the emergency stop cord on a train when I was young. The inconvenience and occasional danger of unwarranted emergency stops is almost certain to be outweighed by the lives saved in the rarer cases where the E-stop is necessary.
technothrasher
> I got into a lot of trouble once for pulling the emergency stop cord on a train when I was young.
You are the hero of every young child that has ever been on a train and wondered what would happen when they pulled that cord. I know that certainly was a dream of mine when I was four or so. I couldn't understand why my dad was so insistent I not try it.
mattlondon
Presumably opening the door would stop it? I would hope that would be something they thought about and do not lock the passengers in...
LocalH
I wouldn't assume anything about a modern system
oneeyedpigeon
He said "I can't get out the car", which suggests otherwise.
kragen
Maybe he was afraid to try because he thought that opening the door of a moving car would be dangerous? It is.
madmulita
How would a malfunctioning AI decide it's safe to stop?
kragen
It wouldn't decide whether it's safe to stop. A separate E-stop processor would have a preprogrammed stop routine that stops the vehicle as fast as it can stop safely, which is to say, without skidding and without making rear-end collisions inevitable if you're on the freeway.
crazygringo
> and without making rear-end collisions inevitable if you're on the freeway.
That means it has to decide whether it's safe to stop.
Stopping safely is no less complicated than the self-driving software in general. Sure you can have backup driver software, but it's just going to be the same software.
null
ajross
There was no emergency! The car wasn't "spinning", it was just driving around a parking lot and failing to find a location to stop. This was a navigation failure, not a control failure. And the solution is to call the human operators who can control it remotely. Which is what happened, and they did. Pulling an "emergency stop" level would have made things worse and not better.
Is it embarassing for Waymo? Sure. But the BBC basically lied to you here, and you bought it. The circumstance you're imagining is not what happened. Just watch the video.
kragen
I wasn't imagining a car spinning out of control. Being trapped inside a car indefinitely is dangerous: you could get carsick and have to throw up, urgently need to use a restroom, have a painful leg cramp that causes a lasting muscle injury, miss picking up your child from school, miss your appointment with your probation officer, be unable to obtain essential medication, suffer from heatstroke, or die of dehydration. An adult should never need to "call human operators" and ask their permission to get out of a car. Unless they're being arrested or something.
And if you hear that the car has run over a child that the operator didn't notice and is dragging it down the pavement under the car, it's essential for you to be able to hit the E-stop button rather than calling an operator to eventually stop the car.
sacado2
> the solution is to call the human operators who can control it remotely
Sounds like a recipe for failure, to be honest. In a potentially life or death situation, the last thing I want is to rely on a remote human being. Plus, if the device entered an erroneous state, I certainly don't trust it to correctly interpret a remote "emergency stop" signal.
If the machine goes crazy (and there is no world where driving around a parking lot until the end of time is the rational expected behavior), the only safe option is a big, red, cut-circuit emergency stop button.
kragen
I agree. Unfortunately just cutting circuits isn't enough; you also need to brake and possibly unlock the doors.
beanjuiceII
i am not sure i would say it was "just driving around a parking lot" it was not spinning but it _was_ just going in circles, watch the steering wheel and the background.. right turn pause right turn pause etc you see the same buildings etc. my spouse would have probably barfed all over that back seat
belter
So if there is no connectivity?
ajross
Presumably it pulls over and stops if the backend connection drops, no? This is a kind of goalpost motion: you're countering an argument about an incorrectly-identified failure mode by positing that the condition would still obtain if there was a dual mode failure instead (i.e. that the car would get stuck in a navigation loop and have a bug in its recovery from connectivity issues).
That's bad engineering, because you can always get to some kind of catastrophe if you keep piling on layers. Might as well just skip to "What if the motor circuit fails open and it accelerates forever?!" in that case.
dist-epoch
One problem is if you are on a freeway, you can't just stop.
Also passenger might be inebriated and randomly push the button.
While I agree with you on principle, it's a hard problem to solve.
Zak
> Also passenger might be inebriated and randomly push the button.
Modern societies are reasonably able to cope with potential misuse of publicly-accessible emergency equipment. Buildings have fire alarms. Airplanes have emergency exits with self-deploying slides. Trains have emergency brakes passengers can activate.
Misusing any of these can create a danger to others and is sure to be very disruptive. People do on occasion, and face punishment for their actions.
kragen
I didn't get arrested or fined, to be clear. I just got told not to do it again in a very stern voice.
kragen
Yes, I agree that pushing the E-stop button is not free of risk. But it's a risk under the control of the person who will suffer most of the consequences.
crazygringo
That's not true. If you're on the freeway you could wind up killing multiple people behind you. It could be extremely dangerous.
hleszek
Sorry, I'm afraid I can't do that, Dave, you're drunk.
psychlops
You could always tug the steering wheel. That's supposed to stop it.
belter
"...Additionally, you may not sit in the driver’s seat, touch the steering wheel, pedals, or driving controls.
Violating these rules will impact your account standing and may lead to permanent revocation of ride privileges..."
netsharc
"Well, I got in a wreck and had 3rd degree burns, but at least my account is in good standing!" (Yes I know it's a totally different scenario to the article's)
kragen
Yeah, safety regulators would also need to prosecute companies that discourage people from using the E-stop button with terms like those.
jordansmithnz
While this did happen and obviously isn’t great, there’s a more skeptical side too.
The ‘trapped’ man is the CEO of an AI consulting firm. He posted the video to linked in, didn’t press the abort button, and had a relatively successful customer support experience (5 minutes delayed and not charged for the trip). Apparently he is using a PR firm to handle questions.
Make of that what you will. Waymo can clearly do better, although for what it’s worth — my worst uber experiences have been much worse than this (legitimately unsafe driving decisions).
belter
The Abort button does not Abort. It looks for a place to stop. Why would that work on this situation: https://youtu.be/5QFGGqrA5os?t=3136 (52:16)
LinuxBender
What is the psychological term for people that defend defective heavy machinery that are not yet ready for production and yet being tested in production and in the general public? This will be the most highly unpopular opinion with this audience but if a human drove around in circles potentially blocking other drivers even in a parking lot a cop would pull them over and possibly have them take a sobriety test. If they passed they would get a warning and the incident would be logged with dispatch. Continued behavior would result in a ticket at best and drivers license suspension at worst. Please stop giving exciting tech a break and expect more. I am starting to lean towards requiring hardware and software developers and their respective companies that work on public heavy machinery to have federal certifications that can be revoked if they are moving fast and breaking things in public. Broken navigation in heavy machinery is a risk to the general public. This case was benign but the next one may not be.
I expect much more by now. I would expect that if anything cars would be actively protecting humans. If a toddler wanders into the road without a parent I would expect them by now to use a PA speaker to alert people and actively create a blockade around the child with only room for humans to pass, flash all their lights continuously and video stream to safety personnel on duty with the vehicle maker.
Zak
There's a belief in some circles that because technologies like self-driving have the potential to save lives, any failure to develop that technology as quickly as possible is akin to manslaughter.
I don't think it's a mental illness, just an extreme form of utilitarian ethics combined with an unreasonable faith in the outcome.
LinuxBender
That makes sense but I think that if they wish to sacrifice humans in the name of innovation that in theory can save lives, it should be tested in locations where all the humans opted-in to this level of sacrifice. Surely there must be a tech community of people that think like this where such in-production testing can occur. Every port of entry into that zone can have checkpoints requiring vehicles and pedestrians to display an opt-in QR code and some incentive to get them to participate. I know we can't waive negligence but at least we could contain the blast radius to a known zone.
shinryuu
Reminds me of this enlightening video mapping out a possible future where self-driving cars have dominance.
https://www.youtube.com/watch?v=040ejWnFkj0
It's very possible that it won't be the utopia it is portrayed to be.
klibertp
Utopia? I never experienced it myself, but the footage of the American roads in the video looks as dystopian as it can possibly get already... Somehow, the elevated freeways and what's below them remind me of the worst cyberpunk sprawl nightmares. I guess being free from the constraints of dealing with any buildings or areas older than 200 years doesn't necessarily lead to better designs...
derr1
Knew I wouldn't have to scroll far to find the link to this video.
I'm genuinely worried for the future if we keep going down this path. And all the hard work by urbanists will be undone.
NJB radicalised me, in the best way possible.
shinryuu
Sincerely hope that more people in this forum watches it. Since we are at the epicenter of it.
eknkc
Never been in one but these things should have a big red stop button on the roof or something. Don't they?
dudeinjapan
The big red button on the roof triggers the ejector seat. To stop you need to press the small blue button next to the cup holder.
jchw
I wish the video didn't end so quickly, because I'm actually curious what exactly had to happen next. I'm guessing having active involvement in both the passenger in the car and the support staff is a feature, since you wouldn't want rogue support operators to be able to just bring cars to abrupt halts or worse, so if the robot driver is unable to pull over safely, it sort of makes sense to require at least some sanity checking from the passengers themselves.
Of course, it's reasonable to be frustrated, especially if you're in a hurry, but anyone who has been in a vehicle accident in an Uber or taxi will know that there's just some inherent risk involved when you're getting into a car with another driver, just as there is risk involved when getting into a car and driving yourself, and I think the idea that a robot driver could fail and get stuck is something that you just have to be mindful of and account for, the same way you have to be mindful of the risk of getting stuck on an elevator. Without seeing the remainder of the altercation it's hard to judge if Waymo's design is lacking for fallbacks here. Another question would be what can be done if the passenger can't access the Waymo app; would they have to jump into the driver's seat?
tantalor
It's driving in a loop, not spinning in circles.
0x1ceb00da
Not just any loop, an infinite loop.
Ma8ee
How do you know it’s infinite?
TheDong
I ran it through my halting problem solver and it told me it doesn't halt, so I know it's infinite
tromp
I'm sure it's finite given the finite capacity of the car battery (or fuel tank).
secondcoming
By watching the video?
LandR
There's a halting problem joke in here somewhere.
londons_explore
Waymo rider support seems to be overly limited in their abilities.
Requiring the user to press buttons on an app, and not being able to do it for them is just one example. Plenty of other videos show they can't control the car, and all they can do is talk to the user.
At a minimum, they should have the ability to stop the car and drive it, remote control, at 1 mph, to get it unstuck or out of the way.
acomjean
I’m torn. If waymo can remote control drive the security implications….
Maybe the solution is a car rental service where the car drives to you, you drive where you want to go, then just leave the car.
In Boston anyway navigating pickup/ drop off at the airport is tricky for people.
whstl
> Waymo rider support seems to be overly limited in their abilities.
Yep. Like virtually every phone support person ever. You would think that with driverless cars this would be different.
voidUpdate
Were the doors locked, or could he have just opened one and gotten out at any time (obviously when it was going slowly) ? It just seems irresponsible to lock someone inside a car, but then if they weren't, then he wasn't exactly "trapped"
redeux
You're suggesting that he should what? Tuck and roll out of the vehicle and risk being run over as the first course of action?
voidUpdate
I mean... yeah. the car isn't exactly going fast, and if you get out of the other side then you're on the outside of the circle so you're not really likely to get run over, especially if you just do a little jump rather than rolling on the floor
hmmm-i-wonder
Seems like he could have ended the ride but didn't, and refused when instructed to knowing he wouldn't get his video to post online.
Not sure acting the ignorant consumer makes me trust the supposed dangerous situation you are in isn't manufactured in some way.
qgin
From the headline, you wouldn't know that they stopped the car about 15 seconds after he was connected to the remote assistance.
The guy seemed really impatient, talked over the operator while she was trying to give instructions, and then refused to follow instructions given by the phone operator.
Judging by how short the video is, I’m assuming that when he just did the 2-click thing the operator told him to do on his phone, the Waymo probably stopped and let him out.
I get vibes he did it on purpose for social media. For example, you’re supposed to be at the airport an hour before your flight, and here he is winging about a Waymo going in circles for a couple mins and threatening to send the bill for his flight to the operator? Eh?