Predictions Scorecard, 2025 January 01

I can tell you as someone that crosses paths almost everyday with a Waymo car, they absolutely due work. I would describe their driving behavior as very safe and overly cautious. I’m far more concerned of humans behind the wheel.

Agreed Waymo cars can drive. Also I don't believe that, say, when a city bus stops on a narrow street near a school crosswalk, that the decision to edge out and around it is made on board the car, as I saw recently. The "car" made the right decision, drove it perfectly, and was safe at all times, but I just don't think anyone but a human in a call center said yes to that.

Which structural limits of TF2 and PyTorch were fixed via the Jax ecosystem?

Does Waymo run on JAX?

I disagree that regular interventions every two trips where you have no control over pickup or dropoff points counts as full self driving.

But that definition doesn’t even matter. The key factor is whether the additional overhead, whatever percentage it is, makes economic sense for the operator or the customer. And it seems pretty clear the economics aren’t there yet.

In what sense is self-driving “here” if the economics alone prove that it can’t get “here”? It’s not just limited coverage, it’s practically non-existent coverage, both nationally and globally, with no evidence that the system can generalize, profitably, outside the limited areas it’s currently in.

Does Wayne operate in heavy rain and any kind of snow or ice conditions?

To apply this benchmark, you'd have to believe that Waymo is paying operators to improve the quality of the ride, not to make the ride possible at all. That is, you'd have to believe that the fully autonomous car works and gets you to your destination safely and in a timely manner (at the level of a median professional human driver), but Waymo decided that's not good enough and hired operators to improve beyond that. This seems very unlikely to me, and some of the (few) examples I've seen online were about correcting significant failures, such as waiting behind a parked truck indefinitely (as if it were stopped at a red light) or looping around aimlessly in a parking lot.

You'd also have to believe that when you wished to change how your Uber driver drove, you'd actually have improved things rather than worsened them.

Human interventions are some non zero number in current self driving cars and will likely be that way for a while. Does this mean self driving is a scam and in fact it is just a human driving, and that these are actually ADAS. Maybe in some pedantic sense, you are right but then your definition is not useful, since it lumps cruise control/ lane-keeping ADAS and Waymo’s in the same category. Waymo is genuinely, qualitatively a big improvement above any ADAS/ self driving system that we have seen. I suspect Rodney did not predict even Waymo’s to be possible, but gave himself enough leeway so that he can pedantically argue that Waymo’s are just ADAS and that his prediction was right.

Yeah, I think semi-autonomous vehicles are a huge milestone and should be celebrated but the jump from semi-autonomous to fully-autonomous will, I think, feel noticeably different. It will be a moment future generations have trouble imagining a world where drunk or tired driving was ever even an issue.

You can make exactly the opposite argument as well: You think that we can still define cars as human-driven even when they have self-driving features (e.g. lane keeping). If the car is self-driving in even the smallest way, then they simply are not human-operated by any sensible definition.

That's how all innovation works. Ford never said people asked for a faster horse, but the theory holds. It doesn't matter what benchmarks you set, the market finds an interesting way to satisfy people's needs.

The prediction is:

> First driverless "taxi" service in a major US city, with dedicated pick up and drop off points, and restrictions on weather and time of day.

Their 2025 analysis is: "This is unlikely to happen in the first half of this century."

The prediction is clear. The evaluation is dishonest.

It’s interesting that in my reading of the post I felt like he hardly talked about Tesla at all.

He calls out that Tesla FSD has been “next year” for 11 years, but then the vast majority of the self-driving car section is about Cruise and Waymo. He also minorly mentions Tesla’s promise of a robotaxi service and how it is unlikely to be materially different than Cruise/Waymo. The amount of space allocated to each made sense as I read it.

For the meat of the issue: I can regularly drive places without someone else intervening. If someone else had to intervene in my driving 1/100 miles, even 1/1000 miles, most would probably say I shouldn’t have a license.

Yes, getting stuck behind a parked car or similar scenario is a critical flaw. It seems simple and non-important because it is not dangerous, but it means the drive would not be completed without a human. If I couldn’t drive to work because there was a parked car on my home street, again, people would question whether I should be on the road, and I’d probably be fired.

Interesting, that wasn't my takeaway from the article at all!

Direct quote from the article:

> Then I will weave them together to explain how it is still pretty much business as usual, and I mean that in a good way, with steady progress on both the science and engineering of AI.

There are some extremely emotional defences of Waymo on this comment thread. I don't quite understand why? Are they somehow inviolable to constructive criticism in the SV crowd?

> I believe some devs were probably replaced by AI, but not a large amount.

I'm not even sold on the idea that there were any. The media likes to blame AI for the developer layoffs because it makes a much more exciting story than interest rates and arcane tax code changes.

But the fact is that we don't need more than the Section 174 changes and the end of ZIRP to explain what's happened in tech. Federal economic policy was set up to direct massive amounts of investment into software development. Now it's not. That's a real, quantifiable impact that can readily explain what we've seen in a way that the current productivity gains from these tools simply can't.

Now, I'll definitely accept that many companies are attributing their layoffs to AI, but that's for much the same reason that the media laps the story up: it's a far better line to feed investors than that the financial environment has changed for the worse.

It's pretty much impossible to get work as a copywriter now

I imagine there aren't really layoffs, but slowing/stopping of hiring as you get more productivity out of existing devs. I imagine in the future, lots of companies will just let their employee base slowly attrition away.

Yeah, the AgentForce thing is a classic example. Internal leaks say Salesforce is using it as cover for more regular (cost cutting based) layoffs. People who've actually evaluated AgentForce don't think it's ready for prime time. It's more smoke and mirrors (and lots of marketing).

Solid state batteries. Prototypes work, but high-volume manufacturing doesn't work yet. The major battery manufacturers are all trying to get this to production. Early versions will probably be expensive.

Maybe a 2x improvement in kwh/kg. Much less risk of fire or thermal runaway. Charging in < 10 mins.

I think there are ~3 major battery improvements to watch out for.

1. Solid state batteries. Likely to be expensive, but promise better energy density.

2. Some really good grid storage battery. Likely made with iron or molten salt or something like that. Dirt cheap, but horrible energy density.

3. Continued Lithium ion battery improvements, e.g. cheaper, more durable etc.

I think the difference is that a helicopter is extremely technical to fly requiring complex and expensive training, and the eVTOL is supposed to be extremely simple to fly. Also the eVTOL in principle is really cheap to make if you just consider the materials and construction costs- probably eventually much cheaper than a car.

I was curious so I looked up how much you can buy the cheapest new helicopters for, and they are cheaper than an eVTOL right now- the XE composite is $68k new, and things like that can be ~25k used. I'm shocked one can in principle own a working helicopter for less than the price of a 7 year old Toyota Camry.

Can you imagine thousands of flying cars flying low over urban areas?

Skill level needed for "driving" would increase by a lot, noise levels would be abysmal, security implications would be severe (be they intentional or mechanical in nature), privacy implications would result in nobody wanting to have windows.

This is all more-or-less true for drones as well, but their weight is comparable to a todler, not to a polar bear. I firmly believe they'll never reach mass usage, but not because they're impossible to make.

I had a friend who used to (still does) fly RC helicopters; that requires quite a bit of skill. Meanwhile, I think anybody can fly a DJI drone. I think that's what will transform "flying" when anybody, not just a highly skilled pilot, can "drive" a flying car (assuming it can be as safe as a normal car... which somehow I doubt)

If the predictions are meant to be bold, then yes. If they're meant to be fairly obvious, then no.

For example, saying that flying cars will be in widespread use NET 2025 is not much of a prediction. I think we can all say that if flying cars will be in widespread use, it will happen No Earlier Than 2025. It could happen in 2060, and that NET 2025 prediction would still be true. He could mark it green in 2026 and say he was right, that, yes, there are no flying cars, and so mark his scorecard another point in the correct column. But is that really a prediction?

A bolder prediction would be, say "Within 1-2 yrs of XX".

So what is Rodney Brooks really trying to predict and say? I'd rather read about what the necessary gating conditions are for something significant and prediction-worthy to occur, or what the intractable problems are that would make something not be possible within a predicted time, rather than reading about him complain about how much overhype and media sensation there is in the AI and robotics (and space) fields. Yes, there is, but that's not much of a prediction or statement either, as it's fairly obvious.

There's also a bit of an undercurrent of complaint in this long article about how the not-as-sexy or hyped work he has done for all those years has gone relatively unrewarded and "undeserving types" are getting all the attention (and money). And as such, many of the predictions and commentary on them read more as rant than as prediction.