Skip to content(if available)orjump to list(if available)

A computer upgrade has shut down BART

A computer upgrade has shut down BART

164 comments

·September 5, 2025

VonGuard

Sooooo much snark, and so little interest into what BART actually runs on!

Originally, BART was a master stroke of digital integration in the 70's, and it's digital voices announcing the next trains were a thing of the future: An early accessibility feature before we even knew what those were, really.

Reading:

https://www.bart.gov/about/history

https://www.bart.gov/about/projects/traincontrol#:~:text=To%...

null

[deleted]

ninetyninenine

I mean despite it's history the snark is well deserved. With so many companies and people in the bay paying taxes, where the hell does all the money go?

Interesting, tidbit you added here. But snark is needed for this situation.

IshKebab

Yeah I was pretty blown away when I visited San Francisco just how archaic it was. In the same place you have driverless cars you have a metro payment system from like 70s USSR or something.

zolland

How did you pay? I made a Clipper account that I fill up with my credit card and tap my phone to pay...

xattt

[delayed]

thephyber

It’s almost like all of the Bay Area tech companies are too busy working on Blockchain / shitcoins, MetaVerse, or hyper optimization of advertising…

Also worth throwing some blame at VCs who are chasing hype cycles instead of investing in boring companies that would actually improve quality of life for the people around them.

jeffbee

The mag stripe 1960s technology worked much better than the new one, I'm sorry to report.

SuperHeavy256

snark is not productive.

semiquaver

What form of comment on an online forum _would be_ productive?

buckle8017

A significant amount of BARTs budget goes to inflated salaries for operators and ticketing staff.

They have very little money left for paying engineering and construction staff.

jeffbee

It certainly doesn't go to Bay Area software companies. When BART originally began letting out the contract to redesign the then-already-obsolete control system in 1992, they awarded it the Hughes Aircraft. That project failed. The current attempt to deploy CBTC was awarded to Hitachi. The supplier of their fare gate system integration was originally IBM and is now CUBIC, a San Diego defense contractor.

If anything the Bay Area has utterly failed to provide systems software of lasting value to address public needs like these.

lokar

Those types of contracts have much worse margins then Bay Area tech companies expect (or aspire to)

inferiorhuman

Likewise neither Rohr nor Westinghouse are Bay Area based.

octernion

your tax money broadly speaking doesn't go to BART; it's massively underfunded. not sure why they are the target of the snark.

nradov

Under funded relative to what? What would the optimal amount of funding be? Are there ways that BART could cut costs to free up budget for IT upgrades?

I'm not trying to be snarky, it's just that for regular citizens who don't have time to attend BART BoD and committee meetings it's almost impossible to tell whether existing money is being wisely spent. So people get the impression that taxes are going up while service quality declines and assume the money must be going into someone's pocket.

esalman

I lived mostly car free in Atlanta because the Marta station is one flight of stairs down from the airport terminal, and I could get to my lab in GSU in downtown Atlanta in less than 30 minutes, midtown Georgia tech campus in similar time, my first apartment in Lindberg in 40 minutes, and my second apartment in Sandy Springs on the other side of the city in less than an hour from the airport. Commute to and from my school/lab/apartment was always under 30 minutes and always faster by train compared to car.

These days I fly to the bay area to my office in East Bay. It's 2+ hours commute from either SFO or even OAK because you need to change buses 2 or 3 times. Add 1 more if you count taking the airport shuttle to the BART station. And SJC does not even have a BART connection.

There's fundamental design flaw in public transportation in the US, they almost never connect the population centers. Part of the reason why people are discouraged from using them and they don't get the funding to stay up to date.

linguae

I travel to Japan twice a year for business and for vacation, and coming back to the Bay Area and dealing with its transportation infrastructure is always jarring.

I find the Bay Area very difficult to get around. The roads are jammed with commuters who live far from their workplaces due to the housing situation. There is not enough housing near job centers, which bids up the prices of available housing to very high levels that requires FAANG-level salaries to clear unless one wants to have an army of roommates. Thus, many people have to commute, some from far-flung exurbs and even from Central Valley cities like Stockton and Modesto.

Public transportation in the Bay Area is better than most American cities, but it’s still underpowered for the size of the metro area. Not all residences are served by trains, and bus service is often infrequent and subject to delays. Missing a connection can lead to major inconveniences (such as a long 30-60 minute wait) or even being unable to reach your destination without an über-expensive Uber or Lyft ride. There’s also matters of safety and cleanliness on public transportation; every now and then I smell unpleasant odors like marijuana and urine, and occasionally I see sketchy people.

It’s a major step down from Tokyo, where public transportation is ultra-convenient, reliable in non-emergency situations, impeccably clean, and generally safe.

The sad thing is the reason the Bay Area lacks Tokyo-style transit is not technology, but social and political issues. If it were merely technology, we’d have solutions by now.

holmesworcester

One way to look at this is that the Bay Area focuses on transportation technology that works and scales regardless of the rare socio-political star alignment that makes HSR and subways possible.

And the Bay Area, largely, eats its own dogfood.

There is no faster, more powerful public transportation system than a city that allows Uber to offer mototaxi service. Uber was allowed to turned that on in Rio at some point in the last couple years and it puts busses and subways to shame. The number of cities where a subway is consistently faster than a skilled motorcyclist who can lane-split is very small if not zero.

halfmatthalfcat

Chicago (Blue Line from O'Hare) and NYC (M60 from Laguardia or Skytrain to MTA/LIRR from JFK) are also good in that regard.

dylan604

Part of the reason why people are discouraged s/from/by/ using them and they don't get the funding to stay up to date.

People are constantly being encouraged to take public transpo, but once they finally do, they realize why they hadn't before.

bigmattystyles

Broke the read-only Friday rule…

jkingsman

I know this is a tongue in cheek casual comment, but this article is a really good and important counterpoint: https://charity.wtf/2019/05/01/friday-deploy-freezes-are-exa...

jjice

Not to jump on your comment (since there have been quite a few other replies already) but just to add another personal anecdote: having been on the more senior end of a junior merge/deploy gone wrong and losing a Friday night or a weekend ping, I'm okay with an additional empty day throughout the week.

I've found that little things like that breed a growing resentment and stress that compounds, until someone wants to leave the company. Thursday night outage that I have to hop on? Much smaller deal than a weekend where I have established plans.

One can argue "why was the PR approved in the first place", but sometimes people make mistakes. It especially sucks when there are limited people that know how to troubleshoot and resolve the production issues with a system, even more so when the on-call individual may have not even reviewed the code initially.

All that said - I'd love to deploy as normal on Fridays! I've just found that the type of businesses I've worked at can wait until Monday, and that makes our weekends less risky.

anonymars

Perhaps. But what's the risk-reward? No matter how good your CI/CD is, the risk is nonzero. Do I really need to ship this today and open a can of worms this afternoon?

tossandthrow

It is not about fear, it is about risk management.

As an engineer I have absolutely no issue deploying on a friday. But friday bar starts at 4pm, and after that I am not sober before monday.

So leadership don't want me to do it - which is probably wise.

green-salt

I enforce a work/life balance and this is how the team loses a weekend when something goes wrong.

dogleash

I hate how people hear "read only friday" and decide to turn it into a CI/CD dick measuring contest.

For "read only friday" to have been a novel idea in the first place, you needed a starting point where conventional practice already was making changes live without stopping to consider the time/day of week.

I really suspect the detractors represent a workflow that would break (or at least introduce pain) if unable to push to production for a few days. So they have to give the hard sell on the benefits of continuous deployment.

null

[deleted]

jidar

To counter the counterpoint. Even if you are better at pushing to production than 90% of the rest of your industry it is still elevated risk and stress so you should avoid it for the sake of your employees. Productivity vs life. If your counterpoint is to claim that you are just as stable pushing to production as you are when you don't, then I would just suggest you're delusional or lying.

dilyevsky

this is just mindless blogospam/clickbait/"buy my thing" - the author even admits shipping big changes on friday is a bad idea

yacthing

This reads like someone who works on a small and simple system.

"Deploy on every commit" lmao

"Shipping software and running tests should be fast. Super fast. Minutes, tops." hahah

dilyevsky

> "Shipping software and running tests should be fast. Super fast. Minutes, tops." hahah

You mean to tell me not everyone works on some SaaS product outside of critical path?

sampullman

Deploy to what? Staging on every merged PR (commit to stg), and prod deploy on every commit to main? That sounds reasonable to me, and I've done some variation of it on most projects for the last 10 years or so without issue.

ForOldHack

Wait... (Obligatory) Did they forget to mount a scratch monkey?

Buuntu

Everyone here blaming BART and bureaucracy for being inefficient when in reality it's starved for funding due to our own voting (and zoning preventing housing/badly needed ridership near transit stops). Yes it's expensive to build transit just like it's expensive to build anything in America, which we should fix but that is not unique to BART.

It's quite possible the system will collapse next year if we don't pass increased taxes to fund it in 2026 https://www.bart.gov/about/financials/crisis.

Just last year we failed to pass a common sense bill to make it so we only need a 51% majority for transit bills in the future, indicative of how opposed we still are to transit in the Bay Area https://www.cbsnews.com/sanfrancisco/news/california-proposi....

Not to mention the fact that Silicon Valley opted out of BART and chose car dependent sprawl instead.

So let's be clear, most of the issues with BART are due to anti-transit and suburban voters starving it of support.

nradov

The failure of Proposition 5 doesn't indicate that California voters are opposed to transit. That was a very broad proposition which lowered the voting threshold for local governments to issue bonds for a wide variety of projects, not just transit. Local governments are already facing debt problems and making it easier to take on more debt would set them up for serious future fiscal problems.

jjice

> when in reality it's starved for funding due to our own voting

Everyone wants more services and lower taxes, but they vote for the lower taxes and get made when there are no services. Those things often don't go together. It's okay to either accept fewer services with less tax burden, or higher taxes with more services (the side I generally lean towards, within reason).

lokar

True, but it ignores the point of who various services are for. Wealthy professionals in the suburbs tend to vote against mass transit they don't plan on using.

dilyevsky

Hilarious that from 2020 and to this day ridership has collapsed but BART operating expenses went up despite that and all the efficiencies they talk about in your link. Kind of tells you everything you need to know about where the money is actually going...

Just to compare with another expensive city - BART serves 1/20th of London's Tube rides while operating on 1/5th of the Tube's budget.

Buuntu

That is mostly a zoning issue, have you seen the density around Tube stations? Compare that to the density around half of the BART stations which are big parking lots surrounded by single family houses. Of course it's cheaper to run a transit system in a city with twice the population density and population in the metro area.

Costs are an America issue, not a BART issue: https://transitcosts.com/new-data/

BART is one of the most cost efficient systems in the US: https://www.reddit.com/r/transit/comments/1d27dvo/us_cost_pe.... It's so efficient that pre-pandemic it got the majority of its funding through fares, not taxes.

By the way it costs exorbitant amounts to build highways too and you don't see people criticizing all of our highways around the area do you.

So quite frankly you don't know what you're talking about.

jen20

> have you seen the density around Tube stations?

As a former tube-commuter and occasional BART-user, I'd wager that possibly a majority of the commuting trips in zone 1 are taking people from a mainline train station to somewhere, and then back in the evening. That option barely even exists in the Bay Area - indeed every time I look at how to use Caltrain from SFO I give up and rent a car instead.

dilyevsky

BART service area population is comparable to Greater London

> Costs are an America issue, not a BART issue: https://transitcosts.com/new-data

If by "America" you mean NYC/SFBA then sure. You can see in your own link there's massive spread across the locales with some being cheaper than UK per km

> you don't see people criticizing all of our highways around the area do you

uhm what?

chuckadams

It's pretty hard to keep from drowning in despair when one realizes that almost everywhere else in the USA except maybe NYC, the situation is worse.

jjice

Hey, the Boston T runs some of the time!

Jokes aside, I'd like to see a stack ranking of US public transit. I'd assume NYC and DC are top dogs, but I'm curious about other cities.

dylan604

> Not to mention the fact that Silicon Valley opted out of BART and chose car dependent sprawl instead.

Didn't bigTech start buses going directly to their campus as a perk?

Buuntu

Yeah this is basically the private market filling in for our lack of transit down south. Most every other major city doesn't have this, you just take the metro to work like a normal person.

esseph

What metro :/

crooked-v

Good ol' Prop 12, guaranteeing that everything will be underfunded one way or another.

dilyevsky

didn't realize cage free pigs lead to such dramatic second order effects =)

peterbecich

must have meant Prop 13

ForOldHack

Except, as always bureaucratic pay raises.

Animats

Are there any technical details yet? What was upgraded?

nilsbunger

Seems like BART should do these upgrades only at low traffic times, like overnight Saturday night.

ForOldHack

They did. They started before yesterday's shutdown, and worked all night, they tried to bring up the system for startup, and it came up, then crashed.

It was state of the art on 1962 when it was designed, and remained state of the art until the 1980s, when the signal system started breaking down, and the the late 80s upgrade which had a train presence glitch, which caused almost all the system to get resignaled.

So by the 2000s again it's showing its age, and they got a 32 processor zSeries mainframe.

Brake problem last week, and the this on Friday? Now it's getting like New York, even more. Whatsmatteryou?

MangoToupe

What on earth does it do that requires a mainframe?

jasonjmcghee

I'm curious what percent of HN is based in the bay area for this to hit the front page so quickly. I suppose it could in part be that it was posted when people are commuting in?

0xffff2

HN has always had a huge bay area focus.

MBCook

A tech failure taking down a big government thing is certainly HN worthy. And BART is relatively famous, as such things go. It’s a name people know, as opposed to of it was the Minnesota DMV system. That would be a fine story too but no one knows the name for that.

zdragnar

Ironically, the Northstar rail line (one part boondoggle, one part "would have worked if it went all the way to st cloud", depending on who you ask) is shutting down Jan 3 or 4 in 2026, so I wouldn't be surprised to see articles on it and/or the met council before then.

ryukoposting

That's a shame. The Twin Cities set a relatively high bar for American public transit, too. The light rail is fantastic. I only wish you could take the green line all the way out to SLP or Plymouth.

hopelite

BART specifically is also a kind of lighting rod of the political, social, economic fissure that runs through American culture; the difference in perspective of the adversarial camps, like different tribes.

It is a microcosm, a bit of a litmus test, and an ideological battlefield of the embattled sides. But this example specifically is also a kind of infighting, of the more anarcho-libertarian tech camp that enjoys highlighting and dripping with self-righteousness about any tech related failure of government, i.e., or at least government that does not align with their ideology or control over it.

This fault line of America runs right through things like BART like an effigy or idol that America performs a kind of ritual form of battle on as proxies for all out civil war. Think of tribes you may have seen videos of where they do all kinds of elaborate dances and blustering displays and fake charges to demonstrate their power.

The glee about this outage happening to BART is very much because the libertarian tech progressive types are amused and validated by it, where something more like rashes of violent attacks on BART riders by menaces to society might be something that the "heartland" may become gleeful about, as evidence for how the ideology of SF is messed up. In the cases of violent attacks on BART riders, another camp/tribe would come out and demonstrate their fierceness; the "socially liberal" types from all over the country and even world, would rush to the defense of their ideological idols with a bewildering storm of rationalization, delusion, and excuse making for violent attackers and in defense of their ideology/cult.

It's just elaborate war dances around an idol/ideology to demonstrate how fierce and powerful each party is. BART is just one of the idols in America around which these displays of simulated conflicts happen.

nottorp

I'm not even on the same continent but I'm still reading this, including the comments...

darth_avocado

Is it causation or correlation? Maybe bart being down caused all the people to browse HN while waiting for the issue to be resolved, thereby making this show up on the main page.

slowhadoken

Yeah the BART needs some love.

coldest_summer

sorry using throwaway for this.

When GitHub was constantly failing, I finally got fed up and now I use my own private Gitea. It’s near-zero maintenance and has never had any unexpected downtime. Never looked back.

Stories like these make me feel the same way about California, which I called home for almost 20 years. So much to love, so many reasons never to live there again. Great place to visit when there’s not an active disaster unfolding.

outlore

mate we can't self host BART

wiml

Isn't that kinda what people are doing when they commute in private cars?

outlore

fair enough!

benced

I’m not frustrated that this happened, I’m frustrated that it seems likely this won’t get better (witness their protracted incredibly high constructions costs that have not improved). I hope they prove me wrong.

giardini

Windows, upgrading again?

fmbb

Nah, we upgraded the network configuration. Should have no impact. No there is no source control.

RcouF1uZ4gsC

Public transportation is inherently centralized.

Cars are anti-fragile and decentralized.

Cars fail open in the short term.

loire280

Buses are the resilient backup for trains, especially if road infrastructure has been designed to prioritize transit (e.g. Chicago highways with shoulders designed to let Pace buses bypass traffic jams).

rafram

Tell that to someone in a two-hour traffic jam on the highway.

formerly_proven

Traditional train systems themselves are extremely decentralized, though scheduling is not. Traditional interlockings form a mirroring mesh network parallel to the physical network of steel rails itself.

xnx

Train tracks are a form of centralization. Without the ability to reroute around disruptions (like cars and buses) a single stopped train (e.g. due to mechanical or passenger issues) can stop everything.