Cloudflare Global Network experiencing issues
595 comments
·November 18, 2025itzjacki
sakisv
Well, you can never be sure that he didn't:
itzjacki
Oh don't you worry. We are very much talking about the global outage as if he was the root cause. Like good colleagues :)
rapnie
Hmm, wait a minute.. maybe he was the cause! (no, kidding. just upping the pressure as a good peer :)
srmarm
> May 12, we began a software deployment that introduced a bug that could be triggered by a specific customer configuration under specific circumstances.
I'd love to know more about what those specific circumstances were!
Freak_NL
Is there a word for that feeling of relief when someone else fucked up after initially thinking it was you?
jspash
The problem is, I still get the wrong end of the stick when AWS or CF go down! Management doesn't care, understandably. They just want the money to keep coming in. It's hard to convince them that this is a pretty big problem. The only thing that will calm them down a bit is to tell them Twitter is also down. If that doesn't get them, I say ChatGPT is also down. Now NOBODY will get any work done! lol.
adriand
Haha yeah I just got off the phone and I said, look, either this gets fixed soon or there's going to be news headlines with photographs of giant queues of people milling around in airports.
graemep
Who decided to go with AWS of CF? If its a management decision tell them you need the resources to have a fallback if they want their system to be more reliable than AWS or CF.
spamizbad
What’s funny is as I get older this feeling of relief turns more like a feeling of dread. The nice thing about problems that you cause is that you have considerable autonomy to fix them. Cloudflare goes down you’re sitting and waiting for a 3 party to fix something.
jpmonette
phewphoria
Sholmesy
Well, at least something good came out of this incident.
Perfect.
zzzeek
it has to sound like a german word though
Freak_NL
Yup, that works.
null
stonecharioteer
It's close enough to Schadenfreude but not really.
mcphage
Maybe this isn’t great, but I get a hint of that feeling when I’m on an airplane and hear a baby crying. For a number of years, if I heard a baby crying, it was probably my baby and I had to deal with it. But now my kids are past that phase, so when I hear the crying, after that initial jolt of panic I realize that it isn’t my problem, and that does give me the warm fuzzies. Even though I do feel bad for the baby and their parents.
adriand
Related situation: you're at a family gathering and everyone has young kids running around. You hear a thump, and then some kid starts screaming. Conversation stops and every parent keenly listens to the screams to try and figure out whose kid just got hurt, then some other parent jumps up - it's not your kid! #phewphoria
shortrounddev2
When I'm debugging something, I'm not usually looking for the solution to the problem; I'm looking for sufficient evidence that I didn't cause the problem. Once I have that, the velocity at which I work slows down
StanAngeloff
Schadenfreude
gnfargbl
Nah, that's delight in someone else's misfortune. This is delight that the misfortune wasn't yours, which is slightly different.
simonklitj
Not quite, that’s more like taking pleasure in the misfortune of someone else. It’s close, but the specific relief bit that it is not _your_ misfortune is not captured
cromka
vindication?
nrhrjrjrjtntbt
The company where this colleague works? Cloudflare.
disconnection
Good that he worked it out so quick. I recently spent a day debugging email problems on Railway PaaS, because they silently closed an SMTP port without telling anyone.
NitpickLawyer
You missed a great opportunity to dead-pan him with something like "No, Bob, not just our site, you brought down the entire Internet, look at this post!"
null
bamboozled
How do we know your colleagues changes didn't take down Cloudflare though?
itzjacki
Good point. We should probably assume they did, until proven otherwise.
belter
Wait for the post mortem ... It is a technical possibility, race condition propagates one customer config to all nodes... :-)
farhadhf
Pretty much everything is down (checking from the Netherlands). The Cloudflare dashboard itself is experiencing an outage as well.
Not-so-funny thing is that the Betterstack dashboard is down but our status page hosted by Betterstack is up, and we can't access the dashboard to create an incident and let our customers know what's going on.
Edit: wording.
celltalk
I think there is a big business opportunity here. Make a site that let companies put their status update on local vps for $100.
alt227
Atlassian has this business model sewn up
colinbartlett
Even the Cloudflare status page, hosted by Atlassian Statuspage, is suffering. Probably due to the traffic crush.
nrhrjrjrjtntbt
Status pigeons.
ramon156
on-demand status balancing!
pell
It’s that time of the year again where we all realize that relying on AWS and Cloudflare to this degree is pretty dangerous but then again it’s difficult to switch at this point.
If there is a slight positive note to all this, then it is that these outages are so large that customers usually seem to be quite understanding.
isodev
Unless you’re say at airport trying to file a luggage claim … or at the pharmacy trying to get your prescription. I think as a community we have a responsibility to do better than this.
sigilis
You aren’t cloudflare’s customer in these examples. It depends on the companies that are actually paying for and using the service to complain. Odds are that they won’t care on your behalf due to how our society is structured.
Not really sure how our community is supposed to deal with this.
tommica
> It’s that time of the year again
It's monthly by now
lbreakjai
If I choose AWS/cloudflare and we're down with half of the internet, then I don't even need to explain it to my boss' bosses, because there will be an article in the mainstream media.
If I choose something else, we're down, and our competitors aren't, then my overlords will start asking a lot of questions.
stevepotter
Yup. AWS went down at a previous job and everyone basically took the day off and the company collectively chuckled. Cloudflare is interesting because most execs don’t know about it so I’d imagine they’d be less forgiving. “So what does cloudflare do for us exactly? Don’t we already have aws?”
jfengel
And if everyone else is down, and you are not, you will get no credit.
timeon
In reality it is not half of the internet. That is just marketing. I've personally noticed one news site while others were working. And I guess sites like that will get the blame.
fusl
Happy to hear anyone's suggestions about where else to go or what else to do in regards to protecting from large-scale volumetric DDoS attacks. Pretty much every CDN provider nowadays has stacked up enough capacity to tank these kind of attacks, good luck trying to combat these yourself these days?
q3k
Just accept that a DDoS might happen and that there's nothing you can do about it. It's fine, it's just how the Internet works.
isodev
Anubis and/or Bunny are good alternatives/combination depending on your exact needs
weird-eye-issue
Oh no, we had 30 minutes of downtime this year :(
pell
I do think this is tenable as long as these services are reliable. Even though there have been some outages I would argue that they’re incredibly reliable at this point. If though this ever changes the costs to move to a competitor won’t be as simple as pushing a repository elsewhere, especially for AWS. I think that’s where some of the potential danger lies.
postalcoder
Cloudflare dashboard is down-ish, not totally down. If you're persistent you can turn off the turnstile and proxy.
It took a few minutes but I got https://hcker.news off of it.
farhadhf
I'm already logged in on the cloudflare dashboard and trying to disable the CF proxy, but getting "404 | Either this page does not exist, or you do not have permission to access it" when trying to access the DNS configuration page.
null
skywhopper
Not saying not to do this to get through, but just as an observation, it’s also the sort of thing that can make these issues a nightmare to remediate, since the outage can actually draw more traffic just as things are warming up, from customers desperate to get through.
But then, that’s what Cloudflare signed up to be.
biinjo
Same here. We’re using OhDear. The status page is available but I can’t post an incident because their service is also behind Cloudflare.
Mojah
Co-founder here, we'll be working on better ways to handle this over the coming days.
esskay
When its back up, do yourself a favour and rent a $5/mo vps in another country from a provider like OVH or Hetzner and stick your status page on that.
"Yes but what if they go down" - it doesnt matter, having it hosted by someone who can be down for the same reason as your main product/service is a recipe for disaster.
jwr
Or use a service like https://updown.io/ (I host my status page there).
hcaz
https://cachethq.io/ is great for this
jwr
Amusingly enough, it is down right now because of Cloudflare :-)
fusl
Been using Cachet for quite a while before inevitably migrating to Atlassian's Statuspage.io. I'm a huge fan of self-hosting and self-managing every single thing in existence but Cachet was just such a PITA to maintain and there was just no other good alternative to Cachet that was also open source.
ablation
This is a big one.
csomar
Seems like workers are less affected and maybe betterstack has decided to bypass cloudflare "stuff" for the status pages? (maybe to cut down costs). My site is still up though some GitHub runners did show it failed at certain points.
chrisandchris
I don't get why you need such a service for a status page with 99.whatever% uptime. I mean, your status page only has to be up if everything else is down, so maybe 1% uptime is fine.
/s
rollulus
Classic. I see issues. Vendor’s status page is all green. Go to HN to find the confirmation. Applies to AWS, GH, everyone.
Edit: beautiful, this decentralised design of the internet.
markild
I get the feeling that all "serious" businesses have manual processes for publicly facing status pages, for political reasons.
I don't like it.
skywhopper
At some level, the status updates have to be manual. Any automation you try to build on top is inevitably going to break in a crisis situation.
its_notjack
Ironically, DownDetector seems to be down because it protects its site with Cloudflare Turnstile... which is also down!
imdsm
I noticed this too!
null
thrdbndndn
The report there for AWS also skyrocketed, but I guess it's probably false positives?
Macha
Even many non tech people have begun to associate Internet wide outages with “aws must be down” so I imagine many of them searching “is aws down” and for down detector, a hit is a down report, so it will report aws impacts even when the culprit is cloudflare in this case
mailund
interesting, maybe "AWS is down" will become the new "the server is down" that some non-tech people throw around when anything unexpected happen on their computer?
gwd
I do appreciate the visual "mea culpa":
Your browser: Working
Host: Working
Cloudflare: Error
mallets
Might be the first time I have ever seen that. Though in my case the "Host" is Cloudflare's own Pages service.
udev4096
Yeah, I was shocked. Disbelief that the host was up, which is what usually happens when the cloudflare's page show up
Neil44
I noticed that refreshing honesty too, not that the users did (our wifi is down fix it pls urgent)
Lukas_Skywalker
They still blame the customers when you click on "Cloudflare":
> If the problem isn’t resolved in the next few minutes, it’s most likely an issue with the web server you were trying to reach.
mig4ng
In terms of probability looking at the history, it is correct. It's mostly me messing up with the web server.
mig4ng
That is really good to be honest!
I have Cloudflare running in production and it is affecting us right now. But at least I know what is going on and how I can mitigate (e.g. disable Cloudflare as a proxy if it keeps affecting our services at skeeled).
jap
That page has special if/endif HTML comments to handle if your browser is IE 6, IE 7, IE 8...
janmo
And at the bottom:
What can I do?
Please try again in a few minutes.
null
djsjajah
I went to check how many services are being impacted on down detector, but it was down.
rco8786
Is it me or has there been a very noticeable uptick in large scale infra-level outages lately? AWS, Cloudflare, etc have all been way under whatever SLA they publish.
alt227
For me the only silver lining to all these cloud outages is now we know that their published SLA times mean absolutely nothing. The number of 9's used to at least give an indication of intent of reliability, now they are twisted to whatever metric the company wants to represent and dont actually represent guaranteed uptime anywhere.
bojangleslover
So true. AWS for example gives only platform credits in the event of an outage. Basically no recourse or insurance.
dimastopel
GCP was down recently as well
guyfromfargo
[dead]
Mojah
Interestingly, also noticing that websites that use Cloudflare Challenge (aka "I'm not a Robot") are also throwing exceptions with a message as "Please unblock challenges.cloudflare.com to proceed" - even though it's just responding with an HTTP/500.
spapas82
Cloudflare seems to have degrated performance. Half the requests for my site throw cloudflare 500x errors, the other half work fine.
However the https://www.cloudflarestatus.com/ does not really mention anything relevant. What's the point of having a status page if it lies ?
Update Ah I just checked the status and now I get a big red warning (however the problem existed for like 15 minutes before 11:48 UTC):
> Investigating - Cloudflare is aware of, and investigating an issue which potentially impacts multiple customers. Further detail will be provided as more information becomes available. Nov 18, 2025 - 11:48 UTC
maushu
> What's the point of having a status page if it lies ?
Status pages are basically marketing crap right now. The same thing happened with Azure where it took at least 45 minutes to show any change. They can't be trusted.
vinishbhaskar
Tried checking Cloudflare’s status on Downdetector, but Downdetector was also behind Cloudflare. Internet checkmate.
virajk_31
I was shouting at network guy/colleague, how come challenges.cloudflare.com got blocked!! damn, I must apologise to him.
yipbub
Probably better not to shout in the first place.
virajk_31
It was friendly fire, nothing serious. haha
weird-eye-issue
> friendly fire
Yeah I don't think you are using this phrase correctly
uniq7
Even if he blocked it by accident, that is not a reason to shout.
Shouting will not prevent errors, and you are only creating a hostile work environment where not acting is better than the risk of making a mistake and triggering an aggressive response from your part.
virajk_31
It wasn't aggressive exchange, but will definitely consider your comment.
enidjv
Don't worry beer gonna fix everything
Fiveplus
Please don't yell at your colleagues.
A colleague of mine just came bursting through my office door in a panic, thinking he brought our site down since this happened just as he made some changes to our Cloudflare config. He was pretty relieved to see this post.