GitHub Is Down
71 comments
·January 30, 2025japhyr
simonw
I do the same - I've been running so many non-software things out of private GitHub Issues. I even have a simple system for creating a new issue every day to use as a personal TODO list, described here: https://til.simonwillison.net/github-actions/daily-planner
I've noticed everything feeling a whole lot less snappy and responsive over the past few days - ironically I think it's because they've done a major rewrite of the frontend presumably with the aim of making it more snappy and responsive!
PapaPalpatine
I’m low-key certain it’s because they started using more React on the front end, especially for the new Issue stuff. The entire UI feels slow and janky like GitLab.
straws
You're not alone. There's been an effort to transparently update the UI to a React implementation over the past year or two, and while I understand the benefits to that approach, they have introduced some flakiness in moving away from a the server-rendered pjax/html-pipeline/simple web components approach that was so cohesive and battle tested over the decade before it.
jrochkind1
I am not experiencing a full outage right now, but I am experiencing very slow (spinner) and flakey (error that works again on repeat).
So perhaps it's the same problem, that's been getting more prevalent until it was noticeable by monitoring. One can hope!
RohMin
That's actually a really interesting way to leverage that feature. Have you found this easier than other services built specifically for this use case?
japhyr
I've tried other tools. But I'm on GitHub most days, so it's been a seamless way to keep track of some things that would otherwise disappear into a calendar, or some tool that I don't use as often.
joseda-hg
For something so critical, does it make sense to self host say a gitea instance or such?
I know you can sync repos, I don't think you can sync issues tho
snapplebobapple
this is what i doits worth the 11 dollars a year for the vps to control my own
null
bigfatfrock
Who's going to kick off the holy war around self hosting?
reddalo
Speaking of self hosting, Codeberg [1] is great. It's open and it looks and behaves pretty much like GitHub (unlike GitLab).
mr_mitm
Did you mean to say Forgejo? Correct me if I'm wrong, but Codeberg is simply hosting a Forgejo instance for you, so not exactly self hosting.
diggan
Seconded. Hosting FOSS on a platform that isn't FOSS itself and run by a for-profit company who tried to work against FOSS for so long (Microsoft) doesn't make much sense to me.
Instead, we should dogfood the FOSS ecosystem on a platform that is FOSS itself, and run as a non-profit. Codeberg, for better or worse, is the best platform for this today.
otter-in-a-suit
Since you asked, https://about.gitea.com/ is a great tool. MIT license.
NewJazz
Would be even better if they didn't require a CLA to contribute.
skydhash
Personal projects are on a vps and accessed through ssh. I’ve self-hosted gitea, then forgejo, then found out that I don’t like the interface or the auth dance. My plan is to ise cgit if I want something to be public and any forge if the purpose is collaboration (sourcehut is nice)
null
null
duxup
Honestly, sounds like a lot of work and I'm not sure I get much benefit, or even less downtime.
I don't know about others but for me github is hardly a service that I'm sweating moment to moment uptime. My apps are still running and so on.
cropcirclbureau
Hosting GitLab is not fun but anyways, M$ can still suck it.
zaruvi
Not Onedev or Forgejo, though it could get tricky depending on what features you require.
rvz
It all began 5 years ago [0] when we wanted to 'centralize everything to GitHub.'
Then, the outages accelerated from there.
mrguyorama
"are you sure your self-hosted solution will have better uptime?"
Hilariously, after 15 years working in self hosted bitbucket systems, YES entirely.
An underfunded university with an incompetent but doing their best IT department? Zero downtime.
A mid sized company full of overly confident and "just build it" nerds building fragile shit? Zero downtime.
A large corporation with a completely outsourced IT department that can't give you access to something unless you do exactly the right undocumented thing in our internal ticketing software? Zero downtime.
That includes self hosted jenkins and literally homebuilt infrastructure with zero documentation and the guy that built it left a while ago.
I have never been able to blame our build and code infra for lack of productivity.
peterldowns
I'm seeing issues with repo pages entirely (getting Unicorn "We couldn't respond to your request in time" errors) but the status page hasn't updated to show that yet.
cjonas
Bitbucket was down for a few hours a week ago and 0 fuss was made about it in the media. Atlassian stock actually went up over the ourage
diggan
Besides projects/companies/people stuck on Bitbucket, is anyone actually voluntary using Bitbucket? I remember using it back when I was poor and it was the only choice for hosted free private repositories, but the rest of the platform was actively worse in basically every way compared to the alternatives.
crest
Pun intended?
jeffbee
Because literally nobody uses it.
G1N
They're saying PRs and issues but I can't access github.com or github oauth for logging into external services. Need to migrate our org off of using github auth for anything critical I think, this is twice so far this year that we've had these issues.
lawrjone
The industry made a general decision after ZIRP ended to deprioritise availability after years of historic levels of engineering investment.
It's no surprise we're now feeling those effects but damn, GitHub and other services like Slack have been really bad lately.
diggan
Or, GitHub used to favor stability back in the day, and there wasn't a lot of changes. People were complaining that GitHub didn't "improve" enough day-to-day, so after the Microsoft purchase, Microsoft started forcing GitHub to add more features, stability be damned.
Most outages are caused not by stuff just randomly breaking, but updates/upgrades going wrong. If you try to increase the output of new features/changes to a platform, you're bound to have more outages and downtime if you aren't more careful than before.
Microsoft, who never really excelled at engineering, to the surprise of absolutely everyone, choose adding features over stability and since years back, we're seeing the consequences of that choice.
lawrjone
I'm not totally certain you can attribute it to this. I know there's a bunch of work going on to migrate things into Azure after the MS acquisition but it feels like more of an industry trend that we cut engineering spend at the cost (often) of lower quality output and outages like these.
GitHub laid off 10% of their staff in 2023 and like you say, won't have slowed down to account for that.
Ignoring all that though... other than Copilot, what big feature changes have you seen in GitHub? My experience of using their product has been broadly unchanged for years.
diggan
> My experience of using their product has been broadly unchanged for years.
Compared to how fast/slow they were moving 2012-2018, they're moving at blazing speed now. It feels like every time I open GitHub now, there are new features/changes and "Beta" available stuff. The platform is almost completely different today than it was in 2018, for better or worse.
mariusor
The move to React was done in 2022. I noticed that one for sure.
rohansingh
GitHub's reliability has gotten pretty terrible since the Microsoft acquisition, well before ZIRP ended.
NewJazz
Citation needed?
Is this true beyond github and slack? Both were acquired btw, that could explain their availability issues (just like twitter, now X).
What about e.g. AWS or GCP? Has their availability meaningfully reduced?
lawrjone
I wouldn't say this applies to cloud providers, they have a very different business on their hands.
But for SaaS in general I think the trend is noticeable? Twitter led the way with massive layoffs in engineering often in the roles around reliability. The industry as a whole have aimed to cut costs however possible, and reliability/ops is usually seen as a cost-centre that gets hit hard.
I've watched this in my own space (start/scale-ups and larger companies, I work in incident response tooling) as people start talking very differently about reliability and engineering investment. You hear "do more with less" about five times every day and spend that was previously greenlit by default around reliability/redundancy is under much more scrutiny now.
I see this as a silent mirror of the reduction in open-source efforts from companies now there's been a refocus on business impact and bottom-line.
brunocroh
If I'm not wrong, this is the third week in a row with GitHub incidents, isn't it?
floydian10
The incidents history page is really damning. I didn't realize it gotten so bad
hedora
Work was impacted by a few outages that they didn’t report there.
ChrisArchitect
Direct incident link: https://www.githubstatus.com/incidents/nm83zrdky73y
null
hedora
The last few days, the “create pr” screen kept hanging for me. I figured out I could reload it, click “create”, grab a coffee, and then come back and it’d be at the next screen.
Maybe it’s time to learn their CLI (and then patiently wait for work to switch to another git service).
ferriswil
Maybe we can spend this downtime researching self-hosting options
There's a not-entirely-small part of me that hopes Issues breaks entirely for long enough that people at GitHub rethink their implementation.
GH issues have been so useful for the better part of a decade, that I have an empty repo called my_life just so I can make issues about things like home maintenance. In the last few months, the UI has become so slow and flaky that I don't use it for nearly as much as I used to. And when I have to for a project, it slows me down noticeably.
I thought I was alone in this, but asking around I found that this is a common frustration among developers I communicate regularly with.