Meta is using the Linux scheduler designed for Valve's Steam Deck on its servers
118 comments
·December 23, 2025Fiveplus
MarleTangible
Over time they're going to touch things that people were waiting for Microsoft to do for years. I don't have an example in mind at the moment, but it's a lot better to make the changes yourself than wait for OS or console manufacturer to take action.
asveikau
I was at Microsoft during the Windows 8 cycle. I remember hearing about a kernel feature I found interesting. Then I found linux had it for a few years at the time.
I think the reality is that Linux is ahead on a lot of kernel stuff. More experimentation is happening.
wmf
I was surprised to hear that Windows just added native NVMe which Linux has had for many years. I wonder if Azure has been paying the SCSI emulation tax this whole time.
7bit
And behind on a lot of stuff. The Microsoft's ACLs are nothing short of one of the best designed permission systems there are.
On the surface, they are as simple as Linux UOG/rwx stuff if you want it to be, but you can really, REALLY dive into the technology and apply super specific permissions.
dijit
yeah, but you have IO Completion Ports…
IO_Uring is still a pale imitation :(
benoau
"It just works" sleep and hibernate.
"Slide left or right" CPU and GPU underclocking.
devnullbrain
I don't understand this comment in this context. Both of these features work on my Steam Deck. Neither of them have worked on any Windows laptop my employers have foisted upon me.
dijit
“it just works” sleep was working, at least on basically every laptop I had the last 10 years…
until the new s2idle stuff that Microsoft and Intel have foisted on the world (to update your laptop while sleeping… I guess?)
pmontra
Sleep and hibernate don't just work on Windows unless Microsoft work with laptop and boards manufacturers to make Windows play nice with all those drivers. It's inevitable that it's hit and miss on any other OS that manufacturers don't care much about. Apple does nearly everything inside their walls, that's why it just works.
Krssst
On my Framework 13 AMD : Sleep just works on Fedora. Sleep is unreliable on Windows; if my fans are all running at full speed while running a game and I close the lid to begin sleeping, it will start sleeping and eventually wake up with all fans blaring.
seba_dos1
Both of these have worked fine for the last 15 years or so on all my laptops.
packetlost
Kernel level anti-cheat with trusted execution / signed kernels is probably a reasonable new frontier for online games, but it requires a certain level of adoption from game makers.
dabockster
This is a part of Secure Boot, which Linux people have raged against for a long time. Mostly because the main key signing authority was Microsoft.
But here's my rub: no one else bothered to step up to be a key signer. Everyone has instead whined for 15 years and told people to disable Secure Boot and the loads of trusted compute tech that depends on it, instead of actually building and running the necessary infra for everyone to have a Secure Boot authority outside of big tech. Not even Red Hat/IBM even though they have the infra to do it.
Secure Boot and signed kernels are proven tech. But the Linux world absolutely needs to pull their heads out of their butts on this.
mstank
Valve... please do Github Actions next
xmprt
I wonder what Valve uses for source control (no pun intended) internally.
guidopallemans
Surely a gaming handheld counts
duped
> I don't have an example in mind at the moment
I do, MIDI 2.0. It's not because they're not doing it, just that they're doing it at a glacial pace compared to everyone else. They have reasons for this (a complete rewrite of the windows media services APIs and internals) but it's taken years and delays to do something that shipped on Linux over two years ago and on Apple more like 5 (although there were some protocol changes over that time).
null
bilekas
I do agree. It's also thanks to gaming that the GPU industry was in such a good state to be consumed by AI now. Game development used to always be the frontier of software optimisation techniques and ingenious approaches to the constraints.
dabockster
> This is the best kind of open source trickledown.
We shouldn't be depending on trickledown anything. It's nice to see Valve contributing back, but we all need to remember that they can totally evaporate/vanish behind proprietary licensing at any time.
baq
I low key hope the current DDR5 prices push them to drag the Linux memory and swap management into the 21st century, too, because hard locking on low memory got old a while ago
captn3m0
My favourite is the Windows futex primitives being shipped on Linux: https://lwn.net/Articles/961884/
delusional
> Valve is practically singlehandedly dragging the Linux ecosystem forward in areas that nobody else wanted to touch.
I'm loving what valve has been doing, and their willingness to shove money into projects that have long been under invested in, BUT. Please don't forget all the volunteers that have developed these systems for years before valve decided to step up. All of this is only possible because a ton of different people spent decades slowly building a project, that for most of it's lifetime seemed like a dead end idea.
Wine as a software package is nothing short of miraculous. It has been monumentally expensive to build, but is provided to everyone to freely use as they wish.
Nobody, and I do mean NOBODY would have funded a project that spent 20 years struggling to run office and photoshop. Valve took it across the finish line into commercially useful project, but they could not have done that without the decade+ of work before that.
mikkupikku
> SCX-LAVD has been worked on by Linux consulting firm Igalia under contract for Valve
It seems like every time I read about this kind of stuff, it's being done by contractors. I think Proton is similar. Of course that makes it no less awesome, but it makes me wonder about the contractor to employee ratio at Valve. Do they pretty much stick to Steam/game development and contract out most of the rest?
ZeroCool2u
Igalia is a bit unique as it serves as a single corporate entity for organizing a lot of sponsored work on the Linux kernel and open source projects. You'll notice in their blog posts they have collaborations with a number of other large companies seeking to sponsor very specific development work. For example, Google works with them a lot. I think it really just simplifies a lot of logistics for paying folks to do this kind of work, plus the Igalia employees can get shared efficiency's and savings for things like benefits etc.
chucky_z
This isn’t explicitly called out in any of the other comments in my opinion so I’ll state this. Valve as a company is incredibly focused internally on its business. Its business is games, game hardware, and game delivery. For anything outside of that purview instead of trying to build a huge internal team they contract out. I’m genuinely curious why other companies don’t do this style more often because it seems incredibly cost effective. They hire top level contractors to do top tier work on hyper specific areas and everyone benefits. I think this kind of work is why Valve gets a free pass to do some real heinous shit (all the gambling stuff) and maintain incredible good will. They’re a true “take the good with the bad” kind of company. I certainly don’t condone all the bad they’ve put out, and I also have to recognize all the good they’ve done at the same time.
Back to the root point. Small company focused on core business competencies, extremely effective at contracting non-core business functions. I wish more businesses functioned this way.
smotched
Whats the bad practices valve is doing in gambling?
mewse-hn
Loot box style underage gambling in their live service games - TF2 hats, counterstrike skins, "trading cards", etc etc
crtasm
Their games and systems tie into huge gambling operations on 3rd party sites
If you have 30mins for a video I recommend People Make Games' documentary on it https://www.youtube.com/watch?v=eMmNy11Mn7g
msh
Lootboxes comes to mind.
tayo42
I feel like I rarely see contacting out work go well. This seems like an exception
OkayPhysicist
The .308 footgun with software contracting stems from a misunderstanding of what we pay software developers for. The model under which contracting seems like the right move is "we pay software developers because we want a unit of software", like how you pay a carpenter to build you some custom cabinets. If the union of "things you have a very particular opinion about, and can specify coherently" and "things you don't care about" completely cover a project, contracting works great for that purpose.
But most of the time you don't want "a unit of software", you want some amorphous blob of product and business wants and needs, continuously changing at the whims of business, businessmen, and customers. In this context, sure, you're paying your developers to solve problems, but moreover you're paying them to store the institutional knowledge of how your particular system is built. Code is much easier to write than to read, because writing code involves applying a mental model that fits your understanding of the world onto the application, whereas reading code requires you to try and recreate someone else's alien mental model. In the situation of in-house products and business automation, at some point your senior developers become more valuable for their understanding of your codebase than their code output productivity.
The context of "I want this particular thing fixed in a popular open source codebase that there are existing people with expertise in", contracting makes a ton of sense, because you aren't the sole buyer of that expertise.
to11mtm
I've seen both good and bad contractors in multiple industries.
When I worked in the HFC/Fiber plant design industry, the simple act of "Don't use the same boilerplate MSA for every type of vendor" and being more specific about project requirements in the RFP makes it very clear what is expected, and suddenly we'd get better bids, and would carefully review the bids to make sure that the response indicated they understood the work.
We also had our own 'internal' cost estimates (i.e. if we had the in house capacity, how long would it take to do and how much would it cost) which made it clear when a vendor was in over their head under-bidding just to get the work, which was never a good thing.
And, I've seen that done in the software industry as well, and it worked.
That said, the main 'extra' challenge in IT is that key is that many of the good players aren't going to be the ones beating down your door like the big 4 or a WITCH consultancy will.
But really at the end of the day, the problem is what often happens is that business-people who don't really know (or necessarily -care-) about specifics enough unfortunately are the people picking things like vendors.
And worse, sometimes they're the ones writing the spec and not letting engineers review it. [0]
[0] - This once led to an off-shore body shop getting a requirement along the lines of 'the stored procedures and SQL called should be configurable' and sure enough the web.config had ALL the SQL and stored procedures as XML elements, loaded from config just before the DB call, thing was a bitch to debug and their testing alone wreaked havoc on our dev DB.
magicalhippo
If you have competent people on both sides who care, I don't see why it wouldn't work.
The problem seems, at least from a distance, to be that bosses treat it as a fire-and-forget solution.
We haven't had any software done by oursiders yet, but we have hired consultants to help us on specifics, like changing our infra and help move local servers to the cloud. They've been very effective and helped us a lot.
We had talks though so we found someone who we could trust had the knowledge, and we were knowledgeable enough ourselves that we could determine that. We then followed up closely.
TulliusCicero
Valve contracts out to actually competent people and companies rather than giant bodycount consulting firms.
abnercoimbre
Nope. Plenty of top-tier contractors work quietly with their clientele and let the companies take the credit (so long as they reference the contractor to others, keeping the gravy train going.)
If you don't see it happening, the game is being played as intended.
bogwog
Valve has a weird obsession with maximizing their profit-per-employee ratio. There are stories from ex-employees out on the web about how this creates a hostile environment, and perverse incentives to sabotage those below you to protect your own job.
I don't remember all the details, but it doesn't seem like a great place to work, at least based on the horror stories I've read.
Valve does a lot of awesome things, but they also do a lot of shitty things, and I think their productivity is abysmal based on what you'd expect from a company with their market share. They have very successful products, but it's obvious that basically all of their income comes from rent-seeking from developers who want to (well, need to) publish on Steam.
tapoxi
Valve is actually extremely small, I've heard estimates at around 350-400 people.
They're also a flat organization, with all the good and bad that brings, so scaling with contractors is easier than bringing on employees that might want to work on something else instead.
mindcrash
Proton is mainly a co-effort between in-house developers at Valve (with support on specific parts from contractors like Igalia), developers at CodeWeavers and the wider community.
For contextual, super specific, super specialized work (e.g. SCX-LAVD, the DirectX-to-Vulkan and OpenGL-to-Vulkan translation layers in Proton, and most of the graphics driver work required to make games run on the upcoming ARM based Steam Frame) they like to subcontract work to orgs like Igalia but that's about it.
everfrustrated
Valve is known to keep their employee count as low as possible. I would guess anything that can reasonably be contracted out is.
That said, something like this which is a fixed project, highly technical and requires a lot of domain expertise would make sense for _anybody_ to contract out.
izacus
This is how "Company funding OSS" looks like in real life.
There have been demands to do that more on HN lately. This is how it looks like when it happens - a company paying for OSS development.
treyd
They seem to be doing it through Igalia, which is a company based on specialized consulting for the Linux ecosystem, as opposed to hiring individual contractors. Your point still stands, but from my perspective this arrangement makes a lot of sense while the Igalia employees have better job security than they would as individual contractors.
redleader55
It's worth mentioning that sched_ext was developed at Meta. The schedulers are developed by several companies who collaborate to develop them, not just Meta or Valve or Italia and the development is done in a shared GitHub repo - https://github.com/sched-ext/scx.
999900000999
That's the magic of open source. Valve can't say ohh noes you need a deluxe enterprise license.
senfiaj
In this case yes, but on the other hand Red Hat won't publish the RHEL code unless you have the binaries. The GPLv2 license requires you to provide the source code only if you provide the compiled binaries. In theory Meta can apply its own proprietary patches on Linux and don't publish the source code if it runs that patched Linux on its servers only.
cherryteastain
Can't anyone get a RHEL instance on their favorite cloud, dnf install whatever packages they want sources of, email Redhat to demand the sources, and shut down the instance?
dfedbeef
RHEL specifically makes it really annoying to see the source. You get a web view.
kstrauser
I'm more surprised that the scheduler made for a handheld gaming console is also demonstrably good for Facebook's servers.
giantrobot
Latency-aware scheduling is important in a lot of domains. Getting video frames or controller input delivered on a deadline is a similar problem to getting voice or video packets delivered on a deadline. Meanwhile housecleaning processes like log rotation can sort of happen whenever.
bigyabai
I mean, part of it is that Linux's default scheduler is braindead by modern standards: https://en.wikipedia.org/wiki/Completely_Fair_Scheduler
3eb7988a1663
Part of that is the assumption that Amazon/Meta/Google all have dedicated engineers who should be doing nothing but tuning performance for 0.0001% efficiency gains. At the scale of millions of servers, those tweaks add up to real dollar savings, and I suspect little of how they run is stock.
accelbred
CFS was replaced by EEVDF, no?
jorvi
I mean.. many SteamOS flavors (and Linux distros in general have) have switched to Meta's Kyber IO scheduler to fix microstutter issues.. the knife cuts both ways :)
bronson
Kyber is an I/O scheduler. Nothing to do with this article.
Brian_K_White
The comment was perfectly valid and topical and applicable. It doesn't matter what kind of improvement Meta supplied that everyone else took up. It could have been better cache invalidation or better usb mouse support.
loeg
Maybe better to go straight to the source and bypass Phoronix blogspam: https://www.youtube.com/watch?v=KFItEHbFEwg
tra3
I'm curious how this came to be:
> Meta has found that the scheduler can actually adapt and work very well on the hyperscaler's large servers.
I'm not at all in the know about this, so it would not even occur to me to test it. Is it the case that if you're optimizing Linux performance you'd just try whatever is available?
laweijfmvo
almost certainly bottom-up: some eng somewhere read about it, ran a test, saw positive results, and it bubbles up from there. this is still how lots of cool things happen at big companies like Meta.
binary132
I'm struggling to understand what workloads Meta might be running that are _this_ latency-critical.
commandersaki
According to the video linked somewhere in this thread indicates WhatsApp Erlang workers that want sub-ms latency.
dabockster
It's Meta. They always push to be that fast on paper, even when it's costly to do and doesn't really need it.
Pr0Ger
It's definitely for ads auctions
tayo42
If you have 50,000 servers for your service, and you can reduce that by 1 percent, you save 50 servers. Multiply that by maybe $8k per server and you have saved $400k,you just paid for your self for a year. With meta the numbers are probably a bit bigger.
stuxnet79
Meta is a humongous company. Any kind of latency has to have a business impact.
tayo42
Interesting to see server workloads take ideas from other areas. I saw recently that some of the k8s specific os do their updates like android devices
alecco
[flagged]
hoppyhoppy2
Generated comments are not allowed on HN.
null
TZubiri
I feel the intent of these rules are to forbid undisclosed aigen comments.
If you ban disclosed usage of AIgen, you will get covert usage of AIgen
null
mort96
You can actually ban both.
bigyabai
> you will get covert usage of AIgen
We get that regardless of how we ban disclosed usage.
wizzwizz4
Such behaviour is extremely obvious. Anyone capable of hiding it is also capable of just… not using AIgen.
alecco
Meanwhile, armies of political bots make comment sections terrible and nothing happens. Yeah, I know talking about it also breaks the rules. But it is the truth!
I don't want HN to become another victim of Dead Internet.
And I explicitly stated my comment was AI generated. No dishonesty. 10k account with 17 years. This is ridiculous.
Boxxed
Posting an AI summary is about as useful as posting Google search results. We can all do it, we don't need anyone to do it for us.
mikkupikku
As well as the points already raised by others, I'd like to make the point that we should be encouraging people to prompt LLMs themselves rather than just accepting the outputs of others. As a social norm, this will make society more robust to misinformation and deception, as it will result in fewer people trusting outputs without knowing how the LLM was actually prompted.
This probably doesn't really matter in this context, but I think it's a general best practice worth reinforcing whenever possible.
littlestymaar
If someone want to ask an LLM about something, good for them, but there's no need to paste its content over the internet, disclosed or not.
Valve is practically singlehandedly dragging the Linux ecosystem forward in areas that nobody else wanted to touch.
They needed Windows games to run on Linux so we got massive Proton/Wine advancements. They needed better display output for the deck and we got HDR and VRR support in wayland. They also needed smoother frame pacing and we got a scheduler that Zuck is now using to run data centers.
Its funny to think that Meta's server efficiency is being improved because Valve paid Igalia to make Elden Ring stutter less on a portable Linux PC. This is the best kind of open source trickledown.