TinyKVM: Fast sandbox that runs on top of Varnish
59 comments
·March 14, 2025nine_k
Oh. It's like Firecracker, only much faster 8-)
What I like most is the ability to instantly reset the state of the VM to a known predefined state. It's like restarting the VM without any actual restart. It looks like an ideal course of action for network-facing services that are constantly under attack: even if an attack succeeds, the result is erased on the next request.
Easy COW page sharing for programs that are not written with that in mind, like ML model runners, is also pretty nice.
chatmasta
It also sounds ideal for resuming memory intensive per-user programs, like LLMs with a large context window. You can basically have an executable (and its memory) attached to a user session, but only pay the cost for it while the user session has an open request.
dividuum
Yes
> TinyKVM can fork itself into copies that use copy-on-write to allow for huge workloads like LLMs to share most memory. As an example, 6GB weights required only 260MB working memory per instance, making it highly scalable.
daralthus
yes that's the durable objects ~ durable agents model that cloudflare is building
bonzini
It's more like gVisor (or QEMU's user mode emulation but it does not support KVM, only dynamic code translation).
ruben_varnish
Original post: https://fwsgonzo.medium.com/tinykvm-the-fastest-sandbox-564a...
You can find a bunch of posts related to this topic there as well.
laurencerowe
This is really exciting. The 2.5us snapshot restore performance is on a par with Wasmtime but with the huge advantage of being able to run native code, albeit with the disadvantage of much slower but still microsecond interop.
I see there is a QuickJS demo in the tinykvm_examples repo already but it'd be great to see if it's possible to get a JIT capable JavaScript runtime working as that will be an order of magnitude faster. From my experiments with server rendering a React app native QuickJS was about 12-20ms while v8 was 2-4ms after jit warmup.
I need to study this some more but I'd love to get to the point where there was a single Deno like executable that ran inside the sandbox and made all http requests through Varnish itself. A snapshot would be taken after importing the specified JS URl and then each request would run in an isolated snapshot.
Probably needs a mechanism to reset the random seed per request.
fwsgonzo
You can run v8 jitless, if you want. It's going to be much faster than QuickJS. Adding JIT support means adding a fixed executable range, which you also can do already, but you can't run it in the dumb CLI example. JITs love to be W+X. So, not sure if it's an afternoon amount of work yet, due to security implications.
I have experience with this from libriscv, where I also embed JIT run-times like v8 and LuaJIT already.
laurencerowe
From my tests v8 jitless was about 50% faster than QuickJS but still almost an order of magnitude slower than with JIT.
Note that I mistranscribed the numbers above: QuickJS was 18-24ms while v8 without warmup was 12-20ms (which I think is similar to jitless perf) and warmed jit was 2-4ms when I benchmarked a couple of years back. https://news.ycombinator.com/item?id=33793181
Thanks for the complexity warning. Sounds like I need to wait for an embedded JIT example using fixed executable range before I start playing around. But it would be fun to try and make Deno run inside it somehow, perhaps building on deno_runtime and hooking the http client user agent to make requests through Varnish. Deno's permission system should allow cleanly disabling unavailable functionality like access to the file system.
I see some examples that seem to use glibc but I was under the impression only musl binaries can be truly static? Can binaries built against glibc be used with TinyKVM?
rwmj
Isn't this basically libkrun? https://github.com/containers/libkrun
incanus77
Not entirely what this is intended for, but does anyone have experience running an X server (or Wayland, I don't care)?
I'm doing some dev (on Mac) against RDP server and occasionally have other needs like that for a client. Currently I use UTM (nice QEMU Mac frontend) along with a DietPi (super stripped-down Debian) VM for these sorts of things.
I'm pretty familiar with Docker, but have a good idea of what sorts of hoop-jumping might be needed to get a graphics server to run there. Wondering if there's a simpler path.
wmf
Fascinating but I'm having trouble understanding the big picture. This runs a user process in a VM with no kernel? Does every system call become a VM exit and get proxied to the host? Or are there no system calls?
cryptonector
IIUC there's no need for system calls because there's no I/O. There's just program arguments and shared memory.
dividuum
You need a few syscalls: to grow your heap size (brk) or to exit your program (exit). I took a quick look at their code and here are the syscalls and arguments implemented: https://github.com/varnish/tinykvm/blob/master/src/functions...
It’s a bit more than running a program under seccomp strict mode, but conceptually similar, so running anything too complicated likely won't work. You certainly won’t be able to sandbox chromium for taking website snapshots for example.
fwsgonzo
There's many ways to go about it, but essentially yes, brk and mmap and a few others just to get into main() for some common run-times.
But you can do whatever you want. For example in libriscv I override the global allocator in my guest programs to use a host-managed heap. That way heap usage has native performance in interpreter mode, while also allowing me full control of the heap from the outside. I wrote about this here: https://medium.com/@fwsgonzo/using-c-as-a-scripting-language...
cryptonector
Well, yes, there's a few system calls like `brk()`/`sbrk()`/`getrandom()`, and what not. But you don't get to open any files or sockets or devices.
winternewt
What do you mean by I/O exactly? Because to me handling HTTP requests definitely requires I/O, no matter how you technically implement it. Does the program start anew with new arguments for each HTTP request, and if so how is that an improvement over I/O syscalls?
nine_k
Handling HTTP requests can be done entirely via stdin + stdout. But it won't be too useful if you could not even talk to a database.
The VM may (and should) be limited to a small subset of what's available on the host though.
cryptonector
I mean you don't get to open files, sockets, devices, etc. in the sandboxed program. You get to do just a few minimal things like I/O on stdin/stdout/stderr, use shared memory, maybe allocate memory.
jkrshnmenon
I believe it ships with its own kernel
> The TinyKVM guest has a tiny kernel which cannot be modified.
tuananh
this is really cool if it works for your use cases.
Some notes from the post
> I found that TinyKVM ran at 99.7% native speed
> As long as they are static and don’t need file or network access, they might just run out-of-the box.
> The TinyKVM guest has a tiny kernel which cannot be modified
jedisct1
And unlike WebAssembly, it can leverage specialized CPU instructions. This is huge for cryptographic implementations, video codecs, LLMs, etc.
notpushkin
This is so cool.
I’m exploring micro-VMs for my self-hosted PaaS, https://lunni.dev/ – and something with such little overhead seems like a really interesting option!
otterley
There's nothing in the article that suggests that it runs on top of Varnish; in fact, the author even says it's not intended to run Varnish in it.
ruben_varnish
There's a contradiction in the text, I'll giv eyou that, bu at the end he clearly links both * a Varnish Module using this <https://github.com/varnish/libvmod-tinykvm> * a set of examples in multiple languages <https://github.com/varnish/tinykvm_examples>
otterley
I still believe the nexus needs to be described clearer and more strongly in the story in order to support the title here in HN that it runs on top of Varnish. Even the blog title itself does not make such a claim.
Tepix
Interesting to see the performance gain. But without file i/o and network access, what are the use cases?
jedisct1
You can call host functions doing whatever you want. Similar to what WebAssembly does.
winternewt
I'm curious: would it be a good idea to switch my desktop Linux pc to using huge pages across the board?
I love this. Please never stop doing what you’re doing.
edit: Of course you’re the top contributor to IncludeOS. That was the first project I thought of while reading this blog post. I’ve been obsessed with the idea of Network Function Virtualization for a long time. It’s the most natural boundary for separating units of work in a distributed system and produces such clean abstractions and efficient scaling mechanisms.
(I’m also a very happy user of Varnish in production btw. It’s by far the most reliable part of the stack, even more than nginx. Usually I forget it’s even there. It’s never been the cause of a bug, once I got it configured properly.)