Launch HN: ParaQuery (YC X25) – GPU Accelerated Spark/SQL
51 comments
·May 12, 2025threeseed
Many of us have been using GPU accelerated Spark for years:
https://developer.nvidia.com/rapids/
https://github.com/NVIDIA/spark-rapids
And it's supported on AWS: https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-spar...
winwang
Indeed, Spark-RAPIDS has been around for a while! And it's quite simple to have a setup that works. Most of the issues come after the initial PoC, especially for teams not wanting to manage infra, not to mention GPU infra.
dogman123
This could be incredibly useful for me. Currently struggling to complete jobs with massive amounts of shuffle with Spark on EMR (large joins yielding 150+ billion rows). We use Glue currently, but it has become cost prohibitive.
threeseed
You should try using an S3 based shuffle plugin: https://github.com/IBM/spark-s3-shuffle
Then mount FSX for Lustre on all of your EMR nodes and have it write shuffle data there. It will massively improve performance and shuffle issues will disappear.
Is expensive though. But you can offset the cost now because you can run entirely Spot instances for your workers as if you lose a node there's no recomputation of the shuffle data.
winwang
Is the shuffle the biggest issue? Not too sure about joins but one of the datasets we're currently dealing with has a couple trillion rows. Would love to chat about this!
mritchie712
> they're saving over 60% off of their BigQuery bill
how big is their data?
A lot of BigQuery users would be surprised to find they don't need BigQuery.
This[0] post (written by founding engineer of BigQuery) has a bit of hyperbole, but this part is inline with my experience:
> A couple of years ago I did an analysis of BigQuery queries, looking at customers spending more than $1000 / year. 90% of queries processed less than 100 MB of data. I sliced this a number of different ways to make sure it wasn’t just a couple of customers who ran a ton of queries skewing the results. I also cut out metadata-only queries, which are a small subset of queries in BigQuery that don’t need to read any data at all. You have to go pretty high on the percentile range until you get into the gigabytes, and there are very few queries that run in the terabyte range.
We're[1] built on duckdb and I couldn't be happier about it. Insanely easy to get started with, runs locally and client-side in WASM, great language features.
threeseed
> A lot of BigQuery users would be surprised to find they don't need BigQuery.
No they wouldn't.
a) BigQuery is the only managed, supported solution on GCP for SQL based analytical workloads. And they are using it because they started with GCP and then chose BigQuery.
b) I have supported hundreds of Data Scientists over the years using Spark and it is nothing like BigQuery. You need to have much more awareness of how it all fits together because it is sitting on a JVM that when exposed to memory pressure will do a full GC and kill the executor. When this happens at best your workload gets significantly slower and at worst your job fails.
winwang
Hopefully, we can be another managed solution for those on GCP.
And as for your second point, yep, Spark tuning is definitely annoying! BigQuery is a lot more than jusr the engine, and building a simple interface for a complicated, high-performance process is hard. That's a big reason why I made ParaQuery.
threeseed
You may want to look into DataMechanics who is another YC startup who tried something similar. They were acqui-hired by NetApp.
If I remember they focused on SME space because in enterprise you will likely struggle against pre-allocated cloud spend budgets which lock companies into just using GCP services. I've worked at a dozen enterprise companies now and every one had this.
winwang
They have >1PB of data to ETL, with some queries hitting 450TB of pure shuffle.
It's very true that most users don't need something like BigQuery or Snowflake. That's why some startups have come up to save Snowflake cost by "simply" putting a postgres instance in front of it!
In fact, I just advised someone recently to simply use Postgres instead of BigQuery since they had <1TB and their queries weren't super intensive.
random17
Congrats on the launch!
Im curious about what kinds of workloads you see GPU-accelerated compute have a significant impact, and what kinds still pose challenges. You mentioned that I/O is not the bottleneck, is that still true for queries that require large scale shuffles?
winwang
Large scale shuffles: Absolutely. One of the larger queries we ran saw a 450TB shuffle -- this may require more than just deploying the spark-rapids plugin, however (depends on the query itself and specific VMs used). Shuffling was the majority of the time and saw 100% (...99%?) GPU utilization. I presume this is partially due to compressing shuffle partitions. Network/disk I/O is definitely not the bottleneck here.
It's difficult to say what "workloads" are significant, and easier to talk about what doesn't really work AFAIK. Large-scale shuffles might see 4x efficiency, assuming you can somehow offload the hash shuffle memory, have scalable fast storage, etc... which we do. Note this is even on GCP, where there isn't any "great" networking infra available.
Things that don't get accelerated include multi-column UDFs and some incompatible operations. These aren't physical/logical limitations, it's just where the software is right now: https://github.com/NVIDIA/spark-rapids/issues
Multi-column UDF support would likely require some compiler-esque work in Scala (which I happen to have experience in).
A few things I expect to be "very" good: joins, string aggregations (empirically), sorting (clustering). Operations which stress memory bandwidth will likely be "surprisingly" good (surprising to most people).
Otherwise, Nvidia has published a bunch of really-good-looking public data, along with some other public companies.
Outside of Spark, I think many people underestimate how "low-latency" GPUs can be. 100 microseconds and above is highly likely to be a good fit for GPU acceleration in general, though that could be as low as 10 microseconds (today).
_zoltan_
8TB/s bandwidth on the B200 helps :-) [yes, yes, that is at the high end, but 4.8TB/s@H200, 4TB/s@H100, 2TB/s@A100 is nothing to sneeze at either).
winwang
Very true. Can't get those numbers even if you get an entire single-tenant CPU VM. Minor note, A100 40G is 1.5TB/s (and much easier to obtain).
That being said, ParaQuery mainly uses T4 and L4 GPUs with "just" ~300 GB/s bandwidth. I believe (correct me if I'm wrong) that should be around a 64-core VM, though obviously dependent on the actual VM family.
Boxxed
I'm surprised the GPU is a win when the data is coming from GCS. The CPU still has to touch all the data, right? Or do you have some mechanism to keep warm data live in the GPUs?
threeseed
I've used GPU based Spark SQL for many years now and it sounds flashy but it's not going to make a meaningful difference for most use cases.
As you say the issue is that you have an overall process to optimise from getting the data off slow GCS onto the nodes, shuffling it which often then writes it to a slow disk before the real processing even starts then writing back to a slow GCS.
winwang
Yep, CPU has to transfer data because no RDMA setup on GCP lol. But that's like 16-32 GB/s of transfer per GPU (assuming T4/L4 nodes), which is much more than network bandwidth. And we're not even network bound, even if there's no warm data (i.e. for our ETL workloads). However, there is some stuff kept on GPU during actual execution for each Spark task even if they aren't running on the GPU at the moment, which makes handling memory and partition sizes... "fun", haha.
achennupati
this is super cool stuff, and would've been really interesting to apply to spark stuff I had to do for Amazon's search system! How is this different than something like using spark-rapids on AWS EMR with GPU-enabled EC2 instances? Are you building on top of that spark-rapids, or is this a more custom solution?
winwang
Good question -- it depends. For certain workloads, it might look exactly the same! For others, I found that the memory and VM constraints were creating large inefficiencies. Also, many teams simply don't want to manage that level of data infra: managing EMR, instance type optimization, spark optimization (now with GPU configs!), custom images, upgrades, etc.
We take care of that and make it as easy as pie... or so we hope! On top of that, we also deploy an external shuffle service, and deal with other plugins, connectors, etc.
I suppose it's similar to using Databricks Serverless SQL!
Another thing: we ran into an incompatible (i.e. non-accelerated) operation in one of our first real workloads, so we worked with our customer to speed up that workload even more with a small query optimization.
rockostrich
> I started working on actually deploying a cloud-based data platform powered by GPUs (i.e. Spark-RAPIDS)
Based on this, the platform is using Spark-RAPIDS.
latchkey
Great, let me know if you want a 1x AMD MI300x VM to build/test on. Free.
winwang
Nice! I attended a hackathon by Modular last weekend where we got to play with MI300X (sponsored by AMD and Crusoe). My team made a GPU-"accelerated" BM25 in Mojo, but mostly kind of failed at it, haha.
The software stack for AMD is still a bit too nascent for ParaQuery's Spark engine, but certain realtime/online workloads can definitely be programmed pretty fast. They also happen to benefit greatly from the staggering levels of HBM on AMD chips. Hopefully I can take a mini-vacation later in the summer to hack on your GPUs :)
latchkey
Great feedback and thanks for the follow on twitter.
Agreed, their software stack needs work, but thankfully that's why they are sponsoring developer events like you attended. The progress is happening fast and this is something that wasn't happening at all 6-12 months ago. It is a real shift in focus.
If you have specific areas you'd like me to pass up the chain for them in order for you to build support for your engine, please let me know and I'm happy to try to help however I can. It took a while for us to get there, but they are now extremely responsive to us.
billy1kaplan
Congrats on the launch, Win! I remember seeing your prototype a while ago and its awesome to see your progress!
_zoltan_
couple weeks ago at VeloxCon one of the days was dedicated to GPU processing (the other being AI/ML data preprocessing); the cuDF team talked about their Velox integration as well. for those interested, might worth to check it out.
disclaimer: my team is working on this very problem as well, as I was a speaker at VeloxCon.
winwang
Just checked out Velox. It's awesome that you're reducing duplicate eng effort! What was your talk about?
_zoltan_
I was part of the panel discussion at the end of the 2nd day discussing hardware acceleration for query processing. before the panel there were very interesting talks about various approaches on how to get to the end goal, which is to efficiently use hardware (non-x86 CPUs, so mostly either GPUs or FPGAs/custom chips) to speed up queries.
winwang
Since you mentioned non-x86, how are things on the ARM side? I believe I heard AWS's Graviton + Correto combo was a huge increase for JVM efficiency.
FPGAs... I somehow highly doubt their efficiency in terms of being the "core" (heh) processor. However, "compute storage" with FPGAs right next to the flash is really interesting.
jelder
Any relationship with the PG-Strom project?
winwang
No relationship... yet! Hoping to have a good relationship in the future so I have a business reason to fly to Japan :D
Btw, interesting thing they said here: "By utilization of GPU (Graphic Processor Unit) device which has thousands cores per chip"
It's more like "hundreds", since the number of "real" cores is like (CUDA cores / 32). Though I think we're about to see 1k cores (SMSPs).
That being said, I do believe CUDA cores have more interesting capabilities than a typical vector lane, i.e. for memory operations (thank the compiler). Would love to be corrected!
dbacar
Cool project, congratulations.
How would you contrast it against HeavyDB?
winwang
I'm not too familiar with HeavyDB, but here are the main differences:
- We're fully compatible with Spark SQL (and Spark). Meaning little to no migration overhead.
- Our focus is on distributed compute first.
- That means ParaQuery isn't a database, just an MPP engine (for now). Also means no data ingestion/migration needed.
vpamrit2
Very cool and great job! Would love to see more as you progress on your journey!
Hey HN! I'm Win, founder of ParaQuery (https://paraquery.com), a fully-managed, GPU-accelerated Spark + SQL solution. We deliver BigQuery's ease of use (or easier) while being significantly more cost-efficient and performant.
Here's a short demo video demonstrating ParaQuery (vs. BigQuery) on a simple ETL job: https://www.youtube.com/watch?v=uu379YnccGU
It's well known that GPUs are very good for many SQL and dataframe tasks, at least by researchers and GPU companies like NVIDIA. So much so that, in 2018, NVIDIA launched the RAPIDS program and the Spark-RAPIDS plugin (https://github.com/NVIDIA/spark-rapids). I actually found out because, at the time, I was trying to craft a CUDA-based lambda calculus interpreter…one of several ideas I didn't manage to implement, haha.
There seems to be a perception among at least some engineers that GPUs are only good for AI, graphics, and maybe image processing (maybe! someone actually told me they thought GPUs are bad for image processing!) Traditional data processing doesn’t come to mind. But actually GPUs are good for this as well!
At a high level, big data processing is a high-throughput, massively parallel workload. GPUs are a type of hardware specialized for this, are highly programmable, and (now) happen to be highly available on the cloud! Even better, GPU memory is tuned for bandwidth over raw latency, which only improves their throughput capabilities compared to a CPU. And by just playing with cloud cost calculators for a couple of minutes, it's clear that GPUs are cost-effective even on the major clouds.
To be honest, I thought using GPUs for SQL processing would have taken off by now, but it hasn't. So, just over a year ago, I started working on actually deploying a cloud-based data platform powered by GPUs (i.e. Spark-RAPIDS), spurred by a friend-of-a-friend(-of-a-friend) who happened to have BigQuery cost concerns at his startup. After getting a proof of concept done and a letter of intent... well, nothing happened! Even after over half a year. But then, something magical did happen: their cloud credits ran out!
And now, they're saving over 60% off of their BigQuery bill by using ParaQuery, while also being 2x faster -- with zero data migration needed (courtesy of Spark's GCS connector). By the way, I'm not sure about other people's experiences but... we're pretty far from being IO-bound (to the surprise of many engineers I've spoken to).
I think that the future of high-throughput compute is computing on high-throughput hardware. If you think so too, or you have scaling data challenges, you can sign up here: https://paraquery.com/waitlist. Sorry for the waitlist, but we're not ready for a self-serve experience just yet—it would front-load significant engineering and hardware cost. But we’ll get there, so stay tuned!
Thanks for reading! What have your experiences been with huge ETL / processing loads? Was cost or performance an issue? And what do you think about GPU acceleration (GPGPU)? Did you think GPUs were simply expensive? Would love to just talk about tech here!