Brush – A new compatible Gaussian splatting engine
32 comments
·October 24, 2024Groxx
steve_adams_86
I’m realizing this might not cross some developers minds because the current time in that context is attached to the commit when it was added or changed.
I do this quite often. I probably shouldn’t, though. It’s only useful if you’re looking at commit logs or have an inline ‘last changed by [author] on [date]’ helper in your IDE.
Then again, even that could be made wrong by future edits.
Groxx
Yeah, there are frequently ways to figure out what date a relative measure is referring to; it's one of the best things about version control being a true norm in this field.
But it's a few extra steps (depending on the UI), and many will not take those steps. They'll just trust it (far beyond when it's relevant), or think "that's probably old" and doubt it (immediately, because old docs are so common).
It's relatively minor, but it's extremely easy to prevent, and just a better habit when communicating with the future.
codeflo
And that’s assuming the repository never gets reorganized in any way that doesn’t perfectly preserve history — which over long timespans, is bound to happen at some point.
Ameo
Wow - the in-browser demo (https://arthurbrussee.github.io/brush-demo/) runs way more performantly and renders much better-looking results than any other I'd tried in the past.
It loaded my 50MB .ply file almost instantly. Orbiting around the scene is extremely smooth and everything is free of flickering or artifacts.
I never tried out training a Gaussian splat from images/video myself before, but this tool makes me want to give it a go.
ArthurBrussee
Love to hear it!! Most viewers take some shortcuts, like only sorting every so often, it's good to hear the difference is noticable :)
Training a splat requires a lot less setup with this, but does still require running COLMAP(https://github.com/colmap/colmap) first, which is still a big barrier... one thing at a time!
throwaway2562
What are splats actually useful for, and where are they used?
eurekin
Corridor Channel had one great example: https://youtu.be/GaGcLhhhbDs?si=vDyeayLf8EAoE0gf&t=442
Above includes the explanation. Final result is here:
twelvechairs
Making a movable 3d scene from limited initial information. Some pros and cons against a traditional '3d model' approach. Pros - faster/simpler to generate (especially with lots of data), better at dealing with light and reflection. Cons - Having 3d geomtry can be useful eg. for collision detection, volumetric understanding, surface alteration/deformation etc.
Not much widespread use right now - Possible commercial use cases are things like real estate walkthroughs and maybe replacing a google street view with something more interactive.
ArthurBrussee
Its really the latest incarnation in the field of Photogrammetry https://en.wikipedia.org/wiki/Photogrammetry - aka, converting 2D images / video to 3D data.
Imagine one of those house tours on Zoopla on steroids, or street view but smoother.
two_handfuls
Splats are good for generating new images from an existing place even if no photograph exists from that exact viewpoint.
They can be used for video special effects, for 3D images/video, and for VR. The technology is nascent but shows promise.
WhatIsDukkha
One of the things thats held me back from being super interested in this field is that my understanding is that there is likely to be some kind of mesh backing needed for this to progress.
IIRC some researchers had started to back the gaussians with a mesh to provide an editable artifact that would allow the gaussians to be moved and manipulated.
Is this anywhere near being a standard feature yet?
edit - ie https://arxiv.org/abs/2402.04796
neom
Is this kinda tech eventually going to sometimes sub in for compression and/or codecs? Feel like it could be kinda applicable to streaming?
DarmokJalad1701
Gaussian splatting using Burn has been on my side project list for a while now. I guess they beat me to it! :)
jtrueb
async_std is a nonstandard choice these days, no? I assume this is related to the style of blocking spawned work?
ArthurBrussee
I'm a bit out of the loop on async runtimes. I know Tokio is of course the big on, but, that seemed much to heavy to just run some tasks, and isn't very WASM compatbile afaik.
Otherwise there's smol, and maybe others? Would love to hear what a good web WASM compatible async framework is nowadays!
efilife
What's it compatible with?
ArthurBrussee
Devices & operating systems! Windows/Mac/Linux, AMD / nvidia / built-in GPUs, Android/iOs, or running in a browser context.
My bad, I really bungled the original tweet the title is from :)
Alifatisk
Super compatible?
dang
We've made it less super and also deinrusted it in the title above.
(submitted title was "Brush – a new super compatible Gaussian splatting engine in Rust")
rc00
> deinrusted
Out of curiosity, what is the motivation or policy here? This feels like a change in stance.
Groxx
To place a barely-educated guess: because "... In Rust" is enough of a trope around here that it brings very specific crowds of people out to argue the same points each time, whether it's even remotely relevant to the link or not.
Better to just avoid it unless the "in rust" part is somehow intrinsically relevant (e.g. it's in rust for specific reasons that were previously too hard in other languages)
dang
Rust is well enough established at this point that titles don't need the extra juice.
ArthurBrussee
I think the title is taken from my original tweet which I really bungled, my bad :) Hopefully the readme does a better job!
A request for everyone writing docs with content like this:
>NOTE: This only works on desktop Chrome 129+ currently. Firefox and Safari are hopefully [supported soon](link), but currently even firefox nightly and safari technical preview do not work.
This is great, especially with that link! Thank you! But please say when "currently" is, e.g. add an "(Oct 2024)". Stuff like this tends to be time-sensitive on accuracy but not consistently updated and is often years out of date with no easy way for visitors to tell.
And when it's recent, it also tells people that the project is active.