Radiant Foam: Real-Time Differentiable Ray Tracing
10 comments
·February 4, 2025thih9
kvark
In pinhole cameras, straight lines look straight. That's what a regular projection matrix gives you with rasterization. With non-pinhole cameras, straight lines look curved. You can't rasterize this directly. 3D Gaussian splats have an issue with this, addressed by methods like ray tracing. It's very useful to train on non-pinhole cameras, because in real world they can capture a wider field of view.
markisus
In Gaussian Splatting, a first order approximation takes ellipsoids in camera space to ellipses in image space. This works ok for pigeonhole cameras where straight lines remain straight and more generally conic sections are taken to other conic sections. For high distortion models like fisheye, this approximation probably breaks. However this method presumably does not rely on approximation since it is ray traced.
jedbrooke
looks like it’s some sort of fisheye camera with a super wide fov. it might be simulating rays bending due to lens effects. a pinhole camera could just look “normal” ie straight lines stay straight (except for horizon convergence perspective effects)
xnx
How is Google using all these amazing radiant field techniques they're developing?
catapart
I have no idea, but given their stock of pictures of the entire earth (via google maps), I have some ideas about what I HOPE they would use this tech for.
wongarsu
And Google Maps/Google Earth have a long history of trying to create 3d views using all kinds of techniques, from manual modeling to radar satellite data.
CyberDildonics
How do you know they're amazing until you've used them yourself?
TuringTourist
By being amazed when observing it, one can conclude that a thing is amazing.
macawfish
They do look amazing
At the end of the video there is a non-pinhole camera demo; could someone explain what exactly is different about this camera?
I.e. what exactly the video is showing? And what would the video look like if that was a pinhole camera?