3D Gaussian Splatting
A scene representation and rendering technique that models a 3D scene as millions of coloured 3D Gaussians, enabling high-quality, real-time novel view synthesis.
3D Gaussian Splatting is a technique for representing and rendering three-dimensional scenes that reconstructs a scene from a set of ordinary photographs and then renders new views of it in real time. Introduced in a 2023 SIGGRAPH paper, it represents a scene not as a mesh or a neural network but as a large collection of three-dimensional Gaussians, each a small translucent blob with a position, shape, colour, and opacity. The method achieved rendering quality comparable to the best neural radiance field methods while rendering far faster, which made it one of the most influential developments in computer graphics and 3D computer vision of the mid-2020s.
How it works
The pipeline begins with a set of images of a scene taken from different viewpoints. Standard structure-from-motion software estimates the camera positions and produces a sparse cloud of 3D points, which becomes the starting set of Gaussians. Each Gaussian is described by a mean position, a covariance matrix that defines its size and orientation (making it anisotropic, meaning it can be stretched in a direction), an opacity value, and colour represented with spherical harmonics so that appearance can vary with viewing angle.
Training then optimises these parameters by comparing rendered images against the original photographs and adjusting the Gaussians to reduce the difference. During this process an adaptive density control step adds Gaussians in under-represented regions and removes those that contribute little, so detail concentrates where the scene needs it and computation is not wasted on empty space. The parameters are differentiable, allowing optimisation by gradient descent.
Rendering uses a technique called splatting rather than ray marching. Each 3D Gaussian is projected, or splatted, onto the image plane as a 2D ellipse, and the ellipses are sorted by depth and blended front to back. Because this rasterisation-style process maps efficiently onto graphics hardware, it supports interactive and real-time frame rates, which pure neural radiance fields historically could not.
Comparison with neural radiance fields
Both 3D Gaussian Splatting and neural radiance fields solve novel view synthesis, but they differ in representation and speed.
| Property | Neural radiance field | 3D Gaussian Splatting | | --- | --- | --- | | Scene stored as | Weights of a neural network | Explicit set of Gaussians | | Rendering method | Volume ray marching | Rasterised splatting | | Rendering speed | Typically slow | Real-time | | Editability | Harder to edit directly | Gaussians can be manipulated |
The explicit representation is one of Gaussian splatting's practical advantages: because the scene is a concrete set of primitives rather than opaque network weights, it is easier to edit, animate, compress, and integrate into existing graphics engines.
Applications and developments
Gaussian splatting has been applied to virtual and augmented reality, digital twins, cultural heritage capture, film and visual effects, robotics mapping, and product visualisation. A large body of follow-up work has extended it to dynamic scenes that change over time, to accurate surface and mesh reconstruction, to compression for storage and streaming, and to specialised domains such as medical imaging. Ongoing challenges include reducing memory footprint, improving geometric accuracy for measurement tasks, and handling sparse or few-view captures.
References
- Kerbl, B., Kopanas, G., Leimkuehler, T., and Drettakis, G. (2023). 3D Gaussian Splatting for Real-Time Radiance Field Rendering. ACM Transactions on Graphics (SIGGRAPH).
- Huang, B., et al. (2024). 2D Gaussian Splatting for Geometrically Accurate Radiance Fields. SIGGRAPH.
- Inria GraphDeco. (2023). 3D Gaussian Splatting project page. repo-sam.inria.fr.