Neural Radiance Fields (NeRF)

Neural radiance fields (NeRF) is a method for reconstructing a three-dimensional representation of a scene from a set of two-dimensional images. Introduced in 2020, it trains a small neural network to store the geometry and appearance of a scene as a continuous field, and then uses that network to render photorealistic images of the scene from viewpoints that were never photographed, a task known as novel view synthesis. NeRF prompted a wave of research in neural rendering and 3D computer vision and established the family of implicit neural scene representations, sometimes called neural fields.

How it works

A NeRF represents a scene as a continuous function of five variables: the three spatial coordinates of a point and the two angles describing a viewing direction. This function is implemented as a multilayer perceptron, a simple fully connected neural network. Given a point in space and a direction from which it is viewed, the network outputs a colour and a density (opacity) value. Density captures how much light the point blocks, and colour is made view-dependent so that the network can represent effects such as reflections and specular highlights that change with viewing angle.

To turn this function into an image, NeRF uses differentiable volume rendering. For each pixel of a target image, a ray is cast from the camera through the scene, points are sampled along the ray, and the network is queried at each point. The colours and densities along the ray are accumulated, closer and more opaque points contributing more, to produce the final pixel colour. Because this rendering process is differentiable, the whole system can be trained by gradient descent: the network is optimised so that the images it renders match the original photographs. Once trained, the network can be queried from any new camera position to synthesise a fresh view.

A notable property of NeRF is that it is self-supervised in the sense that it requires only images and their known camera poses. It does not need explicit 3D models, depth sensors, or manual labelling, which lowers the barrier to capturing detailed 3D scenes.

Strengths, limitations, and successors

NeRF produces strikingly realistic novel views and captures fine detail and view-dependent effects that mesh-based methods struggle with. Its original limitations were practical: training a NeRF for a single scene could take hours, rendering was slow, and each model represented only one scene. A large research effort addressed these issues, producing variants that train and render far faster, handle unbounded outdoor scenes, work from few input images, and represent dynamic scenes that change over time.

In 2023, 3D Gaussian splatting offered an alternative that reached comparable quality with real-time rendering by using an explicit set of primitives rather than a neural network, and it has since become a popular counterpart. NeRF and Gaussian splatting are now often discussed together as the two dominant approaches to radiance-field reconstruction, each with different trade-offs in speed, editability, and representation.

Applications of NeRF and its descendants include virtual and augmented reality, robotics and autonomous navigation, urban mapping, drone surveying, cultural heritage capture, and visual effects.

References

1. Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., and Ng, R. (2020). NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. ECCV.
2. Gao, K., et al. (2022). NeRF: Neural Radiance Field in 3D Vision, A Comprehensive Review. arXiv.
3. Wikipedia contributors. (2025). Neural radiance field. Wikipedia.