We've been delving into the latest advancements in reconstructive 3D rendering, and one innovation that has caught our attention is the emerging world of 3D Gaussian Splatting. This rasterization technique is reshaping the way we capture real-world objects and scenes and seamlessly integrate them into the virtual realms.
As native Unreal Engine (UE) developers, we were immediately drawn to the possibilities of seamlessly integrating these scans into Unreal Engine. Our journey led us to explore the most efficient methods for achieving this integration, and we're excited to share our experiences and insights on how to swiftly bring 3D Gaussian Splatting into your Unreal Engine projects.
What is Gaussian Splatting?
Gaussian Splatting is a rendering technique that represents a 3D scene as a collection of particles, where each particle is essentially a 3D Gaussian function with various attributes such as position, rotation, non-uniform scale, opacity, and color (represented by spherical harmonics coefficients).
The particles are rendered as 2D Gaussians in screen space, creating the appearance of blobs in the rendered scene.
The technique involves two major innovations:
Creating these millions of Gaussian particles that represent a scene, using gradient descent and differentiable rendering techniques.
Developing a fast method for rendering these millions of particles, often utilizing GPU-based rasterization.
The combination of these innovations results in a more flexible and efficient way to render complex 3D scenes, particularly those reconstructed from 2D photos or images. The technique leverages Gaussian functions to capture scene information, and it efficiently renders these particles to produce realistic and detailed scene representations.