【读论文】【速读】4D Gaussian Splatting for Real-Time Dynamic Scene Rendering

1. What

4D Gaussian Splatting (4D-GS) as a holistic representation for dynamic scenes rather than applying 3D-GS for each individual frame.

It uses an encoder and decoder structure to predict the motion of each Gaussian over time. The core idea is to represent the 4D information (x,y,z,t) into 2D HexPlane and then use MLP and decoder to extract the information of the change of Gaussian. This approach allows for efficient processing and storage of high-dimensional data while preserving the necessary spatiotemporal information.

2. Preliminary

There are two dynamic methods in NeRF and one method in Gaussian, as shown below:

As for NeRF, all the dynamic NeRF algorithms can be formulated as:

$$c,\sigma =\mathcal{M}(\mathbf{x},t)$$

1. In Fig. 2 (a), the canonical mapping volume rendering transforms each sampled point into a canonical space: ${\varphi }_t:(\mathbf{x},t)\to \Delta \mathbf{x}$ and calculates the color and density along each ray:

   $$c,\sigma =\mathrm{NeRF}(\mathbf{x}+\Delta \mathbf{x}).$$

2. In Fig. 2 (b), the time-aware volume rendering. It won’t change the rendering path, oppositely, it directly calculates the features of each point at a time:

   $$c,$$