Layered 4D-Rotor Gaussian Splatting

Abstract

We consider the problem of reconstructing long dynamic scenes from multi-view videos in a storage-efficient manner. Recent advances in Gaussian Splatting and its extensions to dynamic scenes have demonstrated impressive visual quality, but remain limited to short duration (<10 s), large storage size (>500 MB), and high GPU VRAM usage. To overcome these limitations, we introduce Layered 4D-Rotor Gaussian Splatting, a novel compressed representation designed for long dynamic scenes. Our approach integrates a layered 4D representation, efficient training, and effective compression into a unified framework. Specifically, 4D Gaussians are first organized into layers based on their temporal extents and then partitioned into discrete temporal buckets. This structure allows for selective access and rendering of only the necessary subsets of 4D Gaussians, substantially reducing GPU memory requirements. To further compress the representation, we apply a series of techniques, Factorized Covariance Quantization, Layered Compression, and Residual Codebook Quantization, achieving a compression ratio of up to 22.3× while preserving high visual fidelity. We implement a highly optimized C++/CUDA framework for efficient training, compression, and real-time rendering, achieving over 500 FPS on an RTX 3090 GPU. Extensive experiments demonstrate the superior storage efficiency, visual quality, and rendering speed of our method, consistently outperforming prior methods in both quantitative metrics and perceptual quality on real-world long dynamic scenes.

Flame Salmon

Full and Compact Dynamic Detail

The full representation preserves crisp flame structure and fast temporal changes, while the compact variant keeps the same scene motion with a smaller representation.

This view contrasts Ours with Ours Small on the same dynamic sequence, emphasizing how compression affects transient flame regions and stable background content.

Ours Ours Small

Interactive viewer

Onscreen rendering

Real-Time Onscreen Rendering

Live Playback at Interactive Rates

The renderer streams only the active temporal buckets needed for the current timestamp, keeping long dynamic scenes responsive during playback.

These two clips are shown as regular video panels rather than a slider, preserving their different aspect ratios while making the real-time output easy to compare side by side.

Overview

Training pipeline

To train long dynamic scenes efficiently, we introduce a Triple-buffer strategy with a GPU double buffer and a CPU bucket buffer. For each sampled timestamp, only the visible Gaussians are loaded into the GPU render buffer, while Gaussians unused for several steps are offloaded back to CPU buckets. This design reduces frequent CPU-GPU memory transfers. We also use a Dynamic-Aware Rotor Learning Rate (DARLR), which assigns smaller temporal-rotor learning rates to Gaussians with larger temporal extents, stabilizing static regions in long sequences.

Layering and compression

To represent long dynamic scenes efficiently, we organize 4D Gaussians into temporal layers and buckets based on their temporal extent τ and mean time μt. For each timestamp, only the current bucket and its neighboring buckets are loaded for rendering. The layered representation is then compressed using FCQ, layer-wise compression, and RCQ, which respectively factorize covariance parameters, adapt quantization to layer-specific distributions, and refine bucket-block codebooks with compact residuals.

BibTeX

@inproceedings{xu2026layered,
  title={Layered 4D-Rotor Gaussian Splatting: A Compressed Representation for Long Dynamic Scenes},
  author={Xu, Hanjie and Duan, Yuanxing and Dai, Qiyu and Li, Ge and Chen, Baoquan and Wang, He},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={18958--18967},
  year={2026}
}