Our code is based on 3D Gaussian Splatting.
We place a specific emphasis on two key objectives: reducing the number of Gaussian points without sacrificing performance and compressing the Gaussian attributes. To this end, we propose a learnable mask strategy that significantly reduces the number of Gaussians while preserving high performance. In addition, we propose a compact but effective representation of view-dependent color by employing a grid-based neural field rather than relying on spherical harmonics. Finally, we learn codebooks to compactly represent the geometric attributes of Gaussian by vector quantization.
We additionally implement straightforward post-processing techniques on the model attributes: 1) Applying 8-bit min-max quantization to opacity and hash grid parameters. 2) Pruning hash grid parameters with values below 0.1. 3) Applying Huffman encoding on the quantized opacity and hash parameters, and R-VQ indices.
As a result, our model is further downsized by over 40 % regardless of dataset, achieving more than 25x compression from 3DGS, while maintaining high performance.
For installation:
git clone https://github.com/maincold2/Compact-3DGS.git --recursive
conda env create --file environment.yml
conda activate c3dgs
We used Mip-NeRF 360, Tanks & Temples, Deep Blending, and NeRF synthetic datasets.
python train.py -s <path to COLMAP> --eval
Applying post-processings for compression.
More Command Line Arguments for train.py
Weight of masking loss to control ma the number of Gaussians masking control factor, 0.01 by default
Learning rate of masking parameter, 0.01 by default
Learning rate for the neural field, 0.01 by default
Step schedule for training the neural field, [5000, 15000, 25000] by default
Maximum hashmap size (log) of the neural field, 19 by default
Codebook size in each R-VQ stage, 64 by default
The number of R-VQ stages, 6 by default
Refer to other arguments of 3DGS.
Some different hyper-parameters are required for synthetic scenes.
python train.py -s <path to NeRF Synthetic dataset> --eval --max_hashmap 16 --lambda_mask 4e-3 --mask_lr 1e-3 --net_lr 1e-3 --net_lr_step 25000
python render.py -m <path to trained model> --max_hashmap <max hash size of the model>
python metrics.py -m <path to trained model>
The original SIBR interactive viewer of 3DGS can not support neural fields for view-dependent color. We would like to support and update this shortly if possible.
Currently, to use the viewer, you have two options: either bypass the neural field for view-dependent color by only applying masking and the geometry codebook, or train neural fields to represent spherical harmonics without inputting view direction (slightly lower performance). After this, you can save the output in a PLY format, similar to 3DGS.
@article{lee2023compact,
title={Compact 3D Gaussian Representation for Radiance Field},
author={Lee, Joo Chan and Rho, Daniel and Sun, Xiangyu and Ko, Jong Hwan and Park, Eunbyung},
journal={arXiv preprint arXiv:2311.13681},
year={2023}
}