3D Gaussian Splatting (3DGS) has shown convincing performance in rendering speed and fidelity, yet the generation of Gaussian Splatting remains a challenge due to its discreteness and unstructured nature. In this work, we propose DiffGS, a general Gaussian generator based on latent diffusion models. DiffGS is a powerful and efficient 3D generative model which is capable of generating Gaussian primitives at arbitrary numbers for high-fidelity rendering with rasterization. The key insight is to represent Gaussian Splatting in a disentangled manner via three novel functions to model Gaussian probabilities, colors and transforms. Through the novel disentanglement of 3DGS, we represent the discrete and unstructured 3DGS with continuous Gaussian Splatting functions, where we then train a latent diffusion model with the target of generating these Gaussian Splatting functions both unconditionally and conditionally. Meanwhile, we introduce a discretization algorithm to extract Gaussians at arbitrary numbers from the generated functions via octree-guided sampling and optimization. We explore DiffGS for various tasks, including unconditional generation, conditional generation from text, image, and partial 3DGS, as well as Point-to-Gaussian generation. We believe that DiffGS provides a new direction for flexibly modeling and generating Gaussian Splatting.
三维高斯喷涂 (3D Gaussian Splatting, 3DGS) 在渲染速度和保真度方面表现出色，但由于其离散性和非结构化特性，高斯喷涂的生成仍然是一个挑战。本文提出了一种基于潜在扩散模型的通用高斯生成器，称为 DiffGS。DiffGS 是一种功能强大且高效的三维生成模型，能够生成任意数量的高斯基元，以实现基于光栅化的高保真渲染。 核心思想在于通过三个新颖的函数以解耦的方式表示高斯喷涂，分别建模高斯概率、颜色和变换。通过对 3DGS 的这种新颖解耦，我们使用连续高斯喷涂函数来表示离散和非结构化的 3DGS，并训练潜在扩散模型，以生成无条件和有条件的高斯喷涂函数。此外，我们引入了一种离散化算法，通过八叉树引导的采样和优化，从生成的函数中提取任意数量的高斯基元。 我们在多个任务中探索了 DiffGS，包括无条件生成、基于文本、图像和部分 3DGS 的条件生成，以及从点到高斯的生成。我们相信，DiffGS 为灵活建模和生成高斯喷涂开辟了新的方向。