Semantic Fusion for 3D Gaussian Splatting

Tarun Gangadhar Vadaparthi
📄 Write up (PDF) 💻 Code

Abstract

We extend 3D Gaussian Splatting (3DGS) with semantic awareness by integrating 2D segmentation from the Segment Anything Model (SAM). Novel views are rendered from a 3DGS scene and segmented; masks are projected into 3D and fused with multi-view voting to assign per-splat labels. The resulting semantic 3DGS enables object/region-level rendering and editing without requiring 3D ground-truth labels.

Visual Comparison

Original 3DGS Semantic-Fused 3DGS
Original
Semantic-Fused