Simone Rossi
← All publications

DrivingVoxels: Compositional Sparse Voxel Rasterization for Dynamic Driving Scene Reconstruction

  • Computer Vision

DrivingVoxels reconstructs dynamic driving scenes with separate sparse voxel octrees for the background and moving objects, composed in a single rendering pass.

Paper BMVC 2026

Tania Aguirre1, Luis Roldão2, Moussab Bennehar2, Nathan Piasco2, Dzmitry Tsishkou2, Simone Rossi3, Pietro Michiardi3

  1. Huawei Paris Research Center & EURECOM
  2. Huawei Paris Research Center
  3. EURECOM
PublicationPDF

Reconstructing a driving scene requires consistent geometry for both the stationary environment and independently moving actors. We introduce DrivingVoxels, an explicit sparse voxel representation with a separate octree for each rigid object and one for the background. A single rendering pass composes their contributions in depth order. On ten PandaSet sequences, this construction improves geometric reconstruction over the reproduced Gaussian-splatting baselines while reducing training time, with a trade-off in perceptual detail.

The work is accepted at BMVC 2026.

StreetGS and DrivingVoxels scene reconstructions with geometry metrics, training times, and separate static and dynamic components.
The illustrated scene compares reconstruction and geometry, then separates the background from moving actors. DrivingVoxels reduces floating artifacts in this example; the displayed timing and geometry values describe this scene.

Motion in separate coordinate frames

A static voxel grid fixes its spatial partition, making independently moving objects difficult to represent. We retain sparse voxel rasterization inside each octree and solve composition between octrees along camera rays.

The background octree Obg\mathcal{O}_{bg} lives in world coordinates. Object ii has a canonical octree Oi\mathcal{O}_i and a tracked rigid transformation Tti∈SE(3)T_t^i\in SE(3) at time tt. Each leaf voxel stores spherical-harmonic appearance coefficients and eight corner densities, interpolated within the voxel. A spherical environment map represents the sky.

DrivingVoxels pipeline showing camera rays, background and object octrees, sorted ray segments, and RGB, depth, and normal outputs.
Bounding-box intersections divide each ray into ordered background and object intervals. Transforming object intervals into local coordinates allows independent octrees to contribute to one occlusion-consistent rendering pass.

LiDAR-based geometry initialization

We prune background voxels outside the training-camera frustums or below the height threshold, then subdivide near LiDAR points. This allocates resolution around observed surfaces while retaining coarse empty space. Initial raw corner density depends on the nearest LiDAR distance dd:

vgeoinit=vfar+(vclose−vfar)exp⁡ ⁣(−2d2s2).\mathbf{v}_{\mathrm{geo}}^{\mathrm{init}} =\mathbf{v}_{\mathrm{far}} +(\mathbf{v}_{\mathrm{close}}-\mathbf{v}_{\mathrm{far}}) \exp\!\left(-\frac{2d^2}{s^2}\right).

Here ss is the minimum voxel size, vclose=−2\mathbf{v}_{\mathrm{close}}=-2, and vfar=−10\mathbf{v}_{\mathrm{far}}=-10. The smooth prior seeds density near measurements while leaving densities trainable. Object octrees instead start from uniform grids bounded by annotated boxes, avoiding LiDAR initialization errors from temporal misalignment.

Three initialization stages showing camera-frustum pruning, LiDAR-guided subdivision around surfaces, and distance-based corner densities.
Panels (a–c) show the visible scene volume, finer cells near LiDAR surfaces, and the density prior. Concentrating initial voxels around measured geometry gives optimization a structured starting point.

Ray accumulation across multiple octrees

For a ray r(τ)=o+τd\mathbf{r}(\tau)=\mathbf{o}+\tau\mathbf{d} with origin o\mathbf{o}, direction d\mathbf{d}, and distance τ\tau, we sort object-box entry and exit distances. Background intervals use world coordinates; object intervals use (Tti)−1r(T_t^i)^{-1}\mathbf{r}. The renderer carries accumulated color, depth, normals, and residual transmittance between intervals. After voxel integration, sky compositing gives

C=Cvoxels+TfinalCsky.\mathbf{C}=\mathbf{C}_{\mathrm{voxels}} +T_{\mathrm{final}}\mathbf{C}_{\mathrm{sky}}.

TfinalT_{\mathrm{final}} is the fraction of light transmitted through all visited volumes. It weights the sky contribution, preserving occlusion across coordinate boundaries.

The joint training objective combines image and geometric evidence:

L=λcolorLcolor+λdepthLdepth+λnormalLnormal+λskyLsky.\begin{aligned} \mathcal{L}={}&\lambda_{\mathrm{color}}\mathcal{L}_{\mathrm{color}} +\lambda_{\mathrm{depth}}\mathcal{L}_{\mathrm{depth}}\\ &+\lambda_{\mathrm{normal}}\mathcal{L}_{\mathrm{normal}} +\lambda_{\mathrm{sky}}\mathcal{L}_{\mathrm{sky}}. \end{aligned}

The λ\lambda coefficients weight color reconstruction, sparse LiDAR depth, dense pseudo-normal alignment, and a sky-transmittance penalty. Background subdivision stops when SSIM improvement plateaus, followed by pruning and refinement.

Image quality and geometric reconstruction

The evaluation uses the front camera of ten PandaSet sequences, with 40 training and 40 held-out frames per sequence. Camera poses and object tracks are fixed, and actors are modeled as rigid. Calibration and tracking errors affect reconstruction accuracy. Geometry is compared against accumulated LiDAR. The adaptive configuration achieves 0.242 m Chamfer distance and 65.6% F1 at 0.1 m, versus StreetGS's 0.605 m and 54.8%. Mean training time is 0.41 hours versus 1.80 hours on RTX 3090-equivalent hardware; StreetGS renders faster, at 57 versus 26 FPS.

Five driving scenes comparing ground truth, OmniRe, StreetGS, SaLF, and DrivingVoxels, with unavailable SaLF examples marked.
Comparison with image-based and geometric baselines. DrivingVoxels preserves coherent geometry in difficult regions, while StreetGS can retain sharper foreground appearance. SaLF image examples come from its published evaluation; its unavailable code prevents a reproduced geometry comparison.

On five-sequence ablations at 20,000 iterations, removing LiDAR subdivision worsens perceptual and geometric metrics. Binary density initialization performs similarly to the smooth prior. Removing depth supervision increases Chamfer distance from 0.231 to 0.644 m; removing normal supervision reduces F1 from 66.3% to 49.0%.

Depth-map ablations comparing omitted supervision terms and the full DrivingVoxels configuration.
Depth reconstructions show complementary effects of geometric supervision and sky modeling. The first two panels carry the same normal-loss label; the ablation values above quantify the separate contributions of depth and normal supervision.
LPIPS against training time for DrivingVoxels with one, two, and three million voxels and StreetGS.
Larger voxel budgets improve LPIPS with a modest increase in training time in this comparison. The plot exposes the quality–cost choice available through an explicit voxel budget.

Editing reconstructed actors

Separate octrees support actor removal, pose changes, and replacement after reconstruction.

A reconstructed driving scene followed by actor removal, pose modification, and vehicle replacement.
Editing an object's octree or transformation changes the rendered scene without retraining, demonstrating a practical consequence of explicit scene decomposition.