Rendering self-supervision

The network is never shown a 3D label. It renders its SDF and flow fields back into the camera and is scored against 2D cues. Press the image to follow one ray.

One ray

SDF sᵢweight wᵢrendered depth d

Depth d = Σ wᵢdᵢ: —
Σ wᵢ: —
Flow f = Σ wᵢfᵢ: —
Optical flow: —
Static flow: —
Dynamic flow |f_d|: —

Sharpness ξ24

αᵢ = max((Φ(sᵢ) − Φ(sᵢ₊₁)) / Φ(sᵢ), 0), with Φ(s) = 1 / (1 + e^−ξs). A larger ξ gathers the weight onto the surface.

Dynamic disentanglement

Threshold τ2.0 px

Movable (mask): —
Dynamic: —
M_all / M_d: —

Shares of the rays that hit a surface. Dynamic rays are rare, so their flow loss is scaled by M_all / M_d, capped (eq. 11).

Reported results

SemanticKITTI occupancy and depth (Table 1)
Method	RayIoU ↑	Abs Rel ↓
MonoScene 3D labels	37.19	—
SceneRF	37.58	0.146
OccNeRF	41.99	0.132
SelfOcc	36.27	0.152
Let Occ Flow	47.06	0.117

nuScenes occupancy flow (Table 3)
Method	RayIoU ↑	mAVE ↓
OccNet 3D labels	39.66	1.61
OccNeRF-C* C	21.61	1.53
Ours-C C	30.12	1.42
RenderOcc* L	36.67	1.63
OccNeRF-L* C+L	31.68	1.59
Ours-L C+L	40.48	1.45

Illustrative simulation. A procedural street at the paper's nuScenes volume and voxel size stands in for the network's predictions. The camera view is rendered from it the way the paper renders its fields (NeuS weights, eqs. 9, 10, 13, 14), with an exact flow prior, so every loss here would be zero. The method itself is on the project page and in the paper.