LSNIF: Locally-Subdivided Neural Intersection Function
0 likes128 views
Neural representations have shown the potential to accelerate ray casting in a conventional ray-tracing-based rendering pipeline. We introduce a novel approach called Locally-Subdivided Neural Intersection Function (LSNIF) that replaces bottom-level BVHs used as traditional geometric representations with a neural network. Our method introduces a sparse hash grid encoding scheme incorporating geometry voxelization, a scene-agnostic training data collection, and a tailored loss function. It enables the network to output not only visibility but also hit-point information and material indices. LSNIF can be trained offline for a single object, allowing us to use LSNIF as a replacement for its corresponding BVH. With these designs, the network can handle hit-point queries from any arbitrary viewpoint, supporting all types of rays in the rendering pipeline. We demonstrate that LSNIF can render a variety of scenes, including real-world scenes designed for other path tracers, while achieving a memory footprint reduction of up to 106.2x compared to a compressed BVH. https://arxiv.org/abs/2504.21627
![3 |LSNIF: Locally-Subdivided Neural Intersection Function
1. Neural Intersection Function [Fujieda et al. 2023]
Replaces bottom-level BVHs with NNs
Fast and accurate for shadow-ray queries
Needs Online training (viewpoint and lighting)
Primarily limited to shadow-ray usage
2. N-BVH [Weier et al. 2024]
Offline training
Support any types of ray queries
Overfitted to each scene
No support for dynamic contents
Previous Works with Neural Networks](https://image.slidesharecdn.com/202505i3dlsnif-250603101542-2f1ff478/85/LSNIF-Locally-Subdivided-Neural-Intersection-Function-3-320.jpg)
![30 |LSNIF: Locally-Subdivided Neural Intersection Function
Memory Footprint Breakdown
• 1.56 MB / LSNIF
• 4 KB: 32^3 voxels
• 62 KB: MLP with two 128-wide hidden layers
• 1,536 KB: sparse hash grid features with 2 levels of 64^3 and 128^3
• Compressed BVH [Ylitie et al. 2017]
• 8-wide BVH
• 80 bytes / node](https://image.slidesharecdn.com/202505i3dlsnif-250603101542-2f1ff478/85/LSNIF-Locally-Subdivided-Neural-Intersection-Function-30-320.jpg)
LSNIF: Locally-Subdivided Neural Intersection Function
- 1. 1 |LSNIF: Locally-Subdivided Neural Intersection Function LSNIF: Locally-Subdivided Neural Intersection Function 2025/05/09 Shin Fujieda Chih-Chen Kao Takahiro Harada ACM SIGGRAPH SYMPOSIUM ON INTERACTIVE 3D GRAPHICS AND GAMES
- 2. 2 |LSNIF: Locally-Subdivided Neural Intersection Function Introduction • Conventional Ray Tracing Pipeline - Ray queries • Find the ray-geometry intersection • Accelerated with a bounding volume hierarchy (BVH) • Typically, a two-level BVH • BVH traversal exhibits irregular execution patterns • Branch divergence & unpredictable memory access • Hardware cannot fully mitigate high cost of the workload Bottom-level Top-level Two-level BVH
- 3. 3 |LSNIF: Locally-Subdivided Neural Intersection Function 1. Neural Intersection Function [Fujieda et al. 2023] Replaces bottom-level BVHs with NNs Fast and accurate for shadow-ray queries Needs Online training (viewpoint and lighting) Primarily limited to shadow-ray usage 2. N-BVH [Weier et al. 2024] Offline training Support any types of ray queries Overfitted to each scene No support for dynamic contents Previous Works with Neural Networks
- 4. 4 |LSNIF: Locally-Subdivided Neural Intersection Function Locally-Subdivided Neural Intersection Function • Extend NIF to address the limitations of previous works Offline training for each object, supporting instanced objects Support scene changes (e.g., camera positions, lighting conditions, rigid transformations) Ray queries for any types of rays other than primary rays Our hybrid path training Ray tracing with BVHs and NNs Complex geometries are replaced with NNs Conventional path tracing Ray tracing with BVHs
- 5. 5 |LSNIF: Locally-Subdivided Neural Intersection Function Locally-Subdivided Neural Intersection Function • Overview of the methodology • Challenge: each input feature must uniquely represent a specific ray Occlusion Hit distance Normal Albedo Material index Sparse hash grid
- 6. 6 |LSNIF: Locally-Subdivided Neural Intersection Function Unique Representation • Rays with the same direction but originating from different points
- 7. 7 |LSNIF: Locally-Subdivided Neural Intersection Function Unique Representation • Rays with the same direction but originating from different points • In NIF: • NN( Grid2D( Position ) + Grid2D( Direction ) ) • Effective only when the viewpoint is fixed Direction Grid Position Grid Different feature Same feature +
- 8. 8 |LSNIF: Locally-Subdivided Neural Intersection Function Unique Representation • Rays with the same direction but originating from different points • In NIF: • NN( Grid2D( Position ) + Grid2D( Direction ) ) • Effective only when the viewpoint is fixed • One possible solution: • NN( Grid4D( Position, Direction ) ) • Computational expensive 4D Grid Distinct feature
- 9. 9 |LSNIF: Locally-Subdivided Neural Intersection Function Unique Representation • Rays with the same direction but originating from different points • In NIF: • NN( Grid2D( Position ) + Grid2D( Direction ) ) • Effective only when the viewpoint is fixed • One possible solution: • NN( Grid4D( Position, Direction ) ) • Computational expensive • LSNIF solution: • Input more data to the MLP • Voxelize the AABB to use intersections points as inputs Distinct feature
- 10. 10 |LSNIF: Locally-Subdivided Neural Intersection Function Locally-Subdivided Neural Intersection Function Occlusion Hit distance Normal Albedo Material index Sparse hash grid
- 11. 11 |LSNIF: Locally-Subdivided Neural Intersection Function Locally-Subdivided Neural Intersection Function 1. Compute the intersection points of a ray with the AABB 2. Calculate the hit points of the ray on the surface of voxels • DDA algorithm based on the intersection points on the AABB Occlusion Hit distance Normal Albedo Material index Sparse hash grid
- 12. 12 |LSNIF: Locally-Subdivided Neural Intersection Function Locally-Subdivided Neural Intersection Function 3. Encode the intersections with a multi-resolution sparse hash grid 4. Get the concatenated feature vector 5. Feed the feature vector to an MLP (Multilayer Perceptron) to predict geometric properties Occlusion Hit distance Normal Albedo Material index Sparse hash grid Feature vectors are located only on the voxel boundary Concatenated feature vector MLP
- 13. 13 |LSNIF: Locally-Subdivided Neural Intersection Function Key Benefits of Voxelization and Encoding in LSNIF 1. Voxelization • Distinguish each input that carry different characteristics • No explicit voxel occupancy checks compared to sampling within regular intervals 2. Multi-resolution Sparse Hash Grid Encoding Sparse hash grid
- 14. 14 |LSNIF: Locally-Subdivided Neural Intersection Function Key Benefits of Voxelization and Encoding in LSNIF 1. Voxelization • Distinguish each input that carry different characteristics • No explicit voxel occupancy checks compared to sampling within regular intervals 2. Multi-resolution Sparse Hash Grid Encoding • Reduce memory footprint: all features are located on the voxel boundaries • Reduce number of queries: improve inference performance 2D “Standard” Grid Query 4 times for bilinear interpolation 2D Sparse Grid Query 2 times for linear interpolation Feature vector on the voxel boundary Feature vector not on the voxel boundary vs. Feature vector : ignored
- 15. 15 |LSNIF: Locally-Subdivided Neural Intersection Function The Pipeline of LSNIF Primary rays • LSNIF leads to large visual errors • Only for non-primary rays • Use rasterization • Store vertex and face information • No need for BVH (1) Prepare G-Buffers
- 16. 16 |LSNIF: Locally-Subdivided Neural Intersection Function The Pipeline of LSNIF (1) Prepare G-Buffers (2) Ray trace using BVH (3) Ray trace using LSNIF Non-primary rays • Simple geometries • Two-level BVH • Shallow and small BVH • Complex Geometries • LSNIF • DDA + Sparse grid + Inference • Dedicated top-level BVH • Two-phase approach 1. Collect hits b/w ray and LSNIF objects 2. Execute LSNIF
- 17. 17 |LSNIF: Locally-Subdivided Neural Intersection Function The Pipeline of LSNIF (1) Prepare G-Buffers (2) Ray trace using BVH (3) Ray trace using LSNIF (4) Combine and shade
- 18. 18 |LSNIF: Locally-Subdivided Neural Intersection Function Comparison between BVH and LSNIF vertex index vertex index BVH BVH vertex index vertex index BVH LSNIF Up to 106x reduction This is not needed if we use LSNIF for primary visibility too
- 19. 19 |LSNIF: Locally-Subdivided Neural Intersection Function Results
- 20. 20 |LSNIF: Locally-Subdivided Neural Intersection Function Reference (PT)
- 21. 21 |LSNIF: Locally-Subdivided Neural Intersection Function Rendering without LSNIF
- 22. 22 |LSNIF: Locally-Subdivided Neural Intersection Function Rendering with LSNIF
- 23. 23 |LSNIF: Locally-Subdivided Neural Intersection Function Reference (PT)
- 24. 24 |LSNIF: Locally-Subdivided Neural Intersection Function Error
- 25. 25 |LSNIF: Locally-Subdivided Neural Intersection Function Reference (PT)
- 26. 26 |LSNIF: Locally-Subdivided Neural Intersection Function Rendering without LSNIF
- 27. 27 |LSNIF: Locally-Subdivided Neural Intersection Function Rendering with LSNIF
- 28. 28 |LSNIF: Locally-Subdivided Neural Intersection Function Reference (PT)
- 29. 29 |LSNIF: Locally-Subdivided Neural Intersection Function Error
- 30. 30 |LSNIF: Locally-Subdivided Neural Intersection Function Memory Footprint Breakdown • 1.56 MB / LSNIF • 4 KB: 32^3 voxels • 62 KB: MLP with two 128-wide hidden layers • 1,536 KB: sparse hash grid features with 2 levels of 64^3 and 128^3 • Compressed BVH [Ylitie et al. 2017] • 8-wide BVH • 80 bytes / node
- 31. 31 |LSNIF: Locally-Subdivided Neural Intersection Function Compression Ratio & Performance Memory footprint comparison Rendering time comparison
- 32. 32 |LSNIF: Locally-Subdivided Neural Intersection Function Limited Scene Change Support
- 33. 33 |LSNIF: Locally-Subdivided Neural Intersection Function Limitations and Future Extensions • Rasterization for primary rays • Retain vertex and face information of geometries • Higher quality of reconstructed geometries • Deformation support • Require additional training • Texture support • Errors in texture-coordinate prediction directly result in visible artifacts • LOD support • Simple solution is to store multiple LSNIFs for different LODs with different voxel resolutions
- 34. 34 |LSNIF: Locally-Subdivided Neural Intersection Function LSNIF Summary • Replace bottom-level BVHs with NNs • Offline training for each object • Support limited scene changes • Transforms of objects, lights, camera • Hybrid rendering pipeline uses • Rasterization • Ray tracing • Machine learning • 106x memory reduction compared to compressed BVH • 1.56 MB / LSNIF • Outperform BVH ray tracing for complex scenes Rendering with LSNIF Rendering w/o LSNIF
- 35. Thank you!