AI News Feed
Market watch
Computer Vision

MarkTechPost Tutorial Builds Hierarchical NeRF With JAX3D for Volumetric Rendering and 3D Reconstruction

MarkTechPost tutorial builds a hierarchical NeRF with JAX, Flax, and Optax for rendering and 3D reconstruction.

The implementation first constructs a synthetic multi-view dataset from an analytic scene containing volumetric geometry and view-dependent radiance. It uses jax3d's sample_along_rays and volume_rendering functions to establish the forward rendering process. Camera poses are generated on a dome using golden-angle azimuths and monotone elevations. The rays_from_pose function returns origins and unit-length directions, so the depths returned by jax3d's sampler are true world-space distances.

The NeRF model includes positional encoding, skip connections, separate coarse and fine networks, and view-direction conditioning. Hierarchical importance sampling is handled through sample_piecewise_constant_pdf, which refines sampling in the fine network after the coarse pass. Training uses JAX JIT compilation, Adam optimization, exponential learning-rate decay, and gradient clipping.

The tutorial's default configuration uses 64 by 64 images, 24 training views, and 3 test views, with a camera radius of 3.2, a 40-degree field of view, near and far planes of 1.9 and 4.7, 256 ground-truth samples, 64 coarse samples, and 64 fine samples. The network has positional encoding degrees of 10 for position and 4 for direction, a width of 128, a depth of 6, and a skip connection at layer 3. Training uses batches of 2,048 rays for 2,500 steps, with learning rates decaying from 5e-4 to 5e-6, a chunk size of 4,096, and a grid resolution of 96.

Evaluation covers novel-view synthesis with PSNR, depth and opacity visualization, sampling diagnostics, 360-degree rendering, and marching-cubes geometry extraction. The code installs dependencies, clones the google-research/jax3d repository, and loads its volume_rendering.py file directly to avoid pulling in additional dependencies such as gin and tfds.

If no GPU is detected, the tutorial switches to a smaller CPU-friendly configuration, reducing image resolution to 40 by 40, training views to 14, steps to 400, ground-truth samples to 128, coarse and fine samples to 32 each, network width to 64, depth to 4, skip to layer 2, batch rays to 1,024, chunk size to 1,600, and grid resolution to 64. The tutorial notes that users can switch to a T4 GPU runtime for the full version.