nerfview

nerfview is a minimal* web viewer for interactive NeRF rendering. It is largely inspired by nerfstudio's viewer, but with a standalone packaging and simple API to quickly integrate into your own research projects.

*The whole package contains two files and is less than 400 lines of code.

Installation

This package requires Python 3.8+.

For existing project, you can install it via pip:

pip install nerfview

To run our examples, you can clone this repository and then install it locally:

git clone https://github.com/nerfstudio-project/nerfview
# Install torch first.
pip install torch
# Then this repo and dependencies for running examples. Note that `gsplat`
# requires compilation and this will take some time for the first time.
pip install -e .
pip install -r examples/requirements.txt

Usage

nerfview is built on viser and provides a simple API for interactive viewing.

The canonical usage is as follows:

from typing import Tuple

import viser
import nerfview


def render_fn(
    camera_state: nerfview.CameraState, render_tab_state: nerfview.RenderTabState
) -> np.ndarray:
    # Parse camera state for camera-to-world matrix (c2w) and intrinsic (K) as
    # float64 numpy arrays.
    if render_tab_state.preview_render:
        width = render_tab_state.render_width
        height = render_tab_state.render_height
    else:
        width = render_tab_state.viewer_width
        height = render_tab_state.viewer_height

    c2w = camera_state.c2w
    K = camera_state.get_K([width, height])
    # Do your things and get an image as a uint8 numpy array.
    img = your_rendering_logic(...)
    return img

# Initialize a viser server and our viewer.
server = viser.ViserServer(verbose=False)
viewer = nerfview.Viewer(server=server, render_fn=render_fn, mode='rendering')

It will start a viser server and render the image from a camera that you can interact with.

Examples

We provide a few examples ranging from toy rendering to real-world NeRF training applications. Click on the dropdown to see more details. You can always ask for help message by the -h flag.

python examples/00_dummy_rendering.py

python examples/01_dummy_training.py

# Only need to run once the first time.
bash examples/assets/download_dragon_mesh.sh
CUDA_VISIBLE_DEVICES=0 python examples/02_mesh_rendering.py

# Only need to run once the first time.
bash examples/assets/download_gsplat_ckpt.sh
CUDA_VISIBLE_DEVICES=0 python examples/03_gsplat_rendering.py \
    --ckpt examples/assets/ckpt_6999_crop.pt

# Only need to run once the first time.
bash examples/assets/download_colmap_garden.sh
CUDA_VISIBLE_DEVICES=0 python examples/04_gsplat_training.py \
    --data_dir examples/assets/colmap_garden/ \
    --data_factor 8 \
    --result_dir results/garden/

Acknowledgement

This project cannot exist without the great work of nerfstudio and viser. We rely on nvdiffrast for the mesh example and gsplat for the 3DGS examples. We thank the authors for their great work and open-source spirit.