Jiabao Li, Yuqi Li*, Ciliang Sun, Chong Wang, and Jinhui Xiang
Spec-NeRF jointly optimizes the degradation parameters and achieves high-quality multi-spectral image reconstruction results at novel views, which only requires a low-cost camera (like a phone camera but in RAW mode) and several off-the-shelf color filters. We also provide real scenarios and synthetic datasets for related studies.
With recovered spectral information of the scenario, we can achieve several art effects, like
filter.mp4
ssf.mp4
ambient.mp4
We conduct our experiments based on TensoRF, please use the branch named public in our repository and feel free to report issues, we'd really appreciate it!
Download the two types of datasets (real senario and synthetic one) from google drive.
For the real dataset, please first separate the images into pose-based groups since all raw images are in the folder RAW by running python split_allimg2filterfixed_classify.py --scene_dir <your full path>/multi-view-MSI/filter19/xjhdesk --filter_num 20 --legacy 0 --angle_num 9 --img_ext tiff as administrator or root.
This will generate nine folders named pose?img, each contains the images filtered by all color filters, which are the symbolic links point to the ones in RAW folder, so remember do not delete the RAW folder.
Check what's the config file in start/start.bat file and execute ./start | start.bat
or try
python train.py --config ./configs/<your config file>.txt