Colmap to Nerfstudio

[frasiolas]

I extract cameras.bin and images.bin from colmap. I use from colmap_utils colmap_to_json to get the transforms.json file and then i run the ns train command. The problem is that the images are not in the same position and orientation that they are in the colmap GUI. Do you know why this is happening?

[kerrj]

This might be because of frame conventions, see https://docs.nerf.studio/quickstart/data_conventions.html
You can also use ns-process-data directly to call colmap and have it generate the transforms for you

[jb-ye]

You can try to set this variable to True
https://github.com/nerfstudio-project/nerfstudio/blob/main/nerfstudio/process_data/colmap_utils.py#L394
or utilizing json["applied_transform"] in the transform.json

[frasiolas]

I think i found it if you want to keep the same orientation you call colmap dataparser instead of nerfstudio dataparser in ns-train and put the arguments [--method none --center_method none]

[jb-ye]

I think you need to call --orientation_method=none --center_method=none --assume_colmap_world_coordinate_convention=False in colmap parser after this change ( #2793 ). This is to keep the exactly same world coordinate as input colmap project up to a scale.

[Nas-Azzam]

I extract cameras.bin and images.bin from colmap. I use from colmap_utils colmap_to_json to get the transforms.json file and then i run the ns train command. The problem is that the images are not in the same position and orientation that they are in the colmap GUI. Do you know why this is happening?

May be you can try this file
colmap2nerf.zip

[tersite1]

Each NeRF model's are has different form when processing image. So it's natural.
I suggest you try using this.

https://github.com/NVlabs/instant-ngp/blob/master/scripts/colmap2nerf.py

than

python colmap2nerf.py

[LuckyOne09]

I think you need to call --orientation_method=none --center_method=none --assume_colmap_world_coordinate_convention=False in colmap parser after this change ( #2793 ). This is to keep the exactly same world coordinate as input colmap project up to a scale.

@jb-ye Thank you very much for your guidance. But when I follow like this ns-train splatfacto --data {data_dir} --viewer.quit-on-train-completion False --max-num-iterations 10000 --pipeline.model.sh-degree 0 colmap --colmap_path sparse/0 --downscale_factor=1 --orientation_method=none --center_method=none --assume_colmap_world_coordinate_convention=False --auto-scale-poses=False and after I run ns-export, I get a splat.py whose many vertices are NaN. I think it is weird since I have nice results in web viewer, could you give me some hints for debugging?

[LuckyOne09]

I think you need to call --orientation_method=none --center_method=none --assume_colmap_world_coordinate_convention=False in colmap parser after this change ( #2793 ). This is to keep the exactly same world coordinate as input colmap project up to a scale.

@jb-ye Thank you very much for your guidance. But when I follow like this ns-train splatfacto --data {data_dir} --viewer.quit-on-train-completion False --max-num-iterations 10000 --pipeline.model.sh-degree 0 colmap --colmap_path sparse/0 --downscale_factor=1 --orientation_method=none --center_method=none --assume_colmap_world_coordinate_convention=False --auto-scale-poses=False and after I run ns-export, I get a splat.py whose many vertices are NaN. I think it is weird since I have nice results in web viewer, could you give me some hints for debugging?

Update: I found that if I set sh-degree back to 3, all things will be fine. I don't know why but it just happens, maybe it is a potential bug