preprocessing of our custom dataset

[vipinsdk]

Hello,
I am really impressed by this work and want to test this work on our multi-view captured dataset. I wanted to know if there is a preprocessing pipeline available that convert my dataset to the required format to train this model.

I basically want to know how to generate the camera calibration metrics in the required format as I have the OPENCV yml file format for all the cameras together. And in addition to that camera calibration. I wanted to ask if we can train the model entirely on the color images.

Thanks in advance.

[TimoBolkart]

Hello,
dataset processing: unfortunately we were unable to release the code of the processing pipeline to register the dataset to the FLAME mesh topology.
camera parametes: you would need to convert them to camera extrinsics (3x4 matrix with camera rotation and translation) and intrinsics (2x3 matrix with focal length, principal point). The radial distortions can be ignored and set to zero, if the input images are undistorted.
color images: yes, as shown in Table 2 ("Ours color images input") one can train the model on color images. The main reason for its slightly worse performance was our scanner system, where color images are slightly offset in time to the 3D scans used as supervision and the sparser set of cameras (i.e., the system has twice as many gray scale images than color images).

[vipinsdk]

Hello,

Thanks for the reply. I will do the suggested steps to train the model with my data.

Best Regards,
Vippin