How to test the model on my own photo?

[shoutOutYangJie]

No description provided.

[lewizo]

I have same question. And I used 3D_Photography(3d-photo-inpainting in git) to generate a file in mat format containing color and depth information. But the final video is severely distorted. I don't know how I understand the params in mat file, such as w/h/full_w/full_h.

[fmu2]

Thanks for your interest. Here is what the parameters mean in our provided mat files:

• uv: pixel coordinates
• z: per-pixel depth values from 3dphoto
• rgb: per-pixel color values
• h, w: size of input image to 3dphoto
• full_h, full_w: size of 3dphoto output after extrapolation (i.e., out-painting).
• fov: camera field of view in degrees. Default: 60

Suppose your input image is 640x480, and extrapolation_thickness in argument.yml is set to 40, then h=480, w=640, full_h=480+40x2=560, full_w=640+40x2=720. Regarding the LDI, the pixels at the top-left and bottom-right corners will have uv=(-40, -40) and (519, 679), respectively. FOV can be an educated guess or estimated using LeReS as we did in our paper. Hope this helps.

[shoutOutYangJie]

Why does LDI make UV move to negaticve coordinative like(-60, 60).

…

------------------ 原始邮件 ------------------ 发件人: "fmu2/3d_photo_stylization" ***@***.***>; 发送时间: 2022年7月19日(星期二) 晚上8:08 ***@***.***>; ***@***.******@***.***>; 主题: Re: [fmu2/3d_photo_stylization] How to test the model on my own photo? (Issue #3) Thanks for your interest. Here is what the parameters mean in our provided mat files: uv: pixel coordinates z: per-pixel depth values from 3dphoto rgb: per-pixel color values h, w: size of input image to 3dphoto full_h, full_w: size of 3dphoto output after extrapolation (i.e., out-painting). fov: camera field of view in degrees. Default: 60 Suppose your input image is 640x480, and extrapolation_thickness in argument.yml is set to 40, then h=480, w=640, full_h=480+40x2=560, full_w=640+40x2=720. Regarding the LDI, the pixels at the top-left and bottom-right corners will have uv=(-40, -40) and (519, 679), respectively. FOV can be an educated guess or estimated using LeReS as we did in our paper. Hope this helps. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

[fmu2]

The image center remains unchanged. We simply extend pixels beyond the borders to account for extrapolation. You can think of it as padding the input image with extrapolated pixels.

[lewizo]

Thanks a lot for your reply. It's helpful. But the final video didn't achieve the best effect. I think I have missed some content. In 3DP, I can get the color and position of points after inpainting. And I put them as "uvz" and "rgb". But there are extra parameters named "faces". I don't know how to use the "faces". Are they necessary?

[cellautomaton]

Can you post the command you used to generate the file in mat format? It seems it can only do npy and png formats.

[zcdliuwei]

I have same question. And I used 3D_Photography(3d-photo-inpainting in git) to generate a file in mat format containing color and depth information. But the final video is severely distorted. I don't know how I understand the params in mat file, such as w/h/full_w/full_h.

May I ask how to obtain the mat file？
I used 3D_Photography(3d-photo-inpainting in git)， but I didn't see that function that can convert the depth and ply files into mat files