Import Custom 3D Model

[lchakkei]

Is it possible to import a custom 3D model into the scene?

[philipturner]

https://github.com/philipturner/ar-multipendulum uses custom 3D model functionality to render a simulation that changes in real-time.

You can make custom GPU shaders, which render a 3D model. I designed the framework to be this way, but cut the tutorial series short. I didn't think anybody would have enough advanced skills to do this.

I can guide you through the process and provide some example Metal shaders, if you wish to make a proof-of-concept. What type of data is in your 3D model? Just a wireframe? Colors? Depth maps?

[lchakkei]

I've recently downloaded your app from the App Store and I must say, it's truly impressive.

I'd like to import the Cartonnage Mask USDZ from Sketchfab into the project. Here’s the link to the model: Cartonnage Mask.

[philipturner]

You would start by creating a USDZ renderer in Metal or searching for an open-source one. This part has no relation to AR and can be debugged on a desktop computer. Test the renderer and upload an image of it working on your computer.

Next, copy the Metal code into ARHeadsetKit. I can provide further guidance once you reach this point.

[lchakkei]

After following some tutorials, I successfully render a usdz model in Metal.

[philipturner]

Awesome! Could you show me the source code? I’ll need to inspect the Metal shaders to figure out how to adapt them for ARHK.

[lchakkei]

https://github.com/lchakkei/metal-render.git

I’ve pushed the project to GitHub. As a beginner in Metal, it’s quite challenging for me to comprehend all the code.

Hope you can integrate it into your project.

[philipturner]

This code is extremely simple. You just have vertices and a 2D texture image. Each vertex specifies a 3D spatial position and 2D texture coordinates. The biggest piece of work is the CPU-side code for deserializing USDZ and generating the textures. That part has little relation to ARHK.

I would begin by following the very last tutorial in the tutorial series. If that is difficult to understand, try some of the tutorials before it. Then, try modifying the shader to render a tetrahedron with custom spatial coordinates.

Hope you can integrate it into your project.

This would belong in your client app. USDZ is out of scope for the main codebase, which just covers basic low-level functionality. It was designed to defer stuff like USDZ support to the library user's app.