Skeleton & skin avatar textures do not swap visibility properly

[jamesdeantv]

I've run into a wall

Let me do my best to explain my process. Hopefully, you will be able to see where I've errorred.

Open up blender and a create a model.

The model should contain a single mesh with two textures.

Here's a sample of the textures I used:

The first texture is a copy of the diffuse texture with the rgb red channel active only.

The second is a copy of the diffuse texture with all channels active.

In blender, you will have to add the red channel texture first, or it will be the active texture layer when you export.

This is not the result you want.

Once you've added both textures, switch from Blender's Blender Render to Cycles Render.

I've found that exporting in Cycles leads to a more performant model when switching between animations.

We'll cover that soon enough.

In Cycles, you want to activate the nodes editor.

I tend to use gLTF Metallic Roughnesss node instead of the base Diffuse node as it allows you to add PBR texturing.

I also add an image texture and texture mapping nodes to make sure the texture uvmaps properly.

Next, we're going to add a few animations.

For the sake of brevity, I wont go through my animation workflow. Just know that we'll be using a simple tpose and an idle animation for variety.

At this point, we are ready to export our model to gLTF.

We'll be using the Kupoman Blender gLTF Exporter. It is the only gltf exporter that properly exports the data that we need to build an avatar.

Name your file to export skeleton.gltf.

Once the export is complete, you should have a total of 4 files including a gltf, bin & two textures.

Spin up your code editor of choice and open glavatar-pipeline.js.

You will find it in the tools folder.

I found that there is a bit of clean up that needs to happen before you will be able to retrieve the avatar files with a desirable effect.

First, you want to comment the line:

gltf.extensionsUsed.push("gl_avatar");

This line adds a duplicate extension reference which will throw an error if included.

Next, just below:
// add bodyidlut

Add the following:

fs.writeFileSynce(outputFilename, JSON.stringify(gltf));

This will ensure that the skeleton file is actually exported.

Jump over to your terminal and run the following command to produce a skeleton avatar file:

node tools/glavatar-pipeline.js /path/to/your/skeleton/file/skeleton.gltf -S -o /path/to/your/skeleton/file/skeleton-avatar.gltf

Check that you have a total of 5 files including the newly created skeleton-avatar.gltf.

Up unto this point, we have only discussed the skeleton file. I will assume that you will be able to produce a clothing avatar using the method provided in the original instructions.

In the terminal run the following to produce a clothing avatar:

node tools/glavatar-pipeline.js /path/to/your/clothing/file/clothing.gltf -s -o /path/to/your/clothing/file/clothing-avatar.gltf

Note the lower case -s for skin versus -S for skeleton.

Open the skeleton-avatar.gltf file in the code editor and pretty-format the code. There is a bit of work that needs to be completed before you can produce an acceptable result.

First, within the skins object, update the main object value from 0 to 1.

Next, add an array with a key value of "visibilityId2Name. Inside this array add alias values to represent different parts of the avatar's mesh. I've simply copied the values from one of the included avatar gltf files within this repository.

Search for the materials array and include an object with a key value of extensions.gl_avatar.bodyIdLUT: 0.

Search for the mesh object that contains your main body. Mines is named Mesh. To this object add an object with a key value of extensions.gl_avatar.skin: [0, 1]. This array is a reference to the total number of skins used in the avatar including the clothes & the skin of the mesh body.

Now jump over to the clothing-avatar.gltf file and copy everything inside of the linkedSkeletons array and paste it into the skins array of the skeleton-avatar.gltf array. It is important to paste it below the first object in the array as this is how the skins array above is referenced.

Save both files and add references to them inside the GLTFAvatarSystem.js file within this repository. If you pay attention to the structure that is already setup inside that file, it shouldn't be too difficult to get it loaded properly.

At this point, npm run build the project and serve the files. I use a basic python http server by running:

python -m SimpleHTTPServer

When the files are served, you will get an invisible man effect. A partial outline of the avatar is visibile because of the shody photoshop worksmenship. But this visualises an issue.

So, I will explain the next few examples and what is happening so you can better understand my debugging process.

So, I figured that because the model was showing up invisible yet the clothing was not that it had to be an issue with the visibility layer of the extension. I attempted numerous pairs of hiding all the layers while keeping the first layer visible, vice versa, and running build. The results speak for themselves.

With the last attempt, I zeroed all the visibility layers and then added each inside the visibility-control gui folder one by one. As you can see, with each, layer clicked, the skin of the model becomes partially visible.

So, now we have arrived at the wall.

The avatar skeleton & skin textures do not swap visibility properly.

Your thoughts?

[jamesdeantv]

For your reference, attached are gist links to the skeleton and clothing avatar gltf files.

Skeleton

https://gist.github.com/jamesdeantv/0d0d9d0333ffa4ac4ed5c80dbe738120

Clothing

https://gist.github.com/jamesdeantv/0a963a760c7d8bde46a3b505c122ea4e

[shrekshao]

Sorry you run into this issue.

I'm not sure what is the reason... But let me make a guess

your body id texture should be a texture of which each pixel's red channel represent it's "body-id".

Here's an illustration. the number represents the red channel value at that region (the value is not correct here they are just an example).

Say that chest region is labeled as 20 in the image. Then the color of that region need to be RGB=(20, 0, 0)

[jamesdeantv]

Ah ha! Thanks. And, thank you for your assistance.