Add base color texture node to tileset materials

[lilleyse]

Follow up to #394 and builds upon #391

Adds a new MDL node cesium_base_color_texture that retrieves the base color texture of the tile. This can be used to create more complex materials than before like clipping out regions of a tileset. Here Cesium World Terrain is being clipped by the Cesium logo with clamp-to-edge sampling.

tileset-materials-complex.zip

(Make sure to set app.useFabricSceneDelegate = false and enable fabric when the popup appears, otherwise it may not recognize the custom material. This is a known issue with USD material population in Fabric.)

How it works:

If a material doesn't have any cesium_base_color_texture nodes it works the same as before: the Fabric prims reference the existing material and that's it.

If a material has cesium_base_color_texture nodes we duplicate all the prims in the material network, reconnect the prims, and add a prim to read from the base color texture. Since materials can't reference texture paths via primvars we can't share the material like before, and instead need to duplicate the material X number of times depending on the size of the material pool. This also means you can't tweak materials dynamically unless you're able to watch for changes and update all materials in the pool.

Future work:

• Figure out how to hide cesium_base_color_texture inputs
• Update materials in the pool when the main material changes
• Delete material pools that are no longer in use. This is especially important now that the tileset material is part of the material definition.
• Add other cesium nodes like cesium_base_color_factor, cesium_base_color_texture_alpha, cesium_emissive_factor, etc. We can expose whatever material values we want, I just didn't want to clutter this PR.
• Add MDL node to lookup color from an imagery layer. Support for multiple raster overlays #84 would need to be implemented first. This would let us do more complex things like water materials using the CWT water mask.

[weegeekps]

(Make sure to set app.useFabricSceneDelegate = false and enable fabric when the popup appears, otherwise it may not recognize the custom material. This is a known issue with USD material population in Fabric.)

Should we create a temporary app that is derived from our test app that sets this to false while this is broken? It seems like it would make launching from our editors easier, but maybe it's overkill?

[r-veenstra]

It appears that if I try to switch between materials, or remove the material entirely, the tileset gets stuck with the first custom material I applied. It appears to only happen to the final level of detail that gets loaded, as the tile reloads you can see the new material coming in to the coarser detail levels, but it still ends up with the old material. Best shown in this capture.

Screen.sharing.-.2023-08-02.10_02_15.AM.mp4

[lilleyse]

@r-veenstra I fixed the bug with switching materials. It was a bug in UsdNotificationHandler. When a material is added to a tileset for the first time it's assigned the MaterialBindingAPI schema which triggers a resync, which triggers a second OmniTileset to be added. The fix was to not add another OmniTileset when that happens. 0dcb35e

[lilleyse]

Should we create a temporary app that is derived from our test app that sets this to false while this is broken? It seems like it would make launching from our editors easier, but maybe it's overkill?

Or we could just disable fabric by default. Though that would mean having to click yes on the popup every time...

[lilleyse]

I still need to fix the texture lookup functions to support y-up and z-up (probably via a uniform), and then this should be ready for review.

Edit: I may tackle "Update materials in the pool when the main material changes" in a follow up PR

[weegeekps]

Should we create a temporary app that is derived from our test app that sets this to false while this is broken? It seems like it would make launching from our editors easier, but maybe it's overkill?

Or we could just disable fabric by default. Though that would mean having to click yes on the popup every time...

I suspect this would break unit testing.

[lilleyse]

I added a uniform called up_axis for selecting between y-up and z-up.

[lilleyse]

A couple changes to the mdl. Existing .usda's will need to be updated.

• Renamed min_world_xy to min_world
• Renamed max_world_xy to max_world
• Renamed lookup_world_texture_float4 to cesium_lookup_world_texture_float4
• Renamed lookup_world_texture_float3 to cesium_lookup_world_texture_float3
• Renamed lookup_world_texture_color to cesium_lookup_world_texture_color

[lilleyse]

@weegeekps this is ready for review now. For testing you can check out the .zip posted in the PR description.

[lilleyse]

I converted this to a draft. I'm experimenting with a way for tiles to share a single material, and depending on how that goes the approach here may change.

[lilleyse]

This is ready for review again. The basic approach is still the same, but I changed what MDL functions are exposed.

• cesium_base_color_texture_float4: returns the base color texture as a float4
• cesium_imagery_layer_float4: returns the imagery layer at the given index as a float4
• cesium_material: a simple wrapper around gltf_material. This is useful since gltf/pbr.mdl is not automatically included in the omniverse MDL search paths.

Together you can create an MDL like this:

imagery-layer-blending-test.usda.zip

[lilleyse]

@corybarr you can ignore the sample data in the PR description. Some of those nodes have been removed temporarily, to be added back in a later PR. See the sample data in #406 (comment) instead.

[corybarr]

Super nit, but I'd probably do this and the above line as an initializer list. Any thoughts on what to do for consistency or other reasons?

[lilleyse]

Fixed in 67024a8

[corybarr]

Again, OV MDL isn't an area I've spent a lot of time in. Duplicating and reconnecting is an interesting solution. Code makes sense and LGTM.