ResourceLoader creating too many jobs and crashes

[mbalajee]

⚠️ Issues not using this template will be systematically closed.

Describe the bug
ResourceLoader creates a job for each primitive to compute tangent. This crashes filament when loading a larger model, even on Pixel 8 (which has been the best case device from our testing on various models).

Computing tangents for all primitives in a single job works (no longer crashes) but I guess that's not ideal, jobs can run in parallel ? So may we run them in batches to avoid creating way too many jobs?

To Reproduce
Steps to reproduce the behavior:
Load a model with 50+K renderables

Expected behavior
ResourceLoader#computeTangents shouldn't crash

Logs

Forks/Filament/libs/utils/src/JobSystem.cpp:315: void utils::JobSystem::put(WorkQueue &, Job *): assertion "job" failed
Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 16042 (d.filament.gltf), pid 16042 (d.filament.gltf)
--------------------------
Cmdline: com.google.android.filament.gltf
pid: 16042, tid: 16042, name: d.filament.gltf  >>> com.google.android.filament.gltf <<<
      #02 pc 0000000001347b3c  /data/app/~~U4sA4_83il1vugyjLUtxAQ==/com.google.android.filament.gltf-7XL-yaMWB6RRRz7fPMZBsQ==/lib/arm64/libgltfio-jni.so (utils::JobSystem::put(utils::WorkStealingDequeue<unsigned short, 16384ul>&, utils::JobSystem::Job*)+76) (BuildId: 31e18d7884d531004ea8c9533895e3c2af57bf4e)
      #03 pc 0000000001348378  /data/app/~~U4sA4_83il1vugyjLUtxAQ==/com.google.android.filament.gltf-7XL-yaMWB6RRRz7fPMZBsQ==/lib/arm64/libgltfio-jni.so (utils::JobSystem::run(utils::JobSystem::Job*&)+56) (BuildId: 31e18d7884d531004ea8c9533895e3c2af57bf4e)
      #04 pc 00000000012db8c4  /data/app/~~U4sA4_83il1vugyjLUtxAQ==/com.google.android.filament.gltf-7XL-yaMWB6RRRz7fPMZBsQ==/lib/arm64/libgltfio-jni.so (utils::JobSystem::run(utils::JobSystem::Job*&&)+36) (BuildId: 31e18d7884d531004ea8c9533895e3c2af57bf4e)
      #05 pc 00000000012ea730  /data/app/~~U4sA4_83il1vugyjLUtxAQ==/com.google.android.filament.gltf-7XL-yaMWB6RRRz7fPMZBsQ==/lib/arm64/libgltfio-jni.so (filament::gltfio::ResourceLoader::Impl::computeTangents(filament::gltfio::FFilamentAsset*)+1408) (BuildId: 31e18d7884d531004ea8c9533895e3c2af57bf4e)
      #06 pc 00000000012e9794  /data/app/~~U4sA4_83il1vugyjLUtxAQ==/com.google.android.filament.gltf-7XL-yaMWB6RRRz7fPMZBsQ==/lib/arm64/libgltfio-jni.so (filament::gltfio::ResourceLoader::loadResources(filament::gltfio::FFilamentAsset*, bool)+572) (BuildId: 31e18d7884d531004ea8c9533895e3c2af57bf4e)
      #07 pc 00000000012ead28  /data/app/~~U4sA4_83il1vugyjLUtxAQ==/com.google.android.filament.gltf-7XL-yaMWB6RRRz7fPMZBsQ==/lib/arm64/libgltfio-jni.so (filament::gltfio::ResourceLoader::asyncBeginLoad(filament::gltfio::FilamentAsset*)+68) (BuildId: 31e18d7884d531004ea8c9533895e3c2af57bf4e)
      #08 pc 0000000001346574  /data/app/~~U4sA4_83il1vugyjLUtxAQ==/com.google.android.filament.gltf-7XL-yaMWB6RRRz7fPMZBsQ==/lib/arm64/libgltfio-jni.so (Java_com_google_android_filament_gltfio_ResourceLoader_nAsyncBeginLoad+48) (BuildId: 31e18d7884d531004ea8c9533895e3c2af57bf4e)
      #14 pc 0000000000001d64  [anon:dalvik-classes3.dex extracted in memory from /data/app/~~U4sA4_83il1vugyjLUtxAQ==/com.google.android.filament.gltf-7XL-yaMWB6RRRz7fPMZBsQ==/base.apk!classes3.dex] (com.google.android.filament.gltfio.ResourceLoader.asyncBeginLoad+0)
      #19 pc 0000000000026d54  [anon:dalvik-classes6.dex extracted in memory from /data/app/~~U4sA4_83il1vugyjLUtxAQ==/com.google.android.filament.gltf-7XL-yaMWB6RRRz7fPMZBsQ==/base.apk!classes6.dex] (com.google.android.filament.utils.ModelViewer.loadModelGlb+0)
      #24 pc 0000000000004018  [anon:dalvik-classes8.dex extracted in memory from /data/app/~~U4sA4_83il1vugyjLUtxAQ==/com.google.android.filament.gltf-7XL-yaMWB6RRRz7fPMZBsQ==/base.apk!classes8.dex] (com.google.android.filament.gltf.MainActivity.createDefaultRenderables+0)
      #29 pc 0000000000004484  [anon:dalvik-classes8.dex extracted in memory from /data/app/~~U4sA4_83il1vugyjLUtxAQ==/com.google.android.filament.gltf-7XL-yaMWB6RRRz7fPMZBsQ==/base.apk!classes8.dex] (com.google.android.filament.gltf.MainActivity.onCreate+0)
------------------------

Smartphone (please complete the following information):

• Device: Pixel 8 / Android 14

[poweifeng]

Do you have the crash log?

And an example gltf would definitely be helpful.

[mbalajee]

Do you have the crash log?

And an example gltf would definitely be helpful.

Added logs in the description. Will try to get a test model that i can share here

[mbalajee]

@poweifeng here is the test model

[pixelflinger]

@poweifeng FYI we currently have a limit of 16384 jobs -- we should probably not create that many :-)