[Epic] FBX animation deterioration

[tellypresence]

There are a number of FBX animation related issues open currently including:

• FBX: Regression importing animations #3390
• [5.2.3 Regression] Importing a FBX with animation show a twisted model #4487
• Bug: broken fbx animation #5449

Possible cause

What seems to have happened is that there was a bug introduced on 27 Oct 2019, which went unreported for several months. Code continued to be added incrementally, and then another bug was introduced on 24 Mar 2020; this bug also went unreported for many months and in the meantime a lot of complexity had been added to the FBX importer.

Also, it's important to understand that bugs in the FBX code don't affect all FBX models. Some FBX models are affected by some bugs, while other FBX models are affected by other bugs; in the examples below, it can be observed that

• "Nathan" model was unaffected by the first bug
• "Sophia" and "Manuel" models were affected by both bugs*

If a developer is only testing against a small set of models, it is unfortunately likely that new code may contain bugs which don't affect the local test FBX model(s), but which may affect other FBX models...

*Manuel worked properly in release v5.3.1 while Nathan was broken (details table below), so I suspect v5.3.1 contained only the 2nd bug and not the first; for some reason "Manuel" showed artifacts due to 2nd bug on master branch but not in release 5.3.1

Bisect report: the first bug introduced on 27 Oct 2019

Git commit	Model	Screenshot	Notes
29f7ea0 27 Oct 2019	55-rp_nathan_animated_003_walking_fbx		All 3 models animate correctly
	22-rp_manuel_animated_001_dancing_fbx
	35-rp_sophia_animated_003_idling_fbx
168ae22 27 Oct 2019	55-rp_nathan_animated_003_walking_fbx		"Nathan" model still correct at this commit (highlighting that different FBX models are (un)affected by different bugs), but the others show obvious deterioration
	22-rp_manuel_animated_001_dancing_fbx
	35-rp_sophia_animated_003_idling_fbx

Bisect report: the second bug introduced on 24 Mar 2020

Git commit	Model	Screenshot	Notes
b962af9 24 Mar 2020	55-rp_nathan_animated_003_walking_fbx		Animations appear to be consistent with the state after first bug introduced on 27 Oct 2019; "Nathan" unaffected, and "Manuel" and "Sophia" look about the same as when the first bug was introduced in Oct 2019
	22-rp_manuel_animated_001_dancing_fbx
	35-rp_sophia_animated_003_idling_fbx
14b8d12 24 Mar 2020	55-rp_nathan_animated_003_walking_fbx		Now "Nathan" is broken, and "Sophia" artifacts are much worse; note that "Manuel" was also affected by this 2nd bug
	22-rp_manuel_animated_001_dancing_fbx
	35-rp_sophia_animated_003_idling_fbx

(N - 1)st partial fix introduced in 88a9598 on 26 Apr 2022

At commit 88a9598 on 26 Apr 2022, "Manuel" got fixed and "Sophia" became less degraded (though was still "abstract art")

Bisect report: Nth bug introduced in 28 Nov 2023 commit 384db86

"Manuel" broke again, Sophia got worse, and both "SilverDragon" and "TarislandDragon" completely broken
Details in below comment

Anyone have any ideas on how to improve the situation? Another case where bugs got introduced several years ago and may be hard to address with all the additional code added during that period

Observations

To prevent these sorts of things happening in future, would be useful to have a minimal set of carefully selected test FBX models in the repo with guidelines that their animation must not be broken by any code updates (i.e. the set of models would need to be manually tested before committing modifications)

Difficulties correlating assimp releases with repo main branch

Due to the highly selective cherry-picking when creating assimp releases, it is often not possible to compare a release against a commit to the repo main branch even if they are around the same point in time. An example is the table in issue 5449 where it can be seen that the "Manuel" model was still working at release 5.3.1 from 2023, despite having been broken on the main repo branch since 2019; and "Nathan," which was broken at the 24 Mar 2020 commit, was also broken in 5.3.1 -- as mentioned above, my theory (which of course may well be wrong) is that v5.3.1 only had the second bug, not the first, which would have broken all three models ("Sophia", "Nathan" and "Manuel") so it's a mystery why only "Manuel" was not broken in release 5.3.1

Model	Assimp version			Notes
	2d2889f (24 Sep 2019)	release 5.3.1 (25 Sep 2023)	feb861f (21 Mar 2024)
manuel dancing				Worked at commit 2d2889f and release 5.3.1, obviously wrong now
sophia idling				Worked at commit 2d2889f, didn't work in release 5.3.1, obviously worse now
nathan walking				Worked at commit 2d2889f, didn't work in release 5.3.1, no obvious difference now

Note: (like there's not enough going on here already) Sophia at release v5.3.1 looks weird, like there was a 3rd bug introduced at some point that only affects her animation

Get the models

Models mentioned above are available either

• using direct link
• from renderpeople.com: under "3D Animated People" click "Free Download" button, then click the "FBX" button and download zip

Models of interest are

• 22-rp_manuel_animated_001_dancing_fbx
• 35-rp_sophia_animated_003_idling_fbx
• 55-rp_nathan_animated_003_walking_fbx

[tellypresence]

(reserved)

[tellypresence]

(reserved)

[tellypresence]

Models unaffected by the first bug introduced on 27 Oct 2019

Git commit	Model	Screenshot	Notes
29f7ea0 27 Oct 2019	55-rp_nathan_animated_003_walking_fbx		All 5 models animate correctly
	mr-man-walking
	robot_kyle_walking
	tarisland-dragon-high-poly
	silver-dragonkin-mir4
168ae22 27 Oct 2019	55-rp_nathan_animated_003_walking_fbx		All the above models unaffected by this commit
	mr-man-walking
	robot_kyle_walking
	tarisland-dragon-high-poly
	silver-dragonkin-mir4

[tellypresence]

Models affected only by the second bug introduced on 24 Mar 2020

Git commit	Model	Screenshot	Notes
b962af9 24 Mar 2020	55-rp_nathan_animated_003_walking_fbx		Animations consistent with the state after first bug introduced on 27 Oct 2019; no apparent degradation in animations
	mr-man-walking
	robot_kyle_walking
	tarisland-dragon-high-poly
	silver-dragonkin-mir4
14b8d12 24 Mar 2020	55-rp_nathan_animated_003_walking_fbx		All models show some degradation ("Silver Dragon" probably the least affected, mostly the right wing)
	mr-man-walking
	robot_kyle_walking
	tarisland-dragon-high-poly
	silver-dragonkin-mir4

[tellypresence]

(reserved)

[tellypresence]

Models fixed (or at least improved) by commit 88a9598 on 26 Apr 2022

Git commit	Model	Screenshot	Notes
02f0706 21 Apr 2022	35-rp_sophia_animated_003_idling_fbx		Manuel and Sophia broken
	22-rp_manuel_animated_001_dancing_fbx
88a9598 26 Apr 2022	35-rp_sophia_animated_003_idling_fbx		Manuel correct; Sophia still broken but obviously better
	22-rp_manuel_animated_001_dancing_fbx

[tellypresence]

Models affected by the Nth bug introduced in 28 Nov 2023 commit 384db86

Git commit	Model	Screenshot	Notes
77a8f01 22 Nov 2023	35-rp_sophia_animated_003_idling_fbx		Manuel/Silver Dragon/Tarisland Dragon correct; Sophia obviously degraded but mostly recognizable
	22-rp_manuel_animated_001_dancing_fbx
	tarisland-dragon-high-poly
	silver-dragonkin-mir4
384db86 28 Nov 2023	35-rp_sophia_animated_003_idling_fbx		Manuel/Silver Dragon/Tarisland Dragon were correct but show obvious degradation; Sophia already broken and obviously worse
	22-rp_manuel_animated_001_dancing_fbx
	tarisland-dragon-high-poly
	silver-dragonkin-mir4

[tellypresence]

(reserved)

[GitHubDragonFly]

@tellypresence maybe try testing with the attached FBX files or possibly compiling my updates branch.

If this does work for you then take your time analyzing the FBX files, since I did restore and modified mainly FBXConverter files but there could be some minor changes elsewhere (maybe also rely on the GitHub system to see the differences).

This seems to be working in my ASSIMP Viewer.

FBX.zip

[Urri78m]

Hello everyone, we have indeed discovered a lot of problems with animations taken from mixamo.

[lxw404]

@tellypresence Thank you very much for making this post, it's helping me track down these regressions. The 28th of November 2023 regression is fixed via: #5751. I'll be taking a look at the prior ones next week or so.

[tellypresence]

@lxw404 Thank you for that PR, can confirm it fixed two of the affected models, but "Manuel" which was also broken with the 28 Nov 2023 regression is still broken as can be seen in this comment on issue 5449

[kimkulling]

I will add the regression tests to the FBX assets.

[kimkulling]

I guess to be able to test these regressions a model dump will be necessary. I guess doing this with your models will be a little bit too big.

[kimkulling]

And thanks you all for the great work and help figuring this out. A lot of users will love you forever I guess!