Incorrect thumb poses due to missing degrees of freedom

[Android-xD]

Hi @SeaOtocinclus ,

Thank you for sharing this well-documented and easy-to-use dataset!

I am currently working with the MANO data provided in the mano_hand_pose_trajectory.jsonl files and have noticed an issue with the thumb poses. It seems like all joints are being modeled as hinge joints, which introduces large errors for the thumb.

On a similar note, in the umetrack format, the thumb seems to be implemented with 4 DOF, like the other fingers.

Question:

Are there any plans to address this issue or is there a way to fix the issue myself?

Are there any plans to use the last two values in the umetrack_hand_pose_trajectory.jsonl for the thumb?

Thank you!

Steps to Reproduce (MANO):

1. Use the provided tutorial notebook: HOT3D Tutorial on the sequence 'P0003_c701bd11'

2. Uncomment the following lines in section 5:

   # hand_mesh_vertices = hand_data_provider.get_hand_mesh_vertices(
   #    hand_pose_data
   # ).tolist()

3. Remove the line in section 5:

   hand_mesh_vertices = np.array([])

Observed Behavior:

The thumb pose appears to be incorrect due to the assumption that all joints behave like hinge joints. There is only one value stored per joint, which is insufficient to capture the full range of thumb motion, as seen in the screenshot.

Umetrack Data has a similar issue

To my knowledge, the UmeTrack model assumes 4 DOF per digit. Since the thumb has 5 DOF, it is not able to model all thumb poses accurately, as seen in the screenshot.

The thumb has at least 5 DOF from the three joints of the thumb: interphalangeal (IP), metacarpophalangeal (MCP), and trapeziometacarpal (TM) joints. The IP joint is commonly considered a hinge joint with 1 degree of freedom. The MCP joint is commonly considered a condyloid joint with 2 degrees of freedom (a high range of motion around one axis and a smaller range of motion around the other) and the TM is usually considered a saddle joint that allows movements similar to the condyloid joint but with a greater range of motion.

On a side note: The TM has been proven to have 3 degrees of freedom since there is also a slight axial rotation around the TM joint, that allows full opposition of the thumb. "The mean range-of-motion of the trapeziometacarpal joint was 41 degrees for flexion-extension, 51 degrees for abduction-adduction, and 21 degrees for axial rotation." [1] Considering this as well, the thumb would have 6 DOF.

Suggested Solution:

A better approach would be to store a general rotation for each joint in axis-angle representation (as done in the H2O Dataset). For example, a 45-dimensional vector, where each triple represents an SO(3) rotation in axis-angle format for each joint, would allow for more accurate thumb poses.

References:

[1] https://www.researchgate.net/publication/222892809_Normal_range-of-motion_of_trapeziometacarpal_joint