Tweaking the Hip Tracking of the FBT

[theboom1]

We want hip tracking to be one to one for proper hip rotation tracking.

[shiftyscales]

Thanks for the report, this is one of the more well-known/often requested issues with FBT in Neos at the moment.

The issue is that there are numerous passes in the current IK, and some of them are suppressing the rotations along the X, and Z axis making some hip movements slightly more 'stiff'.

There is also additional issue in the way the hips are tracked- currently they are positioned while offset from the physical tracker, which causes the neck scrunching/stretching issue.

These issues are known, but it is good to have them logged here.

[RoxyBoxxy]

any updates on this

[Toxic-Cookie]

If growing the population is the focus right now, overhauling the IK would be an excellent decision as many are willing to play but are discouraged by muted hips and other quirks such as legs frequently going 180 degrees.

[Frooxius]

There is no plan to overhaul the whole IK, I don't really believe there is any need for it, as it mostly works fine, save for a few issues like this that can be fixed instead.

Growing the population is always the focus, but there are many other problems and features that help with that as well, so it's just a question of how much of a bigger priority this should be over other things. I suggest using the voting system on GitHub as recommended in the README to help bump up the priority. So far this only has 1 votes, while many other issues that came up quickly gather several.

I've had this on my todo list for a bit though, but it always ends up pushed off by something else that comes up with higher urgency. I'll see if I can push it up soon, it's it's a particularly tedious thing to work on though.

[Frooxius]

Also could you clarify what do you mean by the legs going 180 degrees? Is there an existing GitHub issue for it that I missed? If not could you please make one?

[shiftyscales]

If they're talking about what I think they are, it is something that comes up during fairly extreme poses, e.g. lying on your back and placing your legs upward into the air, or just bringing the legs above the head in general which can come up while dancing with a pole.

It was something I was shown before in the context of doing yoga poses. It's because of a lack of constraints on the bones, so joint roll can occur due to twisting from the IK.

Similarly there are certain angles that one can hold their wrist/hand that will cause the IK to twist the joint at their wrist even while just using IK. @Frooxius

[Frooxius]

That's more about using the right bend/twist vectors. The way the IK system works is that it doesn't (and pretty much cannot) rely on constraints like that - the constraints come naturally from your own body. It sounds like a bit more obscure bug though, but when I was originally implementing and fixing issues, I did test a lot of crazy positions like that and it worked fine.

It might be something avatar-specific, so I'd probably need some samples and replication steps. But it's better for another topic separate from the hips.

[RoxyBoxxy]

@Frooxius i shall make some videos showing the issues if you like

[RoxyBoxxy]

This is slouching in a chair

this is sitting perpendicular in a chair

[TehTurk]

Wanted to Chime in on this too but for Hip Tracking there are a few issues, thought we had one already but I guess this is it:

• The Rotations of your trackers aren't correctly presented/felt as hip rolls, or even pivoting doesn't really show nor have an effect on the system
• If the Hip Tracker ever is flicked or is just calibrated in a weird spot, it drags the players head down, the Hip Tracker should have as much tracking priority as the Head/Hands when on. If not it feels like I shouldn't be using it as without a hip the System is that much more comfortable to use at 2 point vs 3 points.
• Rotations of the entire body are tied too much to the head, if I want to turn my head right, but keep my feet forward this isn't the case.

Besides that the rest of the system works fine!

I can showcase and make a video for this, because the hip tracking is a ###1 Issue for Many FBT Users who'd like to migrate over to the platform. People just like throwing FBT and chilling, and this has been brought up before multiple times in the discord but priority of other features took attention at the time. With the Downtime Froox has had due to some of the Unity Bugs, this would be a good opportunity to get some leeway on it

[RoxyBoxxy]

I agree fully with @TehTurk i had friends come over and they would come back to neos more often when the hip roll is fixed/working more efficient

EDIT: the FBT is awesome as it is also as other platforms have some sort of jiggle when jumping, but in neos is less jiggly

[alexderpyfox]

It's when you move your head while using fbt it also rotates the hip it shouldn't rotate the hip at all unless the actual hip tracker is rotated

[Reactantvr]

I too believe this is a major issue for retaining users. I've met countless new players over the last year that were very disappointed with the hip issue and went back to other games. I think that with the user base starting to really increase, it may be essential to try to retain these users. This is especially important because many of the dancers and streamers have full-body and won't play Neos because of this precise issue. It could have a negative impact on Neos if they were to make videos showing off the problem. The IK is otherwise great except for this one nagging issue.

[Frooxius]

Just to note, this is planned to be addressed when possible. We had some members of team have a look at this already in the past months without success. I plan on looking at it myself when I can.

One thing that could potentially help with the fixing process is using the avatar recorder by guheheP if the latest version supports hip tracking as well. Recording a short clip that reproduces the issue with a sample avatar and then sending me the recorded copy. I could use that to more easily tweak the IK and repeatedly iterate with the same conditions.

[ProbablePrime]

This might be helpful its a video from Rezillo about the differences between Neos FBT and VRC FBT.

https://youtu.be/n8jgqFAQXjs

another video:

https://www.youtube.com/watch?v=OMJamsnBhqs

[RoxyBoxxy]

Any ETA, Just wondering

[TehTurk]

Following up from an earlier discussion this to note for Frooxius in the future, I've sent 6-7 examples of the avatars to you in a message for whenever your free to work on this. If there's an ideal scenario that would cover all grounds or things I need to pay attention for do please let me know. I may also do another round but put it into a world and have a different method of setup that's come to me more recently.

[Frooxius]

No ETA sorry, it'll get done when it gets done. I'm working up my way towards tackling this as it's high on the list, it's just a mentally difficult thing to work on.

[alexderpyfox]

Maybe you could ask coffee or outsider for help with Programming the hip fix so it would be less of a headache 😄

[StarfishHidari]

No ETA sorry, it'll get done when it gets done. I'm working up my way towards tackling this as it's high on the list, it's just a mentally difficult thing to work on.

thanks for the update on the status 😃

[RoxyBoxxy]

Well i for one would be willing to tests builds for you

[Frooxius]

@alexderpyfox As I mentioned above I already did that a few months back, but this falls out of their area, so I need to have a look at it myself.

@RoxyBoxxy Testing builds isn't the issue, the biggest hurdle with this is the iteration speed when working on a solution. For something like this I need to go quickly between making a change to code and testing the results, sharing the builds with another person and waiting for feedback would slow down the process significantly.

My usual approach was putting on body trackers and testing it myself right after tweaking code, but even that's a bit tedious.

With the avatar recording systems the process can be make quite quicker so I can completely skip that for the most part and just test from the desktop mode. Turk has sent me some samples that I can work with. If you can record any yourself that demonstrate the issue as well, that can help iterate on the issue significantly and make sure I have better coverage. I recommend checking up my post above for some details on how to record it.

[guheheP]

Hi, I'm guheheP.
I'm working on a new version of the Motion Capture Tool(avatar recorder).
In the new version, You can choose which IK to record.
That might be useful for development, so I'll share it.

[alexderpyfox]

https://youtu.be/yGn9AzBz3gw here i have a solution video for the fbt issue it seems to work pretty well other than not being able to move the hip target anymore via the calibrator but it's still moveable via the inspector so that's good

[alexderpyfox]

I'm willing to help where do I sign up?

[alexderpyfox]

The mocap tool from guehep broke btw and they said this

So now it doesn't work anymore no way to record fullbody mocap anymore so without this it can't be fixed

[alexderpyfox]

I made a new video with a in-game Logix hip fix
setup tutorial specifically the blueprint https://youtu.be/ELCwI38PaA4 it also includes a link to the Logix blueprint that contains the Logix fix and video in-game

[ProbablePrime]

Hey Alex,

There's no sign up or anything like that, just provide recorded data, logs, videos etc as has been asked for.

Remember when doing this to follow the principles of avoiding the XY Problem (https://xyproblem.info/) in that you should try to demonstrate the problem in your logs, videos etc rather than a proposed solution.

Thank you.

[alexderpyfox]

Hey Alex,

There's no sign up or anything like that, just provide recorded data, logs, videos etc as has been asked for.

Remember when doing this to follow the principles of avoiding the XY Problem (https://xyproblem.info/) in that you should try to demonstrate the problem in your logs, videos etc rather than a proposed solution.

Thank you.

All the data is already given we can't do much more than wait and create our own temporary solutions at this point

[FuzzyFoxe]

I'm with alex at this point. Unless we're given a guide on what info is needed, there's literally no more info we can give. It's been explained multiple times now exactly what people are asking for, and until there's a fix, people are finding their own way to address the issue, since there's been no helpful response in how to gather the data that's needed other than "logs" and "videos". Like, what are we supposed to do to give the best logs? It's not like there's an error happening, it's simply the way the feature is implemented right now that isn't specifically desirable.

[shiftyscales]

@FuzzyFoxe:

One thing that could potentially help with the fixing process is using the avatar recorder by guheheP if the latest version supports hip tracking as well. Recording a short clip that reproduces the issue with a sample avatar and then sending me the recorded copy. I could use that to more easily tweak the IK and repeatedly iterate with the same conditions.

As Frooxius mentioned earlier in this thread. Reportedly according to Alex, that tool had broke had some point?

So a good first step would be to determine if the tool is still usable, or otherwise, what broke on it so it can be fixed (or reported to us in another GitHub issue so we can fix it depending on what broke.)

Their Motion Capture Tool folder inside of their public folder contains the recorder: neosrec:///U-guheheP/R-00ad625c-7401-4e58-8ae6-f2dfdca59b89

[FuzzyFoxe]

@shiftyscales

We want hip tracking to be one to one for proper hip rotation tracking.

Sorry, I thought we were still on the subject of the original post. So, fair enough. My post still stands about the fbt stuff though, hip tracking issues have been explained for some time now, asking for more info about it is almost pointless unless there's a method suggested to attain more info.

I'll see if I can test the issue with the mocap item next time I'm on. Maybe I can help figure it out.

[shiftyscales]

@FuzzyFoxe - I'm thoroughly versed on the issues too as I use FBT on a near daily basis. I know exactly what the issues are, I feel Frooxius does too- but to put it in context, basically, Frooxius needs these samples as they allow him to tweak the IK code, check the result, then repeat, making the iteration time much faster.

To this end, having those samples presented in a Neos world he can just have open up at launch would make it much easier for him to quickly iterate through test builds to see the results of the code he tweaks.

If anyone is able to figure out the tool, it would certainly help.

[alexderpyfox]

I can still adjust the hip position after the recording on the record avatar but then it will not be 1:1 with the original recording so then fixes might be made on a avatar that's adjusted the hip position afterwards i can send the raw recording too with the hip completely mangled in the recording but either way it records the hip wrong on the mocap tool

[Nordwig]

Hello the Recorder Version 2.1 in guheheP Public Folder works for me when my UserScale is set to 1 otherwise it has some problems with the hip position.
I created a Public Folder ( neosrec:///U-Nordwick/R-0b06c6bb-4397-49b9-95a9-c325458077f0 ) with a World http://cloudx.azurewebsites.net/open/world/U-Nordwick/R-cb47b2af-3882-480e-9625-dacfd78aed99 and left some Records.
I hope its set up right so everyone can open it and save if necessary.

One off the main issues I see is the offset of the ViewPort to the Armature. This Offset works like a lever. When you look down it lifts the avatar up. And when u look up it compresses the avatar. With out a hip tracker the legs are compensating the movement very nicely. With the hips tracker the neck and chest bones are rotating out of the way to compensate, which can look wired.
If you try "pull" on the Hips to straiten the Spine, it pulls the avatars head down through the ViewPort.

I propose the ViewPort should be the Tracker with the highest priority and the hips should have a hi influence on rotation.
But some leeway in position a bit like a rubber band that keeps the spine straight and pulls in the gravitational direction.

I hope it helps. please tell me if I got something wrong

• edited the world added captures without any adjustments to the calibration

[alexderpyfox]

The head target should be the most important and have priority over all other trackers not to become disconnected from the avatars head

[alexderpyfox]

Looking down = head target misaligned with avatar head.
Looking up = scrunching.
So you need to compensate the hip tracker for both issues because the avatar is attached to the hip tracker so when you look up the spine becomes collapsed because it folds in on itself when you look down the spine can't stretch far enough so pushes the whole avatar down making the head target become mis aligned it needs to dynamically change the hip target to compensate for both situations

[epicEaston197]

has anyone noticed this issue is not pinned anymore?

[NjnaGrimsdottir]

Is there any status on this issue, or is this still one of the top piorities?
https://twitter.com/ProbablePrime/status/1517020165698772992

[DoubleStyx]

Is there any status on this issue, or is this still one of the top piorities? https://twitter.com/ProbablePrime/status/1517020165698772992

It's the second most voted issue currently and is still on the development roadmap.

[NjnaGrimsdottir]

It's the second most voted issue currently and is still on the development roadmap.

coming up on 2 years. Is there even a rough eta.. like 2022?... 2023?

[ProbablePrime]

We're unable to provide ETAs at the moment. But thank you for you interest. You can keep an eye on our discord's announcements channel or steam updates for any news. Or this issue.

Thank you.

[Kodufan]

I discovered what could possibly be the issue for the hip tracker in Neos, ironically enough, with VRChat. With their updated IK system, they, for some reason, allow you to decide whether to lock FBT to your head, hips, or both. It is when locking FBT to both in VRChat that you see the exact same issue plaguing Neos. Here is a short video I recorded showing the difference between locking to the head, hips, and both.

What this tells me at least is that the root cause of the issue is that tracking is locked to the hips the same way it is locked to hands, heads, and feet. While I don't know what an exact solution would entail and how VRChat does it, the conceptual solution is to allow the hips proxy to "float" from the actual hip tracker so that small changes won't surpass a "crunch" threshold, bending the neck. People smarter than I may be able to come up with the exact way to do this, however, I strongly believe that this is the solution we've been looking for years to find.

[Nytra]

I discovered what could possibly be the issue for the hip tracker in Neos, ironically enough, with VRChat. With their updated IK system, they, for some reason, allow you to decide whether to lock FBT to your head, hips, or both. It is when locking FBT to both in VRChat that you see the exact same issue plaguing Neos. Here is a short video I recorded showing the difference between locking to the head, hips, and both.

What this tells me at least is that the root cause of the issue is that tracking is locked to the hips the same way it is locked to hands, heads, and feet. While I don't know what an exact solution would entail and how VRChat does it, the conceptual solution is to allow the hips proxy to "float" from the actual hip tracker so that small changes won't surpass a "crunch" threshold, bending the neck. People smarter than I may be able to come up with the exact way to do this, however, I strongly believe that this is the solution we've been looking for years to find.

Someone previously posted a video in this thread about a LogiX solution to allow the hip proxy to float up and down based on the up and down rotation of the head. I tried it out and it does work but requires manual tweaking for each individual avatar. The video they made about it is here: https://youtu.be/ELCwI38PaA4

[Kodufan]

This inspired me to actually come up with a potential solution to the issue. I believe I'm onto something. Here is a demonstration of my system dubbed "FBT+". I'll be bug testing it and tweaking it before making an installer and distributing publicly.