Hi everyone,
If you use the TrackingAction RealSense script in Unity to power the rotation of objects then you may notice that the object will sometimes jump out of the desired position. This occurs particularly when the camera gains and loses tracking of the hand or face. In the case of using RealSense with avatar bodies, it can mean that a character's arm leaps sideways or one eyelid half-closes whilst the other remains open.
My company has been researching and experimenting with this problem for over a year and found that there is little that can be done with constraints, rules or custom scripting to minimize the snapping. Even if you manage to deactivate the TrackingAction at the instant that a snap is about to occur to prevent the object from mis-rotating, the TrackingAction initiates the delayed snap-movement as soon as it is switched on again and re-acquires tracking.
This is because it reads the current position of the hand or face landmark that a particular TrackingAction is set to and then pulls the rotation angle of the object towards that landmark. So if you switch off the TrackingAction when your hand is at the side of your body, move your hand towards the center of your chest and switch the TrackingAction back on again, the object will flip sidewards towards the chest where your hand currently is.
Another phenomenon that we had observed was that if the Virtual World box values of a TrackingAction that is configured for rotation are set to '0' then they are automatically changed to '1' as soon as tracking is acquired.
If you are wondering why we were using '0' for our rotational Virtual World Box values instead of the default '100': it is because having a Virtual World Box with perfectly spherical objects (like the limb joints in our avatar) seems to achieve nothing at all except generate unsightly giant red boxes in the editor. We speculate that this is because a sperical object spins on the spot and so can never touch the sides of the box, and that the Virtual World Box would work best with irregularly shaped objects that can make contact with the Bos's boundaries when rotated and have their movement blocked.
Anyway ... since we had observed that a "snap" in the settings of the TrackingAction when the program is running - such as the loss and re-gaining of tracking - can pull an object away from where you want it to be, we speculated that something similar might be happening with the automatic change of our Virtual World Box value from '0' to '1' at program start-up.
It was possible that the change in VBW values caused a change in the rotational joints at the very start of the program that it could not recover from, making the controls prone to going wrong. We therefore decided that the best way to avoid this start-up snap might be to simply set the VWB values of every single one of our rotational TrackingActions - both hand and face ones' - to the '1' value by default so that they did not have to be altered by the TrackingAction when tracking was acquired.
When we performed testing of the avatar with the new '1' setting, we found that the smoothness of the joint rotations had increased considerably, with much less jolting during movement and easier achieving of more difficult actions such as lifting the avatar arms above its head. It wasn't a perfect cure, since the TrackingAction still has other issues to be dealt with, such as producing unwanted direction changes when the wrist is turned slightly, even if the joint is not set to track the wrist joint. But eliminating the automatic updating of the RWB value had certainly improved the controls of our avatar.
Another way you can precisely customize your Virtual World Box settings is in the 'Center Reference' menu. By default it is set to 'Self.' This means that the TrackingAction automatically generates values that will move the center-point of your Virtual World Box away from the absolute center of your object. Sometimes the values are too tiny to make much of a difference, and sometimes they are as large as '10' or '-10', which can cause a noticable offset in the box's boundaries.
You can avoid this auto-generation of the Center values by choosing the 'Custom' option from the 'Center Reference' menu and then setting all of the Center values manually to '0.' When your program is run, your Virtual World Box will remain perfectly on-center no matter what.
CONCLUSION
Once you understand how the TrackingAction script can change the rotation angles of your object based on factors such as jumping to a landmark's new position after tracking is acquired or how much you are unconsciously turning your hands sidewards when using your hands to control an object, it makes it easier to design counter-measures for these unwanted angle changes to produce smoother animation and more refined control.
Best of luck! If you have questions, feel free to ask them in the comments below.
Link Copied
Do the arms on your avatar move? If they do, please make sure that the left arm's TrackingAction is set to Index 0 and the right arm is set to Index 1. If your right arm has Index 0 (when it should be Index 1), then when the left arm moves, the right arm will move too.
Here is what my arms look like.
不要在你的頭像的胳膊動？如果他們這樣做，請確保左臂的TrackingAction設置為指數0和右臂被設置為指數1。如果你的右臂有索引0（當它應該是指數1），左手臂的動作那麼當，右臂將移動過。
VS.png){kind=link}