Developers who are using Unity and its pre-made TrackingAction and TranslateAction movement will no doubt be aware of the concept of what I call "mis-rotation" - an object moving or rotating when you do not want it to because the head has been moved backwards and forwards or the hand has been turned side-on. It can make precision positioning of objects very difficult.
It is possible to minimize or lock an object's ability to mis-rotate if it is using Position type movement. However, there is no such remedy for Rotation type movement. My company has spent six months trying to solve the issue with a huge number of prototypes and could write a thick book about all of the approaches that don't work. :)
We have not given up on solving this issue. But we have increasingly found that instead of trying to swim against the tide, it works better for our project if we try to intentionally incorporate mis-rotation into the project so that when it occurs, it makes sense to the end-user and seems as though it was always planned that way!
In our project, where we control a full-body avatar character with the RealSense camera, we have found that permitting an object to mis-rotate can be beneficial in the following ways:
* When the user moves their head forward and back, a 'TrackingAction' in the top-most parent object of the upper section of the body causes the entire upper body to lean forward and back as the 'TrackingAction' in that object rotates back and forth when the head moves towards and away from the camera.
* Also during leaning backwards and forwards, the chest, midriff and stomach sections of the avatar - which each have a 'TrackingAction' in them - are constrained so that they mis-rotate up and down when the head moves towards or away from the camera. This gives the impression that the body's fat and muscle is compressing when leaning forwards and stretching out to become taut and muscular when the body is leaning back as the fat and muscle is pulled upwards.
* Likewise, our avatar's male breasts also use this technique to rise and fall during lean-back and lean-forward, just like a real human chest.
* When the palms of the hands are moved towards and away from the camera, the shoulder joints of the arms mis-rotate back and forward and so give the avatar the appearance that its arms are reaching backwards and forwards.
* When the head is bowed, the eyelids roll down a little and the height of the eyebrows dips slightly. Likewise, when the head is raised, the eyelids open wide and the eyebrows rise up. This gives the impression of the sad and happy emotional states typically associated with lowering or raising the head.
With imagination and careful use of Constraint and Invert settings, you can turn the problem of mis-rotation into a real asset for your project!