Hi everyone,
Users of the Unity implementation of the RealSense SDK will undoubtedly be familiar with the TrackingAction scripts that enables objects to be controlled through hand or face tracking. If you have played with the script, you will also be familiar with the concept of the Real World Box.
The Real World box enables the user to define an imaginary box between their body and the position of the RealSense camera in the real world. The hand joint or facial landmark that the user has set in the TrackingAction as being trackable (e.g the wrist joint) will only be actively tracked once the designated body part is at a distance from the camera that places it inside the Real World Box's boundaries. Whilst the body part is outside of the box, nothing will happen.
By default, the TrackingAction script draws a box in front of the user that is X = 100, Y = 100, Z = 100 (100 mm wide, 100 mm high and 100 mm deep).
Above those values is a Center setting for each axis that determines where the center-point of the Real World Box is. By default it is set to a value of '50' in the Z axis (which relates to the 3-dimensional depth of the box). Think of it as looking down to the bottom of a cardboard box from above it. If the Center value was applied to a real cardboard box that was 100 mm deep, then a Center value of 50 would place the center-point halfway down the inside of the box - i.e halfway to the bottom.
Occasionally I have set objects in my full-body avatar project to a Z-center value of '0'. This was done so because this value made the object's control more stable than using '50', though I admittedly did not think too deeply about why this might be happening.
During my latest experimentation however, I decided to set the shoulder rotation joints of my avatar to have a Real World Box Center Z-value of '0'. This time I had put some logical reasoning into the rationale. I theorized that I had perhaps been misunderstanding the purpose of the Center setting. I had got into the mindset somehow that if the center-point was not exactly half that of the size value for that axis - a 50 Center for a 100 mm deep box - then the box would collapse as though it had a black hole inside it!
Thinking through the issue rationally though, I thought: "If I am setting the Center point of the box to be halfway down it, am I really telling the camera not to track my body parts properly until my hand or face is at 50 mm distance from the camera?"
My second rationale was: "If I draw a Real World box that is 100 mm wide, 100 mm high and 100 mm deep, it should still measure 100 x 100 x 100 whether I set a Center value or not. And as long as my body parts are within that boundary, the camera should respond to them".
So I set the Center value of the Z depth axis of my avatar shoulder joint to '0', just like the defaults of the X and Y axes' Center values. Then I ran my project and tested the arms.
I found that the arms of the avatar moved far more gracefully and smoothly than they had when they had a Center value of '50' on the Z-axis. Whereas movement was previously slightly jittery, there was now no pauses in movement at all as I moved my hands to control the arms. They were also moving faster.
Encouraged by the results, I applied the 0 Center value to the rest of the TrackingActions in the avatar, in the finger joints and the facial features such as eyebrows, eyes and mouth / lips. They gained the same increase in responsiveness that the shoulder joints had.
That's not to say that the camera suddenly had infinite tracking distance. Tracking still stalled when the body parts moved out of the F200 camera's maximum trackable distance. But when the body was near enough to the camera to be within its range, the improvement in tracking stability was tremendous, at least in my own project. I hope that you can find improvements in your own project too by trying this simple tip.
Link Copied
