Software Archive
Read-only legacy content
17061 Discussions

Intel RealSense & Unity

Kiriakos_T_
Beginner
383 Views

Using the RealSemse SDK for Intel(R) RealSense (TM) 3D Camera (Front F200) in Unity (version 5.3.1f1) I have encountered serious problems as far as the precision of the hand tracking is concerned. In every little move of the hand, the join between a certain part of the hand and an object that is adjusted on it is lost. It works fine when the hand is standstill but not when it moves. I tried to do some corrections in unity 's inspector options but I didn't make sufficient improvement. Is there anything I can try to have better results?

0 Kudos
5 Replies
MartyG
Honored Contributor III
383 Views

Have you linked your objects together by "childing" them to the object that is being moved?  You do this by going to the left-hand 'Hierarchy' text list of objects, holding the left mouse button down on the name of the object that you want to link to the one that moves, and then dragging and dropping this object onto the name in the list of the object that moves.

If you have linked your objects together, the other thing I can think of is that - assuming you are using a TrackingAction script to generate the motion - the "constraints" section of the TrackingAction script's settings in the Inspector are not correctly configured.  For example, if you wanted an object to only be able to rotate on the spot but not move, it would be able to move if you had not placed a tick-mark in all the Position constraints to lock their movement.

0 Kudos
Kiriakos_T_
Beginner
383 Views

Thank you very much Marty G. for your kind answer,

Let me define the problem. I do use a trackingAction script. The problem is that when I adjust fo exapmle a ring in the finger of the hand.. it doesn't follow the actual move of the hand. It is ok with no or little move but not with a wide range of moves or a slight rotation of the hand. Do you think I can imporve this or it is just the way it works (with poor hand tracking)? Let me know if you have an idea to try it.

0 Kudos
MartyG
Honored Contributor III
383 Views

I think I understand you now.  Yes, TrackingAction - in its default form, at least - is best for general sweeping motions and is poor at small, precise motions that move the object exactly where you want it to go.  Rotating the hand can also mis-rotate the object even when you did not select Wrist as the tracking method. 

It's not hopeless though.  I've been building custom fixes myself for over 2 years and have got it to the point where precise movement that mirrors the real-life hand / arm movements almost exactly is possible with the TrackingAction.  Check out this video I posted last week:

https://www.youtube.com/watch?v=HspFO87NHeY

To use the method that provides this precision though, you will need to learn how to create Animation Clips in Unity so that you can then control the animation in real-time with the TrackingAction to provide the movement instead of controlling.the object directly with a TrackingAction inside it.

If you are not keen to go to this much trouble, you may get greater precision by creating 3 invisible spheres overlaid exactly on top of each other and childing them together in a chain (sphere 2 childed to sphere 1 and sphere 2 childed to sphere 3).  You then put a TrackingAction inside each and set each one to have its rotation dedicated to a particular axis.  

For example, Sphere 1 could have all Rotation constraints except X locked.  Sphere 2 could have all constraints except Y locked.  And Sphere 3 could have all constraints except Z locked.  Then you child your hand object onto Sphere 3 (the end of the chain).  This gives much greater control over the direction of the hand, because when you move your RL hand in a particular direction, it's much harder for the object to move in the other directions.  Whereas if you try to move an object with a single TrackingAction with X, Y and Z rotation open, movement can be chaotic.

The above method is the one I used to use successfully in my avatar before I created my Animation Clip based control system in the video.

Edit: if you try the sphere-chain method, make sure that you remove the TrackingAction inside the hand or you will get chaotic movement, since the hand will be trying to move independently of the propulsion being provided by the spheres.

0 Kudos
Kiriakos_T_
Beginner
383 Views

Thank you very much for your help Marty G. There is some improvement. Do you have any idea how to make a really strong join between an object and the wrist of the hand? I mean to make the perfect adjustment between them.

0 Kudos
MartyG
Honored Contributor III
383 Views

Glad it helped!

The strongest join possible between two objects is basic childing with no Rigidbody in the child object, so that it is unable to drift away from the parent object that it is attached to.

If you need a Rigidbody (for example, to give the overall assembly gravity) then put the Rigidbody in the parent object at the top of the hierarchy.  The effects of the Rigidbody propagate down through the hierarchy, so if the parent is falling under gravity then any non-Rigidbody object childed to it should fall with it.  Adding additional Rigidbodies as well as the one in the top-level parent can cause the parent's Rigidbody to malfunction, resulting in effects such as the object turning in big arcs when rotating or floating away.

In the wrist joint of my avatar that turns the hand left-right, I use an ordinary TrackingAction.

1.jpg 

0 Kudos
Reply