Unity - can you trigger script with hand gesture?

[tony2]

Hello!

I am playing with the RSDK + 3D cam and Unity and I wonder if there is a way to trigger scripts with hand gestures? I know there are the Activate/Deactivate/Hide actions provided by RSDK, but you can only add game objects to them... Creating an empty game object and adding a script to it, then adding the object to the Action won't do the job.

Thanks!

[Richard_F_2]

Not quite sure what you're asking...

You can call a method using the actions, with the SendMessage action.  It will do as advertised and do a SendMessage call on the provided method name.  Of course this implies that you have an attached script to the object in question that has the method specified in the Action.

As an example, I currently have the SendMessage Action attached to a cube, and I also have a QuitGame Monobehavior script attached to it as well.  Inside that script I have a method called DoQuit() which has a call to Application.Quit().  On the Action side I set the method name to DoQuit.  So now when I wave my hand at the camera, the game closes.  Is something like that what you're trying to do?

[tony2]

I am actually just now testing the SendMessage action, which works as intended. 

Yes, you are right, I am trying to do something similar to what you described here.

Thanks for your reply and example! 

[MartyG]

Hi Tony,

I had a quick play around in Unity to see if I could come up with an approach to your problem.  One way to trigger a script in Unity using the camera is as follows:

1.  Create your Empty GameObject.  Name it MovingObject.

2.  Add a box collider field to it by highlighting the Empty GameObject and selection Component > Physics > Box Collider. The object needs to have a collider so that it can trigger an event when it passes through the collide field of another object.

3.  Place the TrackingAction script inside the Empty GameObject.

If you highlight the Empty GameObject and do a test-run in the editor with the triangular play button, the Empty GameObject may be invisible but if you look at the object's Position coordinates at the top of the Inspector panel, you can see them changing as you move the invisible cube around with your hand using the camera.

4.  Make another Empty GameObject called TriggerObject and give that a box collider too.  This is going to be the object that your TrackingAction-driven object collides with to trigger an event.

Also on this object, place a tick in the 'Is Trigger' box in the Inspector.  This tells Unity that something is supposed to be triggered when an object collides with the trigger-enabled object's collider field.

5.  In the TriggerObject object, create a JavaScript file called 'TriggerOn'.  Open the file in the Monodevelop script editor, erase the default block of code and paste the following script in.

function OnTriggerEnter (other: Collider) {
var go = GameObject.Find("MovingObject");
go.GetComponent(Event).Start();
}

Note: if the script you want to trigger uses an Update type function instead of a Start type function, write line 3 like this:

go.GetComponent(Event).Update();

This script basically tells Unity to go look for a script called Event inside your moving GameObject when the collider field is passed through by the moving object.

6.  Place a JavaScript script of the thing you want to trigger inside your MovingObject object and call the script Event.  When your moving object hits the collider field of the TriggerObject object, Unity will run the script inside MovingObject.

If you want Unity to run a script inside an object other than MovingObject, edit the script inside TriggerObject to give the name of that object, the name of the script and the function type at the start of that script (Start(), Update(), etc).

Hope that info is of some use to you!

[Serhiy_Posokhin]

Hi Marty!
Thanks for your instruction.

Say me, please, did you test tap, swipe and wave gestures in Unity? It's really work?

Best regards,
Serhiy

[MartyG]

Hi Serhiy,

I haven't tested those gestures in Unity.  Other hand gestures and facial recognitions I have tried have worked though, so I have no reason to think that the ones you mention wouldn't work too.

[Serhiy_Posokhin]

I have tested all gestures.
Tap, swipe, wave don't work.

Could you check them on your computer?

[Richard_F_2]

Tap works, I've tested it on my machine quite a bit.  Wave works, but it is a little twitchy, my general experience is you need to wave 3-4 times before it will pick it up.

Swipe?  Swipe is a jerk.  I can't get that gesture to work reliably on the gesture sample program, much less in Unity where I can't see my hand skeleton.

[Richard_F_2]

And while the video isn't much, here's a visual example of triggering some VFX using the the spread fingers gesture and turning it off when the hand is lost.

https://www.youtube.com/watch?v=zY4lW7o_H2s&feature=youtu.be

And I swear the video looked better on my machine before I uploaded it :P  Still figuring out Open Broadcaster.

[Serhiy_Posokhin]

Thanks Richard.

Nice visual example of triggering.