How to control UI in Unity with SR300

[春董]

I saw the demo of "FF_TouchlessControllerViewer.cs". It can control the UI by hand. This application was developed by C# , I want to ask if this function can be realized in Unity.

[MartyG · Accepted solution]

The documentation page for the Touchless Controller does list a 'C# Unity' reference for the Controller, though this is no guarantee that it still works. Indeed, the Touchless Controller was removed from the current '2016 R3' SDK, so you would have to use '2016 R2' in order to be able to access the Touchless Controller.

https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/pxctouchlesscontroller.html TouchlessController

The functions of most RealSense C# and C++ sample programs can be recreated in Unity if you have the time and programming knowledge to do so.

In my own Unity project, I simulate the functions of the Touchless Controller by using a couple of other scripts that come pre-made in the RealSense Unity Toolkit - 'TrackingAction' for moving the cursor, and 'SendMessageAction' for triggering actions such as a mouse click when a particular gesture is made with the hand.

[春董]

Thank you ! Your answer gave me some ideas. I'll try to use 'TrackingAction' and 'SendMessageAction' to control my GUI in Unity.

[MartyG]

You're very welcome. If you are using the '2016 R2' SDK, I have authored a large range of guides to using RealSense in Unity with R2.

https://software.intel.com/en-us/forums/realsense/topic/676139 Index of Marty G's RealSense Unity How-To Guides

[idata]

Hi shirley_dong,

Let us know if you were able to control the GUI in Unity with the answer provided by MartyG, we'll be looking forward to your response. Have a great day! Regards, Leonardo L

[春董]

I tried，but it did not succeed yet .

I will give me 4 days to studied this.

And now I am thinking if it can controlled by voice.

[春董]

I have not solved my question. Can you give me a little demo. I want to know how you controlled the GUI in detail.

I am a student of Tianjin University of Science & Technology. Hope your answer!

[MartyG]

This was the original guide to creating a camera-controlled menu in the '2016 R2' SDK and Unity.

https://software.intel.com/en-us/forums/realsense/topic/587339 Unity TIp: Creating a RealSense-Powered Menu In Unity

Later on, I helped someone to create gesture-controlled activation for the menu options using SendMessageAction.