Community
cancel
Showing results for 
Search instead for 
Did you mean: 
MartyG
Honored Contributor III
144 Views

Unity Tip: Switching RealSense Action Scripts On And Off With Scripting

Hi everyone,

You may come across situations in your Unity projects with the RealSense camera where you want to be able to switch the SDK's 'Action' scripts (such as TrackingAction and TranslateAction) on and off at will, and you need more options to do so than the built-in 'Start' and 'Stop' rule settings can provide.  There's good news if you are in that situation - you can control the on-off state of the C#-based Action scripts with scripting!

To do this, we need to create a new C#-format script   Whilst JavaScript can also be used to activate and deactivate other JavaScripts at will, it does not work with C# scripts like the Action ones.

Important note: You must place your script file inside the same Unity object that the Action script that you want to turn on or off is located in, otherwise Unity will not be able to locate the exact Action script that you want to affect.  This is the same principle as the 'SendMessageAction' SDK Action script, which must also be in the same object that it is making reference to..

Important note 2: when creating a new script file, ensure that you select 'CSharp' from the script creation interface's drop-down menu, otherwise you may create a JavaScript type file and find that your C# script will not run.

2.jpg

If you are in the habit of mostly using JavaScripts in your project, also make sure you switch the script setting back to 'JavaScript' as soon as you have finished making C# scripts or you will end up creating C# script files when you meant to make JavaScript!

So, continuing on ... C# is a bit more complicated than JavaScript, so I will post the code for my script and then explain what each part does.

*****

using UnityEngine;
using System.Collections;

public class TrackingAction_Off : MonoBehaviour
{

    private MonoBehaviour ScriptOnOff;


    void Start()
    {

        ScriptOnOff = GetComponent<TrackingAction>();

        ScriptOnOff.enabled = false;

    }

}

*****

The first two lines of the script - the 'Using' statements - are mandatory for C# scripts in Unity.  They simply tell Unity where to look for instructions that it needs to make the script run.  The lines will be created automatically every time you make a new script file.  So you can just leave those alone.

 

The third line is where it starts to get a bit more complicated.

public class TrackingAction_Off : MonoBehaviour

The word that you use directly after 'Public Class' is always the name that you gave your C# script file when you created it.  Unlike JavaScript, C# scripts care very much about what the name of your script is.  We chose to give our script the name 'Trackingaction_Off' to reflect its intended function of turning a TrackingAction off.  And so that is the name that we have to use at the end of 'Public Class'.

So when writing your own script, the format basically is:

public class NAMEOFMYSCRIPT : MonoBehaviour

A little further down, we are asked to define a variable whose contents will be accessed by the script further down in its code.

    private MonoBehaviour ScriptOnOff;

The first word after 'Private' tends to refer to the type of object that you want to turn on or off (e.g a light, a script or a graphics renderer.)  Unity usually refers to scripts by the name 'MonoBehavior', and as we want to turn a script on and off, that's the term we choose to use in our line of code.  If we wanted to toggle a light, we would have used 'Light' and if it was a renderer component, 'Renderer.'

The second part of the name is the name of a variable referring to what we want that variable to do.  It can be any word that you like.  For example, if the variable was going to be used to turn a light on and off, you could call it 'LightOnOff.'  But since we are altering the on-off state of a script, we simply chose to call it 'ScriptOnOff' - a name that tells us at a glance exactly what it does.

So the format basically is:

    private TYPEOFCOMPONENT NAMEOFYOURVARIABLE;

Next, we come to the type of function that the script is going to use.  If you have used JavaScript in Unity and are comfortable with using the terms 'Start()' (activate the script once) or 'Update()' (loop the script indefinitely until it is stopped somehow) then it is pretty similar in C#.  Except instead of having the word 'function' in front of the function type, you use the word 'void' instead.

    void Start()
    {

In our TrackingAction on-off script, we want the script to run once to turn the TrackingAction on or off and then stop.  We therefore use the 'Start()' type function.  If the 'Update()' function is used then when your project is run, the TrackingAction will toggle on and off rapidly as the script loops over and over.

Next, we use our custom-defined variable ('ScriptOnOff' in our case) for the first time.

     ScriptOnOff = GetComponent<TrackingAction>();

At the end of the line, you will notice the word 'TrackingAction'.  This is basically telling Unity the name of the C# script file that it needs to switch on or off.

So the format is:

   NAMEOFYOURVARIABLE= GetComponent<NAMEOFSCRIPTTOAFFECT>();

Note: if you want your on-off script to affect any C# script inside a particular object instead of just a single specific script, you can use the generic term 'MonoBehaviour' instead:

    ScriptOnOff = GetComponent<MonoBehaviour>();

Finally, in the next line, we can tell Unity whether to enable or disable the targeted Action script.

To activate the Action script, we use:

  ScriptOnOff.enabled = true;

And to deactivate it, we use:

  ScriptOnOff.enabled = false;

As usual, replace our variable name 'ScriptOnOff' with the name of your own custom variable name when writing your own script.  E.g.

NAMEOFVARIABLE.enabled = true;

You should create two separate versions of the script, one with the True statement in it and one with the 'False' statement in.  This will enable you to turn an Action script off, turn it back on and then switch it off again.

We called our pair of C# scripts 'TrackingAction_On' and 'TrackingAction_Off.'

1.jpg

Having got to this point, you will now be ready to run your project and test out the new scripts.

Ensure that your two on-off scripts are inactive by default by clicking on their tick-boxes to remove the ticks and make them empty.  Also ensure that the Action script has a tick beside it to show that it is active right from the start of running the project.

Select the object that your scripts are contained within and then use the triangular 'Play' run-button in the Unity editor.  This will let you access the script settings in the Inspector whilst the project is running.

3.jpg

Now left-click on the tick box beside your 'off' script.  The Action script's tick box will immediately un-tick, meaning that the Action script is now switched off and is no longer tracking your body!

4.jpg

Un-ticking the 'off' script and ticking the 'on' script turns the Action script back on.

5.jpg

Having the ability to activate and deactivate Action scripts would not be much use unless there was a way to activate the on and off scripts whilst the project is running in full mode instead of the editor preview.  Fortunately, the RealSense SDK provides us with an easy way to do this, via the 'SendMessageAction' Action script.

Add a SendMessageAction to the same object that your on / off C# scripts and the Action script  you are affecting are in.  

6.jpg

The type of event that you define in the SendMessageAction to activate your C# script is up to you - hand or face gesture, animation, etc.  The most important thing is that you place the function name of your script in the text box beside the 'Function Name' option (SendMessageAction locates scripts by their function name, not their file-name.)

Because we used a 'Start()' type function in our script, we must put Start into the text box.

7.jpg

If we want to have a Off script and an On script in the same object, SendMessageAction's approach to script recognition causes us some problems.  We can only have one script with the name Start or the SendMessageAction will not know which of the two scripts (On or Off) to trigger.  And C# is much fussier about function names than JavaScript is, and disables the tick-box if we use an alternate function name (e.g 'StartUp').

However, it turned out that things were not as bad as they looked.  Even though Unity removes the tick-box from the Inspector panel when a name other than Start is used, as long as you put that new variable name into the SendMessageAction then the gesture that you have defined as the script activation trigger will still run the script and turn your Action script on or off.  Problem averted.  Phew!

8.jpg

ALTERNATIVE TRIGGERS TO SENDMESSAGEACTION

An alternative trigger mechanism to using a SendMessageAction script is to give the script an 'OnTrigger' type function instead of a Start one.  With this function, the Action on or off script will be activated when an object enters, stays in or exits the collider field of an object.

The "Messages" section of the following Unity documentation page explains the different type of OnTrigger functions available.

http://docs.unity3d.com/ScriptReference/Collider.html

A version of our script that uses an OnTriggerEnter function (activating the on or off script when an object "enters" the collider field of another object) would look like this:

using UnityEngine;
using System.Collections;

public class TrackingAction_Off : MonoBehaviour
{

    private MonoBehaviour ScriptOnOff;


    void OnTriggerEnter(Collider other)
    {

        ScriptOnOff = GetComponent<TrackingAction>();

        ScriptOnOff.enabled = false;

    }

}

FINAL NOTE

Some of you may have JavaScripts that you wish to be able to activate and deactivate at will.  Whilst this version of the script cannot affect Action scripts, we will post it below for your benefit.  The JavaScript version is much simpler than its C# cousin.  And like the C# script, putting the JavaScript function name (Start) into SendMessageAction will also trigger the script.

function Start () {

GetComponent.<NAMEOFYOURSCRIPT>().enabled = false;

}

Or 

function Start () {

GetComponent.<NAMEOFYOURSCRIPT>().enabled = true;

}

For example:

function Start () {

GetComponent.<ElevatorControl_HallwayToGarden>().enabled = false;

}

If you have any questions about this subject, please post them in the comments below.  Best of luck!

0 Kudos
5 Replies
MartyG
Honored Contributor III
144 Views

Just wanted to add that switching movement-based Action scripts such as TrackingAction on and off can be used to temporarily create new types of movement animation not normally accessible to the user, like mixing paint colors together to make a new palette shade.  This mixing process can be termed as "scripting alchemy".  

For example, in my avatar's shoulder joints, the primary joint is constrained to only be able to move in the X axis direction (it is a secondary joint childed to it that enables the arm to move fully freely with precision in the X-Y-Z directions - the primary joint's purpose is to provide stability.)

However, if I add a second TrackingAction within the primary joint that is inactive by default and then turn that TrackingAction on with a particular gesture via a SendMessageAction script, the new axis mixes with the existing X axis to move the arm in a different direction for the duration that the second TrackingAction is active.  When the gesture is lost, the secondary TrackingAction switches off again and the arm's movement behavior returns to normal.

Scripting alchemy enables you to add additional actions to your existing controls without building a separate control mechanism for that special action.

MartyG
Honored Contributor III
144 Views

Further research yielded an interesting new use for the TrackingAction on-off technique in this article: simulating the movement of an avatar's hair strands with the hand.  

STEP ONE

Create strands of hair on the head of your avatar using normal solid objects (not cloth).

STEP TWO

Place a TrackingAction script inside each stand and set its constraints so that all Position constraints and the Z rotation constraint are locked (leaving the strand free to rotate left/right and up/down in the X and Y directions.  Set the TrackingActions to track the Center (palm) joint of the hand.

Untick each of the TrackingActions in the Inspector so that they are inactive by default.  This is because we only want the hair to move when the hand comes into contact with it.

STEP THREE

Place in each of the hair-strand objects a pair of the C# scripts for switching the TrackingAction in that object on and off.  Set the 'on' script to be an OnTriggerStay type function (so that the TrackingAction switches on when the hand enters the strand's collider field and remains on for the duration that the hand is in contact with the hair).

Then set the 'off' script to be an 'OnTriggerExit' type function (so that the TrackingAction is switched off again when the hand moves away from the hair strand and exits the strand's collide field.)

STEP FOUR

Place a Rigidbody physics component in the object that represents the hand that will be touching the hair-strand object (whether this is an actual model of a hand or some kind of cursor-type targeting object.)  The Rigidbody is what will cause the on and off scripts to activate when the hand containing the Rigidbody enters the hair strand's collider field.

STEP FIVE

Run your project and move the representation of the hand over the hair strand.  For the duration that the hand is in contact with the hair strands, their TrackingAction will switch on and start moving in response to tracking of the palm of the hand.  Because the hand is so close to the strand, it will give the visual illusion that it is the hand that is physically moving the hair, especially as the hair movement will cease when the hand moves away from it..

Edit: as a proof of concept, we expanded our initial hair experiment with a couple of strands into fully equipping our full-body avatar with TrackingAction-equipped hair strands that activate upon hand touch and move into new shapes.  It worked excellently.

movehair.jpg

MartyG
Honored Contributor III
144 Views

In another experiment, the TrackingAction on-off scripts were placed in the avatar's finger rotation joints, along with a Sphere type collider field set as an Is Trigger.

By doing so, the TrackingActions in the joints would turn off - stopping the fingers from bending - when the Sphere colliders came into contact with a Rigidbodied object such as the hand's palm, or a Rigidbodied object that the fingers were closing around.  The fingers would then re-activate their bending when the joint colliders were no longer touching the palm or an object.  This method gives the hand true physicality by stopping the fingers when they press against a surface instead of them passing through the object.  

It also makes the hand look better in general when the arm is in motion.  Because there is currently no easy way to stop TrackingAction-powered objects wrongly rotating when the real-life wrist is turned (even if the wrist joint is not set as the landmark), the fingers tend to flap around during arm movement as they bend and unbend on their own.  With these collision scripts in place, there is less finger-flapping because if the fingers auto-bend during arm movement then their TrackingActions switch off when the fingers bend into the Rigidbodied palm.

MartyG
Honored Contributor III
144 Views

If you have two or more joints whose tracking you want to turn on and off, you can batch-toggle the TrackingAction scripts by using a single long collider field as a 'nerve' that connects the joints by overlapping them.  Below, we illustrate a test where we connected a nerve between an upper arm joint of our avatar and the elbow joint of the same arm.

STEP ONE

We selected the upper arm joint and created an 'Empty Child' object (an object connected to the joint that is invisible because it has no physical form.)  It is created by going to the 'GameObject' menu of Unity and selecting the 'Create Empty Child' sub-option.

3.jpg

STEP TWO

Another trait of Empty GameObjects, aside from having no physical form, is that they do not have a collider field by default.  So we add one by going to the 'Component' menu and selecting the 'Physics' and 'Box Collider' sub-options.  

4.jpg

We use a Box type collider because it is the easiest type of collider to change the length and width.

STEP THREE

With the collider created, we edited the collider's X-Y-Z size settings in the Inspector panel until it was thin enough to be buried within the arm flesh (so that other objects outside of the arm cannot make contact with it)  and tall enough to encompass the upper and lower arm joints inside it.

5.jpg

1.jpg

We do not want our 'nerve' to activate and turn the TrackingActions in the arm joints on or off until we are ready to trigger it at will.  We therefore un-ticked the Box collider to render it inactive by default when the project is first run.

8.jpg

STEP FOUR

Next, we placed a Rigidbody physics component inside the Empty GameObject and set it with gravity and angular drag disabled and all Position and Rotation constraints locked with tick-marks.

2.jpg

Normally, locking all constraints on an object is something that we try to avoid because although it prevents the object from moving or rotating at all (keeping it fixed in position), it also makes other objects pass straight through it.  In cases like this where we are only using an object as a trigger mechanism and do not need it to collide solidly with another object, locking all constraints is fine.

What the Rigidbody does is to act as the "match" that will ignite the TrackingAction on-off scripts when our 'nerve' collider comes into contact with the colliders surrounding the arm joint.  Without the Rigidbody, that activation will not take place.

STEP FIVE

With the setup of our nerve now completed, we turned our attention to the arm joints.  With each of the two joints that were now immersed within the Empty GameObject's box collider field, we placed a tick in the 'Is Trigger' box of the joints' colliders to inform Unity that an event is supposed to be triggered when the collider field of another object (such as our Empty GameObject) interacts with the joint colliders.

6.jpg

STEP SIX

We placed the TrackingAction on and off scripts created at the top of this article inside both of our TrackingAction-powered arm joints.

7.jpg

STEP SEVEN

When we ran the project in the editor preview mode and selected the arm joints, we found that they were switched on by default.  This was because we had rendered the Box collider inactive, so that it could not affect the colliders around the arm joints even though the Box collider overlapped them.

9.jpg

We selected the Empty GameObject in the Hierarchy panel and ticked its Box collider to manually switch it on.  Our 'nerve' connecting the two joints was now active.

10.jpg

When we subsequently checked the two joints, both of their TrackingAction scripts had been simultaneously switched off, rendering tracking inactive.  Our connective nerve was a success and had batch-altered the status of two TrackingActions at the same time!

11.jpg

This technique can be expanded to additional TrackingAction powered objects.  Any such object that is within the Box collider's boundaries will have its TrackingAction deactivated or activated in a single action.

Furthermore, you can batch-activate or deactivate multiple nerves - a few of them, many of them or all of them - by using a script to define a list of Empty GameObjects to have their colliders turned on or off in a single action.  

Just a few example applications of this could be a medical tool for exploring nerve surgery techniques; a training tool for people working in the field of physical disability so that they can experience the effects of a particular limb disability with a realistic avatar; or an injury simulator for a military infantry simulator or first-person shooter game where being injured in particular body areas impairs or disables those body sections.

 

MartyG
Honored Contributor III
144 Views

Whilst switching batches of TrackingAction scripts on and off using 'nerve' connections between objects is fun, we subsequently found a more efficient way of defining enable / disable lists for C# scripts (e.g TrackingActions) inside specific objects.  Here's the template of the C# script.

using UnityEngine;
using System.Collections;

public class NAME-OF-YOUR-SCRIPT : MonoBehaviour
{
	
	public NAME-OF-1ST-SCRIPT-TO-AFFECT VARIABLE NAME 1;
	public NAME-OF-2ND-SCRIPT-TO-AFFECT VARIABLE NAME 2;
	
	
	void Start()
		
	{
		
VARIABLE NAME 1 = GameObject.Find(NAME-OF-OBJECT-THE-SCRIPT-TO-AFFECT-IS-IN").GetComponent<TrackingAction>();
VARIABLE NAME 1.enabled = false;		
VARIABLE NAME 2 = GameObject.Find("NAME-OF-OBJECT-THE-SCRIPT-TO-AFFECT-IS-IN").GetComponent<TrackingAction2>();
VARIABLE NAME 2.enabled = false;
		
	}
}

As a reference, here is a version of the script from our actual project.  When activated, it turns off the TrackingAction script and a copy of that script called TrackingAction2 that reside inside a specific object in our avatar's structure named "Shoulder Blade Right - Rotation Joint".

using UnityEngine;
using System.Collections;

public class TrackingAction_Off : MonoBehaviour
{
	
	public TrackingAction ScriptOff;
	public TrackingAction2 ScriptOff_2;
	
	
	void Start()
		
	{
		
		ScriptOff = GameObject.Find("Shoulder Blade Right - Rotation Joint ").GetComponent<TrackingAction>();
		
		ScriptOff.enabled = false;

		
		ScriptOff_2 = GameObject.Find("Shoulder Blade Right - Rotation Joint ").GetComponent<TrackingAction2>();
		
		ScriptOff_2.enabled = false;
		
	}
}

.Let's break the script down step by step.

public class TrackingAction_Off : MonoBehaviour
{

The word after 'public class' should be the exact same file-name that you have given your C# script file.

	public TrackingAction ScriptOff;
	public TrackingAction2 ScriptOff_2;
	

Here, you define a list of variable names relating to the list of scripts that you want to turn on or off.  Each object that you are targeting should have its own unique variable name so that Unity does not get confused.  So if you have five TrackingActions in your project then you will define a list of five variable names.  If there are ten TrackingActions that you want to target then define a list of ten variable names, and so on.

In the first half of each statement after the word 'public', put the name of the script that you want to turn on or off.  In our example, we want to affect 'TrackingAction' and 'TrackingAction2', so the first variable in the list is dedicated to 'TrackingAction' and the second is dedicated to 'TrackingAction2.'

In the second half of the statement, create a unique variable name that you want to link to a specific script.  As this is a script to switch off a C# script such as a TrackingAction, we simply use the names 'ScriptOff' and 'ScriptOff_2' for the two items in our list.

void Start()
		
	{

The 'Start' type function tells Unity to run the script once and then stop processing it.  If it used the 'Update' function type then the script would keep looping indefinitely, which we do not want.

		ScriptOff = GameObject.Find("Shoulder Blade Right - Rotation Joint ").GetComponent<TrackingAction>();
		
		ScriptOff.enabled = false;

		
		ScriptOff = GameObject.Find("Shoulder Blade Right - Rotation Joint ").GetComponent<TrackingAction2>();
		
		ScriptOff.enabled = false;

In this final section of code, we tell Unity how to find the object that each specific script we are targeting for activation or deactivation is located in.  Because our list has two objects, we have two sets of object references (2 blocks of 2-line code.)  If you want to add further object locations to your list then you just paste in further 2-line sets.  For example:

ScriptOff = GameObject.Find("Shoulder Blade Right - Rotation Joint ").GetComponent<TrackingAction>();
		
		ScriptOff.enabled = false;

		ScriptOff = GameObject.Find("Shoulder Blade Right - Rotation Joint ").GetComponent<TrackingAction2>();
		
		ScriptOff_2.enabled = false;

		ScriptOff = GameObject.Find("Shoulder Blade Right - Rotation Joint ").GetComponent<TrackingAction3>();
		
		ScriptOff_3.enabled = false;

In our example, the scripts that we are seeking are all located inside the same object.  If you want to target scripts that are in different objects then you just have to change the name of the object in quotation marks after 'GameObject.Find.'

For example:

ScriptOff = GameObject.Find("Shoulder Blade Right - Rotation Joint ").GetComponent<TrackingAction>();
		
		ScriptOff.enabled = false;

		ScriptOff = GameObject.Find("Shoulder Blade Left - Rotation Joint ").GetComponent<TrackingAction2>();
		
		ScriptOff_2.enabled = false;

		ScriptOff = GameObject.Find("Left Lower Arm - Rotation Joint ").GetComponent<TrackingAction3>();
		
		ScriptOff_3.enabled = false;

If you want your script to activate the targeted scripts, such as TrackingActions, then change the word 'false' to 'true':  E.g

		ScriptOff_3.enabled = true;

You do not have to have all-enable and all-disable scripts.  Depending on the needs of your project, you could have some scripts be disabled and some of them enabled.  E.g:

ScriptOff = GameObject.Find("Shoulder Blade Right - Rotation Joint ").GetComponent<TrackingAction>();
		
		ScriptOff.enabled = false;

		ScriptOff = GameObject.Find("Shoulder Blade Right - Rotation Joint ").GetComponent<TrackingAction2>();
		
		ScriptOff_2.enabled = true;

In our project, we have been using JavaScript versions of such activation-deactivation lists for a long time.  JavaScript on-off calls do not work with C#-based scripts like the Action scripts supplied in the RealSense SDK though.  So it is exciting for us that we have now worked out how to program complex configurations of on-off sequences in C# so that we can really get precise control over the camera!

Reply