Hi everyone,
You may come across situations in your Unity projects with the RealSense camera where you want to be able to switch the SDK's 'Action' scripts (such as TrackingAction and TranslateAction) on and off at will, and you need more options to do so than the built-in 'Start' and 'Stop' rule settings can provide. There's good news if you are in that situation - you can control the on-off state of the C#-based Action scripts with scripting!
To do this, we need to create a new C#-format script Whilst JavaScript can also be used to activate and deactivate other JavaScripts at will, it does not work with C# scripts like the Action ones.
Important note: You must place your script file inside the same Unity object that the Action script that you want to turn on or off is located in, otherwise Unity will not be able to locate the exact Action script that you want to affect. This is the same principle as the 'SendMessageAction' SDK Action script, which must also be in the same object that it is making reference to..
Important note 2: when creating a new script file, ensure that you select 'CSharp' from the script creation interface's drop-down menu, otherwise you may create a JavaScript type file and find that your C# script will not run.
If you are in the habit of mostly using JavaScripts in your project, also make sure you switch the script setting back to 'JavaScript' as soon as you have finished making C# scripts or you will end up creating C# script files when you meant to make JavaScript!
So, continuing on ... C# is a bit more complicated than JavaScript, so I will post the code for my script and then explain what each part does.
*****
using UnityEngine;
using System.Collections;
public class TrackingAction_Off : MonoBehaviour
{
private MonoBehaviour ScriptOnOff;
void Start()
{
ScriptOnOff = GetComponent<TrackingAction>();
ScriptOnOff.enabled = false;
}
}
*****
The first two lines of the script - the 'Using' statements - are mandatory for C# scripts in Unity. They simply tell Unity where to look for instructions that it needs to make the script run. The lines will be created automatically every time you make a new script file. So you can just leave those alone.
The third line is where it starts to get a bit more complicated.
public class TrackingAction_Off : MonoBehaviour
The word that you use directly after 'Public Class' is always the name that you gave your C# script file when you created it. Unlike JavaScript, C# scripts care very much about what the name of your script is. We chose to give our script the name 'Trackingaction_Off' to reflect its intended function of turning a TrackingAction off. And so that is the name that we have to use at the end of 'Public Class'.
So when writing your own script, the format basically is:
public class NAMEOFMYSCRIPT : MonoBehaviour
A little further down, we are asked to define a variable whose contents will be accessed by the script further down in its code.
private MonoBehaviour ScriptOnOff;
The first word after 'Private' tends to refer to the type of object that you want to turn on or off (e.g a light, a script or a graphics renderer.) Unity usually refers to scripts by the name 'MonoBehavior', and as we want to turn a script on and off, that's the term we choose to use in our line of code. If we wanted to toggle a light, we would have used 'Light' and if it was a renderer component, 'Renderer.'
The second part of the name is the name of a variable referring to what we want that variable to do. It can be any word that you like. For example, if the variable was going to be used to turn a light on and off, you could call it 'LightOnOff.' But since we are altering the on-off state of a script, we simply chose to call it 'ScriptOnOff' - a name that tells us at a glance exactly what it does.
So the format basically is:
private TYPEOFCOMPONENT NAMEOFYOURVARIABLE;
Next, we come to the type of function that the script is going to use. If you have used JavaScript in Unity and are comfortable with using the terms 'Start()' (activate the script once) or 'Update()' (loop the script indefinitely until it is stopped somehow) then it is pretty similar in C#. Except instead of having the word 'function' in front of the function type, you use the word 'void' instead.
void Start()
{
In our TrackingAction on-off script, we want the script to run once to turn the TrackingAction on or off and then stop. We therefore use the 'Start()' type function. If the 'Update()' function is used then when your project is run, the TrackingAction will toggle on and off rapidly as the script loops over and over.
Next, we use our custom-defined variable ('ScriptOnOff' in our case) for the first time.
ScriptOnOff = GetComponent<TrackingAction>();
At the end of the line, you will notice the word 'TrackingAction'. This is basically telling Unity the name of the C# script file that it needs to switch on or off.
So the format is:
NAMEOFYOURVARIABLE= GetComponent<NAMEOFSCRIPTTOAFFECT>();
Note: if you want your on-off script to affect any C# script inside a particular object instead of just a single specific script, you can use the generic term 'MonoBehaviour' instead:
ScriptOnOff = GetComponent<MonoBehaviour>();
Finally, in the next line, we can tell Unity whether to enable or disable the targeted Action script.
To activate the Action script, we use:
ScriptOnOff.enabled = true;
And to deactivate it, we use:
ScriptOnOff.enabled = false;
As usual, replace our variable name 'ScriptOnOff' with the name of your own custom variable name when writing your own script. E.g.
NAMEOFVARIABLE.enabled = true;
You should create two separate versions of the script, one with the True statement in it and one with the 'False' statement in. This will enable you to turn an Action script off, turn it back on and then switch it off again.
We called our pair of C# scripts 'TrackingAction_On' and 'TrackingAction_Off.'
Having got to this point, you will now be ready to run your project and test out the new scripts.
Ensure that your two on-off scripts are inactive by default by clicking on their tick-boxes to remove the ticks and make them empty. Also ensure that the Action script has a tick beside it to show that it is active right from the start of running the project.
Select the object that your scripts are contained within and then use the triangular 'Play' run-button in the Unity editor. This will let you access the script settings in the Inspector whilst the project is running.
Now left-click on the tick box beside your 'off' script. The Action script's tick box will immediately un-tick, meaning that the Action script is now switched off and is no longer tracking your body!
Un-ticking the 'off' script and ticking the 'on' script turns the Action script back on.
Having the ability to activate and deactivate Action scripts would not be much use unless there was a way to activate the on and off scripts whilst the project is running in full mode instead of the editor preview. Fortunately, the RealSense SDK provides us with an easy way to do this, via the 'SendMessageAction' Action script.
Add a SendMessageAction to the same object that your on / off C# scripts and the Action script you are affecting are in.
The type of event that you define in the SendMessageAction to activate your C# script is up to you - hand or face gesture, animation, etc. The most important thing is that you place the function name of your script in the text box beside the 'Function Name' option (SendMessageAction locates scripts by their function name, not their file-name.)
Because we used a 'Start()' type function in our script, we must put Start into the text box.
If we want to have a Off script and an On script in the same object, SendMessageAction's approach to script recognition causes us some problems. We can only have one script with the name Start or the SendMessageAction will not know which of the two scripts (On or Off) to trigger. And C# is much fussier about function names than JavaScript is, and disables the tick-box if we use an alternate function name (e.g 'StartUp').
However, it turned out that things were not as bad as they looked. Even though Unity removes the tick-box from the Inspector panel when a name other than Start is used, as long as you put that new variable name into the SendMessageAction then the gesture that you have defined as the script activation trigger will still run the script and turn your Action script on or off. Problem averted. Phew!
ALTERNATIVE TRIGGERS TO SENDMESSAGEACTION
An alternative trigger mechanism to using a SendMessageAction script is to give the script an 'OnTrigger' type function instead of a Start one. With this function, the Action on or off script will be activated when an object enters, stays in or exits the collider field of an object.
The "Messages" section of the following Unity documentation page explains the different type of OnTrigger functions available.
http://docs.unity3d.com/ScriptReference/Collider.html
A version of our script that uses an OnTriggerEnter function (activating the on or off script when an object "enters" the collider field of another object) would look like this:
using UnityEngine;
using System.Collections;
public class TrackingAction_Off : MonoBehaviour
{
private MonoBehaviour ScriptOnOff;
void OnTriggerEnter(Collider other)
{
ScriptOnOff = GetComponent<TrackingAction>();
ScriptOnOff.enabled = false;
}
}
FINAL NOTE
Some of you may have JavaScripts that you wish to be able to activate and deactivate at will. Whilst this version of the script cannot affect Action scripts, we will post it below for your benefit. The JavaScript version is much simpler than its C# cousin. And like the C# script, putting the JavaScript function name (Start) into SendMessageAction will also trigger the script.
function Start () {
GetComponent.<NAMEOFYOURSCRIPT>().enabled = false;
}
Or
function Start () {
GetComponent.<NAMEOFYOURSCRIPT>().enabled = true;
}
For example:
function Start () {
GetComponent.<ElevatorControl_HallwayToGarden>().enabled = false;
}
If you have any questions about this subject, please post them in the comments below. Best of luck!
Link Copied
Further research yielded an interesting new use for the TrackingAction on-off technique in this article: simulating the movement of an avatar's hair strands with the hand.
STEP ONE
Create strands of hair on the head of your avatar using normal solid objects (not cloth).
STEP TWO
Place a TrackingAction script inside each stand and set its constraints so that all Position constraints and the Z rotation constraint are locked (leaving the strand free to rotate left/right and up/down in the X and Y directions. Set the TrackingActions to track the Center (palm) joint of the hand.
Untick each of the TrackingActions in the Inspector so that they are inactive by default. This is because we only want the hair to move when the hand comes into contact with it.
STEP THREE
Place in each of the hair-strand objects a pair of the C# scripts for switching the TrackingAction in that object on and off. Set the 'on' script to be an OnTriggerStay type function (so that the TrackingAction switches on when the hand enters the strand's collider field and remains on for the duration that the hand is in contact with the hair).
Then set the 'off' script to be an 'OnTriggerExit' type function (so that the TrackingAction is switched off again when the hand moves away from the hair strand and exits the strand's collide field.)
STEP FOUR
Place a Rigidbody physics component in the object that represents the hand that will be touching the hair-strand object (whether this is an actual model of a hand or some kind of cursor-type targeting object.) The Rigidbody is what will cause the on and off scripts to activate when the hand containing the Rigidbody enters the hair strand's collider field.
STEP FIVE
Run your project and move the representation of the hand over the hair strand. For the duration that the hand is in contact with the hair strands, their TrackingAction will switch on and start moving in response to tracking of the palm of the hand. Because the hand is so close to the strand, it will give the visual illusion that it is the hand that is physically moving the hair, especially as the hair movement will cease when the hand moves away from it..
Edit: as a proof of concept, we expanded our initial hair experiment with a couple of strands into fully equipping our full-body avatar with TrackingAction-equipped hair strands that activate upon hand touch and move into new shapes. It worked excellently.
If you have two or more joints whose tracking you want to turn on and off, you can batch-toggle the TrackingAction scripts by using a single long collider field as a 'nerve' that connects the joints by overlapping them. Below, we illustrate a test where we connected a nerve between an upper arm joint of our avatar and the elbow joint of the same arm.
STEP ONE
We selected the upper arm joint and created an 'Empty Child' object (an object connected to the joint that is invisible because it has no physical form.) It is created by going to the 'GameObject' menu of Unity and selecting the 'Create Empty Child' sub-option.
STEP TWO
Another trait of Empty GameObjects, aside from having no physical form, is that they do not have a collider field by default. So we add one by going to the 'Component' menu and selecting the 'Physics' and 'Box Collider' sub-options.
We use a Box type collider because it is the easiest type of collider to change the length and width.
STEP THREE
With the collider created, we edited the collider's X-Y-Z size settings in the Inspector panel until it was thin enough to be buried within the arm flesh (so that other objects outside of the arm cannot make contact with it) and tall enough to encompass the upper and lower arm joints inside it.
We do not want our 'nerve' to activate and turn the TrackingActions in the arm joints on or off until we are ready to trigger it at will. We therefore un-ticked the Box collider to render it inactive by default when the project is first run.
STEP FOUR
Next, we placed a Rigidbody physics component inside the Empty GameObject and set it with gravity and angular drag disabled and all Position and Rotation constraints locked with tick-marks.
Normally, locking all constraints on an object is something that we try to avoid because although it prevents the object from moving or rotating at all (keeping it fixed in position), it also makes other objects pass straight through it. In cases like this where we are only using an object as a trigger mechanism and do not need it to collide solidly with another object, locking all constraints is fine.
What the Rigidbody does is to act as the "match" that will ignite the TrackingAction on-off scripts when our 'nerve' collider comes into contact with the colliders surrounding the arm joint. Without the Rigidbody, that activation will not take place.
STEP FIVE
With the setup of our nerve now completed, we turned our attention to the arm joints. With each of the two joints that were now immersed within the Empty GameObject's box collider field, we placed a tick in the 'Is Trigger' box of the joints' colliders to inform Unity that an event is supposed to be triggered when the collider field of another object (such as our Empty GameObject) interacts with the joint colliders.
STEP SIX
We placed the TrackingAction on and off scripts created at the top of this article inside both of our TrackingAction-powered arm joints.
STEP SEVEN
When we ran the project in the editor preview mode and selected the arm joints, we found that they were switched on by default. This was because we had rendered the Box collider inactive, so that it could not affect the colliders around the arm joints even though the Box collider overlapped them.
We selected the Empty GameObject in the Hierarchy panel and ticked its Box collider to manually switch it on. Our 'nerve' connecting the two joints was now active.
When we subsequently checked the two joints, both of their TrackingAction scripts had been simultaneously switched off, rendering tracking inactive. Our connective nerve was a success and had batch-altered the status of two TrackingActions at the same time!
This technique can be expanded to additional TrackingAction powered objects. Any such object that is within the Box collider's boundaries will have its TrackingAction deactivated or activated in a single action.
Furthermore, you can batch-activate or deactivate multiple nerves - a few of them, many of them or all of them - by using a script to define a list of Empty GameObjects to have their colliders turned on or off in a single action.
Just a few example applications of this could be a medical tool for exploring nerve surgery techniques; a training tool for people working in the field of physical disability so that they can experience the effects of a particular limb disability with a realistic avatar; or an injury simulator for a military infantry simulator or first-person shooter game where being injured in particular body areas impairs or disables those body sections.
