I thought I'd share a neat trick that my research into using interactive cloth in Unity with RealSense uncovered.
Normally, interactive cloth material is not movable with scripting unless you switch off the cloth function, change the rotation / position and re-activate the cloth physics. This means that if you put a 'TrackingAction' SDK script into a cloth object then the position and rotation values appear to be changing in the Inspector panel but the object itself is not actually moving.
However, I found that if you child-link a solid object to the cloth and also define that solid as an attachment in the 'Attached Colliders' section of the cloth's Inspector panel settings, then the cloth is then able to be moved by a 'TrackingAction' script inside it.
Here's a quick visual setup guide.
Go to the 'GameObject' menu at the top of the Unity window and select '3D Object' and 'Cloth.'
A 'Plane' type flat sheet of cloth material will be created by doing so.
Go to the 'Mesh' section of the cloth settings in the Inspector and click on the tiny circle icon at the end of the row labeled 'Mesh' to bring up a 'mesh selector' that enables you to change the shape of the cloth.
Click on the 'Sphere' button to change the flat Plane sheet into a round Sphere.
Go back to the 'GameObject' menu and this time select the '3D Object' option and then one of the solid 3D object types. We believe it is good for both cloth and solid object to be the same type if they are going to be inter-meshed, so we choose 'Sphere.'
The solid Sphere appears on the construction stage beside the cloth Sphere.
Select the 'Scaling' tool at the top of the Unity window and scale the size of the solid sphere down by a small amount so it is buried within the cloth sphere.
Go to the 'Attached Colliders' section of the cloth's settings in the Inspector panel. Place a '1' into the text box labeled 'Size' and press the Enter key to create an 'Element' slot. We are going to use this to attach the cloth firmly to the solid sphere inside it so that the two cannot pull apart.
Hold the left mouse button down on the solid and drag and drop it onto the text box beside the 'Collider' label (the one with the text 'None (Collider)' in it by default.
Make the solid object a child object of the cloth one by holding down the left mouse button on the solid and drag-and-dropping it onto the cloth in the left-hand Hierarchy panel of Unity.
This step is optional. We recommend adding an irregularly patterned texture to the outer cloth sphere so that you can clearly see whether it is moving if you are only using rotation-based movement to test it rather than positional movement. This is not really an issue if you are using any other shape, as it is easy to see such a shape rotating, whereas a sphere is always the same no matter which way it turns.
Add a 'TrackingAction' SDK script to the cloth. Go to the 'Constraints' section and place a tick in all of the Position constraints but leave the Rotation constraints unticked so that the cloth is free to rotate in any direction.
We are just using rotation-only settings as an example for this test. If you want to use Position settings in your own project, that is fine.
The set-up process is now complete. Run the program and move your hand. You should see the cloth rotating in all directions - something that would not normally have been possible without connecting the solid object to it.
A PRACTICAL EXAMPLE
Using this technique, we grafted a cloth version of our full-body avatar's upper arm flesh onto our avatar so that the camera-powered arms could touch the cloth and see it compress and rebound like real flesh.
We fixed the cloth flesh piece to the arm so that it would move and rotate with it by attaching the cloth to the solid spherical rotation joints of the upper and lower arm, so that the cloth was pinioned between them.
When we ran our game, the cloth flesh moved with the rest of the body but had a little trouble staying in perfect alignment with the solid flesh section on the lower arm. We'll keep working on it though to see if we can get the bugs out!
Aside from simulating muscle, skin and fat movement better, one can also imagine that other advantages of cloth could be harnessed. For instance, a medical simulation could make cuts in the cloth via the Tearing function to open up a passage to inner-body areas below the flesh piece. Or in an action game, bullets could puncture or nick the flesh. The possibilities are huge and exciting.
Best of luck with your projects!
I've just learned that Unity have removed Interactive Cloth physics in Unity 5, citing how costly it was in terms of processing resources. Only the simpler Skinned Cloth option remains, renamed simply as 'Cloth'.
There is currently little documentation on the new 'Cloth' in Unity 5. I will share my findings so far though.
* There is no longer the option to create a special cloth object from the GameObjects menu. Instead, you create a solid object and go to 'Component > Physics > Cloth' to add the 'Cloth' and 'Skinned Mesh Renderer' components to the solid.
Doing so seems to make the solid object invisible though. After some testing, I found a solution to make the cloth visible.
1. Go to the 'Skinned Mesh Renderer' settings in the Inspector.
2. Click on the tiny circle icon at the end of the 'Mesh' option row to bring up a mesh selector pop-up. Select a mesh type.
3. The cloth object will now be visible when you run the program.
Whilst the pink texture created by adding the mesh is rather unsightly, you can get rid of it just by dragging and dropping a texture onto the object.
* The new 'Cloth' can be affected by other objects equipped with colliders, but it cannot exert a two-way force back onto the colliding object in the way that cloth in Unity 4.6 and earlier could.
* You can only attach objects with Sphere and Capsule type colliders to the cloth. This is done with new 'Sphere Colliders' and 'Capsule Colliders' options at the base of the 'Cloth' component's settings in the Inspector panel.
* The new 'Cloth' material still responds to 'TrackingAction' scripts placed inside it, thankfully. You no longer need to attach a solid to the cloth like we had to in the original guide for Unity 4.6 and earlier.