Gestures Packages

[Rogério_G_]

hi,
It is to create new gestures packages in RealSense?
And if so how do I develop?

[Xusheng_L_Intel]

This is no such package or tool to create the new gesture now. Here is the sample code you can create your own gesture.

This code was developed by us as a sample of a possible implementation of your gesture, thinking about kids imitating the Muppet gesture.

1. For each finger the algorithm calculates the pinchness and checks that it isn’t folded.
2. The pinchness of each finger is the dot product of the hand’s plane normal and the vector from the center of the hand to the tip of the finger.
3. The pinchness value is the average of the pinchness of all the fingers.

        static float FOLD_RANGE = 0.01f;
        static float PINCH_RANGE = 0.01f;
        static float[] fingerLengths = {9.23f, 8.6f, 9.6f, 8.5f, 6.9f};

        private float norm(float x, float fromMin, float fromMax, float toMin, float toMax)
        {
            return  (x - fromMin) * (toMax - toMin) / (fromMax - fromMin) + toMin;
            
        }

        private Vector3D getVec(PXCMPoint3DF32 point)
        {
            return new Vector3D(point.x, point.y, point.z);
        }
        
        public float IsGesture(PXCMHandData.JointData[] jointData)
        {
            Vector3D vec0 = getVec(jointData[0].positionWorld); //wrist
            Vector3D vec1 = getVec(jointData[1].positionWorld); //center
            Vector3D vec2 = getVec(jointData[6].positionWorld); //index base

            float pinchness = 0;
            PXCMRangeF32 pinchRange = new PXCMRangeF32();
            Vector3D planeNormal = Vector3D.CrossProduct(vec1 - vec0, vec2 - vec0);
           
            for (int i = 0; i < PXCMHandData.NUMBER_OF_FINGERS; i++)
            {
                Vector3D baseVec = getVec(jointData[i * 4 + 2].positionWorld);
                Vector3D tipVec = getVec(jointData[i * 4 + 5].positionWorld);

                //check that foldedness of every finger isn't above FOLD_RANGE
                float fingerFoldness = Constants.fingerLengths - (float)(tipVec-baseVec).Length;
                fingerFoldness = norm(fingerFoldness , 0, Constants.fingerLengths, 0f, 1f);
                if (fingerFoldness > FOLD_RANGE)
                    return -1;

                //check pinch of every finger
                Vector3D centerToTip = tipVec - vec1;
                float fingerPinchness = 1 - (float)Vector3D.DotProduct(centerToTip, planeNormal);
                pinchRange.max = Math.Max(fingerPinchness, pinchRange.max);
                pinchRange.min = Math.Min(fingerPinchness, pinchRange.min);
                if ((pinchRange.max - pinchRange.min) > PINCH_RANGE)
                    return -1;
                pinchness += fingerPinchness * 0.2f;

            }
            return pinchness;
        }

[AndreCarlucci]

If you are a c# dev, here is an example of a punch using SharpSenses:

var cam = Camera.Create();
var mov = Movement.Forward(cam.LeftHand, 8);
mov.Completed += () => {
      Console.WriteLine("Left Punch!");
};
mov.Activate();
cam.Start();

Cheers!