Direct Gradient Extraction in OpenVINO IR Format

[Xanph]

Context:

A convolutional network is making predictions on whether there is motion occurring on a live stream. This is running on a lightweight (resource-wise) edge device.

Problem & Goal:

Needing the model to pinpoint where the 'motion' (class) is occurring that influenced the model's prediction.

Targeted Possible Solution:

Use the gradients outputted by the third convolution layer to generate Grad-CAM visualisations (a heatmap over the image of the motion class activation).

---

The problem, however, is that OpenVINO IR does not allow for direct access to gradients (based on my research), because the model is designed for efficient inference, understandably.

As a result, the original keras model has been modified to contain the third convolution layer as a second output. So we now have a binary output for motion and a feature map output from the third conv layer.

I then need to approximate what the gradients are by using the Central Differencing Scheme numerical calculation. This requires perturbing the input, re-running inference twice to then calculate the estimated gradient, like so:

(python)

def numerical_gradient(infer_request, input_data, output_index, h=1e-3
    grad = np.zeros_like(input_data)
    for i in range(input_data.size):
        input_data_plus_h = np.copy(input_data)
        input_data_minus_h = np.copy(input_data)
        input_data_plus_h.flat[i] += h
        input_data_minus_h.flat[i] -= h

        # Run inference with perturbed inputs
        infer_request.infer(inputs={input_layer: input_data_plus_h})
        f_x_plus_h = infer_request.get_tensor(dense_output_layer).data
        infer_request.infer(inputs={input_layer: input_data_minus_h})
        f_x_minus_h = infer_request.get_tensor(dense_output_layer).data

        if f_x_plus_h.ndim == 3:
            f_x_plus_h = f_x_plus_h[0, -1, 0]

        if f_x_minus_h.ndim == 3:
            f_x_minus_h = f_x_minus_h[0, -1, 0]

        grad.flat[i] = (f_x_plus_h - f_x_minus_h) / (2 * h)
    return grad

So now the problem is that this implementation involves three inference steps, for each frame inside an LSTM sequence of 50, on a live stream - not very efficient:

• First Inference: The initial inference to get the prediction.
• Second Inference: A perturbed input with a small positive perturbation for gradient calculation.
• Third Inference: A perturbed input with a small negative perturbation for gradient calculation.

As an alternative, I could try using Forward Finite Difference, where it only needs a second inference, but this still isn't the best for an edge device with limited GPU resources.

I wanted to see if there are any other suggestions on how I can reach my goal of getting the model to state where the motion class is appearing in the scene. Alternatively, I can fall back on to background subtraction, for getting the positions.

I also appreciate that there's quite a lot of math in this, for which I am learning on the spot about too!

Many thanks,

Xanph

(Using OpenVINO Nightly)

[Vipin_Singh1]

Hi Flynn, could you please provide us with the following details?

• The exact name and build version of the Intel® Toolkit you’re using.
• The operating system and its build version.
• Whether the product has been installed.
• A detailed explanation of your query, along with a screenshot if possible.

To assist you further, we would require these details.

[Iffa_Intel]

Hi,

If you still need help with this issue,

Please help to clarify & share :

1. Your model framework (eg: Tensorflow,ONNX,etc)
2. Does the model is a custom model? Could you elaborate the custom part?
3. Your conversion commands
4. Relevant model files
5. Which OpenVINO sample app did you use for inferencing? (if custom please share the specific code)
6. Your issue right now is, you are not satisfied with the model's inferencing result and wish to improvise am I right?

Cordially,

Iffa

[Iffa_Intel]

Hi,

Thank you for your question. If you need any additional information from Intel, please submit a new question as Intel is no longer monitoring this thread. 

Cordially,

Iffa