Intel® Distribution of OpenVINO™ Toolkit
Community support and discussions about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all things computer vision-related on Intel® platforms.
5961 Discussions

How to generate a heatmap using OpenVINO? (Layer-wise relevance, Grad-CAM?)

AHewi2
Beginner
1,210 Views

We use OpenVINO C++ Package for our inference tasks. Our users require a heatmap visualization to gain trust in the classifications of our CNN. Can a heatmap generation technique like "layer-wise relevance propagation" or "Grad-CAM" be implemented using OpenVINO?

0 Kudos
1 Solution
MAURICIOAL_R_Intel
Moderator
1,152 Views

Hi Alexander,

 

You can use class activation maps to visualize the "decision-making." In order to calculate Class Activation Maps in OpenVINO, you need to access the output feature maps of the last layer and the corresponding weights. This can be done through the CNNNetwork: addOutput() function call when initializing the network. After running the inference, you calculate the weighted sum of the weights with the feature maps and use this to generate the visualization.

 

Regards,

Mauricio R.

View solution in original post

5 Replies
MAURICIOAL_R_Intel
Moderator
1,152 Views

Thanks for reaching out.

 

The following project shows how to implement a heatmap visualization for a people counter using OpenVINO™ toolkit: https://github.com/intel-iot-devkit/store-aisle-monitor-cpp

 

Regards,

Mauricio R.

AHewi2
Beginner
1,152 Views

Hi Mauricio! Thank you for the quick response, but the people counter project uses a accumulation buffer to generate a motion heatmap. The generated heatmap has nothing to do with the neural network and its decisions.

 

What my users need, is a heatmap showing the pixelwise relevance/ importance of the input image for the result of the classification, as described at www.heatmapping.org

In the best case, if the network is working as expected, the heatmap shows hotspots in the input image where faults are located. That way the user can gain trust in the networks decisions.

MAURICIOAL_R_Intel
Moderator
1,152 Views

Hi Alexander,

We are consulting this with engineering, we will get back to you as soon as we get an answer.

 

Regards,

Mauricio R.

MAURICIOAL_R_Intel
Moderator
1,153 Views

Hi Alexander,

 

You can use class activation maps to visualize the "decision-making." In order to calculate Class Activation Maps in OpenVINO, you need to access the output feature maps of the last layer and the corresponding weights. This can be done through the CNNNetwork: addOutput() function call when initializing the network. After running the inference, you calculate the weighted sum of the weights with the feature maps and use this to generate the visualization.

 

Regards,

Mauricio R.

AHewi2
Beginner
1,105 Views

Hi Mauricio!

Just wanted to let you know I was  able to implement class activation mapping based on OpenVINO. Selecting the correct weights and activations from the CNNNetwork is not very convenient, but I managed it in the end. Thanks for your help.

1.png

Reply