Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.
6415 Discussions

How to generate a heatmap using OpenVINO? (Layer-wise relevance, Grad-CAM?)

AHewi2
Beginner
2,748 Views

We use OpenVINO C++ Package for our inference tasks. Our users require a heatmap visualization to gain trust in the classifications of our CNN. Can a heatmap generation technique like "layer-wise relevance propagation" or "Grad-CAM" be implemented using OpenVINO?

0 Kudos
1 Solution
MAURICIOAL_R_Intel
Moderator
2,690 Views

Hi Alexander,

 

You can use class activation maps to visualize the "decision-making." In order to calculate Class Activation Maps in OpenVINO, you need to access the output feature maps of the last layer and the corresponding weights. This can be done through the CNNNetwork: addOutput() function call when initializing the network. After running the inference, you calculate the weighted sum of the weights with the feature maps and use this to generate the visualization.

 

Regards,

Mauricio R.

View solution in original post

0 Kudos
5 Replies
MAURICIOAL_R_Intel
Moderator
2,690 Views

Thanks for reaching out.

 

The following project shows how to implement a heatmap visualization for a people counter using OpenVINO™ toolkit: https://github.com/intel-iot-devkit/store-aisle-monitor-cpp

 

Regards,

Mauricio R.

0 Kudos
AHewi2
Beginner
2,690 Views

Hi Mauricio! Thank you for the quick response, but the people counter project uses a accumulation buffer to generate a motion heatmap. The generated heatmap has nothing to do with the neural network and its decisions.

 

What my users need, is a heatmap showing the pixelwise relevance/ importance of the input image for the result of the classification, as described at www.heatmapping.org

In the best case, if the network is working as expected, the heatmap shows hotspots in the input image where faults are located. That way the user can gain trust in the networks decisions.

0 Kudos
MAURICIOAL_R_Intel
Moderator
2,690 Views

Hi Alexander,

We are consulting this with engineering, we will get back to you as soon as we get an answer.

 

Regards,

Mauricio R.

0 Kudos
MAURICIOAL_R_Intel
Moderator
2,691 Views

Hi Alexander,

 

You can use class activation maps to visualize the "decision-making." In order to calculate Class Activation Maps in OpenVINO, you need to access the output feature maps of the last layer and the corresponding weights. This can be done through the CNNNetwork: addOutput() function call when initializing the network. After running the inference, you calculate the weighted sum of the weights with the feature maps and use this to generate the visualization.

 

Regards,

Mauricio R.

0 Kudos
AHewi2
Beginner
2,643 Views

Hi Mauricio!

Just wanted to let you know I was  able to implement class activation mapping based on OpenVINO. Selecting the correct weights and activations from the CNNNetwork is not very convenient, but I managed it in the end. Thanks for your help.

1.png

0 Kudos
Reply