How can we verify and make sure our model running in GNA?

[Roxy1]

Hello,

Acer can run own speech model in Live Speech Recognition Demo app(CPU information: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz). While selecting engine GNA, it can be inferenced. But how can we verify and make sure our model running in GNA? Such as cmd to query GNA status or others?
Any advices from Intel side will do good help for us, many thanks.

[Wan_Intel]

Hi Roxy1,

Thanks for reaching out to us.

You can observe the device info in the command prompt.

When the CPU is selected for running the Live Speech Recognition Demo, MKLDNNPlugin is loaded.

While GNA is selected for running the demo, you can observe that MKLDNNPlugin and GNAPlugin are loaded as GNA plugin is not intended to replace CPU but is designed to save power and free CPU resources.

Regards,

Wan

[Roxy1]

Hi Wan,

We can check it from "[ INFO ] GNA: GNAPlugin ver. 2.1", great!

Can I double confirm that this log information get from HW layer?

[Wan_Intel]

Hi Roxy1,

Thanks for your patience.

By default, there are several execution modes in this live speech recognition demo:

·      CPU – All calculations are performed on a CPU device using a CPU Plugin.

·      GPU – All calculations are performed on a GPU device using GPU Plugin.

·      GNA – GNA hardware is used if available and the driver is installed. Otherwise, the GNA device is emulated in fast-but-not-bit-exact mode.

For your information, the option of GNA in this demo is GNA_AUTO. You may refer to the speech_engine.py file in Line 59 and Line 60. The file location of speech_engine.py is located under the following directory:

<path_to_live_speech_recognition_demo>\live_speech_recognition_demo\speech_library\

However, there are other execution modes that you may use in this demo. For example:

·      GNA_HW – GNA hardware is used if available and the driver is installed. Otherwise, an error will occur.

·      GNA_SW_FP32 – Substitutes parameters and calculations from low precision to floating-point (FP32).

·      GNA_SW_EXACT – GNA device is emulated in bit-exact mode.

You may have these extra execution modes in the demo by adding these execution modes to Line 75 in the main_controller.py file. The file location of main_controller.py is located under the following directory:

<path_to_live_speech_recognition_demo>\live_speech_recognition_demo\gui\

For example:

self._frame.set_engine_choice(["CPU", "GNA", "GPU", "GNA_HW"], 0)

On another note, full information on GNA Plugin is available at the following page:

https://docs.openvinotoolkit.org/2021.4/openvino_docs_IE_DG_supported_plugins_GNA.html

Hope it helps.

Regards,

Wan

[Wan_Intel]

Hi Roxy1,

Thanks for your question.

This thread will no longer be monitored since we have provided a solution. 

If you need any additional information from Intel, please submit a new question.

Regards,

Wan