Long network loading time on Movidius™ Myriad™ X based Intel® Vision Accelerator Design cards – x8

[E-Hong]

Hi,

I ran the inference on DevCloud Edge benchmark app using OpenVINO LTS. The network loading time for CPU, GPU, FPGA, and NCS2 are around 2000ms, but MX8 uses 40000ms. Why is it so long?

Thank you.

[ChrisB_Intel]

E-Hong,

This is to be expected. It takes longer for the schedule to initialize the Myryd than a CPU. Please take a look at this article for more information. 

https://www.intel.com/content/www/us/en/support/articles/000057722/software/development-software.html

Thanks

Chris

[E-Hong · Accepted solution]

Chris,

I read that issue before. However, the article was on Neural Compute Stick 2 right? For NCS2 (Movidius™ Myriad™ X based Intel® Vision Accelerator Design cards – x1), I am not encountering long loading time, the one I am having problem is Movidius™ Myriad™ X based Intel® Vision Accelerator Design cards – x8.

But, if that article is meant for Movidius™ Myriad™ X based Intel® Vision Accelerator Design cards – x8, how to justify the normal network loading time of Movidius™ Myriad™ X based Intel® Vision Accelerator Design cards – x1 instead?

Thank you.

[ChrisB_Intel]

E-Hong,

Apologies for the delay in getting back to you on this. Firstly, my apologies, I need to learn how to read a little better. I was initially reading the HDDL load time as 4, thousand, not 40 thousand, even though you specifically said 40000 in your post. I ran this a few more times and am seeing the same result. I ran it locally and am seeing a significantly faster result. I am still digging into this, but there appears to be an issues, but as the same model runs faster locally, I am not exactly sure where it is. I have opened a bug and will keep you updated.

Regards,

Chris

[E-Hong]

Hi Chris,

Really appreciate you for looking into this issue, hope to hear from you soon!

Is it convenient to ask for a comparison of results obtained locally from your side?

Thanks again Chris.

Regards.

[ChrisB_Intel]

Sure no problem, they are still looking on this in the backend, but here are the results we saw. 

DevCloud Latest Release:

CPU - load time 988.01

NCS - load time 2554.95

HDDL - load time 42270.43

Local Latest Release:

CPU - load time 460.14

HDDL - load time 2174.48

Thanks

Chris

[E-Hong]

Chris,

Received with thanks, much appreciated!

Regards,

E-Hong