Optimizing custom operations using Intel Intrinsics (SSE, AVX)

[Gouveia__César]

Suppose I have generated a dll with my custom openvino operation (cpu_extension.dll) following your guide, and that my CPU supports AVX. This dll is going to be generated/compiled with a set of operations that will have the HAVE_AVX2 flag activated.

Set of questions:

1 - What happens if I try to run this dll on a CPU which does not support AVX but instead supports SSE (older Intel Intrinsics)? It will crash?

2 - This cpu_extension.dll includes all the .cpp from all openvino supported operations, or it only includes mine?

3 - Supposing that the cpu_extension.dll only includes the .cpp of my operation and that this dll is being generated/compiled by a machine which only supports SSE, will I have all the openvino supported operations compiled with AVX and my custom operation compiled with SSE?

Thanks,

César.

[HemanthKum_G_Intel]

Hi Cesar,

The CPU Plugin supports inference on Intel® Xeon® with Intel® AVX2 and AVX512, Intel® Core™ Processors with Intel® AVX2, Intel Atom® Processors with Intel® SSE. This is what we have validated.

You can use -pc the flag for samples if you would like to know which configuration is used by some layer. This flag shows execution statistics that you can use to get information about layer name, execution status, layer type, execution time and the type of the execution primitive.

[Gouveia__César]

Hi Hemanth Kumar,

Starting by your response to the first question:

The CPU Plugin supports inference on Intel® Xeon® with Intel® AVX2 and AVX512, Intel® Core™ Processors with Intel® AVX2, Intel Atom® Processors with Intel® SSE. This is what we have validated.

I think you didn´t answered my question, but correct me if I'm wrong. What I need to know is if this HAVE_AVX flag is set at compile-time or at run-time. Suppose the flag is set at compile-time and you compiled your custom operation using a machine with AVX support, the layers will be optimized by using AVX instructions. What happens if you run the same dll in another machine which does not support AVX but only supports SSE? It will crash stating that there is illegal instructions (AVX instructions)? 

In the second paragraph of your question you mention a flag that I didn't tested (-pc):

You can use -pc the flag for samples if you would like to know which configuration is used by some layer.

I added this flag to my benchmark.exe script, however I noticed something strange, my custom dot operation (which uses AVX) have an execType set to unknown_FP32. Why is this happening if I'm using the same flags as your layers (HAVE_AVX, HAVE_SSE, HAVE_FMA) and AVX instructions as well (e.g. _mm256_setzero_ps)? I can send you my code if necessary.

Thanks,

César.