Intel® oneAPI Math Kernel Library
Ask questions and share information with other developers who use Intel® Math Kernel Library.

Are DNN functions thread safe?

xuesong_g_
Beginner
220 Views

It seems that dnnExecute_F32 can not be called in multi-thread, does it?

0 Kudos
1 Solution
Ying_H_Intel
Employee
220 Views

Hi Xuesong,

Right, you can't create Global DNN primitives, but use them in multiply different threads. You need to take care of the synchronize of the involve data and internal structure (like any global variable, you use it in different threads , you need to add protect mechanism to use it) .  But you can create private DDN primitives  for each threads, it is thread safe.  That is why we suggest to use MKL build the topology directly instead of manage the threads yourself.

Best Regards,

Ying

View solution in original post

7 Replies
Gennady_F_Intel
Moderator
220 Views

dnnExecute_F32 is threaded by OpenMP and will use OpenMP runtime lib if you linked the application with libiomp5 lib

xuesong_g_
Beginner
220 Views

Gennady F. (Intel) wrote:

dnnExecute_F32 is threaded by OpenMP and will use OpenMP runtime lib if you linked the application with libiomp5 lib

I use sequential mkl. I want to control thread outside. win32 thread or pthread. Does MKL support this way?

Ying_H_Intel
Employee
220 Views

Hi Xuesong,

If needed,  right, you can control thread outside by win32 thread or pthread etc.  MKL function include MKLDNN function support that way.  for example, MKL user guide discuss : https://software.intel.com/en-us/mkl-linux-developer-guide-avoiding-conflicts-in-the-execution-envir....

And if you are build one NN topology, please refer to MKL NN sample under MKL install directory,  /mkl/example/*.tgz, unzip it and the folder dnnc

Best Regards,

Ying

xuesong_g_
Beginner
220 Views

Hi Ying.

Can I create global DNN primitives, then use them in multiple diffrent threads? In my test case, I found this will lead to unstable output.

Ying_H_Intel
Employee
221 Views

Hi Xuesong,

Right, you can't create Global DNN primitives, but use them in multiply different threads. You need to take care of the synchronize of the involve data and internal structure (like any global variable, you use it in different threads , you need to add protect mechanism to use it) .  But you can create private DDN primitives  for each threads, it is thread safe.  That is why we suggest to use MKL build the topology directly instead of manage the threads yourself.

Best Regards,

Ying

xuesong_g_
Beginner
220 Views

Thank you Ying, I got it.

While last question: Can I use global conversion primitives? Thank you very much

Ying_H_Intel
Employee
220 Views

Hi Xuesong,

Yes,  the conversion primitives may work as global primitives, if no data conflict in different threads. Not sure how do like design the Muliti-thread NN.   Here is one MKL NN Convolution code vs. general C code. https://software.intel.com/sites/default/files/managed/b9/e9/mkl_conv_avx2.c for your reference.

Best Regards,
Ying

Reply