topic Thread safety of MKL 10.3 (update 3) in Intel® oneAPI Math Kernel Library

Thread safety of MKL 10.3 (update 3)

AndrewC — Tue, 10 May 2011 15:41:18 GMT

I am checking the thread safety of my code using Intel Inspector XE ( Windows/x64). My code is using omp threading around calls to MKL (parallel)

I get many data race conditions reported in MKL. This is very frustrating as I can never tell what is a real issue and what is bogus.

For example, this code reports many data races such as below.

Time Description Modules Sources
8:24:42 Error:Data race mkl_core.dll mkl_core.dll!0x23a0; mkl_core.dll!0x254d

[bash]#include "mkl.h"
#include "mkl_vml_functions.h"
#include 
#define SIZE ((size_t)10)
//
  // set_diff.cpp
  //

int testMKL()
{
	double *a, *b, *c;
	a = new double [SIZE*SIZE*2];
	b = new double [SIZE*SIZE*2];
	c = new double [SIZE*SIZE*2];
	double alpha[2]={1,0};
	double beta[2]={1,0};
	int m=SIZE, n=SIZE, k=SIZE, lda=SIZE, ldb=SIZE, ldc=SIZE, i=0, j=0;
	char transa='n', transb='n';
	for( i=0; i

Thread safety of MKL 10.3 (update 3)

TimP — Tue, 10 May 2011 16:35:00 GMT

If you call MKL from a single thread region, there is no mistake you can make (except maybe insufficient array sizes or other data overlaps) which would cause race conditions in MKL, so you would have to ignore any indications by Inspector within MKL.
In the early days of the predecessors of Inspector, they used to tell us we could test libraries for thread safety, so you would have a complaint to make in that respect.
If you call dgemm from a threaded region, you would need to assure that the c[] array which is updated by dgemm is a distinct data set in each thread.

Thread safety of MKL 10.3 (update 3)

AndrewC — Tue, 10 May 2011 16:52:25 GMT

I am calling MKL from multiple (OMP) threads, by inspection you can see my sample code is using distinct arrays for each thread ( as they are allocated inside the routine). There should not be any data races if MKL is "thread-safe".

I am using the lastest Inspector XE 2011 and in my experience, when it says "data race" it is correct. When running my application code, all these MKL "data race" errors can crowd out and obscure real errors.

It seems not unreasonable that given that MKL and Inspector XE are both Intel products they should "play nice" with each other. So who do I submit a bug report ? MKL team or Inspector XE team???

Thread safety of MKL 10.3 (update 3)

Gennady_F_Intel — Tue, 10 May 2011 17:37:25 GMT

Andrew,

thanks for reporting this issue. i will try to re-pro this and understand whose issue it is (Inspector or MKL)

What is the level of Inspector analysis you run? is the option to check stack accesses was enabled?

--Gennady

Thread safety of MKL 10.3 (update 3)

AndrewC — Tue, 10 May 2011 19:40:42 GMT

[cpp]// the following will show many "errors" in MKL  in inspector XE. Link against "parallel MKL"

#include "mkl.h"
#include "mkl_vml_functions.h"
#include 
#define SIZE ((size_t)10)
//
  // set_diff.cpp
  //


int testZGETRS()
{


	  #define  NMAX 4
	  int N=NMAX;
	  int LDA=N;
	  int IPIV[NMAX];

	  MKL_Complex16 A[NMAX*NMAX]={ 
	 {-1.34, 2.55}, { 0.28, 3.17}, {-6.39,-2.20}, { 0.72,-0.92},
	 {-0.17,-1.41}, { 3.31,-0.15}, {-0.15, 1.34}, { 1.29, 1.38},
	 {-3.29,-2.39}, {-1.91, 4.42}, {-0.14,-1.35}, { 1.72, 1.35},
	 { 2.41, 0.39}, {-0.56, 1.47}, {-0.83,-0.69}, {-1.96, 0.67} 
	  };

	 MKL_Complex16 B[NMAX]={{26.26, 51.78}, {31.32, -6.70},
	 { 6.43, -8.68}, {15.86, -1.42}};
	 int INFO=0;
	 zgetrf(&N,&N,A,&LDA,IPIV,&INFO);
	 if(INFO==0){
		 const char *TRANS="N";
		 int NRHS=1;
		 int LDB=N;
		 zgetrs(TRANS,&N,&NRHS,A,&LDA,IPIV,B,&LDB,&INFO);
	 }
	 return INFO;
}

int testzgemm3m()
{
	double *a, *b, *c;
	a = new double [SIZE*SIZE*2];
	b = new double [SIZE*SIZE*2];
	c = new double [SIZE*SIZE*2];
	double alpha[2]={1,0};
	double beta[2]={1,0};
	int m=SIZE, n=SIZE, k=SIZE, lda=SIZE, ldb=SIZE, ldc=SIZE, i=0, j=0;
	char transa='n', transb='n';
	for( i=0; i

Thread safety of MKL 10.3 (update 3)

Gennady_F_Intel — Wed, 11 May 2011 18:45:21 GMT

and Inspector reported: "No Problems detected".

- The stack frame which I used == 32 ( maximum)

- checked on Core2Duo CPU.

Thread safety of MKL 10.3 (update 3)

AndrewC — Wed, 11 May 2011 19:02:33 GMT

Hi Gennady,

You must be doing something different to me. I am using the identical Inspector XE version.

None of my MKL code runs without some many "complaints" in XE when called from multiple threads (threaded by OpenMP)

Are you sure you
- Compiled with the Intel Compiler
- Compiled with Generate Parallel Code (/Qopenmp) ON.
- Do you see multiple threads created in the XE log.
- You are checking "Threading Errors" not "Memory Errors"

Can I see your compiler command line?

Andrew

Thread safety of MKL 10.3 (update 3)

Gennady_F_Intel — Fri, 13 May 2011 06:41:21 GMT

Hi Andrew,

yeap, it was my fault :) - I was in a hurry and missed to add the /Qopenmp option.

now I see ~similar report with the last example you provided:

test.exe!mkl_serv_setxer - test.exe:44061

test.exe!zgetrf - test.exe:8389

test.exe!testZGETRS - test.cpp:30

test.exe!main - test.cpp:78

test.exe!main - test.cpp:76

test.exe!_tmainCRTStartup - crtexe.c:586

test.exe!mainCRTStartup - crtexe.c:402

test.exe!mkl_serv_setxer - test.exe:44061test.exe!zgetrf - test.exe:8389test.exe!testZGETRS - test.cpp:30test.exe!main - test.cpp:78test.exe!main - test.cpp:76test.exe!_tmainCRTStartup - crtexe.c:586test.exe!mainCRTStartup - crtexe.c:402

We need to check where is the problem - mkl and/or Inspector.

--Gennady

Thread safety of MKL 10.3 (update 3)

AndrewC — Fri, 13 May 2011 15:32:36 GMT

Easy mistake to make as the Intel compiler does not complain about the #pragma omp even when /Qopenmp is not specified
Will be interested to hear what you discover...

Thread safety of MKL 10.3 (update 3)

Gennady_F_Intel — Thu, 19 May 2011 08:49:55 GMT

Andrew,

The initial investigations shown that is sort of the benign race in our library whentwo or more threads are writing into same varaible same value. Indeed , such situation should be specially handled in Inspector to be not exposed and not confuse users.

--Gennady

Thread safety of MKL 10.3 (update 3)

AndrewC — Mon, 15 Aug 2011 21:17:38 GMT

Hi Gennady,
The latest update to MKL 10.3.5 seems to have fixed these issues. That's great!

Andrew