Hi, dear engineers,
Here is a little bit background: in our solver, we have a precondtioner made up of a block diagonal matrix and a sparse matrix. We use tbb+mkl to LU factorize the block diagonal matrix and Parsido to factorize the sparse matrix. To avoid the racing between mkl and tbb, we use tbb to control the threading; we assign mkl zgetrf just one thread for each diagonal block. The problem is we could not find a good way to control the Pardiso threading. This really screwed up the parallelization efficiency analysis. NOTE our company also charges customers differently for number of threads: 1k for 1-thread, 1.1k for 2-thread, etc. (the numbers are made up but you get my point)
According to the pardiso library, the thing we should do is to somehow control MKL_NUM_THREADS. I do not know if below is what we should do, please make comments:
1) In the block diagonal factorization, we set MKL_NUM_THREADS =1 (since TBB will decide how many threads are using) and each zgetrf should only access one thread.
2) for the sparse matrix, we should set MKL_NUM_THREADS = N, where N is the number of threads that was specified in the license file. And after Pardiso is done, we should reset MKL_NUM_THREADS =1.
Is this the right way to do it? It is quite clumsy, to me. thinking of we have to switch between these two matrices thousands of times in the back substitution when the preconditioner is fired up in matrix solve. If there are better way of doing it, I really appreciate your kindness if you can share the tip with me.
I also strongly recommend that we can control the threading with one of the iparam, it is just a number....
btw, we are using MKL 11.0.5
From first slight for your model you describe correct way to use different threading model for different MKL components. But could you also check you messages? I've send you several question to understand your usage model deeply.