topic Re:Execution gets stuck in batch GEMM when using AVX and libgomp in Intel® oneAPI Math Kernel Library

Execution gets stuck in batch GEMM when using AVX and libgomp

guillaumekln — Thu, 25 Feb 2021 17:27:42 GMT

Hi,

We are facing an issue with the function cblas_sgemm_batch_strided from oneMKL 2021.1. The execution gets stuck in this function when the code is run on a Intel CPU with AVX and compiled with GNU OpenMP (tested on Ubuntu 18.04 and CentOS 7). The same code used to work in Intel MKL 2020.4.

The issue can be reproduced by setting MKL_CBWR=AVX (see below).

Code to reproduce:

#include <mkl.h> int main() { const MKL_INT batch_size = 256; const CBLAS_TRANSPOSE transa = CblasNoTrans; const CBLAS_TRANSPOSE transb = CblasTrans; const MKL_INT m = 1; const MKL_INT n = 1; const MKL_INT k = 64; const MKL_INT lda = k; const MKL_INT ldb = k; const MKL_INT ldc = n; const MKL_INT stridea = m * k; const MKL_INT strideb = k * n; const MKL_INT stridec = m * n; const float alpha = 1; const float beta = 0; const float* a = new float[batch_size * m * k]; const float* b = new float[batch_size * n * k]; float* c = new float[batch_size * m * n]; cblas_sgemm_batch_strided(CblasRowMajor, transa, transb, m, n, k, alpha, a, lda, stridea, b, ldb, strideb, beta, c, ldc, stridec, batch_size); delete [] a; delete [] b; delete [] c; }

Compilation (Ubuntu 18.04):

MKLROOT=/opt/intel/oneapi/mkl/2021.1.1 g++ -o gemm_batch gemm_batch.cc -L${MKLROOT}/lib/intel64 -Wl,--no-as-needed -lmkl_intel_ilp64 -lmkl_gnu_thread -lmkl_core -lgomp -lpthread -lm -ldl -DMKL_ILP64 -m64 -I"${MKLROOT}/include"

Execution:

MKL_CBWR=AVX LD_LIBRARY_PATH=${MKLROOT}/lib/intel64 ./gemm_batch

Thanks for looking into this issue,

Guillaume

Re:Execution gets stuck in batch GEMM when using AVX and libgomp

Gennady_F_Intel — Fri, 26 Feb 2021 05:10:59 GMT

Guillaume,

Is that possible to check if the problem will still exist with Intel OMP threading? ( libmkl_intel_thread)

Re: Execution gets stuck in batch GEMM when using AVX and libgomp

Gennady_F_Intel — Fri, 26 Feb 2021 06:20:30 GMT

I checked and manage to reproduce the problem on my end. the problem happens with any threading runtime libraries. The issue would be investigated and the tread would keep updated as soon as possible.

Re: Execution gets stuck in batch GEMM when using AVX and libgomp

guillaumekln — Fri, 26 Feb 2021 13:12:24 GMT

Hi Gennady,

Thanks for looking into this issue.

According to my tests the issue happens only with GNU OpenMP and not with Intel OpenMP.

Re:Execution gets stuck in batch GEMM when using AVX and libgomp

Gennady_F_Intel — Fri, 26 Feb 2021 15:22:35 GMT

it seems that the behavior depends on the specific CPU type as the problem happens when we change the code path ( by using MKL_CBWR).

Re: Re:Execution gets stuck in batch GEMM when using AVX and libgomp

guillaumekln — Tue, 16 Mar 2021 17:48:18 GMT

Hi Gennady,

I'm just wondering if we can expect a fix to be included in oneMKL 2021.2?

Also to complete the first post, we always set MKL_CBWR=AUTO,STRICT when running our application. So I guess it turns into MKL_CBWR=AVX,STRICT on AVX systems.

As a workaround, we changed the OpenMP runtime to Intel and it seems to work for us.

Re:Execution gets stuck in batch GEMM when using AVX and libgomp

Gennady_F_Intel — Wed, 17 Mar 2021 03:02:57 GMT

Hi Guillaume,

I don't see this issue fixed into the coming 2021 update2 and very likely the fix is targeting to the update3.

Re:Execution gets stuck in batch GEMM when using AVX and libgomp

Gennady_F_Intel — Fri, 23 Apr 2021 14:54:11 GMT

update - the fix of the issue is targeted to be available in the next update of oneMKL. We will keep this thread updated with the status of this release.

Re:Execution gets stuck in batch GEMM when using AVX and libgomp

Gennady_F_Intel — Sun, 04 Jul 2021 04:17:45 GMT

The fix of the issue available in the official release of MKL 2021.3 which has happened yesterday.

You could take it to try and let us know the results.

Re:Execution gets stuck in batch GEMM when using AVX and libgomp

Gennady_F_Intel — Sun, 04 Jul 2021 04:24:38 GMT

here is the mkl verbose mode output

export MKL_CBWR=AVX

MKL_VERBOSE oneMKL 2021.0 Update 3 Product build 20210617 for Intel(R) 64 architecture Intel(R) Advanced Vector Extensions (Intel(R) AVX) enabled processors, Lnx 2.20GHz ilp64 gnu_thread

MKL_VERBOSE SGEMM_BATCH_STRIDED(T,N,1,1,64,0x7ffd330bba98,0x22ab040,64,64,0x229b030,64,64,0x7ffd330bbaa0,0x22bb050,1,1,256) 20.38ms CNR:AVX Dyn:1 FastMM:1 TID:0 NThr:44

...test passed ...

Re: Execution gets stuck in batch GEMM when using AVX and libgomp

guillaumekln — Mon, 05 Jul 2021 07:09:40 GMT

I confirm it is now working as expected.

Thanks for the help!

Re:Execution gets stuck in batch GEMM when using AVX and libgomp

Gennady_F_Intel — Mon, 05 Jul 2021 07:27:48 GMT

Thanks for the update.

This issue has been resolved and we will no longer respond to this thread. If you require additional assistance from Intel, please start a new thread. Any further interaction in this thread will be considered community only.