topic Re:OneAPI MKL vs parallel XE Cluster Edition MKL : mkl_sparse_s_mv inspector/executor performance in Intel® oneAPI Math Kernel Library

OneAPI MKL vs parallel XE Cluster Edition MKL : mkl_sparse_s_mv inspector/executor performance

Joy7 — Fri, 27 Aug 2021 16:28:11 GMT

Hi,

I have tested the performance of mkl_sparse_s_mv executor using CSR format on a linux machine after setting the hints and optimizing via mkl_sparse_optimize for two different versions of Intel MKL distributed via OneAPI MKL version 2021.3.0 and Parallel XE Cluster edition 2020.1.217

I found that mkl_sparse_s_mv executor performance for the latest version is significantly slower than the previous version. This is observed especially for the matrices which are probably converted to the DIA format internally after the inspector stage. Please look into the matter.

Re:OneAPI MKL vs parallel XE Cluster Edition MKL : mkl_sparse_s_mv inspector/executor performance

Gennady_F_Intel — Sat, 28 Aug 2021 03:04:38 GMT

Hi,

This is unexpected behavior.

Could You share more details on this matter? Having the reproducer+input matrix is the best and fastest way to check the problem on our end. Are there any specific CPU types did You run? What is performance differences did You observe? OS? Which threading runtime have You used?

Re: OneAPI MKL vs parallel XE Cluster Edition MKL : mkl_sparse_s_mv inspector/executor performance

Joy7 — Mon, 30 Aug 2021 12:43:37 GMT

Thank you for the reply Gennady_F_Intel.

For example, following is the observed performance for a sparse matrix Watson/Bauman, https://sparse.tamu.edu/Watson/Baumann

Parallel XE Cluster edition 2020.1.217 : 21 GFLOPS

OneAPI MKL version 2021.3.0 : 12 GLOPS

Next, the system is Intel Core i7-3930K with 6 × 3.20GHz cores and Ubuntu Linux 18.04.5 OS.

The gcc/icc compiler options used to link libraries are -lpfm -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread -lm.

Re:OneAPI MKL vs parallel XE Cluster Edition MKL : mkl_sparse_s_mv inspector/executor performance

Gennady_F_Intel — Tue, 31 Aug 2021 03:51:58 GMT

thanks, we will check the case on our end and keep this thread updated.

Re:OneAPI MKL vs parallel XE Cluster Edition MKL : mkl_sparse_s_mv inspector/executor performance

Gennady_F_Intel — Tue, 31 Aug 2021 14:54:30 GMT

I checked this workload with double-precision ( mkl_sparse_d_mv) routine and don't see the issue:

.... matrix name = Baumann.mtx .....

SIZE == 112211, NNZ ==760631

MKL 2020.0 MKL 2020.1 MKL 2021.3

1.050142e-05 sec 1.013826e-05 sec 1.004625e-05sec

-- more details

Major version: 2020

Minor version: 0

Update version: 0

Product status: Product

Build: 20191122

Platform: Intel(R) 64 architecture

Processor optimization: Intel(R) Advanced Vector Extensions 512 (Intel(R) AVX-512) enabled processors

SIZE == 112211, NNZ ==760631

....IE SpBLAS MV Execution Time == 1.050142e-05 sec

***********************************

Major version: 2020

Minor version: 0

Update version: 1

Product status: Product

Build: 20200208

Platform: Intel(R) 64 architecture

Processor optimization: Intel(R) Advanced Vector Extensions 512 (Intel(R) AVX-512) enabled processors

SIZE == 112211, NNZ ==760631

....IE SpBLAS MV Execution Time == 1.013826e-05 sec

***********************************

Major version: 2021

Minor version: 0

Update version: 3

Product status: Product

Build: 20210617

Platform: Intel(R) 64 architecture

Processor optimization: Intel(R) Advanced Vector Extensions 512 (Intel(R) AVX-512) enabled processors

SIZE == 112211, NNZ ==760631

....IE SpBLAS MV Execution Time == 1.004625e-05 sec

>>>>

here are some CPU and OS specific details:

CPU: 2x Xeon Gold 6148 2.4Ghz 20c (NP=40) ( Skylake-SP : )

MEMORY: 192GB 2666Mhz DDR4 Dual-rank

OS: CentOS Linux release 7.9.2009 (Core)

>>>

lp64, intel OpenMP threading

export KMP_AFFINITY=compact,1,0,granularity=fine

--Gennady

Re: Re:OneAPI MKL vs parallel XE Cluster Edition MKL : mkl_sparse_s_mv inspector/executor performanc

Joy7 — Wed, 01 Sep 2021 10:52:54 GMT

Thanks Gennady for checking.

However, I tried it once again on my end while making sure that I only change the linked libraries from one version to another.

I confirm that I see the correct vector output on both the runs, but unfortunately I continue to see the performance degradation in the newer version. Also, I want to point out that I run the benchmark multiple times to increase the running time for a more accurate measurement. Also, I run the experiment 30 times to calculate the variance.

I also inform this via mkl_sparse_set_mv_hint (A_csr, SPARSE_OPERATION_NON_TRANSPOSE, descr, num_runs * 30);

Re:OneAPI MKL vs parallel XE Cluster Edition MKL : mkl_sparse_s_mv inspector/executor performance

Gennady_F_Intel — Wed, 01 Sep 2021 11:58:15 GMT

Please give us your reproducer to run and check how it works on our side. We also run the 1000 times and report the average execution time. the pseudocode looks like as follow:

double t1 = dsecnd();

for (i=0;i<ncount;i++)

{

mkl_sparse_d_mv ( .....);

}

double t2 = dsecnd();

Re: Re:OneAPI MKL vs parallel XE Cluster Edition MKL : mkl_sparse_s_mv inspector/executor performanc

Joy7 — Fri, 03 Sep 2021 12:46:17 GMT

Please find attached the tarball. Please follow the commands to compile and execute.

$ make float

$ ./run_float <PATH-TO-MATRIX-FILE> <NUMBER-OF-THREADS>

e.g.

$ ./run_float ~/Watson/Baumann.mtx 5

Output : Standard_Deviation, Performance in MFLOPS, Fletcher_Sum

Re:OneAPI MKL vs parallel XE Cluster Edition MKL : mkl_sparse_s_mv inspector/executor performance

Gennady_F_Intel — Fri, 03 Sep 2021 12:50:00 GMT

ok, thanks for the tarball. We will check the case.