OneAPI MKL vs parallel XE Cluster Edition MKL : mkl_sparse_s_mv inspector/executor performance

[Joy7]

Hi, 

I have tested the performance of mkl_sparse_s_mv executor using CSR format on a linux machine after setting the hints and optimizing via mkl_sparse_optimize for two different versions of Intel MKL distributed via OneAPI MKL version 2021.3.0 and Parallel XE Cluster edition 2020.1.217

I found that mkl_sparse_s_mv executor performance for the latest version is significantly slower than the previous version. This is observed especially for the matrices which are probably converted to the DIA format internally after the inspector stage. Please look into the matter.

[Gennady_F_Intel]

Hi,

This is unexpected behavior. 

Could You share more details on this matter? Having the reproducer+input matrix is the best and fastest way to check the problem on our end. Are there any specific CPU types did You run? What is performance differences did You observe?  OS? Which threading runtime have You used?

[Joy7]

Thank you for the reply Gennady_F_Intel.

For example, following is the observed performance for a sparse matrix Watson/Bauman, https://sparse.tamu.edu/Watson/Baumann

Parallel XE Cluster edition 2020.1.217 : 21 GFLOPS

OneAPI MKL version 2021.3.0 : 12 GLOPS

Next, the system is Intel Core i7-3930K with 6 × 3.20GHz cores and Ubuntu Linux 18.04.5 OS.

The gcc/icc compiler options used to link libraries are -lpfm -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread -lm.

[Gennady_F_Intel]

thanks, we will check the case on our end and keep this thread updated.

[Gennady_F_Intel]

I checked this workload with double-precision ( mkl_sparse_d_mv) routine and don't see the issue:

.... matrix name = Baumann.mtx .....

SIZE == 112211, NNZ ==760631  

MKL 2020.0      MKL 2020.1    MKL 2021.3

1.050142e-05 sec  1.013826e-05 sec 1.004625e-05sec

-- more details

Major version:      2020

Minor version:      0

Update version:     0

Product status:     Product

Build:          20191122

Platform:        Intel(R) 64 architecture

Processor optimization: Intel(R) Advanced Vector Extensions 512 (Intel(R) AVX-512) enabled processors

SIZE == 112211, NNZ ==760631  

....IE SpBLAS MV Execution Time == 1.050142e-05 sec

***********************************

Major version:      2020

Minor version:      0

Update version:     1

Product status:     Product

Build:          20200208

Platform:        Intel(R) 64 architecture

Processor optimization: Intel(R) Advanced Vector Extensions 512 (Intel(R) AVX-512) enabled processors

 SIZE == 112211, NNZ ==760631  

....IE SpBLAS MV Execution Time == 1.013826e-05 sec

***********************************

Major version:      2021

Minor version:      0

Update version:     3

Product status:     Product

Build:          20210617

Platform:        Intel(R) 64 architecture

Processor optimization: Intel(R) Advanced Vector Extensions 512 (Intel(R) AVX-512) enabled processors

 SIZE == 112211, NNZ ==760631  

....IE SpBLAS MV Execution Time == 1.004625e-05 sec

>>>>

here are some CPU and OS specific details:

CPU:    2x Xeon Gold 6148 2.4Ghz 20c (NP=40) ( Skylake-SP : )

 MEMORY:  192GB 2666Mhz DDR4 Dual-rank

 OS:    CentOS Linux release 7.9.2009 (Core)

>>>

lp64, intel OpenMP threading

export KMP_AFFINITY=compact,1,0,granularity=fine

--Gennady

[Joy7]

Thanks Gennady for checking. 

However, I tried it once again on my end while making sure that I only change the linked libraries from one version to another.

I confirm that I see the correct vector output on both the runs, but unfortunately I continue to see the performance degradation in the newer version. Also, I want to point out that I run the benchmark multiple times to increase the running time for a more accurate measurement. Also, I run the experiment 30 times to calculate the variance.

I also inform this via mkl_sparse_set_mv_hint (A_csr,  SPARSE_OPERATION_NON_TRANSPOSE,  descr,  num_runs * 30);

[Gennady_F_Intel]

Please give us your reproducer to run and check how it works on our side. We also run the 1000 times and report the average execution time. the pseudocode looks like as follow:

  double t1 = dsecnd();

for (i=0;i<ncount;i++)

{

       mkl_sparse_d_mv ( .....);

}

  double t2 = dsecnd();

[Joy7]

Please find attached the tarball.  Please follow the commands to compile and execute.

$ make float

$ ./run_float <PATH-TO-MATRIX-FILE> <NUMBER-OF-THREADS>

e.g.

$ ./run_float ~/Watson/Baumann.mtx 5

Output : Standard_Deviation, Performance in MFLOPS, Fletcher_Sum

[Gennady_F_Intel]

ok, thanks for the tarball. We will check the case.