topic Hi Nguyen, in Intel® oneAPI Math Kernel Library

Create a random matrix with MKL library

Tuyen__Nguyen — Thu, 02 Apr 2020 01:26:20 GMT

When I use vdRngUniform() routine to create a random matrix, the maximum size matrix I can use only 40000 x 40000. Do we have any other routine from MKL to create with the bigger size? I can create by the normal way on C but it affects performance. So please guide me with other routines from MKL

Thanks.

Hi Nguyen,

Pavel_D_Intel1 — Thu, 02 Apr 2020 12:27:37 GMT

Hi Nguyen,

I suggest to continue use vdRngUniform() routine with generation by blocks.
Here I mean that you can generate matrix of the needed size block by block, as shown below:

VSLStreamStatePtr stream;
vslNewStream( &stream, BRNG, SEED );

const int64_t size = 40000*40000;
const int64_t block_size = 1000;

double* matrix;
matrix = (double*) malloc (size*sizeof(double));

    for(int64_t i = 0; i < size; i+=block_size) {
          vdRngUniform( VSL_RNG_METHOD_UNIFORM_STD, stream, block_size, matrix + i, 0.0, 1.0 );
  }

free(matrix);
vslDeleteStream( &stream );

Please, let me know, if it addresses your question.

Best regards,
Pavel

plus two cents - I am not

Gennady_F_Intel — Thu, 02 Apr 2020 14:47:25 GMT

plus two cents - I am not sure if using RNGUniform is important to your case, but you may take a look at the https://software.intel.com/en-us/mkl-developer-reference-c-lagge routine which generates M x N matrix. You may link with ILP64 mkl API and generate matrix beyond of 40k by 40K

Hi Pavel,

Tuyen__Nguyen — Fri, 03 Apr 2020 08:02:39 GMT

Hi Pavel,

Your program is correct with size 40000x40000 but with the bigger size as 50000x50000 it will be fault "Segmentation fault (core dumped)".

Please tell me if you have another way to solve this problem.

Thanks.

Hi Gennady F.,

Tuyen__Nguyen — Fri, 03 Apr 2020 08:32:00 GMT

Hi Gennady F.,

Thanks for your recommendation but I also have a problem, please see my program:

#include "stdio.h"
#include "mkl.h"
#include "mkl_lapack.h"


#define LAPACK_ROW_MAJOR  101
#define LAPACK_COL_MAJOR  102

#define MIN(a,b) ((a < b) ? (a) : (b))

double  main()
{
    int m = 5000;
    int n = 5000;
    int kl = m-1 ;
    int ku = n-1;
    int len_d = MIN(m,n);
    const double *d ;
    double *a;
    int lda = n ;
    int * iseed;
    *iseed = 999;
    a = (double *)malloc(m*n*sizeof(double));
    d = (double *)malloc(len_d*sizeof(double));
    LAPACKE_dlagge (LAPACK_ROW_MAJOR , m , n , kl , ku , &d[0] , &a[0] , lda , iseed );
    free(a);

}

icc -I/opt/intel/compilers_and_libraries_2018.5.274/linux/mkl/include/ -mkl -lmkl_intel_ilp64 matrix.c -o matrix

In this case, I want sure it works well with the small size before I check bigger than 40000x40000. But it shows me the error "Intel MKL ERROR: Parameter 4 was incorrect on entry to DLAGGE."

Do you know how to fix the problem?

Thanks.

Hi Nguyen,

Pavel_D_Intel1 — Fri, 03 Apr 2020 09:12:47 GMT

Hi Nguyen,

I think that the root cause of your problem is integer overflow.

To avoid it you can change the line with size in my program as follows:

const int64_t size = (int64_t)50000*(int64_t)50000;

I checked the program with 100 000 x 100 000 size – it works fine.

Hope it helps you.

Best regards,
Pavel

if you link with ILP64 lib,

Gennady_F_Intel — Fri, 03 Apr 2020 09:14:39 GMT

if you link with ILP64 lib, then you have to use -DMKL_ILP64 compiler option. Please refer the mkl linker adviser or mkl user guide for more details