Create a random matrix with MKL library

[Tuyen__Nguyen]

When I use vdRngUniform() routine to create a random matrix, the maximum size matrix I can use only 40000 x 40000. Do we have any other routine from MKL to create with the bigger size? I can create by the normal way on C but it affects performance. So please guide me with other routines from MKL

Thanks.

[Pavel_D_Intel1]

Hi Nguyen,

I suggest to continue use vdRngUniform() routine with generation by blocks.
Here I mean that you can generate matrix of the needed size block by block, as shown below:

    VSLStreamStatePtr stream;
    vslNewStream( &stream, BRNG,  SEED );       

    const int64_t size = 40000*40000;
    const int64_t block_size = 1000;

    double* matrix;
    matrix = (double*) malloc (size*sizeof(double));

    for(int64_t i = 0; i < size; i+=block_size) {
            vdRngUniform( VSL_RNG_METHOD_UNIFORM_STD, stream, block_size, matrix + i, 0.0, 1.0 );
    }

    free(matrix);
    vslDeleteStream( &stream );

Please, let me know, if it addresses your question. 

Best regards,
Pavel

[Gennady_F_Intel]

plus two cents - I am not sure if using RNGUniform is important to your case, but you may take a look at the https://software.intel.com/en-us/mkl-developer-reference-c-lagge routine which generates M x N matrix. You may link with ILP64 mkl API and generate matrix beyond of 40k by 40K

[Tuyen__Nguyen]

Hi Pavel,

Your program is correct with size 40000x40000 but with the bigger size as 50000x50000 it will be fault "Segmentation fault (core dumped)". 

Please tell me if you have another way to solve this problem.

Thanks.

[Tuyen__Nguyen]

Hi Gennady F.,

Thanks for your recommendation but I also have a problem, please see my program:

#include "stdio.h"
#include "mkl.h"
#include "mkl_lapack.h"


#define LAPACK_ROW_MAJOR  101
#define LAPACK_COL_MAJOR  102

#define MIN(a,b) ((a < b) ? (a) : (b))

double  main()
{
    int m = 5000;
    int n = 5000;
    int kl = m-1 ;
    int ku = n-1;
    int len_d = MIN(m,n);
    const double *d ;
    double *a;
    int lda = n ;
    int * iseed;
    *iseed = 999;
    a = (double *)malloc(m*n*sizeof(double));
    d = (double *)malloc(len_d*sizeof(double));
    LAPACKE_dlagge (LAPACK_ROW_MAJOR , m , n , kl , ku , &d[0] , &a[0] , lda , iseed );
    free(a);

}

icc -I/opt/intel/compilers_and_libraries_2018.5.274/linux/mkl/include/ -mkl -lmkl_intel_ilp64 matrix.c -o matrix

In this case, I want sure it works well with the small size before I check bigger than 40000x40000. But it shows me the error "Intel MKL ERROR: Parameter 4 was incorrect on entry to DLAGGE." 

Do you know how to fix the problem?

Thanks.

[Pavel_D_Intel1]

Hi Nguyen,

I think that the root cause of your problem is integer overflow.

To avoid it you can change the line with size in my program as follows:

    const int64_t size = (int64_t)50000*(int64_t)50000;

I checked the program with 100 000 x 100 000 size – it works fine.

Hope it helps you.

Best regards,
Pavel

[Gennady_F_Intel]

if you link with ILP64 lib, then you have to use -DMKL_ILP64 compiler option. Please refer the mkl linker adviser or mkl user guide for more details