MKL Invalid Pointer in dsyevr (novice question)

mspecian — Sat, 03 Jul 2010 21:40:18 GMT

I am a young researcher without much CS experience trying to use the MKL routine "dsyevr" to diagonalize a real symmetric matrix. The matrix has approximately 6e9 8-byte elements (about 32.5GB). I have successfully executed this routine before for smaller matrices, but since moving to a new machine have experienced problems. If someone could point me in the right direction, I would be most appreciative.

Tech specs: I'm running MKL version 9 (I think) on a machine with16 64-bitXeon cores using em64t. I have 128GB of memory at my disposal. I call dsyevr from FORTRAN, but access the FORTRAN subroutine from C. Here's the C code:

////////////////////////////////////////////////////////////////////////////////////////////

// This routine calculates the eigenvalues and eigenvectors of the symmetric matrix

// of doubles fwritten to the file "fnameANM" using the dsyevr LAPACK procedure. Since

// memory constraints restrict the creation of a huge, complete eigenvector file, only

// the il-th through iu-th eigenvalues and eigenvectors are returned where 1<=il<=

// iu<=nrows. The eigenvalues are stored from most positive to most negative. This

// routine is somewhat faster than the packed version.

//Text files are written in matrix format with the eigenvectors in the columns.

// The "il"th eigenvector is in the first column while the "iu"th is in the last.

// Fwritten files store the eigenvectors sequentially. The eigenvector corresponding to

// the largest returned eigenvalue (i.e. the "il"th one takes up elements [0 through

// (nelements-1)]. The next largest eigenvector is stored to elements [nelements through

// (2*nelements-1)] and so forth.

////////////////////////////////////////////////////////////////////////////////////////////

void GetEigenvectors(char *fnameANM, int nrows, int il, int iu) {

double *w, *z, *work, rnrows, working, *EArray;

size_t size, ee, rr;

int ii, info, lz, lzp, lisuppz, lwork, m, *iwork, liwork, cc, iil, iiu;

info = NULL;

printf("\\nThis routine returns eigenmode #%d (larger eigenvalue) through %d (smaller eigenvalue).\\n\\n",il,iu);

//dsyevr puts the largest Evl's Evc last, so this fixes that

int swap = iu;

iu = nrows - il + 1;

il = nrows - swap + 1;

lz = iu-il+1; //Number of eigenvectors to be returned.

lzp = lz; //Same variable, but this doesn't get overwritten.

lisuppz = 4*lz; //Size of the iwork array

lwork = 50*nrows; //Size of work array

liwork= 40*nrows; //Size of lwork array

//////////////////// In case of seg fault, try upping the size of these arrays.

rnrows = nrows;

working = rnrows*rnrows;

size = floor(working+0.0001);

double *a = freadDMatrix(fnameANM,size); //Dynamically allocates memory and initializes matrix

printf("The final element of the matrix is %f.\\n",a[size-1]);

w = (double *)malloc(nrows*sizeof(double));

printf("Regarding memory allocation for w: "); ErrnoDescription(errno);

z = (double *)malloc(nrows*lz*sizeof(double));

printf("Regarding memory allocation for z: "); ErrnoDescription(errno);

work = (double *)malloc(lwork*sizeof(double));

printf("Regarding memory allocation for work: "); ErrnoDescription(errno);

iwork = (int *)malloc(liwork*sizeof(int));

printf("Regarding memory allocation for iwork: "); ErrnoDescription(errno);

printf("Executing dsyevrf.\\n");

dsyevrf_(&nrows, a, &il, &iu, &m, w, z, &lz, &lisuppz, work, &lwork, iwork, &liwork, &info);

}

void dsyevrf_(int nrows, double *matrix, int *il, int *iu, int *m, double *w, double *z, int *lz, int *lisuppz, double *work, int *lwork, int *iwork, int *liwork, int *info);

int main ()

{

//////////////////////////////////////////////////////////////////////////////////////////

///////////////////////////////// GetEigenvectors /////////////////////////////////////

//////////////////////////////////////////////////////////////////////////////////////////

size_t ncells6 = 66113;

char *fnameGCorr_sy = "/home/mspecian/PSnoise/noiseMat_Vec/GCorMat_R6sy_fwrite";

int il = 1;

int iu = 10000;

/*GetEigenvectors[1. Name of symmetric, fwritten ncells by ncells covariance function

to be diagonalized

2. Dimension of above matrix, also # of cells

3. Value >= 1 equaling # of first eigenvector returned (1 is largest)

4. Value >= il and <= ncells equaling # of final eigenvector returned]*/

GetEigenvectors(fnameGCorr_sy, ncells6, il, iu);

return 0;

}

The functiondsyevrf_ is written in FORTRAN. Here is how I define the subroutine:

c********************************************************************

subroutine dsyevrf(n,a,il,iu,m,w,z,lz,lisuppz,work,lwork,iwork,liwork,info)

c--------------------------------------------------------------------

c Calculates selected eigenvalues and eigenvectors of a

c symmetric matrix T.

c Based upon the dsyevr routine.

c This routine is called from the C function GetEigenvectors

c 'N' should be used if only eigenvalues are desired.

c 'V' should be used if both eigenvalues and eigenvalues are

c desired.

c For the second parameter, 'A' means all eigenvalues will be

c found. 'V' means all eigenvalues in the half-open interval

c (vl,vu] will be found. 'I' mean the eigenvalues with indices

c il through iu will be found.

c For the third parameter, 'U' means the symmetric matrix is

c stored in the upper triangle of a and 'L' means its stored in

c the lower triangle.

c "n" is the number of rows (columns) in the symmetric matrix

c "a" is the nXn array housing the symmetric matrix in its upper

c or lower triangle.

c "vl" and "vu" are the lower and upper bounds respectively of the

c interval to be searched for eigenvalues. "vl" <= "vu". These

c values are referenced only if the second parameter = 'V'

c "il" and "iu" are the indices of the smallest and largest eigenvalues

c to be returned such that 1<=il<=iu<=in. These values are referenced

c only if the second parameter = 'I'

c "abstol" is the absolute error tolerance for the eigenvalues.

c "z" is an output parameter. If 'N', this is not referenced. If

c 'V', then this will contain the "lz" eigenvectors specified by

c the range "il" through "iu".

c "m" is the total number of eigenvalues found.

c "w" is the array to which the first m eigenvalues are stored.

c "z" is the array to which the first m eigenvectors are stored.

c "isuppz" contains, upon output, the indices indicating the non-zero

c elements in "z". It has length "lisuppz".

c "work" is a workspace array of length "lwork"

c "iwork" is a workspace array of length "liwork" = 10n

c "info" is an integer. If all is ok, a 0 is output. If i, the

c algorithm did not converge and i indicates the number of

c elements of an intermediate tridiagonal form which did not

c converge to zero. If -i, the ith parameter had an illegal

c value.

c Using LAPACK.

implicit none

integer n,il,iu,m,lz,lisuppz,isuppz(lisuppz),lwork,liwork,iwork(liwork),info

real*8 a(n,n),vl,vu,abstol,dlamch,w(n),z(n,lz),work(lwork)

external dlamch

abstol = dlamch('S')

print *, 'a(final) = ', a(n,n)

print *, 'About to compute eigenvectors',il,'through',iu,'.'

print *, 'n = ',n

call dsyevr('V','I','L',n,a,n,vl,vu,il,iu,abstol,m,w,z,n,isuppz,work,lwork,iwork,liwork,info)

return

end

The code compiles. Upon execution, I receive the following error:

This routine returns eigenmode #1 (larger eigenvalue) through 10000 (smaller eigenvalue).

Regarding memory allocation for /home/mspecian/PSnoise/noiseMat_Vec/GCorMat_R6sy_fwrite: Memory allocated properly.

All values read from /home/mspecian/PSnoise/noiseMat_Vec/GCorMat_R6sy_fwrite in 49 seconds.

The final element of the average noise matrix is 48.310387.

Regarding memory allocation for w: Memory allocated properly.

Regarding memory allocation for z: Memory allocated properly.

Regarding memory allocation for work: Memory allocated properly.

Regarding memory allocation for iwork: Memory allocated properly.

Executing dsyevrf.

a(final) = 48.3103872000000

About to compute eigenvectors 56114 through 66113 .

n = 66113

MKL ERROR: Parameter 10 was incorrect on entry to DSYEVR

Parameter 10 had an illegal value.

*** glibc detected *** ./eigen.o: free(): invalid pointer: 0x00002b12ffffffff ***

======= Backtrace: =========

/lib64/libc.so.6[0x3b2566f444]

/lib64/libc.so.6(cfree+0x8c)[0x3b25672a6c]

./eigen.o[0x40723a]

./eigen.o[0x4082ea]

/lib64/libc.so.6(__libc_start_main+0xf4)[0x3b2561d8a4]

./eigen.o[0x4034a9]

======= Memory map: ========

00400000-004bc000 r-xp 00000000 00:23 2148785222 /home/mspecian/PSnoise/eigen.o

006bb000-006c2000 rw-p 000bb000 00:23 2148785222 /home/mspecian/PSnoise/eigen.o

006c2000-006c9000 rw-p 006c2000 00:00 0

1da45000-1da66000 rw-p 1da45000 00:00 0 [heap]

313fe00000-313fefc000 r-xp 00000000 08:12 196705 /usr/lib64/libfftw3.so.3.2.4

313fefc000-31400fc000 ---p 000fc000 08:12 196705 /usr/lib64/libfftw3.so.3.2.4

31400fc000-3140103000 rw-p 000fc000 08:12 196705 /usr/lib64/libfftw3.so.3.2.4

3acc400000-3acc40d000 r-xp 00000000 08:12 491255 /lib64/libgcc_s-4.1.2-20080825.so.1

3acc40d000-3acc60d000 ---p 0000d000 08:12 491255 /lib64/libgcc_s-4.1.2-20080825.so.1

3acc60d000-3acc60e000 rw-p 0000d000 08:12 491255 /lib64/libgcc_s-4.1.2-20080825.so.1

3b24400000-3b2441a000 r-xp 00000000 08:12 2714721 /lib64/ld-2.5.so

3b24619000-3b2461a000 r--p 00019000 08:12 2714721 /lib64/ld-2.5.so

3b2461a000-3b2461b000 rw-p 0001a000 08:12 2714721 /lib64/ld-2.5.so

3b25600000-3b25746000 r-xp 00000000 08:12 2714722 /lib64/libc-2.5.so

3b25746000-3b25946000 ---p 00146000 08:12 2714722 /lib64/libc-2.5.so

3b25946000-3b2594a000 r--p 00146000 08:12 2714722 /lib64/libc-2.5.so

3b2594a000-3b2594b000 rw-p 0014a000 08:12 2714722 /lib64/libc-2.5.so

3b2594b000-3b25950000 rw-p 3b2594b000 00:00 0

3b25a00000-3b25a82000 r-xp 00000000 08:12 2714723 /lib64/libm-2.5.so

3b25a82000-3b25c81000 ---p 00082000 08:12 2714723 /lib64/libm-2.5.so

3b25c81000-3b25c82000 r--p 00081000 08:12 2714723 /lib64/libm-2.5.so

3b25c82000-3b25c83000 rw-p 00082000 08:12 2714723 /lib64/libm-2.5.so

3b25e00000-3b25e02000 r-xp 00000000 08:12 2714724 /lib64/libdl-2.5.so

3b25e02000-3b26002000 ---p 00002000 08:12 2714724 /lib64/libdl-2.5.so

3b26002000-3b26003000 r--p 00002000 08:12 2714724 /lib64/libdl-2.5.so

3b26003000-3b26004000 rw-p 00003000 08:12 2714724 /lib64/libdl-2.5.so

3b26200000-3b26215000 r-xp 00000000 08:12 2714726 /lib64/libpthread-2.5.so

3b26215000-3b26414000 ---p 00015000 08:12 2714726 /lib64/libpthread-2.5.so

3b26414000-3b26415000 r--p 00014000 08:12 2714726 /lib64/libpthread-2.5.so

3b26415000-3b26416000 rw-p 00015000 08:12 2714726 /lib64/libpthread-2.5.so

3b26416000-3b2641a000 rw-p 3b26416000 00:00 0

2b09f1d32000-2b09f1d35000 rw-p 2b09f1d32000 00:00 0

2b09f1d67000-2b09f2004000 r-xp 00000000 00:20 5944241 /data1/apps0/intel_fortran-11.1.069/x64-linux/mkl/lib/em64t/libmkl_core.so

2b09f2004000-2b09f2104000 ---p 0029d000 00:20 5944241 /data1/apps0/intel_fortran-11.1.069/x64-linux/mkl/lib/em64t/libmkl_core.so

2b09f2104000-2b09f2109000 rw-p 0029d000 00:20 5944241 /data1/apps0/intel_fortran-11.1.069/x64-linux/mkl/lib/em64t/libmkl_core.so

2b09f2109000-2b09f2126000 rw-p 2b09f2109000 00:00 0

2b09f2126000-2b09f2787000 r-xp 00000000 00:20 5944270 /data1/apps0/intel_fortran-11.1.069/x64-linux/mkl/lib/em64t/libmkl_sequential.so

2b09f2787000-2b09f2887000 ---p 00661000 00:20 5944270 /data1/apps0/intel_fortran-11.1.069/x64-linux/mkl/lib/em64t/libmkl_sequential.so

2b09f2887000-2b09f2893000 rw-p 00661000 00:20 5944270 /data1/apps0/intel_fortran-11.1.069/x64-linux/mkl/lib/em64t/libmkl_sequential.so

2b09f2893000-2b09f289f000 rw-p 2b09f2893000 00:00 0

2b09f289f000-2b09f2b41000 r-xp 00000000 00:20 5944250 /data1/apps0/intel_fortran-11.1.069/x64-linux/mkl/lib/em64t/libmkl_intel_ilp64.so

2b09f2b41000-2b09f2c40000 ---p 002a2000 00:20 5944250 /data1/apps0/intel_fortran-11.1.069/x64-linux/mkl/lib/em64t/libmkl_intel_ilp64.so

2b09f2c40000-2b09f2c48000 rw-p 002a1000 00:20 5944250 /data1/apps0/intel_fortran-11.1.069/x64-linux/mkl/lib/em64t/libmkl_intel_ilp64.so

2b09f2c48000-2b09f2c52000 rw-p 2b09f2c48000 00:00 0

2b09f4000000-2b09f4021000 rw-p 2b09f4000000 00:00 0

2b09f4021000-2b09f8000000 ---p 2b09f4021000 00:00 0

2b1216fdd000-2b13547b5000 rw-p 2b1216fdd000 00:00 0

2b13547b5000-2b13552f9000 r-xp 00000000 00:20 5944257 /data1/apps0/intel_fortran-11.1.069/x64-linux/mkl/lib/em64t/libmkl_lapack.so

2b13552f9000-2b13553f9000 ---p 00b44000 00:20 5944257 /data1/apps0/intel_fortran-11.1.069/x64-linux/mkl/lib/em64t/libmkl_lapack.so

2b13553f9000-2b1355409000 rw-p 00b44000 00:20 5944257 /data1/apps0/intel_fortran-11.1.069/x64-linux/mkl/lib/em64t/libmkl_lapack.so

2b1355409000-2b135540a000 rw-p 2b1355409000 00:00 0

2b135540a000-2b135541f000 r--p 00000000 00:20 5944281 /data1/apps0/intel_fortran-11.1.069/x64-linux/mkl/lib/em64t/locale/en_US/mkl_msg.cat

7fffaa40f000-7fffaa43e000 rw-p 7ffffffd0000 00:00 0 [stack]

ffffffffff600000-ffffffffffe00000 ---p 00000000 00:00 0 [vdso]

Abort

This seems like a memory issue, but I can't find where I've gone wrong.

MKL Invalid Pointer in dsyevr (novice question)

Gennady_F_Intel — Sun, 04 Jul 2010 05:27:55 GMT

What exact MKL version you are using? look into \doc\ mklsupport.txt file. what is package_id?

--Gennady

MKL Invalid Pointer in dsyevr (novice question)

mspecian — Mon, 05 Jul 2010 20:05:08 GMT

MKL 10.2 update 4

MKL Invalid Pointer in dsyevr (novice question)

barragan_villanueva_ — Tue, 06 Jul 2010 05:08:11 GMT

Hi,

Could you please show us your link-line using MKL libraries? Please check it with MKL Link Line Adviser on the top of MKL Forum.

MKL Invalid Pointer in dsyevr (novice question)

mspecian — Tue, 13 Jul 2010 21:07:57 GMT

I figured it out. In C I was passing 4-byte integers, while in FORTRAN I was declaring 8-byte integers. This didn't always lead to obvious memory overwrite issues so it took me a while to identify. Thanks to anyone who read through.

topic MKL Invalid Pointer in dsyevr (novice question) in Intel® oneAPI Math Kernel Library

MKL Invalid Pointer in dsyevr (novice question)

MKL Invalid Pointer in dsyevr (novice question)

MKL Invalid Pointer in dsyevr (novice question)

MKL Invalid Pointer in dsyevr (novice question)

MKL Invalid Pointer in dsyevr (novice question)