Community
cancel
Showing results for 
Search instead for 
Did you mean: 
mspecian
Beginner
53 Views

MKL Invalid Pointer in dsyevr (novice question)

I am a young researcher without much CS experience trying to use the MKL routine "dsyevr" to diagonalize a real symmetric matrix. The matrix has approximately 6e9 8-byte elements (about 32.5GB). I have successfully executed this routine before for smaller matrices, but since moving to a new machine have experienced problems. If someone could point me in the right direction, I would be most appreciative.
Tech specs: I'm running MKL version 9 (I think) on a machine with16 64-bitXeon cores using em64t. I have 128GB of memory at my disposal. I call dsyevr from FORTRAN, but access the FORTRAN subroutine from C. Here's the C code:
////////////////////////////////////////////////////////////////////////////////////////////
// This routine calculates the eigenvalues and eigenvectors of the symmetric matrix
// of doubles fwritten to the file "fnameANM" using the dsyevr LAPACK procedure. Since
// memory constraints restrict the creation of a huge, complete eigenvector file, only
// the il-th through iu-th eigenvalues and eigenvectors are returned where 1<=il<=
// iu<=nrows. The eigenvalues are stored from most positive to most negative. This
// routine is somewhat faster than the packed version.
//
//Text files are written in matrix format with the eigenvectors in the columns.
// The "il"th eigenvector is in the first column while the "iu"th is in the last.
// Fwritten files store the eigenvectors sequentially. The eigenvector corresponding to
// the largest returned eigenvalue (i.e. the "il"th one takes up elements [0 through
// (nelements-1)]. The next largest eigenvector is stored to elements [nelements through
// (2*nelements-1)] and so forth.
////////////////////////////////////////////////////////////////////////////////////////////
void GetEigenvectors(char *fnameANM, int nrows, int il, int iu) {
double *w, *z, *work, rnrows, working, *EArray;
size_t size, ee, rr;
int ii, info, lz, lzp, lisuppz, lwork, m, *iwork, liwork, cc, iil, iiu;
info = NULL;
printf("\\nThis routine returns eigenmode #%d (larger eigenvalue) through %d (smaller eigenvalue).\\n\\n",il,iu);
//dsyevr puts the largest Evl's Evc last, so this fixes that
int swap = iu;
iu = nrows - il + 1;
il = nrows - swap + 1;
lz = iu-il+1; //Number of eigenvectors to be returned.
lzp = lz; //Same variable, but this doesn't get overwritten.
lisuppz = 4*lz; //Size of the iwork array
lwork = 50*nrows; //Size of work array
liwork= 40*nrows; //Size of lwork array
//////////////////// In case of seg fault, try upping the size of these arrays.
rnrows = nrows;
working = rnrows*rnrows;
size = floor(working+0.0001);
double *a = freadDMatrix(fnameANM,size); //Dynamically allocates memory and initializes matrix
printf("The final element of the matrix is %f.\\n",a[size-1]);
w = (double *)malloc(nrows*sizeof(double));
printf("Regarding memory allocation for w: "); ErrnoDescription(errno);
z = (double *)malloc(nrows*lz*sizeof(double));
printf("Regarding memory allocation for z: "); ErrnoDescription(errno);
work = (double *)malloc(lwork*sizeof(double));
printf("Regarding memory allocation for work: "); ErrnoDescription(errno);
iwork = (int *)malloc(liwork*sizeof(int));
printf("Regarding memory allocation for iwork: "); ErrnoDescription(errno);
printf("Executing dsyevrf.\\n");
dsyevrf_(&nrows, a, &il, &iu, &m, w, z, &lz, &lisuppz, work, &lwork, iwork, &liwork, &info);
}
void dsyevrf_(int nrows, double *matrix, int *il, int *iu, int *m, double *w, double *z, int *lz, int *lisuppz, double *work, int *lwork, int *iwork, int *liwork, int *info);
int main ()
{
//////////////////////////////////////////////////////////////////////////////////////////
///////////////////////////////// GetEigenvectors /////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////////////////
size_t ncells6 = 66113;
char *fnameGCorr_sy = "/home/mspecian/PSnoise/noiseMat_Vec/GCorMat_R6sy_fwrite";
int il = 1;
int iu = 10000;
/*GetEigenvectors[1. Name of symmetric, fwritten ncells by ncells covariance function
to be diagonalized
2. Dimension of above matrix, also # of cells
3. Value >= 1 equaling # of first eigenvector returned (1 is largest)
4. Value >= il and <= ncells equaling # of final eigenvector returned]*/
GetEigenvectors(fnameGCorr_sy, ncells6, il, iu);
return 0;
}
The functiondsyevrf_ is written in FORTRAN. Here is how I define the subroutine:
c********************************************************************
subroutine dsyevrf(n,a,il,iu,m,w,z,lz,lisuppz,work,lwork,iwork,liwork,info)
c--------------------------------------------------------------------
c Calculates selected eigenvalues and eigenvectors of a
c symmetric matrix T.
c
c Based upon the dsyevr routine.
c
c This routine is called from the C function GetEigenvectors
c
c 'N' should be used if only eigenvalues are desired.
c 'V' should be used if both eigenvalues and eigenvalues are
c desired.
c For the second parameter, 'A' means all eigenvalues will be
c found. 'V' means all eigenvalues in the half-open interval
c (vl,vu] will be found. 'I' mean the eigenvalues with indices
c il through iu will be found.
c For the third parameter, 'U' means the symmetric matrix is
c stored in the upper triangle of a and 'L' means its stored in
c the lower triangle.
c "n" is the number of rows (columns) in the symmetric matrix
c "a" is the nXn array housing the symmetric matrix in its upper
c or lower triangle.
c "vl" and "vu" are the lower and upper bounds respectively of the
c interval to be searched for eigenvalues. "vl" <= "vu". These
c values are referenced only if the second parameter = 'V'
c "il" and "iu" are the indices of the smallest and largest eigenvalues
c to be returned such that 1<=il<=iu<=in. These values are referenced
c only if the second parameter = 'I'
c "abstol" is the absolute error tolerance for the eigenvalues.
c "z" is an output parameter. If 'N', this is not referenced. If
c 'V', then this will contain the "lz" eigenvectors specified by
c the range "il" through "iu".
c "m" is the total number of eigenvalues found.
c "w" is the array to which the first m eigenvalues are stored.
c "z" is the array to which the first m eigenvectors are stored.
c "isuppz" contains, upon output, the indices indicating the non-zero
c elements in "z". It has length "lisuppz".
c "work" is a workspace array of length "lwork"
c "iwork" is a workspace array of length "liwork" = 10n
c "info" is an integer. If all is ok, a 0 is output. If i, the
c algorithm did not converge and i indicates the number of
c elements of an intermediate tridiagonal form which did not
c converge to zero. If -i, the ith parameter had an illegal
c value.
c
c Using LAPACK.
c
implicit none
integer n,il,iu,m,lz,lisuppz,isuppz(lisuppz),lwork,liwork,iwork(liwork),info
real*8 a(n,n),vl,vu,abstol,dlamch,w(n),z(n,lz),work(lwork)
external dlamch
c
abstol = dlamch('S')
print *, 'a(final) = ', a(n,n)
print *, 'About to compute eigenvectors',il,'through',iu,'.'
print *, 'n = ',n
c
call dsyevr('V','I','L',n,a,n,vl,vu,il,iu,abstol,m,w,z,n,isuppz,work,lwork,iwork,liwork,info)
c
return
end
c
c
The code compiles. Upon execution, I receive the following error:
This routine returns eigenmode #1 (larger eigenvalue) through 10000 (smaller eigenvalue).
Regarding memory allocation for /home/mspecian/PSnoise/noiseMat_Vec/GCorMat_R6sy_fwrite: Memory allocated properly.
All values read from /home/mspecian/PSnoise/noiseMat_Vec/GCorMat_R6sy_fwrite in 49 seconds.
The final element of the average noise matrix is 48.310387.
Regarding memory allocation for w: Memory allocated properly.
Regarding memory allocation for z: Memory allocated properly.
Regarding memory allocation for work: Memory allocated properly.
Regarding memory allocation for iwork: Memory allocated properly.
Executing dsyevrf.
a(final) = 48.3103872000000
About to compute eigenvectors 56114 through 66113 .
n = 66113
MKL ERROR: Parameter 10 was incorrect on entry to DSYEVR
Parameter 10 had an illegal value.
*** glibc detected *** ./eigen.o: free(): invalid pointer: 0x00002b12ffffffff ***
======= Backtrace: =========
/lib64/libc.so.6[0x3b2566f444]
/lib64/libc.so.6(cfree+0x8c)[0x3b25672a6c]
./eigen.o[0x40723a]
./eigen.o[0x4082ea]
/lib64/libc.so.6(__libc_start_main+0xf4)[0x3b2561d8a4]
./eigen.o[0x4034a9]
======= Memory map: ========
00400000-004bc000 r-xp 00000000 00:23 2148785222 /home/mspecian/PSnoise/eigen.o
006bb000-006c2000 rw-p 000bb000 00:23 2148785222 /home/mspecian/PSnoise/eigen.o
006c2000-006c9000 rw-p 006c2000 00:00 0
1da45000-1da66000 rw-p 1da45000 00:00 0 [heap]
313fe00000-313fefc000 r-xp 00000000 08:12 196705 /usr/lib64/libfftw3.so.3.2.4
313fefc000-31400fc000 ---p 000fc000 08:12 196705 /usr/lib64/libfftw3.so.3.2.4
31400fc000-3140103000 rw-p 000fc000 08:12 196705 /usr/lib64/libfftw3.so.3.2.4
3acc400000-3acc40d000 r-xp 00000000 08:12 491255 /lib64/libgcc_s-4.1.2-20080825.so.1
3acc40d000-3acc60d000 ---p 0000d000 08:12 491255 /lib64/libgcc_s-4.1.2-20080825.so.1
3acc60d000-3acc60e000 rw-p 0000d000 08:12 491255 /lib64/libgcc_s-4.1.2-20080825.so.1
3b24400000-3b2441a000 r-xp 00000000 08:12 2714721 /lib64/ld-2.5.so
3b24619000-3b2461a000 r--p 00019000 08:12 2714721 /lib64/ld-2.5.so
3b2461a000-3b2461b000 rw-p 0001a000 08:12 2714721 /lib64/ld-2.5.so
3b25600000-3b25746000 r-xp 00000000 08:12 2714722 /lib64/libc-2.5.so
3b25746000-3b25946000 ---p 00146000 08:12 2714722 /lib64/libc-2.5.so
3b25946000-3b2594a000 r--p 00146000 08:12 2714722 /lib64/libc-2.5.so
3b2594a000-3b2594b000 rw-p 0014a000 08:12 2714722 /lib64/libc-2.5.so
3b2594b000-3b25950000 rw-p 3b2594b000 00:00 0
3b25a00000-3b25a82000 r-xp 00000000 08:12 2714723 /lib64/libm-2.5.so
3b25a82000-3b25c81000 ---p 00082000 08:12 2714723 /lib64/libm-2.5.so
3b25c81000-3b25c82000 r--p 00081000 08:12 2714723 /lib64/libm-2.5.so
3b25c82000-3b25c83000 rw-p 00082000 08:12 2714723 /lib64/libm-2.5.so
3b25e00000-3b25e02000 r-xp 00000000 08:12 2714724 /lib64/libdl-2.5.so
3b25e02000-3b26002000 ---p 00002000 08:12 2714724 /lib64/libdl-2.5.so
3b26002000-3b26003000 r--p 00002000 08:12 2714724 /lib64/libdl-2.5.so
3b26003000-3b26004000 rw-p 00003000 08:12 2714724 /lib64/libdl-2.5.so
3b26200000-3b26215000 r-xp 00000000 08:12 2714726 /lib64/libpthread-2.5.so
3b26215000-3b26414000 ---p 00015000 08:12 2714726 /lib64/libpthread-2.5.so
3b26414000-3b26415000 r--p 00014000 08:12 2714726 /lib64/libpthread-2.5.so
3b26415000-3b26416000 rw-p 00015000 08:12 2714726 /lib64/libpthread-2.5.so
3b26416000-3b2641a000 rw-p 3b26416000 00:00 0
2b09f1d32000-2b09f1d35000 rw-p 2b09f1d32000 00:00 0
2b09f1d67000-2b09f2004000 r-xp 00000000 00:20 5944241 /data1/apps0/intel_fortran-11.1.069/x64-linux/mkl/lib/em64t/libmkl_core.so
2b09f2004000-2b09f2104000 ---p 0029d000 00:20 5944241 /data1/apps0/intel_fortran-11.1.069/x64-linux/mkl/lib/em64t/libmkl_core.so
2b09f2104000-2b09f2109000 rw-p 0029d000 00:20 5944241 /data1/apps0/intel_fortran-11.1.069/x64-linux/mkl/lib/em64t/libmkl_core.so
2b09f2109000-2b09f2126000 rw-p 2b09f2109000 00:00 0
2b09f2126000-2b09f2787000 r-xp 00000000 00:20 5944270 /data1/apps0/intel_fortran-11.1.069/x64-linux/mkl/lib/em64t/libmkl_sequential.so
2b09f2787000-2b09f2887000 ---p 00661000 00:20 5944270 /data1/apps0/intel_fortran-11.1.069/x64-linux/mkl/lib/em64t/libmkl_sequential.so
2b09f2887000-2b09f2893000 rw-p 00661000 00:20 5944270 /data1/apps0/intel_fortran-11.1.069/x64-linux/mkl/lib/em64t/libmkl_sequential.so
2b09f2893000-2b09f289f000 rw-p 2b09f2893000 00:00 0
2b09f289f000-2b09f2b41000 r-xp 00000000 00:20 5944250 /data1/apps0/intel_fortran-11.1.069/x64-linux/mkl/lib/em64t/libmkl_intel_ilp64.so
2b09f2b41000-2b09f2c40000 ---p 002a2000 00:20 5944250 /data1/apps0/intel_fortran-11.1.069/x64-linux/mkl/lib/em64t/libmkl_intel_ilp64.so
2b09f2c40000-2b09f2c48000 rw-p 002a1000 00:20 5944250 /data1/apps0/intel_fortran-11.1.069/x64-linux/mkl/lib/em64t/libmkl_intel_ilp64.so
2b09f2c48000-2b09f2c52000 rw-p 2b09f2c48000 00:00 0
2b09f4000000-2b09f4021000 rw-p 2b09f4000000 00:00 0
2b09f4021000-2b09f8000000 ---p 2b09f4021000 00:00 0
2b1216fdd000-2b13547b5000 rw-p 2b1216fdd000 00:00 0
2b13547b5000-2b13552f9000 r-xp 00000000 00:20 5944257 /data1/apps0/intel_fortran-11.1.069/x64-linux/mkl/lib/em64t/libmkl_lapack.so
2b13552f9000-2b13553f9000 ---p 00b44000 00:20 5944257 /data1/apps0/intel_fortran-11.1.069/x64-linux/mkl/lib/em64t/libmkl_lapack.so
2b13553f9000-2b1355409000 rw-p 00b44000 00:20 5944257 /data1/apps0/intel_fortran-11.1.069/x64-linux/mkl/lib/em64t/libmkl_lapack.so
2b1355409000-2b135540a000 rw-p 2b1355409000 00:00 0
2b135540a000-2b135541f000 r--p 00000000 00:20 5944281 /data1/apps0/intel_fortran-11.1.069/x64-linux/mkl/lib/em64t/locale/en_US/mkl_msg.cat
7fffaa40f000-7fffaa43e000 rw-p 7ffffffd0000 00:00 0 [stack]
ffffffffff600000-ffffffffffe00000 ---p 00000000 00:00 0 [vdso]
Abort
This seems like a memory issue, but I can't find where I've gone wrong.
0 Kudos
4 Replies
Gennady_F_Intel
Moderator
53 Views

What exact MKL version you are using? look into \doc\ mklsupport.txt file. what is package_id?
--Gennady
mspecian
Beginner
53 Views

MKL 10.2 update 4
barragan_villanueva_
Valued Contributor I
53 Views

Hi,

Could you please show us your link-line using MKL libraries? Please check it with MKL Link Line Adviser on the top of MKL Forum.

mspecian
Beginner
53 Views

I figured it out. In C I was passing 4-byte integers, while in FORTRAN I was declaring 8-byte integers. This didn't always lead to obvious memory overwrite issues so it took me a while to identify. Thanks to anyone who read through.
Reply