sparse matrix multiplication and segmentation fault?

utab · ‎05-08-2011

Dear all,

I am trying to interface some boost ublas csr matrices to mkl sparse matrix routines, I installed and compiled the library without problems and my code is also compiling and linking fine. However, I am getting a segmentation fault during the operation,

I used g++ as the compiler. Here is the link output of the makefile

g++ -DNDEBUG -O3 -o test_mkl_sparse csr_boost_intel.o boost_matrix_utilities.o -L/home/utab/intel/composerxe-2011.3.174/mkl/lib/intel64 -L /home/utab/thesis/intel_mkl \\
-lboost_filesystem -lboost_system -Wl,--start-group -lmkl_intel_ilp64 -lmkl_gnu_thread -lmkl_core -Wl,--end-group -fopenmp -lpthread

And here is my naive matrix-vector multiplication code:

#include
#include
#include

#include
#include

#include "boost_matrix_utilities.h"
#include "mkl_spblas.h"
#include "mkl_types.h"

using std::map;
using std::cout;
using std::endl;
using VibroSys::read_triplet_data;

int main () {
using namespace boost::numeric::ublas;
compressed_matrix m (3, 3, false);
for (unsigned i = 0; i < m.size1 (); ++ i)
for (unsigned j = 0; j < m.size2 (); ++ j)
m (i, j) = 3 * i + j;
std::cout << m << std::endl;
char transa('n');
MKL_INT n = m.size1();
//
std::vector ra;
std::vector ia;
std::vector ja;
std::vector a;
map > row_col;

read_triplet_data( m , ra, ia, ja, a, row_col ); // ra row pointer array
// ia and ja row and column indices of nnz
// a are the nnz values, row_col is not related in this context.

cout << " Row pointers size : " << ra.size() << endl;
cout << " Ja size :" << ja.size() << endl;
MKL_INT mkl_ia[ra.size()];
MKL_INT mkl_ja[ja.size()];
double mkl_aa[ja.size()];

int c = 0;
for(std::vector::iterator i = ra.begin();
i!=ra.end();++i)
{
mkl_ia = *i;
c++;
}
c = 0;
for(std::vector::iterator k = ja.begin();
k!=ja.end();++k)
{
mkl_ja = *k;
c++;
}
c = 0;
for(std::vector::iterator k = a.begin();
k!=a.end();++k)
{
mkl_aa = *k;
c++;
}
double x[] = {1.,2.,3.};
//
double* y;
// sparse matrix vector multiplication
mkl_cspblas_dcsrgemv(&transa, &n, mkl_aa, mkl_ia, mkl_ja, x, y);
//
return 0;
}

doing a backtrace in gdb results in

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff48c1422 in memset () from /lib/libc.so.6
(gdb) backtrace
#0 0x00007ffff48c1422 in memset () from /lib/libc.so.6
#1 0x00007ffff36d003f in mkl_spblas_dzeros () from /home/utab/intel/composerxe-2011.3.174/mkl/lib/intel64/libmkl_mc.so
#2 0x00007ffff587e00f in mkl_spblas_dzeros () from /home/utab/intel/composerxe-2011.3.174/mkl/lib/intel64/libmkl_core.so
#3 0x00007ffff68b5719 in mkl_spblas_mkl_cspblas_dcsrgemv ()
from /home/utab/intel/composerxe-2011.3.174/mkl/lib/intel64/libmkl_gnu_thread.so
#4 0x00007ffff7317d65 in mkl_cspblas_dcsrgemv__ ()
from /home/utab/intel/composerxe-2011.3.174/mkl/lib/intel64/libmkl_intel_ilp64.so
#5 0x0000000000401a7b in main () at csr_boost_intel.cc:66

I could not resolve this problem, could someone comment on this problem?

Thanks in advance.
Umut

Ps: I have some routines to analyze the structure of the matrices, read_triplet_data, that is tested in other context and gives the row pointer array, row indices, col incided and values of nnz correctly,

utab · ‎05-08-2011

The answer is

double y[3]

so output array should also be allocated...

TimP · ‎05-08-2011

As far as I know, it's not satisfactory to have implied -lgomp (by setting -fopenmp) when you are using an mkl_xxx_thread library, which requires -liomp5 in place of -lgomp. Maybe the mkl_gnu_thread took care of that.
In order to use libmkl_intel_ilp64, you need to assure that all int pointers sent to MKL refer to 64-bit ints (long long, or 64-bit long) and assure that the MKL include directories are the ilp64 ones.
In addition, you told us nothing about whether whether you have checked stack settings.

utab · ‎05-09-2011

first of all I am not doing things in parallel, actually I can use the library in sequential mode. I used the linker advisor from the website and it gave me -fopenmp option most probably and that is the reason why it is there. But now I updated my linker options to

-L$(MKLROOT)/lib/intel64 -Wl,--start-group -lmkl_intel_ilp64 -lmkl_sequential -lmkl_core -Wl,--end-group -lpthread

by changing to sequential dynamic linking on gnu c++ compiler using ilp64 and I did not select any cluster libraries since sequential mpi options are not accesible. Compiles and links fine.

I use the "mkl_types.h" to use the MKL_INT to define my arrays in C++, if this is what you mean by 64 bit integer conformance.

Your last question is not clear to me, what do you mean by stack settings, can your please rephrase that?

One more question since last night.
I am trying to multiply a 36234X36234 matrix by a dense vector with the same code. It thas732322 nnz. I am running into a strange problem

#include
#include
#include
#include
#include

#include
#include
#include

#include "boost_matrix_utilities.h"
#include "matrixMarketIO.h"
#include "mkl_spblas.h"
#include "mkl_types.h"

using std::map;
using std::cout;
using std::endl;
using std::copy;
using std::ostream_iterator;
using VibroSys::read_triplet_data;
using boost::timer;

int main () {
using namespace boost::numeric::ublas;
compressed_matrix K;
matrixMarketIO matrix_market_interface;
matrix_market_interface.loadMatrix( "K.mm", K ) ; // I use another class of mine to inteface matrices
char transa = 'N';
MKL_INT n = K.size1();
//
std::vector ra;
std::vector ia;
std::vector ja;
std::vector a;
map > row_col;
//
read_triplet_data( K,ra,ia,ja,a,row_col );
MKL_INT ra_sz = ra.size();
MKL_INT nnz = ja.size();
MKL_INT mkl_ia[ra_sz];
// MKL_INT mkl_ja[nnz]; // uncommenting these two lines, I get a segfault from size() below?
// double mkl_aa[nnz];

cout << " Size of ra : " << ra.size() << endl;
return 0;
}

works fine if I uncomment the two other array declerations, I start to get a segfault from ra.size(). what is the reason of this segfault, most probably there is a problem related to my mkl usage and its binding to stl.

Best,
Umut

TimP · ‎05-09-2011

As you must be aware, it's often necessary to increase the stack limit in your shell, the exact command depending on the shell ('ulimit -s unlimited' or 'limit stacksize ....'). unlimited doesn't mean what it says, it simply means the largest permitted. I don't think you can tell whether a segfault is due to stack, except by trying an increase, or running under debugger.
I thought you had to set the include path explicitly to the ilp64 directory to get mkl includes to work with ilp64.

utab · ‎05-09-2011

Dear Tim,

Indeed, the stack size was a problem for my previous post, thank you for mentioning that... Now that work works with the stack size set to maximum permitted. I guess this also explains the reason of the segfault.

Best,
Umut

walkershane123 · ‎07-05-2011

hey umut, When the program is executed I tried programming in some test messages and the program appears to execute all the loops but it returns the segmentation fault at the end. Of course, this is all just speculation. Any help would be awesome.

Wedding Hairstyles