C examples in MKL's ScaLAPACK

[Georgios_S_]

I am aware of these examples, but I do not know any Fortran, thus I can not understand much. So finding at least an example with MKL and ScaLAPACK in C would be critical for me. I know there is a C interface. For example p?potrf will be the function I am going to use, for performing a Cholesky factorization.

If someone would reply negatively, that would be also OK, since it would stop me for searching..

Some links that have some interest (and I am reading now):

1. Intel 11
2. MKL ScaLAPACK problems
3. cholesky ScaLAPACK error

[mecej4]

Have you seen http://acts.nersc.gov/scalapack/, where you can see four (small number, but > 0!) C examples?

Converting some Fortran-77 examples to C would be possible, but is there a "market" for such examples, considering that Scalapack was designed for MIMD computers with a message-passing interface, whereas most popular machines today use SIMD capable multi-core processors with shared-memory architecture?

[Ying_H_Intel · Accepted solution]

Hi Georgios, 

You may find one C dgesv sample in https://software.intel.com/en-us/forums/topic/536962. ;

In most of case, it is not problem for  the scalapack call  (just align with the parameters, even you is not familiar with fortran ), more often, there is small error in other part of the coding. 

For  example, i try the C pddttrf sample,  (which you mentioned in   http://stackoverflow.com/questions/23722060/. with Intel Compiler and Intel MPI , Intel MKL .

source /opt/intel/bin/compilervars.sh intel64

>source /opt/intel/impi/5.1.0.042/bin64/mpivars.sh

>mpiicpc Scalapack_pddtrf.cpp -mkl:cluster

[yhu5@prc-mic01 scalapack]$run -np 2 ./a.out
MYPE=0: x[:] =  1.0000  2.0000  3.0000  4.0000
MYPE=1: x[:] =  5.0000  6.0000  7.0000  8.0000

it is run ok  with 2 rank.  as the code designed. 

If run with 1 rank, it return error as expected. 

[yhu5@prc-mic01 scalapack]$ mpirun -np 1 ./a.out
{    0,    0}:  On entry to
PDDTTRF, D&C alg.: only 1 block per proc parameter number    1 had an illegal value
pddttrf problem! Info -1
MYPE=0: x[:] =  1.0000  2.0000  3.0000  4.0000

The user http://stackoverflow.com/questions/23722060/, use MPICH2,  not sure how they compile and link mkl, it seems the code go wrong at first Cblacs_get(0, 0, &context);   his is { -1, -1}:

Best Regards,

Ying

[Georgios_S_]

Thanks both! Regarding the example of Ying, I tried to compile, but I got an error

gsamaras@pythagoras:~/intel/code$ ../bin/icc -mkl nine_by_nine.c -o n
nine_by_nine.c(1): catastrophic error: cannot open source file "mpi.h"
  #include <mpi.h>

what am I missing? Probably it's a linking error. You see, I am an absolute beginner!

This did not help much. However, MPI seems to be OK:

gsamaras@pythagoras:~/intel$ mpiexec --version
HYDRA build details:
    Version:                                 3.1
    Release Date:                            Thu Feb 20 11:41:13 CST 2014
    CC:                              gcc -D_FORTIFY_SOURCE=2 -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wl,-z,relro
    CXX:                             g++ -D_FORTIFY_SOURCE=2 -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wl,-z,relro
    F77:                             gfortran -g -O2 -fstack-protector-strong -Wl,-z,relro
    F90:                             gfortran -g -O2 -fstack-protector-strong -Wl,-z,relro
    Configure options:                       '--disable-option-checking' '--prefix=/usr' '--build=x86_64-linux-gnu' '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--libdir=${prefix}/lib/x86_64-linux-gnu' '--libexecdir=${prefix}/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--enable-shared' '--enable-fc' '--disable-rpath' '--disable-wrapper-rpath' '--sysconfdir=/etc/mpich' '--libdir=/usr/lib/x86_64-linux-gnu' '--includedir=/usr/include/mpich' '--docdir=/usr/share/doc/mpich' '--with-hwloc-prefix=system' '--enable-checkpointing' '--with-hydra-ckpointlib=blcr' 'build_alias=x86_64-linux-gnu' 'MPICHLIB_CFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security' 'MPICHLIB_CXXFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security' 'MPICHLIB_FFLAGS=-g -O2 -fstack-protector-strong' 'MPICHLIB_FCFLAGS=-g -O2 -fstack-protector-strong' 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -O2' 'LDFLAGS=-Wl,-z,relro ' 'CPPFLAGS=-D_FORTIFY_SOURCE=2 -I/build/mpich-Lgqv02/mpich-3.1/src/mpl/include -I/build/mpich-Lgqv02/mpich-3.1/src/mpl/include -I/build/mpich-Lgqv02/mpich-3.1/src/openpa/src -I/build/mpich-Lgqv02/mpich-3.1/src/openpa/src -I/build/mpich-Lgqv02/mpich-3.1/src/mpi/romio/include' 'CXXFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -g -O2 -fstack-protector-strong -Wformat -Werror=format-security' 'F77=gfortran' 'FFLAGS=-g -O2 -fstack-protector-strong -g -O2 -fstack-protector-strong -O2' 'FC=gfortran' 'FCFLAGS=-g -O2 -fstack-protector-strong -g -O2 -fstack-protector-strong' '--cache-file=/dev/null' '--srcdir=.' 'CC=gcc' 'LIBS=-lrt -lcr -lpthread '
    Process Manager:                         pmi
    Launchers available:                     ssh rsh fork slurm ll lsf sge manual persist
    Topology libraries available:            hwloc
    Resource management kernels available:   user slurm ll lsf sge pbs cobalt
    Checkpointing libraries available:       blcr
    Demux engines available:                 poll select

[TimP]

The usual way of satisfying #include <mpi.h>, as Ying said, is to use mpiicc in place of icc (mpicc for most non-Intel MPI implementations).  With mpicc, you won't have available the -mkl link shortcuts, so will need to check docs such as MKL link advisor.

[Georgios_S_]

I tried that (but forgot to edit, sorry), but it fails, because first of all I couldn't find the mpi script that I need to source, let alone the compiler. By the way Tim said "mpiicc", while Ying used "mpiicpc". 

I just want to compile the example, so if I could avoid the link advisor, it would be nice.

Here is my work space:

gsamaras@pythagoras:~/intel$ ls
advisor_xe                   code                    include                       intel_sdp_products.db  licenses                 tbb
advisor_xe_2015              composerxe              inspector_xe                  ipp                    man                      vtune_amplifier_xe
advisor_xe_2015.1.10.380555  composer_xe_2015        inspector_xe_2015             ism                    mkl                      vtune_amplifier_xe_2015
bin                          composer_xe_2015.3.187  inspector_xe_2015.1.2.379161  lib                    parallel_studio_xe_2015  vtune_amplifier_xe_2015.3.0.403110
g

[Ying_H_Intel]

Hi Georgios, 

The mpi.h is from one mpi install directory, You may get Intel MPI library from https://software.intel.com/en-us/intel-mpi-library/ ( on the right of box). 

If you have one mpi installed, you may find it on your machine and compile command like: , 

mpicc *.c -I<MPI install header path> -I/opt/intel/mkl/include   ${MKLROOT}/lib/intel64/libmkl_scalapack_lp64.a -Wl,--start-group ${MKLROOT}/lib/intel64/libmkl_intel_lp64.a ${MKLROOT}/lib/intel64/libmkl_core.a ${MKLROOT}/lib/intel64/libmkl_intel_thread.a -Wl,--end-group ${MKLROOT}/lib/intel64/libmkl_blacs_intelmpi_lp64.a -liomp5 -ldl -lpthread -lm

You can see the details in https://software.intel.com/en-us/articles/intel-mkl-link-line-advisor/

Best Regards,

Ying 

[Georgios_S_]

Hi Ying,

  the product I have already installed is: Intel® Parallel Studio XE Professional Edition for C++ Linux* and as far as I can see from the link you provided, you are suggesting me to download the same product again, not an MPI-Intel standalone one (which I would expect to see in the link). As a result, the MPI should already be installed, since I installed the product, right? The question is where is it?

In post #24, from here, you can see my installation summary. I do not see mpi stated anywhere. From the link you provided, I can see only the Buy option for Intel MPI library. I am a student though, thus I want the package for student use. When on free software tools, it redirects me to a page where I can get Parallel Studio XE, which I already have, so I am stuck again. :/

Thanks,

George

[Ying_H_Intel]

Hi George,

So have you installed one Opensoruce version of MPI?

If Intel MPI, I saw there is 30-day free download version:

Download Linux version

Intel MPI is not in Intel® Parallel Studio XE Professional Edition for C++ Linux*,  it is in Intel® Parallel Studio XE Cluster Edition.

Professional  vs Cluster. 

Best Regards,

Ying

[Georgios_S_]

I saw that too, however, my project is the final of the class, thus it will need to last more than 1 month. Cluster is not provided for students (what a pity!), thus I am forced to install an Opensource version of MPI. Hopefully then I will be able to compile the code as you suggested in your pre-last post.

Can you suggest an Opensource version of MPI? I found this, which is for Ubuntu and I am on Debian. Maybe this for Debian?

Thanks,

George

[TimP]

Either mpich2 or OpenMPI should work, provided you are willing to follow the instructions.  Note that the data types encoding varies from one MPI to another, so if you have pre-built libraries they must support your MPI.

I'm not certain that you couldn't use the Intel MPI trial for more than a month, provided that it is for your own use.  You could ask on the licensing forum.

[Georgios_S_]

Thanks Tim. Well, having Parallel studio should imply access to Intel's MPI, in my humble opinion of course. I made a post in the licensing forum. Hope they let me use it, since it's really important for me.

The answer was no, so I will go for MPICH2, since it has nice instructions. :)

However, my professor can apply for a Intel® Parallel Studio XE (Bundle of Cluster Edition and Professionals Edition) (read more here). Hope MPI will be provided! Moreover, I hope I won't have to set up the MPI cluster myself, like the tutorial I linked does, since it didn't work (if anybody can confirm that studio XE will get me by this problem, that would be great!).

[Ying_H_Intel]

Hi Georgios,

No sure if your professior get the Intel MPI version.  But whatever, you can go ahead for MPICH2, it should be work ok with Opensource Scalapack and mkl Scalapack.

Best Regards,
Ying

[Joe_W_1]

Sorry for reposting if this has been asked before:

Scalapack has a set of TOOLS functions such as: SL_init, descinit, etc.  I didn't see these declared in mkl_scalapack.h, even though they are in libmkl_scalapack_lp64.so.  Is there a separate include file that declared these functions to be used in C/C++?  If not, is there a reason that these functions are not in mkl_scalapack.h?

Thanks.

[Ying_H_Intel]

Hi Joe, 

Thanks for asking this. 

Our doc team plan fix them in the next releases. (the latest release one may  be 2017 update 1) 

Thanks

Ying