open MPI e intel MPI DATATYPE

[diedro]

Dear all,

I have notice small difference between OPEN-MPI and intel MPI. 

For example in MPI_ALLREDUCE in intel MPI is not allowed to use the same variable in send and receiving Buff.

I have written my code in OPEN-MPI, run in on a intel-MPI cluster. 

Now I have the following error:

Fatal error in MPI_Isend: Invalid communicator, error stack:
MPI_Isend(158): MPI_Isend(buf=0x1dd27b0, count=1, INVALID DATATYPE, dest=0, tag=0, comm=0x0, request=0x7fff9d7dd9f0) failed

This is how I create my type:

  CALL  MPI_TYPE_VECTOR(1, Ncoeff_MLS, Ncoeff_MLS, MPI_DOUBLE_PRECISION, coltype, MPIdata%iErr) 
  CALL  MPI_TYPE_COMMIT(coltype, MPIdata%iErr)
  !
  CALL  MPI_TYPE_VECTOR(1, nVar, nVar, coltype, MPI_WENO_TYPE, MPIdata%iErr) 
  CALL  MPI_TYPE_COMMIT(MPI_WENO_TYPE, MPIdata%iErr)

do you believe that is here the problem?

Is also this the way how intel MPI create a datatype?

What do you think?

[Kevin_D_Intel]

I redirecting your question to our Intel® Clusters and HPC Technology to tap the expertise of our MPI team and community.

[Artem_R_Intel1]

Hello,

Which version of Intel MPI Library do you use?

Regarding to:

For example in MPI_ALLREDUCE in intel MPI is not allowed to use the same variable in send and receiving Buff.

You can set I_MPI_COMPATIBILITY=4 environment variable to avoid this limitation.

There's the following note in the Intel(R) MPI Library for Linux* OS Release Notes:

    - Intel(R) MPI Library 4.1 for Linux* OS implements the MPI-2.2
      standard.
      On top of this, the aliasing of the send and receive buffers in the
      following collective routines will be rejected:
      o MPI_Gather, MPI_Gatherv
      o MPI_Scatter, MPI_Scatterv
      o MPI_Allgather, MPI_Allgatherv
      o MPI_Alltoall, MPI_Alltoallv, MPI_Alltoallw

      If your application depends on the pre-MPI-2.2 behavior, set the
      environment variable I_MPI_COMPATIBILITY to 4. If your application
      depends on the pre-MPI-2.1 behavior, set the environment variable
      I_MPI_COMPATIBILITY to 3.

      You will see the diagnostic messages below in case of buffer aliasing
      issue:
      MPIR_Localcopy(381).......: memcpy arguments alias each other
     
      Known applications with such kind of limitations:
      SPEC MPI2007 V2.0, VASP 5.3

MPI_ALLREDUCE isn't mentioned in the note, but this limitation and W/A is applicable for it also.

[diedro]

Dear Artem, Dear all,

when I run 

mpiifort --version

I get

ifort (IFORT) 12.1.4 20120410
Copyright (C) 1985-2012 Intel Corporation.  All rights reserved.

how can i understand MPI version? Is this what you need?

Regardin the othes part:

I use in some part of my code MPI_ALLGATHER.

How can I set  I_MPI_COMPATIBILITY ?

Thanks a lot

[Artem_R_Intel1]

You can get the Intel MPI Library version for example with the command 'mpirun -V'.

How can I set  I_MPI_COMPATIBILITY ?

In Linux you can set the environment variables with 'export' command before the mpirun call:

$ export I_MPI_COMPATIBILITY=4

$ mpirun ...

[TimP]

If you are switching between intel and openmpi you must remember not to mix environment.  You might use modules to manage this.  As the data types encodings differ, you must take care that all objects are built against the same headers.

[diedro]

Dear Tim  Dear all,

The problem is that I am running on my laptop openMPI. This is usefull for me to run amf test my code.The cluster where I run my code uses intel MPI.

what do you mean for

use modules to manage this

?

thanks

Diego

[TimP]

If you are willing to manage it yourself, you remove openmpi (such as your linux distro might provide) from the default paths and build it and install with --prefix, e.g. --prefix=/opt/ompi1.8/x86_64/

then set PATH and LD_LIBRARY_PATH accordingly for each MPI you use (mpivars script will do it for Intel MPI).

You will have trouble with OpenMPI itself if you want to use a better version than your distro provides, if you don't remove that version.

[John_D_6]

the MPI standard (see www.mpi-forum.org) has a special value MPI_IN_PLACE that can be used when the send and receive buffer are the same. For MPI_Allreduce, the standard says:

The “in place” option for intracommunicators is specified by passing the value MPI_IN_PLACE to the argument sendbuf at all processes.

About the MPI types: have you checked the error values of MPI_TYPE_VECTOR and MPI_TYPE_COMMIT?