Intel® MPI Library
Get help with building, analyzing, optimizing, and scaling high-performance computing (HPC) applications.
2153 Discussions

Using oneAPI buffers in MPI calls

germainf
Beginner
3,072 Views

Hello,
As far as I understand, the buffer object is an abstract view of data.
Then, an accessor is needed (with an access mode) to interact with that data.

Most of MPI implementation provide a way to use CUDA device pointers to perform MPI communication.

Is there a way to perform MPI calls with this data abstraction? (with IntelMPI? with OpenMPI?)
Does the oneAPI MKL provide parallel implementation through MPI?

Thanks

Florent Germain

Labels (1)
0 Kudos
1 Solution
PrasanthD_intel
Moderator
2,998 Views

Hi Florent,


Sorry for the late reply.

After logging into the devcloud you will be in a login node. You have to launch MPI in a compute node through a queue instead on a login node.

To run in a compute node, you must submit a job through a queue.

You can submit jobs to the queue using below command (Here you can change the nodes as per requirement, below command will allocate 2 nodes)

qsub -I -l nodes=2:ppn=2


You can see the nodefile(echo $PBS_NODEFILE) to check which nodes are allocated to you.


Let us know if you need any help


Regards

Prasanth


View solution in original post

0 Kudos
8 Replies
PrasanthD_intel
Moderator
3,052 Views

Hi Florent,

 

Yes, we can use both MPI and DPCPP and write a hybrid application. 

I am attaching a sample code and Makefile on how to use DPCPP along with MPI.

You can use OneAPI MKL with dpcpp in which you can use MKL call to launch kernels. I am attaching a GitHub repo link, please refer to this link https://github.com/intel/HPCkit-code-samples

Let us know if you need anything.

Regarding how to use MKL with IMPI I will get back to you.

Regards

Prasanth

0 Kudos
germainf
Beginner
3,044 Views

Hi,

Thanks for the answer and the examples, it will help a lot.

Regards

Florent

0 Kudos
germainf
Beginner
3,025 Views

Hi,

An error occurs when I try to run an MPI programm on devcloud:

mpirun -n 2 bin/vector_copy
Abort(2664079) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init: Other MPI error, error stack:
MPIR_Init_thread(136).........:
MPID_Init(1127)...............:
MPIDI_SHMI_mpi_init_hook(29)..:
MPIDI_POSIX_mpi_init_hook(141):
MPIDI_POSIX_eager_init(2109)..:
MPIDU_shm_seg_commit(261).....: unable to allocate shared memory

 

This seems to be an communication layer initialization failure. Is there any way to run an mpi based programm on the oneapi devcloud? I attach a copy of my source code and Makefile.

Regards

Florent

0 Kudos
PrasanthD_intel
Moderator
2,999 Views

Hi Florent,


Sorry for the late reply.

After logging into the devcloud you will be in a login node. You have to launch MPI in a compute node through a queue instead on a login node.

To run in a compute node, you must submit a job through a queue.

You can submit jobs to the queue using below command (Here you can change the nodes as per requirement, below command will allocate 2 nodes)

qsub -I -l nodes=2:ppn=2


You can see the nodefile(echo $PBS_NODEFILE) to check which nodes are allocated to you.


Let us know if you need any help


Regards

Prasanth


0 Kudos
PrasanthD_intel
Moderator
2,964 Views

Hi Florent,


Since you have marked my response as a solution, I think you are able to use IMPI and DPCPP together.

As I said I will get back to you on how to run IMPI, DPCPP and OneMKL together, here is the command below. You must link MPI and MKL libraries with dpcpp.


dpcpp -g -o<executable> <Foo.cpp> -I$MKLROOT/include $MKLROOT/lib/intel64/libmkl_sycl.a -L$MKLROOT/lib/intel64 -I$I_MPI_ROOT/include/ -L$I_MPI_ROOT/lib/release -DMKL_ILP64 -lmpi -lmkl_intel_ilp64 -lmkl_sequential -lmkl_core -lOpenCL -ldl


Let us know if you face any issues.


Regards

Prasanth


0 Kudos
PrasanthD_intel
Moderator
2,915 Views

Hi Florent,


We haven't heard back from you. If you face any issue while using those commands, please reach out to us.

Else if you found the answer helpful please confirm so we can close this thread.


Regards

Prasanth


0 Kudos
germainf
Beginner
2,908 Views

Hi,

I posted a new topic here: https://community.intel.com/t5/Intel-oneAPI-Base-Toolkit/Use-mathimf-h-with-dpcpp/td-p/1215586 with the new issue I encountered to avoid this thread to diverges.

I think this thread can be closed, thanks for your help, it helped a lot.

Regards

Florent

0 Kudos
PrasanthD_intel
Moderator
2,895 Views

Hi Florent,

Thanks for the confirmation. We are closing this thread. Any further interaction in this thread will be considered community only

Regards

Prasanth


0 Kudos
Reply