topic Re: Re:Cannot run a simple Boost.MPI test code in Intel® MPI Library

Cannot run a simple Boost.MPI test code

geza_szarnyathy — Wed, 07 Apr 2021 14:43:25 GMT

Here is a very simple test program (`mpiboost.cc`) that uses Boost.MPI:

#include <boost/mpi/environment.hpp> #include <boost/mpi/communicator.hpp> #include <iostream> namespace bmpi = boost::mpi; int main(int argc, char* argv[]) { bmpi::environment env(argc, argv); bmpi::communicator world; if (world.rank() == 0) { std::cout << "MPI thread level: " << bmpi::environment::thread_level() << std::endl; } std::cout << "I am process " << world.rank() << " of " << world.size() << "." << std::endl; return 0; }

This is how I compiled it (`/usr/lib/x86_64-linux-gnu` is where the Boost libraries live):

mpiicpc mpiboost.cc -L/usr/lib/x86_64-linux-gnu -lboost_mpi -lboost_serialization -o mpiboost

This is the linker warning that I got:

ld: warning: libmpi.so.12, needed by /glob/development-tools/versions/oneapi/2021.2/inteloneapi/mpi/2021.2.0/lib/libmpicxx.so, may conflict with libmpi.so.20

and then I ran it:

mpirun -np 2 ./mpiboost

And here are the errors that I have gotten.

First a probably harmless complaint about missing network interfaces:

[[50429,1],0]: A high-performance Open MPI point-to-point messaging module was unable to find any relevant network interfaces: Module: OpenFabrics (openib) Host: s001-n028 Another transport will be used instead, although this may result in lower performance. NOTE: You can disable this warning by setting the MCA parameter btl_base_warn_component_unused to 0.

...then the incorrect output:

MPI thread level: single I am process 0 of 1. MPI thread level: single I am process 0 of 1.

[NOTE: The correct output should have been:

MPI thread level: single I am process 0 of 2. I am process 1 of 2.

]

And then, finally, some error messages that I was unable to decipher:

[s001-n028:08428] PMIX ERROR: NO-PERMISSIONS in file src/dstore/pmix_esh.c at line 521 [s001-n028:08428] PMIX ERROR: NO-PERMISSIONS in file src/dstore/pmix_esh.c at line 530 [s001-n028:08428] PMIX ERROR: NO-PERMISSIONS in file src/dstore/pmix_esh.c at line 521 [s001-n028:08428] PMIX ERROR: NO-PERMISSIONS in file src/dstore/pmix_esh.c at line 521 [s001-n028:08428] PMIX ERROR: NO-PERMISSIONS in file src/dstore/pmix_esh.c at line 530 [s001-n028:08428] PMIX ERROR: NO-PERMISSIONS in file src/dstore/pmix_esh.c at line 1048 [s001-n028:08427] PMIX ERROR: NO-PERMISSIONS in file src/dstore/pmix_esh.c at line 521 [s001-n028:08427] PMIX ERROR: NO-PERMISSIONS in file src/dstore/pmix_esh.c at line 530 [s001-n028:08427] PMIX ERROR: NO-PERMISSIONS in file src/dstore/pmix_esh.c at line 521 [s001-n028:08427] PMIX ERROR: NO-PERMISSIONS in file src/dstore/pmix_esh.c at line 521 [s001-n028:08427] PMIX ERROR: NO-PERMISSIONS in file src/dstore/pmix_esh.c at line 530 [s001-n028:08427] PMIX ERROR: NO-PERMISSIONS in file src/dstore/pmix_esh.c at line 1048

I suspect that the Boost libraries on DevCloud might have been compiled with an MPI version that is not quite the same as the one that comes with OneAPI. The `libmpi ` mismatch warning from the linker might be a diagnostic sign.

The test program is correct, I could build and run it on our university cluster using g++ 10.2 and the corresponding `mpic++` compiler plus Boost V 1.75.0.

Any help & suggestions on how to build and link C++ programs with Boost.MPI on the DevCloud would be much appreciated.

Re: Cannot run a simple Boost.MPI test code

SantoshY_Intel — Thu, 08 Apr 2021 12:20:22 GMT

Hi,

Thanks for reaching out to us.

We are able to reproduce the issue at our end. we are looking into it. We will get back to you soon with an update.

Regards,

Santosh

Re: Cannot run a simple Boost.MPI test code

geza_szarnyathy — Wed, 14 Apr 2021 12:24:29 GMT

Thank you for looking into this issue. Any developments so far?

Re: Cannot run a simple Boost.MPI test code

geza_szarnyathy — Fri, 14 May 2021 13:46:57 GMT

Hmm, one month later still no change. Is anyone looking at this issue?

Re: Cannot run a simple Boost.MPI test code

clevels — Tue, 01 Jun 2021 17:43:54 GMT

Hello. I was successfully able to build and run the Boost.MPI test code on DevCloud with the following steps. Please copy the mpiboost.cc file to the cloned "boost" directory and try these steps:

git clone --recursive https://github.com/boostorg/boost.git
cd boost
./bootstrap.sh --with-toolset=gcc --with-libraries=mpi

Added the following to the file project-config.jam:

using mpi : mpiicc :
      <library-path>"path to intel mpi"/lib
      <library-path>"path to intel mpi"/lib/release_mt
      <include>"path to intel mpi"/include
      <find-shared-library>mpi
      <find-shared-library>dl
      <find-shared-library>rt ;

./b2
mpiicpc mpiboost.cc -I<boost include path> -L<boost library path> -lboost_mpi -lboost_serialization -o mpiboost
export LD_LIBRARY_PATH=/<path to /boost/stage/lib/>:$LD_LIBRARY_PATH
mpirun -genv I_MPI_DEBUG 5 -np 2 ./mpiboost

Please let me know if there are any issues or if there is anything else I can help you with.

Re: Cannot run a simple Boost.MPI test code

geza_szarnyathy — Wed, 02 Jun 2021 12:42:33 GMT

Thank you for your efforts. And yes, "there is something you can help me with".

Namely: install Boost on Devcloud centrally so that it works seamlessly with MPI and the OneAPI compilers. Your solution assumes that whoever wants to use Boost.MPI will be happy to clone and build a private copy of Boost for him/herself. This has the following drawbacks:

Technical drawback: it does not scale well (what if 100 people need it?)
PR drawback: if you want to win the hearts of HPC developers, you must demonstrate that the OneAPI tools fit together with other, maybe more traditional, but still very often used HPC tools.

Meanwhile I'll plod along on our university cluster with G++

Cheers, Géza

Re:Cannot run a simple Boost.MPI test code

clevels — Thu, 09 Sep 2021 17:27:00 GMT

Hello. Unfortunately at this time we will not be pre-installing boost on devcloud, but this is certainly something that will be potentially discussed for future releases. Please let me know if there are any other topics you would like to discuss, otherwise I will be closing this ticket.

Re: Cannot run a simple Boost.MPI test code

geza_szarnyathy — Fri, 10 Sep 2021 09:40:49 GMT

Dear "clevels",

Thank you very much for your message. I very much doubt that one can develop serious HPC applications in C++ without Boost and without MPI. You can of course "...potentially discuss this for future releases..." until the cows come home, but then you may lose out in the mid-term. Which would be a pity indeed. So why not assign one person and one working day (max.) to this task and make DevCloud more useful?

Cheers, Géza

Re:Cannot run a simple Boost.MPI test code

clevels — Fri, 08 Oct 2021 11:48:29 GMT

I have reached out directly to a Devcloud developer with your concerns and this was the response I received:

"We have done research on BoostMPI and it looks like an older library without much support or much usage.

We should not recommend or support it, people should migrate to Intel MPI as it is officially supported inside oneAPI: https://software.intel.com/content/www/us/en/develop/tools/oneapi/components/mpi-library.html#gs.2dlkqm

Intel implementation can reach better performance and it supports some special network fabrics.

I can imagine some developer using a project that has dependencies on BoostMPI therefore they might want to install it in DevCloud.

I would not recommend Boost be centrally installed on DevCloud and officially support it(lack of resources is another reason), but if user wants to install on their own user space, they can using :

pip install –user…where the –user direct pip to install it in their user directory"

I really hope that might be useful to you.

This issue has been resolved and we will no longer respond to this thread. If you require additional assistance from Intel, please start a new thread. Any further interaction in this thread will be considered community only”

Re: Re:Cannot run a simple Boost.MPI test code

geza_szarnyathy — Fri, 08 Oct 2021 12:14:52 GMT

"We have done research on BoostMPI and it looks like an older library without much support or much usage."

I respectfully disagree. The Boost libraries play a central role in C++ development. You would know this if you had any idea about C++. But you obviously don't, as the following (non-)recommendation shows:

"I would not recommend Boost be centrally installed on DevCloud and officially support it(lack of resources is another reason), but if user wants to install on their own user space, they can using :

pip install –user…where the –user direct pip to install it in their user directory"

You can't install Boost with pip , userspace or not, because pip is a package manager for Python. Boost, on the other hand, is a C++ library.