topic Re:Disconnecting VPN when running MPMD execution c... in Intel® MPI Library

Disconnecting VPN when running MPMD execution crashes

Figura__Ed — Fri, 24 Jul 2020 09:00:32 GMT

I've read the thread: https://community.intel.com/t5/Intel-oneAPI-HPC-Toolkit/Disconnect-VPN-causes-parallel-computing-with-Intel-MPI-to-stop/td-p/1173381

My problem is slightly different: I have a product which executes as MPMD e.g. mpiexec -localonly -n 1 <main-prog>.exe : -n <X> <worker-prog>.exe

Whilst this is running, if VPN is disconnected, then the application terminates ("connection to proxy 0 failed", etc.)

I can reproduce using the following: mpiexec -localonly -n 4 IMB-MPI1.exe : -n 2 IMB-MPI1.exe

Currently we are running Intel MPI 5.0 (I know, very out-of-date - but there was a historical reason to do with dynamic spawning - no longer used) and will be looking to upgrade. I have actually installed 2019U7 to try but the issue still exists. The VPN disconnection problem for our users on laptops is almost a 'killer issue'. Any advice greatly appreciated.

Re:Disconnecting VPN when running MPMD execution c...

PrasanthD_intel — Mon, 27 Jul 2020 12:13:08 GMT

Hi Ed,

As specified in the thread, you have mentioned the VPN disconnect issue has been resolved in the latest version.

As you were getting an error while using argument sets, we have tried with that too but haven't got any error after disconnecting VPN.

After upgrading to the MPI 2019u7 have you done the prerequisite steps as mentioned in Getting started guide(https://software.intel.com/content/www/us/en/develop/documentation/get-started-with-mpi-for-windows/top.html)?

If yes Could you please provide the command line so we can debug from our side.

Also if you were using Dynamic processes upgrade to the newer version 2019u8 in which PMI2 spawn support was added. (https://software.intel.com/content/www/us/en/develop/documentation/mpi-developer-reference-linux/top/introduction/whats-new.html)

Regards

Prasanth

Re: Re:Disconnecting VPN when running MPMD execution c...

Figura__Ed — Mon, 27 Jul 2020 15:22:44 GMT

Thx for getting back. We're in the middle of a release so I'm a little limited in time at the moment.

Like I said, when I executed the shipped exe as follows:

mpiexec -localonly -n 4 IMB-MPI1.exe : -n 2 IMB-MPI1.exe

and disconnected the VPN, then I got a crash.

However, executing the following works fine:

mpiexec -localonly -n 4 IMB-MPI1.exe

Re: Re:Disconnecting VPN when running MPMD execution c...

Figura__Ed — Thu, 30 Jul 2020 07:41:42 GMT

Note: I am running on Windows 10 (not linux)

Re:Disconnecting VPN when running MPMD execution c...

PrasanthD_intel — Tue, 04 Aug 2020 11:17:27 GMT

Hi Ed,

We have tried to reproduce the error in windows 10.

Could you please provide the debug info by setting I_MPI_DEBUG=5.

set I_MPI_DEBUG=5

Also if possible please update to the latest version 2019u8 and check.

Thanks

Prasanth

Re: Re:Disconnecting VPN when running MPMD execution c...

Figura__Ed — Tue, 04 Aug 2020 14:25:13 GMT

Hi. I can confirm that with 2019 Update 8, I no longer get the crash with the example I posted. I will test this with our software once I get a chance and post any follow ups.

Re:Disconnecting VPN when running MPMD execution c...

PrasanthD_intel — Wed, 05 Aug 2020 05:59:15 GMT

Hi Ed,

Thanks for the confirmation. It should also work with 2019u7 version too.

Anyway if your initial query has been answered please confirm so we can close the case.

Regards

Prasanth

Re: Re:Disconnecting VPN when running MPMD execution c...

Figura__Ed — Wed, 05 Aug 2020 08:00:29 GMT

Sure, close the case. If I have any further issues once I upgrade & integrate with our own code then I'll raise a different query. Thanks for your help.

Re:Disconnecting VPN when running MPMD execution c...

PrasanthD_intel — Thu, 06 Aug 2020 08:40:20 GMT

Hi Ed,

Glad your problem is resolved.

If you require additional assistance from Intel, please start a new thread. Any further interaction in this thread will be considered community only

Regards

Prasanth