Multi-threaded application using Intel MPI Library has 'bad termination' errors when VPN is active

[Roth__CJ]

I have developed a large C++ application for performing large amounts of model calculations, and can successfully run it in multi-threaded mode using the Intel MPI Library, but have problems when using a VPN.

Due to end-users of the application now working from home, VPN connections are being used a lot.  It was discovered that the state change of their VPN connections during the application's multi-threaded mode execution caused it to crash with some specific error.  The application works perfectly normal when the VPN connection was disabled.

I found another forum post that described a very similar behavior; in the replies, it was noted the apparent VPN-related problem had been 'fixed' in the latest release of the MPI Library.

However, after upgrading my MPI Library installation to the most recent (2019u8), I now find that when the VPN connection is active, my application now just hangs right after starting, displaying 'Bad Termination...' error messages for each of the child processes being spawned off of the parent application (using the MPI_Comm_spawn library routine).  The application still works perfectly when the VPN connection is disabled.

At this point, I'm desperate enough to want to try installing the latest 'beta' version of the Intel MPI Library, hoping that these other VPN-related problems have also been fixed.  Where do I go to download it?  I have an active license.

BTW, the VPN software being used is Cisco AnyConnect Secure Mobility Client, version 4.8.02045, on a Windows 10 Pro system, 2004.

[PrasanthD_intel]

Hi,

The VPN issue which you are referring was about getting an error when shifting between VPN states while executing on a local system. But in a multi-node environment when you connect/disconnect VPN while MPI is running the network configuration will change and MPI is bound to fail.

So, are you getting the error while running on the local system or across multiple nodes?

And coming to the bad termination error while using dynamic processes in 2019u8, thanks for reporting we observed similar behaviour. It looks like a bug; we are transferring this to our internal team.

Thanks and Regards

Prasanth

[Roth__CJ]

That previously-encountered error, my application crashing when the VPN state changed, was happening on the local system.  But that version of the application had been using the Intel MPI Library version 2017u3.  Another forum thread had described this VPN-related error, and had been noted to be fixed in a newer version, 2019u7.

To address that issue, I updated the application to use MPI Library version 2019u8.  But now, running the application on the local system _always_ fails when the VPN connection is active.

I am glad to hear that this newest behavior is already known and being investigated.