MPI program intermittently crashes

Matt_Rinella — Fri, 10 Jan 2025 21:15:40 GMT

Below I've provided the MPI part of the program, the debugging information I get when the program fails, and the versions of Visual Studio and the one API and HPC toolkits I'm using. A sincere thanks in advance for any help.

do k=1,5000
if(proc.eq.0)then
   do i=2,nprocs
   call MPI_SEND(xp(i,1:d),d,MPI_real,i-1,1,MPI_COMM_WORLD,ierr)
   enddo
else
   call MPI_recv(xp(1,1:d),d,MPI_real,0,1,MPI_COMM_WORLD, MPI_STATUS_IGNORE,ierr)
endif

call call_hydrus(proc,xp(1,1:d),d,ndata,convergevec(1),sse(1))

If(proc.ne.0)then
   call MPI_SEND(sse(1),1,MPI_real,0,2,MPI_COMM_WORLD,ierr)
else
   do i=2,nprocs
   call MPI_recv(sse(i),1,MPI_real,i-1,2,MPI_COMM_WORLD, MPI_STATUS_IGNORE,ierr)
   enddo
endif
enddo

!****************************************************************************************
subroutine call_hydrus(proc,pars,d,ndata,convergeq,sse)
implicit none
integer :: ndata,d,i,proc
real :: pars(d),sse
logical :: convergeq
CHARACTER(100) :: filenumberstring

write(filenumberstring, '(I5)') proc

open(30,file=Trim("selector"//trim(adjustl(filenumberstring))//".in"),status='old',action='read')
open(32,file=Trim("profile"//trim(adjustl(filenumberstring))//".dat"),status='old',action='read')
open(31,file=Trim("ATMOSPH"//trim(adjustl(filenumberstring))//".in"),status='old',action='read')
open(666,file=Trim("tdr"//trim(adjustl(filenumberstring))//".in"),status='old',action='read')

!this subroutine calls several other subroutines written in fortran 77
call hydrus(proc,pars,sse,convergeq,ndata)

close(30)
close(31)
close(32)
close(666)
end
!****************************************************************************************

'mpiexec.exe' (Win32): Loaded 'C:\Program Files (x86)\Intel\oneAPI\mpi\2021.8.0\bin\mpiexec.exe'.
'mpiexec.exe' (Win32): Loaded 'C:\Windows\System32\ntdll.dll'.
'mpiexec.exe' (Win32): Loaded 'C:\Windows\System32\kernel32.dll'.
'mpiexec.exe' (Win32): Loaded 'C:\Windows\System32\KernelBase.dll'.
'mpiexec.exe' (Win32): Loaded 'C:\Windows\System32\ws2_32.dll'.
'mpiexec.exe' (Win32): Loaded 'C:\Windows\System32\rpcrt4.dll'.
'mpiexec.exe' (Win32): Loaded 'C:\Windows\System32\crypt32.dll'.
'mpiexec.exe' (Win32): Loaded 'C:\Windows\System32\ucrtbase.dll'.
'mpiexec.exe' (Win32): Loaded 'C:\Windows\System32\bcrypt.dll'.
'mpiexec.exe' (Win32): Loaded 'C:\Windows\System32\secur32.dll'.
'mpiexec.exe' (Win32): Loaded 'C:\Windows\System32\mswsock.dll'.
'mpiexec.exe' (Win32): Loaded 'C:\Windows\System32\advapi32.dll'.
'mpiexec.exe' (Win32): Loaded 'C:\Windows\System32\mpr.dll'.
'mpiexec.exe' (Win32): Loaded 'C:\Windows\System32\msvcrt.dll'.
'mpiexec.exe' (Win32): Loaded 'C:\Windows\System32\sechost.dll'.
'mpiexec.exe' (Win32): Loaded 'C:\Windows\System32\IPHLPAPI.DLL'.
'mpiexec.exe' (Win32): Loaded 'C:\Windows\System32\ntdsapi.dll'.
'mpiexec.exe' (Win32): Loaded 'C:\Windows\System32\sspicli.dll'.
'mpiexec.exe' (Win32): Loaded 'C:\Windows\System32\dsparse.dll'.
'mpiexec.exe' (Win32): Loaded 'C:\Windows\System32\NapiNSP.dll'.
'mpiexec.exe' (Win32): Loaded 'C:\Windows\System32\bcryptprimitives.dll'.
'mpiexec.exe' (Win32): Loaded 'C:\Windows\System32\pnrpnsp.dll'.
'mpiexec.exe' (Win32): Loaded 'C:\Windows\System32\dnsapi.dll'.
'mpiexec.exe' (Win32): Loaded 'C:\Windows\System32\nsi.dll'.
'mpiexec.exe' (Win32): Loaded 'C:\Windows\System32\winrnr.dll'.
'mpiexec.exe' (Win32): Loaded 'C:\Windows\System32\wshbth.dll'.
'mpiexec.exe' (Win32): Loaded 'C:\Windows\System32\nlansp_c.dll'.
'mpiexec.exe' (Win32): Loaded 'C:\Windows\System32\msvcp_win.dll'.
onecore\net\netprofiles\service\src\nsp\dll\namespaceserviceprovider.cpp(613)\nlansp_c.dll!00007FFB645AF6CD: (caller: 00007FFB7131ACF6) LogHr(1) tid(afe0) 8007277C No such service is known. The service cannot be found in the specified name space.
onecore\net\netprofiles\service\src\nsp\dll\namespaceserviceprovider.cpp(613)\nlansp_c.dll!00007FFB645AF6CD: (caller: 00007FFB7131ACF6) LogHr(2) tid(afe0) 8007277C No such service is known. The service cannot be found in the specified name space.
onecore\net\netprofiles\service\src\nsp\dll\namespaceserviceprovider.cpp(613)\nlansp_c.dll!00007FFB645AF6CD: (caller: 00007FFB7131ACF6) LogHr(3) tid(afe0) 8007277C No such service is known. The service cannot be found in the specified name space.
'mpiexec.exe' (Win32): Unloaded 'C:\Windows\System32\NapiNSP.dll'
'mpiexec.exe' (Win32): Unloaded 'C:\Windows\System32\pnrpnsp.dll'
'mpiexec.exe' (Win32): Unloaded 'C:\Windows\System32\winrnr.dll'
'mpiexec.exe' (Win32): Unloaded 'C:\Windows\System32\wshbth.dll'
'mpiexec.exe' (Win32): Unloaded 'C:\Windows\System32\msvcp_win.dll'
'mpiexec.exe' (Win32): Unloaded 'C:\Windows\System32\nlansp_c.dll'
The thread 0xb55c has exited with code -1 (0xffffffff).
The thread 0x93f8 has exited with code -1 (0xffffffff).
The thread 0x6e4c has exited with code -1 (0xffffffff).
The thread 0x8b44 has exited with code -1 (0xffffffff).
The thread 0x9fc8 has exited with code -1 (0xffffffff).
The program '[42104] mpiexec.exe' has exited with code -1 (0xffffffff).

Microsoft Visual Studio Community 2019
Version 16.11.17

Product: Intel® oneAPI Base & HPC Toolkit (Single-Node)
Serial Number 1: C6VB-4H4H55R2
SKU: ABS999OSGE01X1Z - Intel® oneAPI Base & HPC Toolkit (Single-Node) - Named-user Commercial (ESD)
Vendor: Climb Channel Solutions
Version: 2022
Quantity: 1
Support Period End: 2023-02-06
Order Number: IRC:414043 T20925690101 X04757840101

Re: MPI program intermittently crashes

TobiasK — Mon, 20 Jan 2025 10:12:00 GMT

@Matt_Rinella please consider contacting priority support to share more details on your code and the failure. With the information you provided here, we are not able to help you.

topic MPI program intermittently crashes in Intel® MPI Library

MPI program intermittently crashes

Re: MPI program intermittently crashes