Crash in __kmp_cleanup_user_locks() when using OpenMP and CRITICAL SECTIONS

[rakkota]

Hi,

we recently experienced a crash when quitting the program in libiomp5md.dll, __kmp_cleanup_user_locks(). Since the architecture is quite complex, we simplified it down to a two-file project:

• a C++ driver, which loads a dll FORTRAN library
• a FORTRAN dll, which executes an OpenMP parallel DO loop, which contains a CRITICAL SECTION  

I also have a VisualStudio project for this (see attached). Note that the driver explicitly loads libiomp5md.dll since, in the original architecture, there are many FORTRAN dll modules which are linked against libiomp5md.dll. By loading this into the driver we considerably reduced the "module switch" time. I mention that we are using ifort 13.1.3.0 for FORTRAN and VC2008 for C++.

Please let me know if there is solution for this, other than not explicitly loading libiomp5md.dll, or if this a bug on the INTEL side. 

Thanks, Rak

[IanH]

I don't see any problems with 14.0.1 and VS2010.

After conversion to VS2010 (which may have jumbled things), I needed to edit the output directory property of the Fortran project in order to run this.  Note that the tests in the C++ program using the assert macro won't be present in a typical Release build, in which case a failure in LoadLibrary or GetProcAddress would go undetected until the attempted call.

I don't understand why preloading the openmp DLL would reduce "module switch time" (module load time?).  I would expect preloading to just shift in time the overhead required to load the DLL.  I guess the openmp runtime might do something different depending on whether it is being load time or run time loaded.

[rakkota]

Thanks Ian for trying. We don't have access yet to version 14 of ifort, such that I cannot confirm that the problem is fixed in this version. I still have to know if in version 13 it is an bug on the INTEL side.

Could you check please the compilation flags for fortran_test.dll project? I added some extra flags that our build is using and they might got lost in the phase of conversion to VC2010 (fortran_test.dll Property Pages -> Configuraion properties -> Fortran -> Command Line -> Additional Options):

/extend-source:80
/intconstant
/fpconstant
/f77rtl
/arch:SSE2
/Qprec-div
/Qprec-sqrt
/fp:strict
/assume:buffered_io
/assume:byterecl
/Qopenmp
/Qopenmp_report:2
/Qpar_report:3
/Qvec-report:1
/Qdiag-disable:6371
/Qdiag-disable:8290
/Qdiag-disable:8291
/heap-arrays0
/integer-size:32
/real-size:32
/O3

Now more about module switching (loading/unloading). In the project I sent as an example, only one module is loaded, only once. In our project, different modules are loaded and unloaded, tens of thousand times. Each module is statically linked against OpenMP libraries, which makes the associated dll's to be loaded/unloaded each time. In our benchmarks, pre-loading INTEL dll's clearly reduced the latency associated to "module switching". 

Thanks, Rak

[Steven_L_Intel1]

It would take significant effort to determine what caused the earlier failure. I can't spot any obvious reports in our bug tracking system, but it could have been described in ways I didn't look for. I can verify that it does work in the current version.

I could see an initial delay in loading dependent DLLs (limiomp5md.dll is just one of several), but I'd expect this time to be taken whether you load it explicitly or let it come in automatically.

[rakkota]

Steve Lionel (Intel) wrote:

It would take significant effort to determine what caused the earlier failure. I can't spot any obvious reports in our bug tracking system, but it could have been described in ways I didn't look for. I can verify that it does work in the current version.

Thanks Lionel. I will try to get a trial of version 14 on a test machine to check if it works.

Steve Lionel (Intel) wrote:

I could see an initial delay in loading dependent DLLs (limiomp5md.dll is just one of several), but I'd expect this time to be taken whether you load it explicitly or let it come in automatically.

It was counter-intuitive for us as well, but we noticed that, in practice, if limiomp5md.dll (together with other redistributable dll's) is explicitly pre-loaded as in the example, the pair of calls "mod=LoadLibrary("some_fortran_dll"); FreeLibrary(mod);" in a loop is clearly faster.

Thanks, Rak