gcc's thread sanitizer reports a race condition in Intel MKL Pardiso

[MBDyn-User]

Dear Intel MKL developers,

I'm using Intel MKL Pardiso as a parallel direct linear solver for the multibody dynamics software MBDyn (https://public.gitlab.polimi.it/DAER/mbdyn). Pardiso provides significant performance improvements compared to all other solvers currently used with MBDyn. Unfortunately this solver suffers from stability issues which are not observed with Umfpack for example. If MBDyn is compiled with gcc's -fsanitize=thread, several warnings about race conditions in Pardiso are reported. Those warnings do not appear if Pardiso is used in single thread mode (e.g. if only one OpenMP thread is used). Since only a single thread in MBDyn is calling pardiso, I suppose that the race condition is caused by Pardiso itself.

Below you can find further information, how Pardiso is used by MBDyn:

The following links show the code which provides the interface between MBDyn and Pardiso:

https://public.gitlab.polimi.it/DAER/mbdyn/-/blob/develop/libraries/libmbwrap/pardisowrap.h

https://public.gitlab.polimi.it/DAER/mbdyn/-/blob/develop/libraries/libmbwrap/pardisowrap.cc

In order to build MBDyn on a Linux system (e.g. Ubuntu 20.04) with support for Pardiso use the following steps:

tar -jxvf mbdyn-pardiso.tar.bz2

./mbdyn-pardiso.sh

Then you should get a message like below. If you need further information, please let me know!

==================
WARNING: ThreadSanitizer: data race (pid=257811)
Write of size 8 at 0x7b4000010000 by main thread:
#0 free ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:707 (libtsan.so.0+0x35f25)
#1 mkl_serv_free_buffers <null> (libmkl_core.so+0x200334)

Previous write of size 8 at 0x7b4000010000 by thread T6:
#0 malloc ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:651 (libtsan.so.0+0x30323)
#1 mm_account_ptr_by_tid..0 <null> (libmkl_core.so+0x1fe133)

Thread T6 (tid=257818, running) created by main thread at:
#0 pthread_create ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:962 (libtsan.so.0+0x5ea79)
#1 <null> <null> (libgomp.so.1+0x1adea)
#2 PardisoSolutionManager<SpGradientSparseMatrixHandler, int>::Solve() mbdyn/libraries/libmbwrap/pardisowrap.cc:270 (mbdyn+0x11bfd04)
#3 LineSearchFull::Solve(NonlinearProblem const*, Solver*, int, double const&, int&, double&, double const&, double&) mbdyn/mbdyn/base/linesearch.cc:707 (mbdyn+0x76281c)
#4 DerivativeSolver::Advance(Solver*, double, double, StepIntegrator::StepChange, std::deque<MyVectorHandler*, std::allocator<MyVectorHandler*> >&, std::deque<MyVectorHandler*, std::allocator<MyVectorHandler*> >&, MyVectorHandler*, MyVectorHandler*, int&, double&, double&) mbdyn/mbdyn/base/stepsol.cc:310 (mbdyn+0x59786a)
#5 Solver::Prepare() mbdyn/mbdyn/base/solver.cc:839 (mbdyn+0x58a8bd)
#6 Solver::Run() mbdyn/mbdyn/base/solver.cc:1632 (mbdyn+0x56fab0)
#7 RunMBDyn mbdyn/mbdyn/mbdyn.cc:1498 (mbdyn+0x50c137)
#8 mbdyn_program mbdyn/mbdyn/mbdyn.cc:942 (mbdyn+0x50c137)
#9 main mbdyn/mbdyn/mbdyn.cc:1168 (mbdyn+0x4ec45b)

SUMMARY: ThreadSanitizer: data race (/lib/x86_64-linux-gnu/libmkl_core.so+0x200334) in mkl_serv_free_buffers
==================
ThreadSanitizer: reported 1 warnings

[VidyalathaB_Intel]

Hi,

Thanks for reaching out to us.

>>race conditions in Pardiso are reported..warnings do not appear if Pardiso is used in single thread mode

You can avoid these race conditions by using mkl_set_num_threads_local function as this only affects the current execution thread of the application.

Please refer to the below link 

https://software.intel.com/content/www/us/en/develop/documentation/onemkl-linux-developer-guide/top/managing-performance-and-memory/improving-performance-with-threading/avoiding-conflicts-in-the-execution-environment.html  

You can get more details regarding the mkl_set_num_threads_local function from the below link.

https://software.intel.com/content/www/us/en/develop/documentation/onemkl-developer-reference-c/top/support-functions/threading-control/mkl-set-num-threads-local.html#mkl-set-num-threads-local

Hope the information provided above helps.

Regards,

Vidya.

[MBDyn-User]

Dear Vidya,

Thank you for your advice, but it does not solve the issue. In this software there is only a single thread in the main program which is calling Pardiso. In addition to that, the race condition also appears if the main program uses only one thread of execution (e.g. no multithreading is used outside Pardiso). According to the link you shared, using mkl_set_num_threads_local should help only in situations where MKL is called from several threads simultaneously.

Best regards

MBDyn-User

[VidyalathaB_Intel]

Hi,

We are looking into this issue, we will get back to you soon.

Regards,

Vidya.

[MBDyn-User]

Dear Vidya,

Thank you for your support! It seems that unstable numerical factorization with pardiso (e.g. zero pivot) is not related to those race condition. Probably the issue was, that the weighted matching (e.g. phase=11) was not executed always when former non-zeros in the matrix pattern were turned to numerical zeros again. The reason for that is, that in MBDyn the non-zero pattern is preserved between iterations, unless new non-zeros appear. Only in the latter case the symbolic factorization (e.g. phase=11) was executed. So, the weighted matching was still based on an old non-zero pattern and zero pivots were the consequence. The new approach is to compute a checksum of the non-zero pattern, considering only numerical non-zeros. Whenever this checksum changes, the symbolic factorization is executed again. See pardisowrap.cc.  However, with this approach pardiso is no longer competitive in some cases because the numerical non-zero pattern changes almost at each iteration. Another alternative would be to compute the backward error of the solution. Whenever the backward error is above a certain limit or zero pivot (e.g. ierror = -4) is detected, the symbolic factorization is repeated. This approach is not fully tested yet.

Nevertheless, it would be fine if also those race condition could be fixed because it could lead to further issues.

Best regards

MBDyn-User

[Khang_N_Intel]

Hi,

Thank you for the additional information!

We are currently working on the issue and will let you know how we will proceed next.

Best regards,

Khang

[Khang_N_Intel]

Hi Reinhard,

I have been trying to reproduce the issue that you mentioned with no success.

I received many errors when running the script mbdyn-pardiso.sh.

I tried it on 2 different systems.

I will do some troubleshooting and will let you know how thing goes.

Best regards,

Khang

[Khang_N_Intel]

Hi Reinhard,

I am still having problem with reproducing the error. Let me gather the error and will send it to you.

Best,

Khang

[MBDyn-User]

Dear Khang,

What kind of problems do you have? Were you able to compile and run the code?

Best regards,

MBDyn User

[Khang_N_Intel]

Hi Reinhard,

The issue is the same on both machines.

When I ran the shell script mbdyn-pardiso.sh, it complained that the bootstrap and configure are not found. Also, the permission is denied for mbdyn/mbdyn.

Khang

[MBDyn-User]

Dear Khang,

I think there was an issue in the first submission that the bootstrap script was called "bootstrap" instead of "bootstrap.sh". If bootstrap is not executed, "configure" and "mbdyn" will not be generated. I have attached a new version of all files.

On Ubuntu you probably need to install additional packages:

sudo apt-get build-dep octave
sudo apt-get install  libmkl-full-dev

Please let me know if you have still problems running the script!

Best regards

Reinhard

[Khang_N_Intel]

Hi Reinhard,

Thank you for providing the updated code!

I will test the new update and will you know the status.

Best regards,

Khang

[Khang_N_Intel]

Hi Reinhard,

I tried to reproduce the issue on the latest version of oneMKL, 2022.1, which came out not too long ago. However, I ran into multiple errors when running the script you mentioned.

At first, I got an error: aclocal: not found. In order to fix this error I had to install two additional packages as followed:

sudo apt-get install autotools-dev

sudo apt-get install automake

I then encountered more errors which I captured in the file error-running-mbdyn-pardiso.sh.txt.

[Khang_N_Intel]

I captured all the errors I encountered when running the script mbdyn-pardiso.sh.

[Khang_N_Intel]

Oops! forgot to print the output

$ ./mbdyn-pardiso.sh

Cloning into 'mbdyn'...

remote: Enumerating objects: 73214, done.

remote: Counting objects: 100% (859/859), done.

remote: Compressing objects: 100% (313/313), done.

remote: Total 73214 (delta 574), reused 814 (delta 545), pack-reused 72355

Receiving objects: 100% (73214/73214), 34.72 MiB | 7.32 MiB/s, done.

Resolving deltas: 100% (59048/59048), done.

+ aclocal

configure.ac:387: warning: macro 'AM_PROG_LIBTOOL' not found in library

acinclude.m4:37: warning: prefer named diversions

+ libtoolize --automake --force --copy

./bootstrap.sh: 6: libtoolize: not found

+ autoheader

acinclude.m4:37: warning: prefer named diversions

+ automake --foreign --add-missing --copy --force-missing

acinclude.m4:37: warning: prefer named diversions

configure.ac:311: installing 'build/compile'

configure.ac:111: installing 'build/config.guess'

configure.ac:111: installing 'build/config.sub'

configure.ac:113: installing 'build/install-sh'

configure.ac:113: installing 'build/missing'

libraries/libann/Makefile.am:6: error: Libtool library used but 'LIBTOOL' is undefined

libraries/libann/Makefile.am:6:  The usual way to define 'LIBTOOL' is to add 'LT_INIT'

libraries/libann/Makefile.am:6:  to 'configure.ac' and run 'aclocal' and 'autoconf' again.

libraries/libann/Makefile.am:6:  If 'LT_INIT' is in 'configure.ac', make sure

libraries/libann/Makefile.am:6:  its definition is in aclocal's search path.

libraries/libann/Makefile.am: installing 'build/depcomp'

libraries/libcolamd/Makefile.am:6: error: Libtool library used but 'LIBTOOL' is undefined

libraries/libcolamd/Makefile.am:6:  The usual way to define 'LIBTOOL' is to add 'LT_INIT'

libraries/libcolamd/Makefile.am:6:  to 'configure.ac' and run 'aclocal' and 'autoconf' again.

libraries/libcolamd/Makefile.am:6:  If 'LT_INIT' is in 'configure.ac', make sure

libraries/libcolamd/Makefile.am:6:  its definition is in aclocal's search path.

libraries/libmbc/Makefile.am:19: error: Libtool library used but 'LIBTOOL' is undefined

libraries/libmbc/Makefile.am:19:  The usual way to define 'LIBTOOL' is to add 'LT_INIT'

libraries/libmbc/Makefile.am:19:  to 'configure.ac' and run 'aclocal' and 'autoconf' again.

libraries/libmbc/Makefile.am:19:  If 'LT_INIT' is in 'configure.ac', make sure

libraries/libmbc/Makefile.am:19:  its definition is in aclocal's search path.

libraries/libmbmath/Makefile.am:6: error: Libtool library used but 'LIBTOOL' is undefined

libraries/libmbmath/Makefile.am:6:  The usual way to define 'LIBTOOL' is to add 'LT_INIT'

libraries/libmbmath/Makefile.am:6:  to 'configure.ac' and run 'aclocal' and 'autoconf' again.

libraries/libmbmath/Makefile.am:6:  If 'LT_INIT' is in 'configure.ac', make sure

libraries/libmbmath/Makefile.am:6:  its definition is in aclocal's search path.

libraries/libmbutil/Makefile.am:6: error: Libtool library used but 'LIBTOOL' is undefined

libraries/libmbutil/Makefile.am:6:  The usual way to define 'LIBTOOL' is to add 'LT_INIT'

libraries/libmbutil/Makefile.am:6:  to 'configure.ac' and run 'aclocal' and 'autoconf' again.

libraries/libmbutil/Makefile.am:6:  If 'LT_INIT' is in 'configure.ac', make sure

libraries/libmbutil/Makefile.am:6:  its definition is in aclocal's search path.

libraries/libmbwrap/Makefile.am:6: error: Libtool library used but 'LIBTOOL' is undefined

libraries/libmbwrap/Makefile.am:6:  The usual way to define 'LIBTOOL' is to add 'LT_INIT'

libraries/libmbwrap/Makefile.am:6:  to 'configure.ac' and run 'aclocal' and 'autoconf' again.

libraries/libmbwrap/Makefile.am:6:  If 'LT_INIT' is in 'configure.ac', make sure

libraries/libmbwrap/Makefile.am:6:  its definition is in aclocal's search path.

libraries/libnaive/Makefile.am:6: error: Libtool library used but 'LIBTOOL' is undefined

libraries/libnaive/Makefile.am:6:  The usual way to define 'LIBTOOL' is to add 'LT_INIT'

libraries/libnaive/Makefile.am:6:  to 'configure.ac' and run 'aclocal' and 'autoconf' again.

libraries/libnaive/Makefile.am:6:  If 'LT_INIT' is in 'configure.ac', make sure

libraries/libnaive/Makefile.am:6:  its definition is in aclocal's search path.

libraries/libobjs/Makefile.am:5: error: Libtool library used but 'LIBTOOL' is undefined

libraries/libobjs/Makefile.am:5:  The usual way to define 'LIBTOOL' is to add 'LT_INIT'

libraries/libobjs/Makefile.am:5:  to 'configure.ac' and run 'aclocal' and 'autoconf' again.

libraries/libobjs/Makefile.am:5:  If 'LT_INIT' is in 'configure.ac', make sure

libraries/libobjs/Makefile.am:5:  its definition is in aclocal's search path.

libraries/liby12/Makefile.am:5: error: Libtool library used but 'LIBTOOL' is undefined

libraries/liby12/Makefile.am:5:  The usual way to define 'LIBTOOL' is to add 'LT_INIT'

libraries/liby12/Makefile.am:5:  to 'configure.ac' and run 'aclocal' and 'autoconf' again.

libraries/liby12/Makefile.am:5:  If 'LT_INIT' is in 'configure.ac', make sure

libraries/liby12/Makefile.am:5:  its definition is in aclocal's search path.

mbdyn/aero/Makefile.am:6: error: Libtool library used but 'LIBTOOL' is undefined

mbdyn/aero/Makefile.am:6:  The usual way to define 'LIBTOOL' is to add 'LT_INIT'

mbdyn/aero/Makefile.am:6:  to 'configure.ac' and run 'aclocal' and 'autoconf' again.

mbdyn/aero/Makefile.am:6:  If 'LT_INIT' is in 'configure.ac', make sure

mbdyn/aero/Makefile.am:6:  its definition is in aclocal's search path.

mbdyn/base/Makefile.am:6: error: Libtool library used but 'LIBTOOL' is undefined

mbdyn/base/Makefile.am:6:  The usual way to define 'LIBTOOL' is to add 'LT_INIT'

mbdyn/base/Makefile.am:6:  to 'configure.ac' and run 'aclocal' and 'autoconf' again.

mbdyn/base/Makefile.am:6:  If 'LT_INIT' is in 'configure.ac', make sure

mbdyn/base/Makefile.am:6:  its definition is in aclocal's search path.

mbdyn/elec/Makefile.am:6: error: Libtool library used but 'LIBTOOL' is undefined

mbdyn/elec/Makefile.am:6:  The usual way to define 'LIBTOOL' is to add 'LT_INIT'

mbdyn/elec/Makefile.am:6:  to 'configure.ac' and run 'aclocal' and 'autoconf' again.

mbdyn/elec/Makefile.am:6:  If 'LT_INIT' is in 'configure.ac', make sure

mbdyn/elec/Makefile.am:6:  its definition is in aclocal's search path.

mbdyn/hydr/Makefile.am:6: error: Libtool library used but 'LIBTOOL' is undefined

mbdyn/hydr/Makefile.am:6:  The usual way to define 'LIBTOOL' is to add 'LT_INIT'

mbdyn/hydr/Makefile.am:6:  to 'configure.ac' and run 'aclocal' and 'autoconf' again.

mbdyn/hydr/Makefile.am:6:  If 'LT_INIT' is in 'configure.ac', make sure

mbdyn/hydr/Makefile.am:6:  its definition is in aclocal's search path.

mbdyn/struct/Makefile.am:6: error: Libtool library used but 'LIBTOOL' is undefined

mbdyn/struct/Makefile.am:6:  The usual way to define 'LIBTOOL' is to add 'LT_INIT'

mbdyn/struct/Makefile.am:6:  to 'configure.ac' and run 'aclocal' and 'autoconf' again.

mbdyn/struct/Makefile.am:6:  If 'LT_INIT' is in 'configure.ac', make sure

mbdyn/struct/Makefile.am:6:  its definition is in aclocal's search path.

mbdyn/thermo/Makefile.am:5: error: Libtool library used but 'LIBTOOL' is undefined

mbdyn/thermo/Makefile.am:5:  The usual way to define 'LIBTOOL' is to add 'LT_INIT'

mbdyn/thermo/Makefile.am:5:  to 'configure.ac' and run 'aclocal' and 'autoconf' again.

mbdyn/thermo/Makefile.am:5:  If 'LT_INIT' is in 'configure.ac', make sure

mbdyn/thermo/Makefile.am:5:  its definition is in aclocal's search path.

utils/Makefile.am:98: error: Libtool library used but 'LIBTOOL' is undefined

utils/Makefile.am:98:  The usual way to define 'LIBTOOL' is to add 'LT_INIT'

utils/Makefile.am:98:  to 'configure.ac' and run 'aclocal' and 'autoconf' again.

utils/Makefile.am:98:  If 'LT_INIT' is in 'configure.ac', make sure

utils/Makefile.am:98:  its definition is in aclocal's search path.

+ autoconf

acinclude.m4:37: warning: prefer named diversions

configure.ac:387: error: possibly undefined macro: AM_PROG_LIBTOOL

   If this token and others are legitimate, please use m4_pattern_allow.

   See the Autoconf documentation.

+ test -d contrib

+ find contrib -name bootstrap.sh

+ echo contrib/n2m/bootstrap.sh

+ sed s/\(.*\)\/bootstrap\.sh/\1/

+ dir=contrib/n2m

+ pwd

+ olddir=/home/tce/Documents/2022/OLD-forum-05214468/mbdyn-pardiso/mbdyn

+ echo => entering subdir 'contrib/n2m'...

=> entering subdir 'contrib/n2m'...

+ cd contrib/n2m

+ sh bootstrap.sh

+ aclocal

configure.ac:7: warning: macro 'AM_PROG_LIBTOOL' not found in library

+ libtoolize --automake --force --copy

bootstrap.sh: 5: libtoolize: not found

+ autoheader

+ automake --foreign --add-missing --copy --force-missing

configure.ac:6: installing './compile'

configure.ac:3: installing './config.guess'

configure.ac:3: installing './config.sub'

configure.ac:4: installing './install-sh'

configure.ac:4: installing './missing'

Makefile.am: installing './depcomp'

+ autoconf

configure.ac:7: error: possibly undefined macro: AM_PROG_LIBTOOL

   If this token and others are legitimate, please use m4_pattern_allow.

   See the Autoconf documentation.

+ echo <= leaving subdir 'contrib/n2m'; reverting to dir '/home/tce/Documents/2022/OLD-forum-05214468/mbdyn-pardiso/mbdyn'...

<= leaving subdir 'contrib/n2m'; reverting to dir '/home/tce/Documents/2022/OLD-forum-05214468/mbdyn-pardiso/mbdyn'...

Configuring MBDyn Devel ...

checking build system type... x86_64-pc-linux-gnu

checking host system type... x86_64-pc-linux-gnu

checking target system type... x86_64-pc-linux-gnu

checking for a BSD-compatible install... /usr/bin/install -c

checking whether build environment is sane... yes

checking for a thread-safe mkdir -p... /bin/mkdir -p

checking for gawk... no

checking for mawk... mawk

checking whether make sets $(MAKE)... yes

checking whether make supports nested variables... yes

checking configure arguments... checking for pkg-config... /usr/bin/pkg-config

checking pkg-config is at least version 0.9.0... yes

checking for gcc... gcc

checking whether the C compiler works... yes

checking for C compiler default output file name... a.out

checking for suffix of executables... 

checking whether we are cross compiling... no

checking for suffix of object files... o

checking whether we are using the GNU C compiler... yes

checking whether gcc accepts -g... yes

checking for gcc option to accept ISO C89... none needed

checking whether gcc understands -c and -o together... yes

checking whether make supports the include directive... yes (GNU style)

checking dependency style of gcc... gcc3

checking for g++... g++

checking whether we are using the GNU C++ compiler... yes

checking whether g++ accepts -g... yes

checking dependency style of g++... gcc3

checking for gfortran... gfortran

checking whether we are using the GNU Fortran compiler... yes

checking whether gfortran accepts -g... yes

checking how to get verbose linking output from gfortran... -v

checking for Fortran libraries of gfortran... -L/opt/intel/oneapi/vpl/2022.0.0/lib/../lib -L/opt/intel/oneapi/mpi/2021.5.0//libfabric/lib/../lib -L/opt/intel/oneapi/mpi/2021.5.0//lib/../lib -L/opt/intel/oneapi/dnnl/2022.0.1/cpu_dpcpp_gpu_dpcpp/lib/../lib -L/opt/intel/oneapi/compiler/2022.0.1/linux/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/9 -L/usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/9/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib -L/opt/intel/oneapi/vpl/2022.0.0/lib -L/opt/intel/oneapi/tbb/2021.5.0/env/../lib/intel64/gcc4.8 -L/opt/intel/oneapi/mpi/2021.5.0//libfabric/lib -L/opt/intel/oneapi/mpi/2021.5.0//lib/release -L/opt/intel/oneapi/mpi/2021.5.0//lib -L/opt/intel/oneapi/mkl/2022.0.1/lib/intel64 -L/opt/intel/oneapi/ipp/2021.5.1/lib/intel64 -L/opt/intel/oneapi/ippcp/2021.5.0/lib/intel64 -L/opt/intel/oneapi/dnnl/2022.0.1/cpu_dpcpp_gpu_dpcpp/lib -L/opt/intel/oneapi/dal/2021.5.1/lib/intel64 -L/opt/intel/oneapi/compiler/2022.0.1/linux/compiler/lib/intel64_lin -L/opt/intel/oneapi/compiler/2022.0.1/linux/lib -L/opt/intel/oneapi/clck/2021.5.0/lib/intel64 -L/opt/intel/oneapi/ccl/2021.5.0/lib/cpu_gpu_dpcpp -L/usr/lib/gcc/x86_64-linux-gnu/9/../../.. -lgfortran -lm -lquadmath

checking for dummy main to link with Fortran libraries... none

checking how to get verbose linking output from gfortran... (cached) -v

checking for Fortran libraries of gfortran... (cached) -L/opt/intel/oneapi/vpl/2022.0.0/lib/../lib -L/opt/intel/oneapi/mpi/2021.5.0//libfabric/lib/../lib -L/opt/intel/oneapi/mpi/2021.5.0//lib/../lib -L/opt/intel/oneapi/dnnl/2022.0.1/cpu_dpcpp_gpu_dpcpp/lib/../lib -L/opt/intel/oneapi/compiler/2022.0.1/linux/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/9 -L/usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/9/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib -L/opt/intel/oneapi/vpl/2022.0.0/lib -L/opt/intel/oneapi/tbb/2021.5.0/env/../lib/intel64/gcc4.8 -L/opt/intel/oneapi/mpi/2021.5.0//libfabric/lib -L/opt/intel/oneapi/mpi/2021.5.0//lib/release -L/opt/intel/oneapi/mpi/2021.5.0//lib -L/opt/intel/oneapi/mkl/2022.0.1/lib/intel64 -L/opt/intel/oneapi/ipp/2021.5.1/lib/intel64 -L/opt/intel/oneapi/ippcp/2021.5.0/lib/intel64 -L/opt/intel/oneapi/dnnl/2022.0.1/cpu_dpcpp_gpu_dpcpp/lib -L/opt/intel/oneapi/dal/2021.5.1/lib/intel64 -L/opt/intel/oneapi/compiler/2022.0.1/linux/compiler/lib/intel64_lin -L/opt/intel/oneapi/compiler/2022.0.1/linux/lib -L/opt/intel/oneapi/clck/2021.5.0/lib/intel64 -L/opt/intel/oneapi/ccl/2021.5.0/lib/cpu_gpu_dpcpp -L/usr/lib/gcc/x86_64-linux-gnu/9/../../.. -lgfortran -lm -lquadmath

checking for g77... no

checking for xlf... no

checking for f77... f77

checking whether we are using the GNU Fortran 77 compiler... yes

checking whether f77 accepts -g... yes

checking for gcc version = 3.3... no

checking for gcc version = 3.2... no

./configure: line 7893: AM_PROG_LIBTOOL: command not found

checking for gcc... (cached) gcc

checking whether we are using the GNU C compiler... (cached) yes

checking whether gcc accepts -g... (cached) yes

checking for gcc option to accept ISO C89... (cached) none needed

checking whether gcc understands -c and -o together... (cached) yes

checking dependency style of gcc... (cached) gcc3

checking how to run the C preprocessor... gcc -E

checking for grep that handles long lines and -e... /bin/grep

checking for egrep... /bin/grep -E

checking for ANSI C header files... yes

checking for sys/types.h... yes

checking for sys/stat.h... yes

checking for stdlib.h... yes

checking for string.h... yes

checking for memory.h... yes

checking for strings.h... yes

checking for inttypes.h... yes

checking for stdint.h... yes

checking for unistd.h... yes

checking arpa/inet.h usability... yes

checking arpa/inet.h presence... yes

checking for arpa/inet.h... yes

checking atomic_ops.h usability... no

checking atomic_ops.h presence... no

checking for atomic_ops.h... no

checking cxxabi.h usability... no

checking cxxabi.h presence... no

checking for cxxabi.h... no

checking fenv.h usability... yes

checking fenv.h presence... yes

checking for fenv.h... yes

checking getopt.h usability... yes

checking getopt.h presence... yes

checking for getopt.h... yes

checking netdb.h usability... yes

checking netdb.h presence... yes

checking for netdb.h... yes

checking netinet/in.h usability... yes

checking netinet/in.h presence... yes

checking for netinet/in.h... yes

checking pwd.h usability... yes

checking pwd.h presence... yes

checking for pwd.h... yes

checking pthread.h usability... yes

checking pthread.h presence... yes

checking for pthread.h... yes

checking sched.h usability... yes

checking sched.h presence... yes

checking for sched.h... yes

checking semaphore.h usability... yes

checking semaphore.h presence... yes

checking for semaphore.h... yes

checking signal.h usability... yes

checking signal.h presence... yes

checking for signal.h... yes

checking for stdint.h... (cached) yes

checking stdio.h usability... yes

checking stdio.h presence... yes

checking for stdio.h... yes

checking for stdlib.h... (cached) yes

checking for string.h... (cached) yes

checking sys/ipc.h usability... yes

checking sys/ipc.h presence... yes

checking for sys/ipc.h... yes

checking sys/mman.h usability... yes

checking sys/mman.h presence... yes

checking for sys/mman.h... yes

checking sys/param.h usability... yes

checking sys/param.h presence... yes

checking for sys/param.h... yes

checking sys/poll.h usability... yes

checking sys/poll.h presence... yes

checking for sys/poll.h... yes

checking sys/pstat.h usability... no

checking sys/pstat.h presence... no

checking for sys/pstat.h... no

checking sys/select.h usability... yes

checking sys/select.h presence... yes

checking for sys/select.h... yes

checking sys/shm.h usability... yes

checking sys/shm.h presence... yes

checking for sys/shm.h... yes

checking sys/socket.h usability... yes

checking sys/socket.h presence... yes

checking for sys/socket.h... yes

checking sys/sysinfo.h usability... yes

checking sys/sysinfo.h presence... yes

checking for sys/sysinfo.h... yes

checking sys/times.h usability... yes

checking sys/times.h presence... yes

checking for sys/times.h... yes

checking for sys/types.h... (cached) yes

checking sys/ioctl.h usability... yes

checking sys/ioctl.h presence... yes

checking for sys/ioctl.h... yes

checking time.h usability... yes

checking time.h presence... yes

checking for time.h... yes

checking for unistd.h... (cached) yes

checking values.h usability... yes

checking values.h presence... yes

checking for values.h... yes

checking termios.h usability... yes

checking termios.h presence... yes

checking for termios.h... yes

checking winsock2.h usability... no

checking winsock2.h presence... no

checking for winsock2.h... no

checking how to run the C++ preprocessor... g++ -E

checking exception usability... yes

checking exception presence... yes

checking for exception... yes

checking stdexcept usability... yes

checking stdexcept presence... yes

checking for stdexcept... yes

checking netcdf usability... no

checking netcdf presence... no

checking for netcdf... no

checking netcdfcpp.h usability... no

checking netcdfcpp.h presence... no

checking for netcdfcpp.h... no

checking netcdf.h usability... no

checking netcdf.h presence... no

checking for netcdf.h... no

checking udunits.h usability... no

checking udunits.h presence... no

checking for udunits.h... no

checking for _aligned_malloc and _aligned_free... no

checking for posix_memalign... yes

checking for memalign... yes

checking for aligned_alloc... yes

checking blitz/blitz.h usability... no

checking blitz/blitz.h presence... no

checking for blitz/blitz.h... no

checking blitz/array.h usability... no

checking blitz/array.h presence... no

checking for blitz/array.h... no

checking blitz/tinyvec.h usability... no

checking blitz/tinyvec.h presence... no

checking for blitz/tinyvec.h... no

checking blitz/tinyvec-et.h usability... no

checking blitz/tinyvec-et.h presence... no

checking for blitz/tinyvec-et.h... no

checking blitz/tinymat.h usability... no

checking blitz/tinymat.h presence... no

checking for blitz/tinymat.h... no

checking blitz/matrix.h usability... no

checking blitz/matrix.h presence... no

checking for blitz/matrix.h... no

checking for clock_nanosleep in -lrt... yes

checking for noexcept... yes

checking for offsetof... yes

checking for main in -lws2_32... no

checking for socket... yes

checking for socklen_t... yes

checking for sig_atomic_t... yes

checking whether bool is defined... yes

checking whether the C++ compiler handles exceptions... yes

checking whether the C++ compiler allows to use namespace std... yes

checking for the pthreads library -lpthreads... no

checking whether pthreads work without any flags... no

checking whether pthreads work with -Kthread... no

checking whether pthreads work with -kthread... no

checking for the pthreads library -llthread... no

checking whether pthreads work with -pthread... yes

checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE

checking if more special flags are required for pthreads... no

checking for PTHREAD_PRIO_INHERIT... yes

checking for threads... yes

checking for usable pthread_setaffinity_np... yes

checking for usable pthread_sigmask... yes

checking ltdl.h usability... no

checking ltdl.h presence... no

checking for ltdl.h... no

configure: WARNING: no dynamic linking capability detected; runtime loading disabled

checking whether to enable multithread... configure: error: need atomic_ops.h (from HP's atomic_ops project)

make: *** No targets specified and no makefile found. Stop.

./mbdyn-pardiso.sh: 19: mbdyn/mbdyn/mbdyn: not found

[MBDyn-User]

OK, it seems that there are still missing packages.

Please try to install the following packages and then run the script again!

sudo apt-get install libatomic-ops-dev

sudo apt-get build-dep octave

[Khang_N_Intel]

Hi Reinhard,

I tried to install the 2 packages you mentioned. The first one went fine. However, it gave me an error when executing the second one:

E: Invalid operation build-dev

I tried to just install octave:

sudo apt-get install octave

I then reran the scipt file. However, I still received the same error as I mentioned in my last post.

[MBDyn-User]

Dear Khang,

The command is

sudo apt-get build-dep octave

not

sudo apt-get build-dev octave

As an alternative you could also use

sudo apt-get install automake autoconf libtool

in order to install missing libtoolize which was missing when you run bootstrap.

Probably you also need

sudo apt-get install libmkl-full-dev

in order to use pardiso.

Best regards

Reinhard

[Khang_N_Intel]

Hi Reinhard,

Thank you for the info. I finally was able to finish running the script, although I couldn't use thecommand:

sudo apt-get build-dep octacve

I had to use the alternative that you also provided.

I also replaced the version of mkl that you provided with the latest version of oneMKL, 2022.0.

The data race issue still exists in oneMKl 2022.0

I will send the issue to the mkl developer team to further inverstigate.

Best regards,

Khang

[Khang_N_Intel]

Hi Reinhard,

After testing the reproducer the engineer suspected that the warning errors:

WARNING: ThreadSanitizer: data race...

could be false positive.

They would like: to have a reproducer of Pardiso stability issue and the expected behavior so that they can locate locate the cause and to verify if the problem is fixed.

Thanks,

Khang

[MBDyn-User]

Dear Khang,

Since I posted this issue, a lot of changes were applied to the code base of MBDyn. Actually I think that the Pardiso solver is stable and does not suffer from stability issues. Indeed gcc's thread sanitizer might report false positives if code which was not  compiled with gcc's -fsanitize=thread option performs atomic variable access. Such access cannot be intercepted by the thread sanitizer and could lead to those false positives .

Nevertheless, I can run the MBDyn/Pardiso solver through a test suite of more than 400 tests in multi-thread mode if you want. In case of a failure in one of those tests, I would provide the files to reproduce that test.