Errors using mpiifx and -check all with OneAPI 2024.1

[Marcos_V_1]

Hi, we've come across two errors compiling with mpiifx and -check all using the latest OneAPI 2024.1 Base and HPC toolkits.

To reproduce these I have made a snippet code that essentially does a "Hello World" test with MPI calling some MPI routines. 

- First issue : comes trying to link the source to MKL (actually not being used here, but same issue found in our simulation code that uses Pardiso). To reproduce compile the attached file with:

$ mpiifx -c -O0 -g -check all -I/${MKLROOT}/include main.f90
$ mpiifx -O0 -g -check all -o test_check_all main.o -Wl,--start-group ${MKLROOT}/lib/libmkl_intel_lp64.a ${MKLROOT}/lib/libmkl_sequential.a ${MKLROOT}/lib/libmkl_core.a ${MKLROOT}/lib/libmkl_blacs_intelmpi_lp64.a -Wl,--end-group -lpthread -lm -ldl

This is what I get in linking phase:

ld: /home/marcosvanella/intel/oneapi/mpi/2021.12/lib/libmpi.a(lib_libmpi_la-init_bindings.o):/build/impi/_buildspace/release/../../src/mpi/init/init_bindings.c:64: multiple definition of `MPIR_C_MPI_STATUS_IGNORE'; /home/marcosvanella/intel/oneapi/mpi/2021.12/lib/libmpifort.a(lib_libmpifort_la-cdesc.o):/build/impi/_buildspace/release/../../src/binding/fortran/use_mpi_f08/wrappers_c/cdesc.c:11: first defined here
ld: /home/marcosvanella/intel/oneapi/mpi/2021.12/lib/libmpi.a(lib_libmpi_la-init_bindings.o):/build/impi/_buildspace/release/../../src/mpi/init/init_bindings.c:65: multiple definition of `MPIR_C_MPI_STATUSES_IGNORE'; /home/marcosvanella/intel/oneapi/mpi/2021.12/lib/libmpifort.a(lib_libmpifort_la-cdesc.o):/build/impi/_buildspace/release/../../src/binding/fortran/use_mpi_f08/wrappers_c/cdesc.c:12: first defined here
ld: /home/marcosvanella/intel/oneapi/mpi/2021.12/lib/libmpi.a(lib_libmpi_la-init_bindings.o):/build/impi/_buildspace/release/../../src/mpi/init/init_bindings.c:66: multiple definition of `MPIR_C_MPI_ARGV_NULL'; /home/marcosvanella/intel/oneapi/mpi/2021.12/lib/libmpifort.a(lib_libmpifort_la-cdesc.o):/build/impi/_buildspace/release/../../src/binding/fortran/use_mpi_f08/wrappers_c/cdesc.c:13: first defined here
ld: /home/marcosvanella/intel/oneapi/mpi/2021.12/lib/libmpi.a(lib_libmpi_la-init_bindings.o):/build/impi/_buildspace/release/../../src/mpi/init/init_bindings.c:67: multiple definition of `MPIR_C_MPI_ARGVS_NULL'; /home/marcosvanella/intel/oneapi/mpi/2021.12/lib/libmpifort.a(lib_libmpifort_la-cdesc.o):/build/impi/_buildspace/release/../../src/binding/fortran/use_mpi_f08/wrappers_c/cdesc.c:14: first defined here
ld: /home/marcosvanella/intel/oneapi/mpi/2021.12/lib/libmpi.a(lib_libmpi_la-init_bindings.o):/build/impi/_buildspace/release/../../src/mpi/init/init_bindings.c:68: multiple definition of `MPIR_C_MPI_ERRCODES_IGNORE'; /home/marcosvanella/intel/oneapi/mpi/2021.12/lib/libmpifort.a(lib_libmpifort_la-cdesc.o):/build/impi/_buildspace/release/../../src/binding/fortran/use_mpi_f08/wrappers_c/cdesc.c:15: first defined here
`.text.__intel_get_fast_memcpy_impl' referenced in section `.text.__real_memcpy_impl_setup' of /home/marcosvanella/intel/oneapi/compiler/2024.1/lib/libirc_msan.a(fastmemcpy.c.o): defined in discarded section `.text.__intel_get_fast_memcpy_impl[__intel_get_fast_memcpy_impl]' of /home/marcosvanella/intel/oneapi/compiler/2024.1/lib/libirc_msan.a(fastmemcpy.c.o)
`.text.__intel_get_fast_memset_impl' referenced in section `.text.__real_memset_impl_setup' of /home/marcosvanella/intel/oneapi/compiler/2024.1/lib/libirc_msan.a(fastmemset.c.o): defined in discarded section `.text.__intel_get_fast_memset_impl[__intel_get_fast_memset_impl]' of /home/marcosvanella/intel/oneapi/compiler/2024.1/lib/libirc_msan.a(fastmemset.c.o)
`.text.addq_abs' referenced in section `.text.__addq' of /home/marcosvanella/intel/oneapi/compiler/2024.1/lib/libirc_msan.a(ia32_addsubq.c.o): defined in discarded section `.text.addq_abs[addq_abs]' of /home/marcosvanella/intel/oneapi/compiler/2024.1/lib/libirc_msan.a(ia32_addsubq.c.o)
`.text.subq_abs' referenced in section `.text.__addq' of /home/marcosvanella/intel/oneapi/compiler/2024.1/lib/libirc_msan.a(ia32_addsubq.c.o): defined in discarded section `.text.subq_abs[subq_abs]' of /home/marcosvanella/intel/oneapi/compiler/2024.1/lib/libirc_msan.a(ia32_addsubq.c.o)

- Second Issue : happens compiling with mpiifx and not MKL linking. To reproduce compile the file and execute as:

$ mpiifx -c -O0 -g -check all main.f90
$ mpiifx -O0 -g -check all -o test_check_all main.o
$ mpirun -n 1 ./test_check_all 

This is what I get:

==16934==WARNING: MemorySanitizer: use-of-uninitialized-value
#0 0x4e870f in for__format_value (/home/marcosvanella/Documents/Firemodels_fork/CHECK_ALL_BUGS/test_check_all+0x4e870f) (BuildId: d2c02c2765ff405477a2fe35deaa3c1fa34f02e9)
#1 0x4af244 in for_write_seq_fmt_xmit (/home/marcosvanella/Documents/Firemodels_fork/CHECK_ALL_BUGS/test_check_all+0x4af244) (BuildId: d2c02c2765ff405477a2fe35deaa3c1fa34f02e9)
#2 0x49149e in MAIN__ /home/marcosvanella/Documents/Firemodels_fork/CHECK_ALL_BUGS/main.f90:14:1
#3 0x40cfd8 in main (/home/marcosvanella/Documents/Firemodels_fork/CHECK_ALL_BUGS/test_check_all+0x40cfd8) (BuildId: d2c02c2765ff405477a2fe35deaa3c1fa34f02e9)
#4 0x7d97f6e29d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
#5 0x7d97f6e29e3f in __libc_start_main csu/../csu/libc-start.c:392:3
#6 0x40cea4 in _start (/home/marcosvanella/Documents/Firemodels_fork/CHECK_ALL_BUGS/test_check_all+0x40cea4) (BuildId: d2c02c2765ff405477a2fe35deaa3c1fa34f02e9)

Uninitialized value was stored to memory at
#0 0x49142f in MAIN__ /home/marcosvanella/Documents/Firemodels_fork/CHECK_ALL_BUGS/main.f90:14:1
#1 0x40cfd8 in main (/home/marcosvanella/Documents/Firemodels_fork/CHECK_ALL_BUGS/test_check_all+0x40cfd8) (BuildId: d2c02c2765ff405477a2fe35deaa3c1fa34f02e9)
#2 0x7d97f6e29d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Uninitialized value was created by an allocation of 'test_check_all_$MY_RANK' in the stack frame
#0 0x490e23 in MAIN__ /home/marcosvanella/Documents/Firemodels_fork/CHECK_ALL_BUGS/main.f90:8:23

SUMMARY: MemorySanitizer: use-of-uninitialized-value (/home/marcosvanella/Documents/Firemodels_fork/CHECK_ALL_BUGS/test_check_all+0x4e870f) (BuildId: d2c02c2765ff405477a2fe35deaa3c1fa34f02e9) in for__format_value
Exiting

Note that none of these occure using the latest mpiifort compiler wrapper with ifort (classic).

Thank you for you time and help!

Marcos

[Ron_Green]

Read This article

Although the uninit bug was fixed in 2024.1, the information in this article regarding the need for ALL third party libraries like MKL, MPI, NetCDF, HDF5, etc need to be built with -fsanitize=memory.  

For your case, change

-check all

to

-check all,nounit

as called out in the article.

[Marcos_V_1]

Thank you Ron, so do we know when to expect MKL and MPI to be released having been compiled with the -fsanitize=memory flag?

Best Regards,

Marcos

[Ron_Green]

typo above, should be

-check all,nouninit

For MKL and Intel MPI it is uncertain when or if that will happen.  I will escalate this request which other customers have made as well.

In defense, -check uninit in ifort had little value, given it's restrictions.  

You can use other LLVM sanitizers on Linux

-fsanitize=address

this will slow runtimes by about 2x

and if your code is threaded and you want to test for race conditions

-fsanitize=threads

this will slow runtimes by 10x to 20x and should be used only for development debug and not production.

[Marcos_V_1]

Thank you Ron, it would be very helpful for us if these libs (MPI,MKL) are compiled with the -fsanitize=memory option.

As part of our continuous integration we run every night 600 verification cases for a few time steps in debug mode, and one of the reasons for this is to catch uninitialized variables.

At this point, we cannot use ifx given this restriction.

Is the reason the libs are not compiled with the -fsanitize=memory flag related to them not working with ifort if this is done?

I will checkout the other options for fsanitize in our optimized targets, they sound interesting.

Thank you,

Marcos