valgrind invalid read mkl_serv_mkl_get_max_threads

[Azua_Garcia__Giovann]

Hello,

I continuously get valgrind errors below "invalid read" while looking for memory errors. It is apparently related to the MKL function mkl_serv_mkl_get_max_threads. Is this a MKL bug or am I missing to initialize something?

I am developing on Mac OSX 10.7.3 gcc 4.6 latest, but this same problem I had half a year ago I just didn't pay attention to it.

TIA,
Best regards,
Giovanni

==82462== Invalid read of size 8
==82462== at 0x3AD688: _dyld_func_lookup (in /opt/intel/composerxe-2011.4.184/compiler/lib/libiomp5.dylib)
==82462== by 0x422196: __kmp_do_serial_initialize() (in /opt/intel/composerxe-2011.4.184/compiler/lib/libiomp5.dylib)
==82462== by 0x409E53: __kmp_get_global_thread_id_reg (in /opt/intel/composerxe-2011.4.184/compiler/lib/libiomp5.dylib)
==82462== by 0x3FE708: omp_in_parallel (in /opt/intel/composerxe-2011.4.184/compiler/lib/libiomp5.dylib)
==82462== by 0x1E13611: mkl_serv_mkl_get_max_threads (in /opt/intel/composerxe-2011.4.184/mkl/lib/libmkl_intel_thread.dylib)
==82462== by 0x5241: __divti3 (in /opt/local/lib/valgrind/vgpreload_core-amd64-darwin.so)
==82462== by 0x3: ???
==82462== by 0x7FFF5FBFE27F: ???
==82462== by 0x25FC14B5F: ???
==82462== Address 0x242bbe998 is 56 bytes inside a block of size 60 alloc'd
==82462== at 0xD562: malloc_zone_malloc (vg_replace_malloc.c:267)
==82462== by 0xBEFE72: __setenv (in /usr/lib/system/libsystem_c.dylib)
==82462== by 0xBEFF51: setenv (in /usr/lib/system/libsystem_c.dylib)
==82462== by 0x3F8346: __kmp_env_set (in /opt/intel/composerxe-2011.4.184/compiler/lib/libiomp5.dylib)
==82462== by 0x42218E: __kmp_do_serial_initialize() (in /opt/intel/composerxe-2011.4.184/compiler/lib/libiomp5.dylib)
==82462== by 0x409E53: __kmp_get_global_thread_id_reg (in /opt/intel/composerxe-2011.4.184/compiler/lib/libiomp5.dylib)
==82462== by 0x3FE708: omp_in_parallel (in /opt/intel/composerxe-2011.4.184/compiler/lib/libiomp5.dylib)
==82462== by 0x1E13611: mkl_serv_mkl_get_max_threads (in /opt/intel/composerxe-2011.4.184/mkl/lib/libmkl_intel_thread.dylib)
==82462== by 0x5241: __divti3 (in /opt/local/lib/valgrind/vgpreload_core-amd64-darwin.so)
==82462== by 0x3: ???
==82462== by 0x7FFF5FBFE27F: ???
==82462== by 0x25FC14B5F: ???
==82462==
==82462== Invalid read of size 8
==82462== at 0x3AD688: _dyld_func_lookup (in /opt/intel/composerxe-2011.4.184/compiler/lib/libiomp5.dylib)
==82462== by 0x42B9B9: __kmp_env_initialize(char const*) (in /opt/intel/composerxe-2011.4.184/compiler/lib/libiomp5.dylib)
==82462== by 0x4226CB: __kmp_do_serial_initialize() (in /opt/intel/composerxe-2011.4.184/compiler/lib/libiomp5.dylib)
==82462== by 0x409E53: __kmp_get_global_thread_id_reg (in /opt/intel/composerxe-2011.4.184/compiler/lib/libiomp5.dylib)
==82462== by 0x3FE708: omp_in_parallel (in /opt/intel/composerxe-2011.4.184/compiler/lib/libiomp5.dylib)
==82462== by 0x1E13611: mkl_serv_mkl_get_max_threads (in /opt/intel/composerxe-2011.4.184/mkl/lib/libmkl_intel_thread.dylib)
==82462== by 0x5241: __divti3 (in /opt/local/lib/valgrind/vgpreload_core-amd64-darwin.so)
==82462== by 0x3: ???
==82462== by 0x7FFF5FBFE27F: ???
==82462== by 0x25FC14B5F: ???
==82462== Address 0x242bbe998 is 56 bytes inside a block of size 60 alloc'd
==82462== at 0xD562: malloc_zone_malloc (vg_replace_malloc.c:267)
==82462== by 0xBEFE72: __setenv (in /usr/lib/system/libsystem_c.dylib)
==82462== by 0xBEFF51: setenv (in /usr/lib/system/libsystem_c.dylib)
==82462== by 0x3F8346: __kmp_env_set (in /opt/intel/composerxe-2011.4.184/compiler/lib/libiomp5.dylib)
==82462== by 0x42218E: __kmp_do_serial_initialize() (in /opt/intel/composerxe-2011.4.184/compiler/lib/libiomp5.dylib)
==82462== by 0x409E53: __kmp_get_global_thread_id_reg (in /opt/intel/composerxe-2011.4.184/compiler/lib/libiomp5.dylib)
==82462== by 0x3FE708: omp_in_parallel (in /opt/intel/composerxe-2011.4.184/compiler/lib/libiomp5.dylib)
==82462== by 0x1E13611: mkl_serv_mkl_get_max_threads (in /opt/intel/composerxe-2011.4.184/mkl/lib/libmkl_intel_thread.dylib)
==82462== by 0x5241: __divti3 (in /opt/local/lib/valgrind/vgpreload_core-amd64-darwin.so)
==82462== by 0x3: ???
==82462== by 0x7FFF5FBFE27F: ???
==82462== by 0x25FC14B5F: ???
==82462==
==82462== Invalid read of size 8
==82462== at 0x3AD688: _dyld_func_lookup (in /opt/intel/composerxe-2011.4.184/compiler/lib/libiomp5.dylib)
==82462== by 0x42B4F3: __kmp_asat_env_initialize(char const*) (in /opt/intel/composerxe-2011.4.184/compiler/lib/libiomp5.dylib)
==82462== by 0x42275C: __kmp_do_serial_initialize() (in /opt/intel/composerxe-2011.4.184/compiler/lib/libiomp5.dylib)
==82462== by 0x409E53: __kmp_get_global_thread_id_reg (in /opt/intel/composerxe-2011.4.184/compiler/lib/libiomp5.dylib)
==82462== by 0x3FE708: omp_in_parallel (in /opt/intel/composerxe-2011.4.184/compiler/lib/libiomp5.dylib)
==82462== by 0x1E13611: mkl_serv_mkl_get_max_threads (in /opt/intel/composerxe-2011.4.184/mkl/lib/libmkl_intel_thread.dylib)
==82462== by 0x5241: __divti3 (in /opt/local/lib/valgrind/vgpreload_core-amd64-darwin.so)
==82462== by 0x3: ???
==82462== by 0x7FFF5FBFE27F: ???
==82462== by 0x25FC14B5F: ???
==82462== Address 0x242bbe998 is 56 bytes inside a block of size 60 alloc'd
==82462== at 0xD562: malloc_zone_malloc (vg_replace_malloc.c:267)
==82462== by 0xBEFE72: __setenv (in /usr/lib/system/libsystem_c.dylib)
==82462== by 0xBEFF51: setenv (in /usr/lib/system/libsystem_c.dylib)
==82462== by 0x3F8346: __kmp_env_set (in /opt/intel/composerxe-2011.4.184/compiler/lib/libiomp5.dylib)
==82462== by 0x42218E: __kmp_do_serial_initialize() (in /opt/intel/composerxe-2011.4.184/compiler/lib/libiomp5.dylib)
==82462== by 0x409E53: __kmp_get_global_thread_id_reg (in /opt/intel/composerxe-2011.4.184/compiler/lib/libiomp5.dylib)
==82462== by 0x3FE708: omp_in_parallel (in /opt/intel/composerxe-2011.4.184/compiler/lib/libiomp5.dylib)
==82462== by 0x1E13611: mkl_serv_mkl_get_max_threads (in /opt/intel/composerxe-2011.4.184/mkl/lib/libmkl_intel_thread.dylib)
==82462== by 0x5241: __divti3 (in /opt/local/lib/valgrind/vgpreload_core-amd64-darwin.so)
==82462== by 0x3: ???
==82462== by 0x7FFF5FBFE27F: ???
==82462== by 0x25FC14B5F: ???
==82462==

[Gennady_F_Intel]

it may be false positive errors produces by profiling tool.

can you give the test case? and which compiler has been used?

[barragan_villanueva_]

Hi,

It's a known issue with valgrind when itcannot correctly recognize/profile codecreated by Intel compiler:
libiomp5.dylib, libmkl_intel_thread.dylib

What version of valgrind did you use?

[Azua_Garcia__Giovann]

Hi Victor,

Thank you for your answer. Here is my setup:

/Users/bravegag/code/fastcode_project/code$ uname -a
Darwin Macintosh-4.local 11.3.0 Darwin Kernel Version 11.3.0: Thu Jan 12 18:47:41 PST 2012; root:xnu-1699.24.23~1/RELEASE_X86_64 x86_64

/Users/bravegag/code/fastcode_project/code$ gcc --version
gcc (GCC) 4.6.3
Copyright (C) 2011 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

/Users/bravegag/code/fastcode_project/code$ valgrind --version
valgrind-3.8.0.SVN

-- Intel MKL was found:
MKL_INCLUDE_DIRS: /opt/intel/composerxe-2011.4.184/mkl/include
MKL_LIBRARY_DIRS: /opt/intel/composerxe-2011.4.184/mkl/lib;/opt/intel/composerxe-2011.4.184/compiler/lib
MKL_LIBRARIES: /opt/intel/composerxe-2011.4.184/mkl/lib/libmkl_rt.dylib;/opt/intel/composerxe-2011.4.184/compiler/lib/libiomp5.dylib;/opt/intel/composerxe-2011.4.184/compiler/lib/libimf.dylib

Compilation:
/usr/bin/gcc -DGTEST_HAS_TR1_TUPLE=0 --std=gnu99 -Wall -Wextra -Wshadow -Wstrict-prototypes -Wmissing-prototypes -g3 -ggdb3 -I/Users/bravegag/code/fastcode_project/code/third_party/googletest/include -I/Users/bravegag/code/fastcode_project/code/third_party/googletest -I/opt/intel/composerxe-2011.4.184/mkl/include -I/Users/bravegag/code/fastcode_project/code/src -I/Users/bravegag/code/fastcode_project/code/build -I/Users/bravegag/code/fastcode_project/code/third_party/genrmf -o CMakeFiles/submodularity.dir/src/matrix.c.o -c /Users/bravegag/code/fastcode_project/code/src/matrix.c

Running valgrind:
/Users/bravegag/code/fastcode_project/code/build$ valgrind --show-reachable=yes --dsymutil=yes ./test_matrix
==8028== Memcheck, a memory error detector
==8028== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==8028== Using Valgrind-3.8.0.SVN and LibVEX; rerun with -h for copyright info
==8028== Command: ./test_matrix
==8028==

I don't compile with Intel compiler but with gcc instead. When I did the initial benchmarks of my project gcc release gave better results (and this is a topic for a separate question :)).

TIA,
Best regards,
Giovanni

[barragan_villanueva_]

Giovanni,

Thanks for the detailed info.
I can see that the latest released valgrind 3.7.0is from 5 November 2011:

for {x86,amd64,arm,ppc32,ppc64,s390x}-linux, arm-android (2.3.x) and {x86,amd64}-darwin (Mac OS X 10.6 and 10.7).

Could you create/isolatesome testcase to look at and analyzeit on our side?
Also, did you try to link withPGI mkl-thread-layer (instead of Intel-thread-layer and libiomp5)together withPGI openmp library?

[Azua_Garcia__Giovann]

Hello Victor,

Please see below the linking step. That's what I am linking with ... I don't invoke any threads explicitly so I didn't link with any specific threading library. The invalid access issue ocurrs as soon as I call a MKL function the first time. Then it doesn't show anymore. I tried the newest MKL version sp1.9.289 to check whether the problem was solved but no it still ocurrs in the latest version.

Linking CXX executable test_matrix
/opt/local/bin/cmake -E cmake_link_script CMakeFiles/test_matrix.dir/link.txt --verbose=1
/usr/bin/c++ -Wall -Wextra -Wshadow -Woverloaded-virtual -g3 -ggdb3 -Wl,-search_paths_first -Wl,-headerpad_max_install_names CMakeFiles/test_matrix.dir/test/matrix_test.cc.o -o test_matrix gtest/libgtest.a gtest/libgtest_main.a libqr_updates.a libsubmodularity.a gtest/libgtest.a -lpthread /opt/intel/composerxe-2011.4.184/mkl/lib/libmkl_rt.dylib /opt/intel/composerxe-2011.4.184/compiler/lib/libiomp5.dylib /opt/intel/composerxe-2011.4.184/compiler/lib/libimf.dylib -lgfortran -lquadmath -lm
/opt/local/bin/cmake -E cmake_progress_report /Users/bravegag/code/fastcode_project/code/build/CMakeFiles 25
[ 89%] Built target test_matrix

I am sorry but I don't have so much time to prepare a test case in isolation since I have lot of presure to deliver, this is my Master thesis.

Best regards,
Giovanni

[Ying_H_Intel]

Hi Giovanni,

We can reproduce the valgrind report on Mac OS. And we checked it withthe OpenMPengineer team. (MKL is threaded by OpenMP runtime library, which is developed by OpenMP team).

Please seeour expert's explanation,

<<
Valgrind usually reports some false positives in Intel OpenMP RTL on Linux, such as:

1. Syscall param sched_setaffinity(mask) points to unaddressable byte(s)

2. Invalid read of size 8

3. Use of uninitialised value of size 8

4. Conditional jump or move depends on uninitialised value(s)

The reasons are different.
First problem shown because we intentionally call system routine with "invalid" NULL parameter. The behavior of the routine is well defined in this case, so no problems expected.
Second and third problems caused by valgrind which haven't got on well with code generated by Intel compiler.
Fourth problem is the consequence of the previous one.

There may also be memory leaks detected in glibc library on each call to setenv() function, but this is known problem of glibc that has nothing with the OpenMP Runtime library.
>>.

So as victor mentioned, it is known and fasle reports. You may ignore them here.

Best Regards,
Ying

[Azua_Garcia__Giovann]

Hello Ying,

Thank you! very good job following up with this.

Best regards,
Giovanni