OMP: Error #13: Assertion failure at kmp_runtime.c(8139)

amir_l_ · ‎11-14-2012

Hi guys,

I compiled my application with ICC 2013 on several machines and it works well.

I recevie an error message only with one machine:

OMP: Error #13: Assertion failure at kmp_runtime.c(8139).
OMP: Hint: Please submit a bug report with this message, compile and run commands used, and machine configuration info including native compiler and operating system versions. Faster response will be obtained by including all program sources. For information on submitting this issue, please see http://www.intel.com/software/products/support/.

I read about similiar problem in the forum and set the enviroment variables:

KMP_SETTINGS=1
KMP_VERSION=1

And I get:

Effective settings:

KMP_ABORT_DELAY=0
KMP_ABORT_IF_NO_IRML=false
KMP_AFFINITY="noverbose,none"
KMP_ALIGN_ALLOC=64
KMP_ALL_THREADPRIVATE=128
KMP_ALL_THREADS=2147483647
KMP_ASAT_DEC=1
KMP_ASAT_FAVOR=0
KMP_ASAT_INC=4
KMP_ASAT_INTERVAL=5
KMP_ASAT_TRIGGER=5000
KMP_ATOMIC_MODE=2
KMP_BLOCKTIME=200
KMP_CPUINFO_FILE: value is not defined
KMP_DETERMINISTIC_REDUCTION=false
KMP_DUPLICATE_LIB_OK=false
KMP_FORCE_REDUCTION: value is not defined
KMP_FOREIGN_THREADS_THREADPRIVATE=true
KMP_FORKJOIN_BARRIER="2,2"
KMP_FORKJOIN_BARRIER_PATTERN="hyper,hyper"
KMP_GTID_MODE=3
KMP_HANDLE_SIGNALS=false
KMP_INIT_AT_FORK=true
KMP_INIT_WAIT=2048
KMP_ITT_PREPARE_DELAY=0
KMP_LIBRARY=throughput
KMP_LOCK_KIND=queuing
KMP_MALLOC_POOL_INCR=1M
KMP_MONITOR_STACKSIZE: value is not defined
KMP_NEXT_WAIT=1024
KMP_NUM_LOCKS_IN_BLOCK=1
KMP_PLAIN_BARRIER="2,2"
KMP_PLAIN_BARRIER_PATTERN="hyper,hyper"
KMP_REDUCTION_BARRIER="1,1"
KMP_REDUCTION_BARRIER_PATTERN="hyper,hyper"
KMP_SCHEDULE="static,balanced;guided,iterative"
KMP_SETTINGS=true
KMP_SSP_MAX_THREADS=0
KMP_STACKOFFSET=0
KMP_STACKSIZE=4M
KMP_STORAGE_MAP=false
KMP_TASKING=2
KMP_TASK_STEALING_CONSTRAINT=1
KMP_USE_IRML=false
KMP_VERSION=true
KMP_WARNINGS=true
OMP_DYNAMIC=false
OMP_MAX_ACTIVE_LEVELS=2147483647
OMP_NESTED=false
OMP_NUM_THREADS: value is not defined
OMP_WAIT_POLICY=PASSIVE

I also verified that only one omp library is linked to my application.

My OS is Ubuntu 12.04.

Please advise

Thanks

Andrey_C_Intel1 · ‎11-15-2012

Hi! Could you please provide a bit more info on the problem? First of all, we would like to see the output with KMP_VERSION=1 setting, in the output you provided this information is not included. Second, what kind of Ubuntu 12.04 you are using? Is it 32-bit or 64-bit OS? Could you provide the output of "uname -a" command? It would be interesting to also see the output of the program with KMP_AFFINITY=verbose setting. Thanks, Andrey

amir_l_ · ‎11-15-2012

Hi, Thanks for your help! I'm using Ubuntu 12.04 64-bit. uname -a: Linux ubuntu1010 2.6.32-11-pve #1 SMP Tue Apr 3 10:21:21 CEST 2012 x86_64 x86_64 x86_64 GNU/Linux With KMP_VERSION=1 the output is: Intel(R) OMP Copyright (C) 1997-2012, Intel Corporation. All Rights Reserved. Intel(R) OMP version: 5.0.20120611 Intel(R) OMP library type: performance Intel(R) OMP link type: dynamic Intel(R) OMP build time: 2012-06-22 14:20:32 UTC Intel(R) OMP build compiler: Intel C++ Compiler 12.1 Intel(R) OMP alternative compiler support: yes Intel(R) OMP API version: 3.1 (201107) Intel(R) OMP tracefile support: no Intel(R) OMP dynamic error checking: no Intel(R) OMP helper thread support: yes Intel(R) OMP thread affinity support: not used Intel(R) OMP debugger support version: 1.1 User settings: KMP_SETTINGS=1 KMP_VERSION=1 Effective settings: KMP_ABORT_DELAY=0 KMP_ABORT_IF_NO_IRML=false KMP_AFFINITY="noverbose,none" KMP_ALIGN_ALLOC=64 KMP_ALL_THREADPRIVATE=128 KMP_ALL_THREADS=2147483647 KMP_ASAT_DEC=1 KMP_ASAT_FAVOR=0 KMP_ASAT_INC=4 KMP_ASAT_INTERVAL=5 KMP_ASAT_TRIGGER=5000 KMP_ATOMIC_MODE=2 KMP_BLOCKTIME=200 KMP_CPUINFO_FILE: value is not defined KMP_DETERMINISTIC_REDUCTION=false KMP_DUPLICATE_LIB_OK=false KMP_FORCE_REDUCTION: value is not defined KMP_FOREIGN_THREADS_THREADPRIVATE=true KMP_FORKJOIN_BARRIER="2,2" KMP_FORKJOIN_BARRIER_PATTERN="hyper,hyper" KMP_GTID_MODE=3 KMP_HANDLE_SIGNALS=false KMP_INIT_AT_FORK=true KMP_INIT_WAIT=2048 KMP_ITT_PREPARE_DELAY=0 KMP_LIBRARY=throughput KMP_LOCK_KIND=queuing KMP_MALLOC_POOL_INCR=1M KMP_MONITOR_STACKSIZE: value is not defined KMP_NEXT_WAIT=1024 KMP_NUM_LOCKS_IN_BLOCK=1 KMP_PLAIN_BARRIER="2,2" KMP_PLAIN_BARRIER_PATTERN="hyper,hyper" KMP_REDUCTION_BARRIER="1,1" KMP_REDUCTION_BARRIER_PATTERN="hyper,hyper" KMP_SCHEDULE="static,balanced;guided,iterative" KMP_SETTINGS=true KMP_SSP_MAX_THREADS=0 KMP_STACKOFFSET=0 KMP_STACKSIZE=4M KMP_STORAGE_MAP=false KMP_TASKING=2 KMP_TASK_STEALING_CONSTRAINT=1 KMP_USE_IRML=false KMP_VERSION=true KMP_WARNINGS=true OMP_DYNAMIC=false OMP_MAX_ACTIVE_LEVELS=2147483647 OMP_NESTED=false OMP_NUM_THREADS: value is not defined OMP_WAIT_POLICY=PASSIVE OMP: Error #13: Assertion failure at kmp_runtime.c(8139). OMP: Hint: Please submit a bug report with this message, compile and run commands used, and machine configuration info including native compiler and operating system versions. Faster response will be obtained by including all program sources. For information on submitting this issue, please see http://www.intel.com/software/products/support/. Aborted After I set the KMP_AFFINITY variable I ran the program and the output is: Intel(R) OMP Copyright (C) 1997-2012, Intel Corporation. All Rights Reserved. Intel(R) OMP version: 5.0.20120611 Intel(R) OMP library type: performance Intel(R) OMP link type: dynamic Intel(R) OMP build time: 2012-06-22 14:20:32 UTC Intel(R) OMP build compiler: Intel C++ Compiler 12.1 Intel(R) OMP alternative compiler support: yes Intel(R) OMP API version: 3.1 (201107) Intel(R) OMP tracefile support: no Intel(R) OMP dynamic error checking: no Intel(R) OMP helper thread support: yes Intel(R) OMP thread affinity support: not used Intel(R) OMP debugger support version: 1.1 User settings: KMP_AFFINITY=verbose KMP_SETTINGS=1 KMP_VERSION=1 Effective settings: KMP_ABORT_DELAY=0 KMP_ABORT_IF_NO_IRML=false KMP_AFFINITY="verbose,none" KMP_ALIGN_ALLOC=64 KMP_ALL_THREADPRIVATE=128 KMP_ALL_THREADS=2147483647 KMP_ASAT_DEC=1 KMP_ASAT_FAVOR=0 KMP_ASAT_INC=4 KMP_ASAT_INTERVAL=5 KMP_ASAT_TRIGGER=5000 KMP_ATOMIC_MODE=2 KMP_BLOCKTIME=200 KMP_CPUINFO_FILE: value is not defined KMP_DETERMINISTIC_REDUCTION=false KMP_DUPLICATE_LIB_OK=false KMP_FORCE_REDUCTION: value is not defined KMP_FOREIGN_THREADS_THREADPRIVATE=true KMP_FORKJOIN_BARRIER="2,2" KMP_FORKJOIN_BARRIER_PATTERN="hyper,hyper" KMP_GTID_MODE=3 KMP_HANDLE_SIGNALS=false KMP_INIT_AT_FORK=true KMP_INIT_WAIT=2048 KMP_ITT_PREPARE_DELAY=0 KMP_LIBRARY=throughput KMP_LOCK_KIND=queuing KMP_MALLOC_POOL_INCR=1M KMP_MONITOR_STACKSIZE: value is not defined KMP_NEXT_WAIT=1024 KMP_NUM_LOCKS_IN_BLOCK=1 KMP_PLAIN_BARRIER="2,2" KMP_PLAIN_BARRIER_PATTERN="hyper,hyper" KMP_REDUCTION_BARRIER="1,1" KMP_REDUCTION_BARRIER_PATTERN="hyper,hyper" KMP_SCHEDULE="static,balanced;guided,iterative" KMP_SETTINGS=true KMP_SSP_MAX_THREADS=0 KMP_STACKOFFSET=0 KMP_STACKSIZE=4M KMP_STORAGE_MAP=false KMP_TASKING=2 KMP_TASK_STEALING_CONSTRAINT=1 KMP_USE_IRML=false KMP_VERSION=true KMP_WARNINGS=true OMP_DYNAMIC=false OMP_MAX_ACTIVE_LEVELS=2147483647 OMP_NESTED=false OMP_NUM_THREADS: value is not defined OMP_WAIT_POLICY=PASSIVE OMP: Warning #121: Error initializing affinity - not using affinity. OMP: Error #13: Assertion failure at kmp_runtime.c(8139). OMP: Hint: Please submit a bug report with this message, compile and run commands used, and machine configuration info including native compiler and operating system versions. Faster response will be obtained by including all program sources. For information on submitting this issue, please see http://www.intel.com/software/products/support/.

SergeyKostrov · ‎11-15-2012

How many OMP threads are created in your application? Also, you need to verify all Not Defined settings like: ... OMP_NUM_THREADS: value is not defined ...

amir_l_ · ‎11-15-2012

How can I check the amount of OMP threads in my application? I didn't create such threads with Intel's API. This source code is also compiled with GCC. The only difference between the two compilations is the use of IPP.

SergeyKostrov · ‎11-15-2012

>>How can I check the amount of OMP threads in my application? On a Windows platform you need to use the Task Manager. On a Linux platform a similar monitoring utility. >>...This source code is also compiled with GCC. The only difference between the two compilations is the use of IPP... Take into account that IPP could create many OMP threads internally to speed up some processing. It actually could be a problem with IPP and try to understand which IPP function is called before the assert message is displayed. It also could be a problem with the OMP_NUM_THREADS value since it is not defined.

SergeyKostrov · ‎11-15-2012

An exact reason of the problem is in 'kmp_runtime.c' source file at a line 8139. Intel Software Engineers should look there.

Andrey_C_Intel1 · ‎11-16-2012

As a workaround, could you please try to set environment KMP_AFFINITY=disabled. In the meantime we will work on the investigation. It is still not clear what is the reason of the issue. Thanks, Andrey