This system is running CentOS

Clay__R_Torsten · ‎09-11-2014

I am trying to get the mic offload samples available at

https://software.intel.com/en-us/articles/offload-programming-fortran-and-c-code-examples

to run. Everything appears to build without error; here is the output of "make mic":

====

ifort -qopenmp  -c leoF00_driver.F90 -o leoF00_driver.o
ifort -qopenmp  -c leoF01_scalar.F90 -o leoF01_scalar.o
ifort -qopenmp  -c leoF02_global.F90 -o leoF02_global.o
ifort -qopenmp  -c leoF03_various_clauses.F90 -o leoF03_various_clauses.o
ifort -qopenmp  -c leoF04_explicit_shape_array.F90 -o leoF04_explicit_shape_array.o
ifort -qopenmp  -c leoF05_assumed_shape_array.F90 -o leoF05_assumed_shape_array.o
ifort -qopenmp  -c leoF06_assumed_size_array.F90 -o leoF06_assumed_size_array.o
ifort -qopenmp  -c leoF07_deferred_shape_array.F90 -o leoF07_deferred_shape_array.o
ifort -qopenmp  -c leoF08_multi_target.F90 -o leoF08_multi_target.o
ifort -qopenmp  -c leoF09_subprograms_callees.F90 -o leoF09_subprograms_callees.o
ifort -qopenmp  -c leoF09_subprograms.F90 -o leoF09_subprograms.o
ifort -qopenmp  -c leoF10_alloc_into.F90 -o leoF10_alloc_into.o
ifort -qopenmp  -c leoF11_async.F90 -o leoF11_async.o
ifort -V leoF00_driver.o leoF01_scalar.o leoF02_global.o leoF03_various_clauses.o leoF04_explicit_shape_array.o leoF05_assumed_shape_array.o leoF06_assumed_size_array.o leoF07_deferred_shape_array.o leoF08_multi_target.o leoF09_subprograms_callees.o leoF09_subprograms.o leoF10_alloc_into.o leoF11_async.o -qopenmp   -o leoF_intro.out
Intel(R) Fortran Intel(R) 64 Compiler XE for applications running on Intel(R) 64, Version 15.0.0.090 Build 20140723
Copyright (C) 1985-2014 Intel Corporation.  All rights reserved.


GNU ld (GNU Binutils) 2.22.52.20120302

GNU ld version 2.20.51.0.2-5.36.el6 20100205
....
Build complete
Run : leoF_intro.out [--help]

===========

But this is what happens when I try to run it (here OFFLOAD_REPORT=3):

===========

System configuration pre-check
    Checking for Intel(R) Xeon Phi(TM) (Target CPU) devices...

    Number of Target devices installed:      2
[Offload] [HOST]          [State]           Initialize logical card 0 = physical card 0
offload error: cannot execute function on the device 0 (error code 13)

============

Some further information:

============
ifort -V
Intel(R) Fortran Intel(R) 64 Compiler XE for applications running on Intel(R) 64, Version 15.0.0.090 Build 20140723
Copyright (C) 1985-2014 Intel Corporation.  All rights reserved.
---------------
MicInfo Utility Log

Created Thu Sep 11 14:40:42 2014


	System Info
		HOST OS			: Linux
		OS Version		: 2.6.32-431.17.1.el6.x86_64
		Driver Version		: 3.1.4-1
		MPSS Version		: 3.1.4
		Host Physical Memory	: 66001 MB

Device No: 0, Device Name: mic0

	Version
		Flash Version 		 : 2.1.03.0386
		SMC Firmware Version	 : 1.15.4830
		SMC Boot Loader Version	 : 1.8.4326
		uOS Version 		 : 2.6.38.8+mpss3.1.4
		Device Serial Number 	 : ADKC32400287

	Board
		Vendor ID 		 : 0x8086
		Device ID 		 : 0x2250
		Subsystem ID 		 : 0x2500
		Coprocessor Stepping ID	 : 3
		PCIe Width 		 : Insufficient Privileges
		PCIe Speed 		 : Insufficient Privileges
		PCIe Max payload size	 : Insufficient Privileges
		PCIe Max read req size	 : Insufficient Privileges
		Coprocessor Model	 : 0x01
		Coprocessor Model Ext	 : 0x00
		Coprocessor Type	 : 0x00
		Coprocessor Family	 : 0x0b
		Coprocessor Family Ext	 : 0x00
		Coprocessor Stepping 	 : B1
		Board SKU 		 : B1PRQ-5110P/5120D
		ECC Mode 		 : Enabled
		SMC HW Revision 	 : Product 225W Passive CS

	Cores
		Total No of Active Cores : 60
		Voltage 		 : 971000 uV
		Frequency		 : 1052631 kHz

	Thermal
		Fan Speed Control 	 : N/A
		Fan RPM 		 : N/A
		Fan PWM 		 : N/A
		Die Temp		 : 49 C

	GDDR
		GDDR Vendor		 : Elpida
		GDDR Version		 : 0x1
		GDDR Density		 : 2048 Mb
		GDDR Size		 : 7936 MB
		GDDR Technology		 : GDDR5 
		GDDR Speed		 : 5.000000 GT/s 
		GDDR Frequency		 : 2500000 kHz
		GDDR Voltage		 : 1501000 uV

Device No: 1, Device Name: mic1
...

==============

Any suggestions on what is wrong? I am able to ssh into the mics and run native code there, so they seem to be functioning normally. But offloading does not seem to be working at all.

Kevin_D_Intel · ‎09-11-2014

Unless I'm mistaken, the error code 13 indicates the application ran out of memory which is unusual and not something we've heard/seen with the samples in the past.

Is there anything unique about your system/setup?
Are you using the pre-built MPSS 3.1.2?
Can you show the values on the host of ulimit -a ?
Can you capture your host environment variable settings and share those with us?
Are there any other applications running on the card at the time of the failure?

Clay__R_Torsten · ‎09-11-2014

As far as I know there isn't anything unique about this system (I am not the sysadmin for this system however). There are no other applications running on the card. The system appears to be using MPSS 3.1.4 (pre-built). I'm wondering if there is a software version mismatch somewhere, or if somehow the mic is misconfigured? But it only affects offloading, I can ssh into the mic and run code in mic native mode.

ulimit -a:

core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 515477
max locked memory       (kbytes, -l) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1023
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 1048576
cpu time               (seconds, -t) unlimited
max user processes              (-u) 1024
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

host environment (hostnames removed):

MKLROOT=/usr/local/intel-2015/composer_xe_2015.0.090/mkl
SWSETUP_HISTORY=intel-14:intel-mpi
MODULE_VERSION_STACK=3.2.10
MANPATH=/usr/local/intel-2015/composer_xe_2015.0.090/man/en_US:/usr/local/intel-2015/composer_xe_2015.0.090/man/en_US:/usr/local/intel-2015/composer_xe_2015.0.090/debugger/gdb/intel64/share/man/:/usr/local/intel-2015/composer_xe_2015.0.090/debugger/gdb/intel64_mic/share/man/:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/man/en_US:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/man/en_US:/usr/man:/usr/share/man:/usr/X11R6/man:/usr/local/man:/usr/local/gnu-64/man:/usr/local/gnu/man:/usr/local/gnu/lib/perl5/man:/usr/local/xfreerdp/man
HOSTNAME=shadow-0027
INTEL_LICENSE_FILE=27015@xxxxx:/usr/local/intel-2015/composer_xe_2015.0.090/licenses:/opt/intel/licenses:/home/tclay/intel/licenses
IPPROOT=/usr/local/intel-2015/composer_xe_2015.0.090/ipp
TERM=vt100
SHELL=/bin/bash
HISTSIZE=100
GDBSERVER_MIC=/usr/local/intel-2015/composer_xe_2015.0.090/debugger/gdb/target/mic/bin/gdbserver
SSH_CLIENT=xxxxx
LIBRARY_PATH=/usr/local/intel-2015/composer_xe_2015.0.090/compiler/lib/intel64:/usr/local/intel-2015/composer_xe_2015.0.090/ipp/../compiler/lib/intel64:/usr/local/intel-2015/composer_xe_2015.0.090/ipp/lib/intel64:/usr/local/intel-2015/composer_xe_2015.0.090/compiler/lib/intel64:/usr/local/intel-2015/composer_xe_2015.0.090/mkl/lib/intel64:/usr/local/intel-2015/composer_xe_2015.0.090/tbb/lib/intel64/gcc4.4:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/compiler/lib/intel64:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/ipp/../compiler/lib/intel64:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/ipp/lib/intel64:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/compiler/lib/intel64:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/mkl/lib/intel64:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/tbb/lib/intel64/gcc4.4
QTDIR=/usr/lib64/qt-3.3
QTINC=/usr/lib64/qt-3.3/include
MIC_LD_LIBRARY_PATH=/usr/local/intel-2015/composer_xe_2015.0.090/compiler/lib/mic:/usr/local/intel-2015/composer_xe_2015.0.090/mpirt/lib/mic:/opt/intel/mic/coi/device-linux-release/lib:/opt/intel/mic/myo/lib:/usr/local/intel-2015/composer_xe_2015.0.090/ipp/lib/lib/mic:/opt/intel/mic/coi/device-linux-release/lib:/opt/intel/mic/myo/lib:/usr/local/intel-2015/composer_xe_2015.0.090/compiler/lib/mic:/usr/local/intel-2015/composer_xe_2015.0.090/mkl/lib/mic:/usr/local/intel-2015/composer_xe_2015.0.090/tbb/lib/mic:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/compiler/lib/mic:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/mpirt/lib/mic:/opt/intel/mic/coi/device-linux-release/lib:/opt/intel/mic/myo/lib:/opt/intel/mic/coi/device-linux-release/lib:/opt/intel/mic/myo/lib:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/compiler/lib/mic:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/mkl/lib/mic:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/tbb/lib/mic
SSH_TTY=/dev/pts/0
HISTFILESIZE=0
USER=tclay
LD_LIBRARY_PATH=/usr/local/intel-2015/composer_xe_2015.0.090/compiler/lib/intel64:/opt/intel/mic/coi/host-linux-release/lib:/opt/intel/mic/myo/lib:/usr/local/intel-2015/composer_xe_2015.0.090/mpirt/lib/intel64:/usr/local/intel-2015/composer_xe_2015.0.090/ipp/../compiler/lib/intel64:/usr/local/intel-2015/composer_xe_2015.0.090/ipp/lib/intel64:/opt/intel/mic/coi/host-linux-release/lib:/opt/intel/mic/myo/lib:/usr/local/intel-2015/composer_xe_2015.0.090/compiler/lib/intel64:/usr/local/intel-2015/composer_xe_2015.0.090/mkl/lib/intel64:/usr/local/intel-2015/composer_xe_2015.0.090/tbb/lib/intel64/gcc4.4:/usr/local/intel-2015/impi/5.0.1.035/lib64:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/compiler/lib/intel64:/opt/intel/mic/coi/host-linux-release/lib:/opt/intel/mic/myo/lib:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/mpirt/lib/intel64:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/ipp/../compiler/lib/intel64:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/ipp/lib/intel64:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/mkl/lib/intel64:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/tbb/lib/intel64/gcc4.4:/usr/lib:/usr/X11R6/lib:/lib:/opt/gnome/lib64:/opt/gnome/lib:/usr/local/lib:/usr/local/gnu-64/lib:/usr/local/gnu/lib:/usr/local/xfreerdp/lib/
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lz=01;31:*.xz=01;31:*.bz2=01;31:*.tbz=01;31:*.tbz2=01;31:*.bz=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:
MIC_LIBRARY_PATH=/usr/local/intel-2015/composer_xe_2015.0.090/compiler/lib/mic:/usr/local/intel-2015/composer_xe_2015.0.090/mpirt/lib/mic:/usr/local/intel-2015/composer_xe_2015.0.090/tbb/lib/mic:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/tbb/lib/mic
CPATH=/usr/local/intel-2015/composer_xe_2015.0.090/ipp/include:/usr/local/intel-2015/composer_xe_2015.0.090/mkl/include:/usr/local/intel-2015/composer_xe_2015.0.090/tbb/include:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/ipp/include:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/mkl/include:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/tbb/include
NNTP=xxxxx
NLSPATH=/usr/local/intel-2015/composer_xe_2015.0.090/compiler/lib/intel64/locale/%l_%t/%N:/usr/local/intel-2015/composer_xe_2015.0.090/ipp/lib/intel64/locale/%l_%t/%N:/usr/local/intel-2015/composer_xe_2015.0.090/mkl/lib/intel64/locale/%l_%t/%N:/usr/local/intel-2015/composer_xe_2015.0.090/debugger/gdb/intel64_mic/share/locale/%l_%t/%N:/usr/local/intel-2015/composer_xe_2015.0.090/debugger/gdb/intel64/share/locale/%l_%t/%N:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/compiler/lib/intel64/locale/%l_%t/%N:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/ipp/lib/intel64/locale/%l_%t/%N:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/mkl/lib/intel64/locale/%l_%t/%N:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/debugger/gdb/intel64_mic/py26/share/locale/%l_%t/%N:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/debugger/gdb/intel64/py26/share/locale/%l_%t/%N:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/debugger/intel64/locale/%l_%t/%N
MODULE_VERSION=3.2.10
MAIL=/var/spool/mail/tclay
PATH=/usr/local/intel-2015/composer_xe_2015.0.090/bin/intel64:/usr/local/intel-2015/composer_xe_2015.0.090/mpirt/bin/intel64:/usr/local/intel-2015/composer_xe_2015.0.090/debugger/gdb/intel64_mic/bin:/usr/local/intel-2015/impi/5.0.1.035/bin64:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/bin/intel64:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/mpirt/bin/intel64:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/debugger/gdb/intel64_mic/py26/bin:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/debugger/gdb/intel64/py26/bin:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/bin/intel64_mic:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/debugger/gui/intel64:/sbin:/bin:/usr/sbin:/usr/bin:/usr/X11R6/bin:/opt/gnome/bin:/usr/local/bin:/usr/local/gnu-64/bin:/usr/local/gnu/bin:.:/usr/local/xfreerdp/bin/:/home/tclay/bin
TBBROOT=/usr/local/intel-2015/composer_xe_2015.0.090/tbb
PWD=/home/tclay
IDB_HOME=/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/bin/intel64
EDITOR=vi
GDB_CROSS=/usr/local/intel-2015/composer_xe_2015.0.090/debugger/gdb/intel64_mic/bin/gdb-mic
LANG=en_US.UTF-8
MODULEPATH=/global/modules
LOADEDMODULES=
LM_LICENSE_FILE=27015@xxxxx
MPM_LAUNCHER=/usr/local/intel-2015/composer_xe_2015.0.090/debugger/mpm/bin/start_mpm.sh
SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass
HISTCONTROL=ignoredups
INTEL_PYTHONHOME=/usr/local/intel-2015/composer_xe_2015.0.090/debugger/python/intel64/
MANPAGER=/usr/bin/less
SHLVL=1
HOME=/home/tclay
LOGNAME=tclay
QTLIB=/usr/lib64/qt-3.3/lib
CVS_RSH=ssh
SSH_CONNECTION=xxxxx
MODULESHOME=/usr/Modules/3.2.10
LESSOPEN=|/usr/bin/lesspipe.sh %s
INFOPATH=/usr/local/intel-2015/composer_xe_2015.0.090/debugger/gdb/intel64/share/info/:/usr/local/intel-2015/composer_xe_2015.0.090/debugger/gdb/intel64_mic/share/info/
DISPLAY=localhost:10.0
INCLUDE=/usr/local/intel-2015/composer_xe_2015.0.090/mkl/include:/usr/local/intel-2013.1/composer_xe_2013_sp1.2.144/mkl/include
G_BROKEN_FILENAMES=1
module=() {  eval `/usr/Modules/$MODULE_VERSION/bin/modulecmd bash $*`
}
_=/bin/env

Kevin_D_Intel · ‎09-12-2014

Very puzzling. Everything is looking fine. It does not seem like there is any software version incompatibility. What is your host OS?

Let's try a couple of other things.

First, let's try a trivial offload example.F90 (see below) and isolate the execution to only Card 0.

Just compile with: ifort example.F90 (make certain to use the uppercase .F90 extension). Before you execute, set the environment variable, OFFLOAD_DEVICES=0

program main
!dir$ attributes offload : mic :: foo

!dir$ offload target (mic:0) optional
call foo()

end

!dir$ attributes offload : mic :: foo
subroutine foo()
use mic_lib
#ifdef __MIC__
   print "((A,I2))","On MIC: ",OFFLOAD_GET_DEVICE_NUMBER()
#else
   print "(A)","On host"
#endif
end

Clay__R_Torsten · ‎09-12-2014

This system is running CentOS 6.5. It is a cluster system- I don't think it is a failure of an individual card, as I get the same problem on all nodes.

I tried your example but it also does not run:

[Offload] [HOST]          [State]           Initialize logical card 0 = physical card 0
offload error: cannot execute function on the device 0 (error code 13)

Here is a simple example I did get to work:

program main
  implicit none
  ! will not run if following line is uncommented!
!!  !dir$ attributes offload:mic :: a,b,c,d
  real(kind=8),allocatable :: a(:),b(:),c(:),d(:)
  integer,parameter :: N=100
  integer :: i

  allocate(a(N))
  allocate(b(N))
  allocate(c(N))
  allocate(d(N))

  do i=1,N
     a(i)=i
     b(i)=i
  enddo

  !dir$ offload begin target(mic) &
  in(a), out(c,d), inout(b)
  !$omp parallel do
  do i=1,N
     c(i) = a(i) + b(i)
     d(i) = a(i) - b(i)
     b(i) = -b(i)
  end do
  !dir$ end offload

end program main

I compiled it with ifort test2.f90 (not .F90 here). It appers to run fine:

[Offload] [HOST]          [State]           Initialize logical card 0 = physical card 0
[Offload] [MIC 0] [File]                    test2.f90
[Offload] [MIC 0] [Line]                    19
[Offload] [MIC 0] [Tag]                     Tag 0
[Offload] [HOST]  [Tag 0] [State]           Start target
[Offload] [HOST]  [Tag 0] [State]           Setup target entry: __offload_entry_test2_f90_19MAIN__ifort236684784zzNxbD
[Offload] [HOST]  [Tag 0] [State]           Host->target pointer data 0
[Offload] [HOST]  [Tag 0] [Signal]          signal : none
[Offload] [HOST]  [Tag 0] [Signal]          waits  : none
[Offload] [HOST]  [Tag 0] [State]           Gather copyin data: base=0xa5faf0 length=800
[Offload] [HOST]  [Tag 0] [State]           Create target buffer: size=800 offset=0
[Offload] [HOST]  [Tag 0] [State]           Gather copyin data: base=0xa60c40 length=800
[Offload] [HOST]  [Tag 0] [State]           Create target buffer: size=3936 offset=3136
[Offload] [HOST]  [Tag 0] [State]           Gather copyin data: base=0xa60910 length=800
[Offload] [HOST]  [Tag 0] [State]           Create target buffer: size=800 offset=0
[Offload] [HOST]  [Tag 0] [State]           Gather copyin data: base=0xa605e0 length=800
[Offload] [HOST]  [Tag 0] [State]           Create target buffer: size=2304 offset=1504
[Offload] [HOST]  [Tag 0] [State]           Host->target pointer data 1600
[Offload] [HOST]  [Tag 0] [State]           Host->target copyin data 256 
[Offload] [HOST]  [Tag 0] [State]           Execute task on target
[Offload] [HOST]  [Tag 0] [State]           Target->host pointer data 2400
[Offload] [MIC 0] [Tag 0] [State]           Start target entry: __offload_entry_test2_f90_19MAIN__ifort236684784zzNxbD
[Offload] [MIC 0] [Tag 0] [Var]             main_$A_V$37  IN
[Offload] [MIC 0] [Tag 0] [Var]             main_$A_V$37  IN
[Offload] [MIC 0] [Tag 0] [Var]             main_$D_V$16  OUT
[Offload] [MIC 0] [Tag 0] [Var]             main_$D_V$16  OUT
[Offload] [MIC 0] [Tag 0] [Var]             main_$C_V$21  OUT
[Offload] [MIC 0] [Tag 0] [Var]             main_$C_V$21  OUT
[Offload] [MIC 0] [Tag 0] [Var]             main_$B_V$2c  INOUT
[Offload] [MIC 0] [Tag 0] [Var]             main_$B_V$2c  INOUT
[Offload] [MIC 0] [Tag 0] [State]           Target->host copyout data   0
[Offload] [HOST]  [Tag 0] [CPU Time]        0.184816(seconds)
[Offload] [MIC 0] [Tag 0] [CPU->MIC Data]   1856 (bytes)
[Offload] [MIC 0] [Tag 0] [MIC Time]        0.000559(seconds)
[Offload] [MIC 0] [Tag 0] [MIC->CPU Data]   2400 (bytes)

However, if I uncomment the 4th line it does not run:

[Offload] [HOST]          [State]           Initialize logical card 0 = physical card 0
offload error: cannot execute function on the device 0 (error code 13)

The problem appears to be some kind of memory issues. If I login to the mic, this is what ulimit -a gives. Should any of these be changed?

core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 61401
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 61401
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

Kevin_D_Intel · ‎09-12-2014

Its strange my smaller program did not run. Maybe the inclusion of mic_lib and the call to the OFFLOAD_GET_DEVICE_NUMBER() bloated the image making it larger than the one you did get to run.

Nothing in those card settings are much different from what I see. I'm beginning to wonder whether the coi_daemon process (on the card) that runs the offload code on behalf of the user has been started on with some shell or other execution limitation that's impacting the user offload executable process. Maybe others with more knowledge in that area are reading and can chime in on other things to check.

Are you running on the same node each time?

Do you have any ability to reboot only the node that you are using?

Kevin_D_Intel · ‎09-29-2014

Let us know if you resolved this issue. Your experience and the resolution may prove beneficial to someone else in the future. Thank you.

Clay__R_Torsten · ‎09-30-2014

No, I have not been able to resolve this issue. I do not have admin access to reboot the system, but always got the same results after the system had been restarted. Every node I tried gave the same results.

At the moment I do not have access to this system- when I can login again I will try some more things.

Clay__R_Torsten · ‎10-09-2014

It appears that my problems have been solved. I have access to the system again and this time the test codes appear to fine now. I am not sure what was changed- perhaps there was a mismatch somewhere in the versions of installed software. I will post back here if I have further problems or figure out what the problem was.

Kevin_D_Intel · ‎10-09-2014

Glad to hear it was resolved.

Cannot get sample code to run