problem to build 32-bit linux executable with ifort/openmp on 64-bit linux

[rookie2010]

I got a problem with ifort using openmp.

When I use ifort static link to create an executable with
ifort 32-bit environment on a 64-bit linux machine, I CANNOT
run that executable on a 32-bit linux. One thing to mention
is that my code use openmp and -openmp is included in the
link command. If I do not invoke openmp (no -openmp in compile
and link command), static link executable works normal
(32-bit executable built in ifort 32-bit environment on a 64-bit
linux runs on 32-bit linux).

The way I built 32-bit version of the executable on 64-bit linux is:
cd /opt/intel/Compiler/11.1/056/bin
source ifortvars.sh ia32
cd /home/user_name/code_dir
ifort -c -traceback code.f
ifort -c -traceback -openmp code_openmp_portion.f
ifort -static -openmp code.o code_openmp_portion.o -o code_static_openmp.out

When code_static_openmp.out is taken to a 32-bit linux, execution
results in Segmentation fault (likely in openmp portion of the code).
SURPROSINGLY, code_static_openmp.out also got Segmentation fault when
it is executed on the machine building it with and without runninng
'source ifortvars.sh ia32'.

Additional info:
a. Without -openmp, it does not link.
b. 'ldd code_static_openmp.out' shows "not a dynamic executable"
c. Prior to 'source ifortvars.sh ia32', 'env | grep LD_' shows
LD_LIBRARY_PATH=/opt/sge/lib/lx24-amd64:/usr/X11R6/lib64:/usr/X11R6/lib
LD_ASSUME_KERNEL=2.4.1

After 'source ifortvars.sh ia32', 'env | grep LD_' shows
LD_LIBRARY_PATH=/opt/intel/Compiler/11.1/056/lib/ia32:/opt/intel/Compil
er/11.1/056/ipp/ia32/sharedlib:/opt/intel/Compiler/11.1/056/mkl/lib/32:
/opt/intel/Compiler/11.1/056/tbb/ia32/cc3.3.3_libc2.3.3_kernel2.6.5/lib
:/opt/sge/lib/lx24-amd64:/usr/X11R6/lib64:/usr/X11R6/lib
LD_ASSUME_KERNEL=2.4.1
DYLD_LIBRARY_PATH=/opt/intel/Compiler/11.1/056/tbb/ia32/cc3.3.3_libc2.3
.3_kernel2.6.5/lib

However, if openmp IS NOT INVOKED at all, I can take the executable built on
64-bit linux WITH STATIC LINK to 32-bit linux. Here are the steps:
cd /opt/intel/Compiler/11.1/056/bin
source ifortvars.sh ia32
cd /home/user_name/code_dir
ifort -c -traceback code.f
ifort -c -traceback code_openmp_portion.f
ifort -static code.o code_openmp_portion.o -o code_static.out

Note that there is no -openmp during compiling and linking. code_static.out
runs fine on 32-bit linux. 'ldd code_static.out' shows "not a dynamic executable".

Another test is to use openmp with static link only with ifort library. 32-bit
executable built on 64-bit linux also works on 32-bit linux. The steps are:
cd /opt/intel/Compiler/11.1/056/bin
source ifortvars.sh ia32
cd /home/user_name/code_dir
ifort -c -traceback code.f
ifort -c -traceback -openmp code_openmp_portion.f
ifort -i-static -openmp code.o code_openmp_portion.o -o code_i-static_openmp.out

Note that -openmp is used during compiling and linking and -i-static links only
ifort libraries from my understanding. code_i-static_openmp.out runs fine on 32-bit linux.
'ldd code_i-static_openmp.out' gives
linux-gate.so.1 => (0xffffe000)
libm.so.6 => /lib/i686/libm.so.6 (0x55588000)
libpthread.so.0 => /lib/i686/libpthread.so.0 (0x555aa000)
libc.so.6 => /lib/i686/libc.so.6 (0x555fb000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x5570d000)
libdl.so.2 => /lib/libdl.so.2 (0x55719000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x55555000)


Is it possible that ifort grabs some kind of 64-bit library file during static link
due to openmp need and mixes it with 32-bit obj codes and 32-bit libraries? I really
want to use static link so that any portability issue can be avoided.

Youropinion is very much appreciated.

Reggie

[timintel]

I didn't see any mention of your investigations on stack setting and stack overflow, which might have been the first things you looked at.
Fully static link isn't generally considered good for portability. -i-static links static libraries from the ifort installation, with the exception of libiomp5. ifort 11.1 had the option -openmp-link static, in case you were interested. Otherwise, you would make sure that LD_LIBRARY_PATH or ldconfig point to libiomp5.so 32-bit.

[Ron_Green]

have you eliminated stack space as an issue, as mentioned here:

http://software.intel.com/en-us/articles/determining-root-cause-of-sigsegv-or-sigbus-errors/

No, you cannot link 64bit objects with 32, linker won't allow it. use command 'file code_static_openmp.out' to verify it's 32bit.

it's probably just a matter of 'ulimit -s unlimited' prior to running the code.

You could try 'size code_static_openmp.out' to see what your static data requirements are, making sure you're fitting into 32bit 2GB limits.

ron

[rookie2010]

Ron,

Thank you for the info.

I've tried 'ulimit -s unlimited' as you suggested but still got Segmentatin fault error.

'file code_static_openmp.out' gives
code_static_openmp.out: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linu
x 2.2.5, statically linked, not stripped

'size code_static_openmp.out'gives
text data bss dec hex filename
10626773 724588 8117112 19468473 12910b9 code_static_openmp.out


I'll study size command tomorrow.

Best regards,

Reggie

[Ron_Green]

Reggie,

Throw in the -g option along with -traceback. This will give you an idea of where the code is dying.

ron

[rookie2010]

Ron,

-g did not help. There is no additional info display with Segmentation fault.
Nor is there a dumped core.

The way I built with -g is:
cd /opt/intel/Compiler/11.1/056/bin
source ifortvars.sh ia32
cd /home/user_name/code_dir
ifort -c -g -traceback code.f
ifort -c -g -traceback -openmp code_openmp_portion.f
ifort -static -openmp code.o code_openmp_portion.o -o code_static_openmp.g.out
ifort -g -static -openmp code.o code_openmp_portion.o -o code_static_openmp.g2.out

code_static_openmp.g.out and code_static_openmp.g2.out are identical.

Thanks.

Reggie

[rookie2010]

Tim,

As I mentioned in one post, I've tried 'ulimit -s unlimited' but still got Segmentation fault.

My company once provided an executable to a user but he could not run it. Once static link
was introduced, the problem was resolved. Since there are so many versions of linux out there,
using static link may not be a bad idea. I would rather fix the static link problem now than
being forced to provide static link executable and then realize static link with openmp does not work.

I believe LD_LIBRARY_PATH has been handled correctly by 'source ifortvars.sh ia32' command
since LD_LIBRARY_PATH shows intel ia32 libraries after the command (please see my 1st posting).

I've tried-openmp-link-static option with the following two link commands.
(-openmp is needed as I described in my first posting.)

ifort -static -openmp -openmp-link-static

ifort -openmp -openmp-link-static

The screen showed
ifort: command line warning #10122: overriding '-openmp-link-static' with '-ocode_static_openmp.out'

and the executable is identical to previous executable without -openmp-link-static with diff command.

Thanks.

Reggie