SEGV with optimization

[dkokron]

All,

I have an application that fails at runtime with a SIGSEGV when a particular source file is compiled with optimization but runs otherwise. This particular source file is a tall-pole performance wise so I would prefer to use full optimization. I list below some results from experimenting with compiler options.

I have put together a test case, but it's rather large at about 13GB uncompressed and needs about 3.3 GB of main memory/process on 224 cores to run. I'll hold off on submitting it to this forum until I get the go-ahead.

Any suggestions?

Dan

ifort -V
Intel Fortran Intel 64 Compiler Professional for applications running on Intel 64, Version 11.0 Build 20090318 Package ID: l_cprof_p_11.0.083
Copyright (C) 1985-2009 Intel Corporation. All rights reserved.

uname -a
Linux pfe1 2.6.16.60-0.37.02schamp-nasa #1 SMP Wed May 20 20:50:30 UTC 2009 x86_64 x86_64 x86_64 GNU/Linux

module list
comp-intel/11.0.083 3) mpi-intel/3.2.011 4) math-intel/mkl_64_10.0.011

Also fails in similar manner if lanczos.f90 is compiled with 11.1.046 while the other source is compiled with 11.0.083. Other compile time and runtime issues prevent use of 11.1.046 on the whole application.

known working compiler flags
-g -O0 -check bounds
-O1
-O2 -no-vec
-O3 -no-vec
-O3 -check bounds (does CB disable vectorization??)

[Ron_Green]

-check bounds will probably disable vectorization -vec-report could tell for sure.

Don't post the code - I don't have a system anywhere near that size to test this. Does this reproduce with smaller problem sizes?

I might suggest !DEC NOVECTOR directives around specific vectorized loops (from -vec-report output). And you might try to find the specific fault point by compiling and linking with -O2 or -O3 and -g -traceback. If you can find 1 loop whose vectorization causes the seg fault then perhaps !DEC NOVECTOR on that loop.

And have you tried another MPI?

ron

[Kevin_D_Intel]

While you investigate the items Ron suggests, will you upload only lanczos.f90 for now.

[dkokron]

Quoting - Kevin Davis (Intel)

While you investigate the items Ron suggests, will you upload only lanczos.f90 for now.

I put together a tarball and sent it to Ron. Please send me your email and I'll forward the tarball to you.

I used "-g -O2 -traceback" and it did implicate a loop at line 520 in lanczos.f90. The !DEC$ NOVECTOR directive didn't change the outcome. I'll try spraying some more directives around and see what happens.

Dan

[Kevin_D_Intel]

Quoting - dkokron

I used "-g -O2 -traceback" and it did implicate a loop at line 520 in lanczos.f90. The !DEC$ NOVECTOR directive didn't change the outcome. I'll try spraying some more directives around and see what happens.

Dan

Thanks Dan, Ron already forwarded it. Will post again when we know more.