Very large structs GCC ok, ICC seg fault, simple example.

[jkwi]

We have a very large structure ~10GB. GCC does not seem to have a problem with it but ICC is seg faulting. I've boiled it down to simple assignment statements. Any ideas?  Files attached. If we don't allocate but declare the structure it works fine. Any problems with Intel allocating large amounts of memory?  We've tried moving things around in the structure definition but it continues to seg fault.

icc -mcmodel=medium -shared-intel -g  test.c -o test
gcc -mcmodel=medium -g test.c -o test

icc: Version 12.1.5.339 Build 20120612
gcc: gcc (GCC) 4.4.6 20120305 (Red Hat 4.4.6-4)

GCC output:
Allocate the grid struct
sizeof of grids 9.393871GB
Address of grids->G2.rain.count 0x7f5d05143190
Address of grids->G2.d0.count 0x7f5d61d35190
Address of grids->G2.Nt.count 0x7f5d96d2d190
Address of grids->G2.dBZm.count 0x7f5f079a0190
location: 0
G2.dBZ.stdev[ii][jj][kk][ll][mm] = 2.001000
Address of grids->G2.dBZm.count 0x7f5f079a0190
assigned G2.dBZm.count
assigend G2.dBZm.mean
assigned G2.dBZm.stdev
Address of G2.d0.count 0x7f5d61d35190
Going to assign d0.count
assigned G2.d0.count
Going to assign d0.mean
assigned G2.d0.mean
Going to assign d0.stdev
assigned G2.d0.stdev
Successful write

Intel ICC:
Allocate the grid struct
sizeof of grids 9.393871GB
Address of grids->G2.rain.count 0x7fc9e070f190
Address of grids->G2.d0.count 0x7fca3d301190
Address of grids->G2.Nt.count 0x7fca722f9190
Address of grids->G2.dBZm.count 0x7fcbe2f6c190
location: 0
G2.dBZ.stdev[ii][jj][kk][ll][mm] = 2.001000
Address of grids->G2.dBZm.count 0x7fcbe2f6c190
assigned G2.dBZm.count
assigend G2.dBZm.mean
assigned G2.dBZm.stdev
Address of G2.d0.count 0x7fca3d301190
Going to assign d0.count
Segmentation fault

[bernaske]

Hi Sergey, i have compile this sample under openSUSE 12.2 Linux 64 Bit with the ParallelStudio XE 2013 ( Compiler icc/icpc 13.1 )

no problems, no compile errors, runs normal the executable result output 77

#include "stdio.h"

typedef struct tagDataSet
{
// 2^26 = 67108864 - Default limit for 32-bit Intel C++ compiler
// 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
__int8 iData[2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2];
} DataSet;

DataSet ds = { 0x0 };

int main( void )
{
ds.iData[0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0] = 77;
printf( "%d\n", ds.iData[0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0] );
return ( int )0;

best regards

Franz

[Mark_S_Intel1]

The original problem reported has been resolved. We will let you know when a compiler update containing the fix is available. 

Thanks,
--mark

[SergeyKostrov]

It is a very good news and thank you for the update.

[Mark_S_Intel1]

The C++ compiler issue was resolved. The latest composer XE 2013 update 5 contains the fix. The Fortran compiler issue has been fixed in the 14.0 compiler schedule to release in September (schedule subject to change).

-mark

[Michael_J_Intel2]

Mark,

is it possible this was "partially fixed"?  I got to this thread via google but in my case I have structs of structs of structs.  (That add up to ~16GB.)

I'll work on a reproducer but I basically have a top-level struct with 4 instances of a second-level struct (which take ~4GB each).  I can cleanly access fields in the first instance.  All three of the other instances seg fault.

I am using ICC 14.0.1 20131008 on Linux (Ubuntu 13.10)

[Michael_J_Intel2]

The following code fails on ICC 14.0.1 20131008 (Linux) depending on if the #include of stdlib.h is commented out or not.  (It seg faults if the include is commented out.)

Does that make any sense???  I only figured that out because GCC complained of "incompatible implicit declaration of built-in function 'malloc'"

Thanks.

#include <stdio.h>
//#include <stdlib.h>

typedef struct _bar
{
    char doug[4L*1024L*1024L*1024L];
}bar;

typedef struct _foo
{
    bar bob[4];
}foo;

int main()
{
    foo* foo0=(foo*)malloc(sizeof(foo));
    foo0->bob[0].doug[0]=1;
    foo0->bob[1].doug[0]=1;
    printf("Hello world\n");
}

[Georg_Z_Intel]

Hello,

I've came across this thread and would like to have a quick look.

Matthias, could you please provide me the following information:

$ icpc -v     [lower case 'v']
...

$ icpc -V   [upper case 'V']
...

$ g++ -v    [lower case 'v']
...

...and the exact output of the failing compiler?

It's also important to know whether your system is 32 or 64 bit. I was running your case on a random 32 bit system with 15.0.4 (which I just had at hand; do not have 16.0 for IA32 currently) and I got a proper error:

$ icpc -mcmodel=large large.cpp icpc: command line warning #10148: option '-mcmodel=large' not supported
large.cpp(7): error: array is too large
  typedef float data[2];
                ^

large.cpp(8): error: array is too large
  typedef complex<float> data_complex[2];
                         ^

compilation aborted for large.cpp (code 2)

I need to understand your configuration before doing more analysis.

Thank you & best regards,

Georg Zitzlsberger