Intel® C++ Compiler
Support and discussions for creating C++ code that runs on platforms based on Intel® processors.

Very large structs GCC ok, ICC seg fault, simple example.

jkwi
Beginner
2,551 Views

We have a very large structure ~10GB. GCC does not seem to have a problem with it but ICC is seg faulting. I've boiled it down to simple assignment statements. Any ideas?  Files attached. If we don't allocate but declare the structure it works fine. Any problems with Intel allocating large amounts of memory?  We've tried moving things around in the structure definition but it continues to seg fault.

icc -mcmodel=medium -shared-intel -g  test.c -o test
gcc -mcmodel=medium -g test.c -o test

icc: Version 12.1.5.339 Build 20120612
gcc: gcc (GCC) 4.4.6 20120305 (Red Hat 4.4.6-4)

GCC output:
Allocate the grid struct
sizeof of grids 9.393871GB
Address of grids->G2.rain.count 0x7f5d05143190
Address of grids->G2.d0.count 0x7f5d61d35190
Address of grids->G2.Nt.count 0x7f5d96d2d190
Address of grids->G2.dBZm.count 0x7f5f079a0190
location: 0
G2.dBZ.stdev[ii][jj][kk][ll][mm] = 2.001000
Address of grids->G2.dBZm.count 0x7f5f079a0190
assigned G2.dBZm.count
assigend G2.dBZm.mean
assigned G2.dBZm.stdev
Address of G2.d0.count 0x7f5d61d35190
Going to assign d0.count
assigned G2.d0.count
Going to assign d0.mean
assigned G2.d0.mean
Going to assign d0.stdev
assigned G2.d0.stdev
Successful write

Intel ICC:
Allocate the grid struct
sizeof of grids 9.393871GB
Address of grids->G2.rain.count 0x7fc9e070f190
Address of grids->G2.d0.count 0x7fca3d301190
Address of grids->G2.Nt.count 0x7fca722f9190
Address of grids->G2.dBZm.count 0x7fcbe2f6c190
location: 0
G2.dBZ.stdev[ii][jj][kk][ll][mm] = 2.001000
Address of grids->G2.dBZm.count 0x7fcbe2f6c190
assigned G2.dBZm.count
assigend G2.dBZm.mean
assigned G2.dBZm.stdev
Address of G2.d0.count 0x7fca3d301190
Going to assign d0.count
Segmentation fault




0 Kudos
48 Replies
bernaske
New Contributor I
236 Views

Hi Sergey, i have compile this sample under openSUSE 12.2 Linux 64 Bit with the ParallelStudio XE 2013 ( Compiler icc/icpc 13.1 )

no problems, no compile errors, runs normal the executable result output 77

#include "stdio.h"

typedef struct tagDataSet
{
// 2^26 = 67108864 - Default limit for 32-bit Intel C++ compiler
// 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
__int8 iData[2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2];
} DataSet;

DataSet ds = { 0x0 };

int main( void )
{
ds.iData[0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0] = 77;
printf( "%d\n", ds.iData[0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0] );
return ( int )0;

best regards

Franz

Mark_S_Intel1
Employee
236 Views

The original problem reported has been resolved. We will let you know when a compiler update containing the fix is available. 

Thanks,
--mark

SergeyKostrov
Valued Contributor II
236 Views
It is a very good news and thank you for the update.
Mark_S_Intel1
Employee
236 Views

The C++ compiler issue was resolved. The latest composer XE 2013 update 5 contains the fix. The Fortran compiler issue has been fixed in the 14.0 compiler schedule to release in September (schedule subject to change).

-mark

Michael_J_Intel2
Employee
236 Views

Mark,

is it possible this was "partially fixed"?  I got to this thread via google but in my case I have structs of structs of structs.  (That add up to ~16GB.)

I'll work on a reproducer but I basically have a top-level struct with 4 instances of a second-level struct (which take ~4GB each).  I can cleanly access fields in the first instance.  All three of the other instances seg fault.

 

I am using ICC 14.0.1 20131008 on Linux (Ubuntu 13.10)

Michael_J_Intel2
Employee
236 Views

The following code fails on ICC 14.0.1 20131008 (Linux) depending on if the #include of stdlib.h is commented out or not.  (It seg faults if the include is commented out.)

Does that make any sense???  I only figured that out because GCC complained of "incompatible implicit declaration of built-in function 'malloc'"

Thanks.

 

#include <stdio.h>
//#include <stdlib.h>

typedef struct _bar
{
    char doug[4L*1024L*1024L*1024L];
}bar;

typedef struct _foo
{
    bar bob[4];
}foo;

int main()
{
    foo* foo0=(foo*)malloc(sizeof(foo));
    foo0->bob[0].doug[0]=1;
    foo0->bob[1].doug[0]=1;
    printf("Hello world\n");
}

Georg_Z_Intel
Employee
236 Views

Hello,

I've came across this thread and would like to have a quick look.

Matthias, could you please provide me the following information:

$ icpc -v     [lower case 'v']
...

$ icpc -V   [upper case 'V']
...

$ g++ -v    [lower case 'v']
...

...and the exact output of the failing compiler?

It's also important to know whether your system is 32 or 64 bit. I was running your case on a random 32 bit system with 15.0.4 (which I just had at hand; do not have 16.0 for IA32 currently) and I got a proper error:

$ icpc -mcmodel=large large.cpp icpc: command line warning #10148: option '-mcmodel=large' not supported
large.cpp(7): error: array is too large
  typedef float data[2];
                ^

large.cpp(8): error: array is too large
  typedef complex<float> data_complex[2];
                         ^

compilation aborted for large.cpp (code 2)

I need to understand your configuration before doing more analysis.

Thank you & best regards,

Georg Zitzlsberger

Reply