Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Beginner
224 Views

Very large structs GCC ok, ICC seg fault, simple example.

We have a very large structure ~10GB. GCC does not seem to have a problem with it but ICC is seg faulting. I've boiled it down to simple assignment statements. Any ideas?  Files attached. If we don't allocate but declare the structure it works fine. Any problems with Intel allocating large amounts of memory?  We've tried moving things around in the structure definition but it continues to seg fault.

icc -mcmodel=medium -shared-intel -g  test.c -o test
gcc -mcmodel=medium -g test.c -o test

icc: Version 12.1.5.339 Build 20120612
gcc: gcc (GCC) 4.4.6 20120305 (Red Hat 4.4.6-4)

GCC output:
Allocate the grid struct
sizeof of grids 9.393871GB
Address of grids->G2.rain.count 0x7f5d05143190
Address of grids->G2.d0.count 0x7f5d61d35190
Address of grids->G2.Nt.count 0x7f5d96d2d190
Address of grids->G2.dBZm.count 0x7f5f079a0190
location: 0
G2.dBZ.stdev[ii][jj][kk][ll][mm] = 2.001000
Address of grids->G2.dBZm.count 0x7f5f079a0190
assigned G2.dBZm.count
assigend G2.dBZm.mean
assigned G2.dBZm.stdev
Address of G2.d0.count 0x7f5d61d35190
Going to assign d0.count
assigned G2.d0.count
Going to assign d0.mean
assigned G2.d0.mean
Going to assign d0.stdev
assigned G2.d0.stdev
Successful write

Intel ICC:
Allocate the grid struct
sizeof of grids 9.393871GB
Address of grids->G2.rain.count 0x7fc9e070f190
Address of grids->G2.d0.count 0x7fca3d301190
Address of grids->G2.Nt.count 0x7fca722f9190
Address of grids->G2.dBZm.count 0x7fcbe2f6c190
location: 0
G2.dBZ.stdev[ii][jj][kk][ll][mm] = 2.001000
Address of grids->G2.dBZm.count 0x7fcbe2f6c190
assigned G2.dBZm.count
assigend G2.dBZm.mean
assigned G2.dBZm.stdev
Address of G2.d0.count 0x7fca3d301190
Going to assign d0.count
Segmentation fault




0 Kudos
48 Replies
Highlighted
Valued Contributor II
24 Views

Thank you, Franz. As I promised here are results of two tests with Intel C++ compiler XE 2013 on WIndows 7 Professional 64-bit: [ TEST 1 - Intel C++ compiler - 32-bit ] - No problems ..>icl.exe Test.cpp Intel(R) C++ Compiler XE for applications running on IA-32, Version 13.0.0.089 Build 20120731 Copyright (C) 1985-2012 Intel Corporation. All rights reserved. Test.cpp Microsoft (R) Incremental Linker Version 9.00.30729.01 Copyright (C) Microsoft Corporation. All rights reserved. -out:Test.exe Test.obj [ Test Output ] 77 [ TEST - 2 Intel C++ compiler - 64-bit ] - No problems ..>icl.exe Test.cpp Intel(R) C++ Intel(R) 64 Compiler XE for applications running on Intel(R) 64, Version 13.0.0.089 Build 20120731 Copyright (C) 1985-2012 Intel Corporation. All rights reserved. Test.cpp Microsoft (R) Incremental Linker Version 9.00.30729.01 Copyright (C) Microsoft Corporation. All rights reserved. -out:Test.exe Test.obj [ Test Output ] 77
0 Kudos
Highlighted
Valued Contributor II
24 Views

Mark, I'm not concerned about Intel(R) C++ Compiler XE ( 32-bit ) Version 12.1.3.300 Build 20120130 because it was just additional verification. Please keep everybody informed regarding status of the original problem with ~9GB structure. Best regards, Sergey
0 Kudos
Highlighted
New Contributor I
24 Views

Hi Sergey, i have compile this sample under openSUSE 12.2 Linux 64 Bit with the ParallelStudio XE 2013 ( Compiler icc/icpc 13.1 )

no problems, no compile errors, runs normal the executable result output 77

#include "stdio.h"

typedef struct tagDataSet
{
// 2^26 = 67108864 - Default limit for 32-bit Intel C++ compiler
// 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
__int8 iData[2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2][2];
} DataSet;

DataSet ds = { 0x0 };

int main( void )
{
ds.iData[0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0] = 77;
printf( "%d\n", ds.iData[0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0][0] );
return ( int )0;

best regards

Franz

0 Kudos
Highlighted
Employee
24 Views

The original problem reported has been resolved. We will let you know when a compiler update containing the fix is available. 

Thanks,
--mark

0 Kudos
Highlighted
Valued Contributor II
24 Views

It is a very good news and thank you for the update.
0 Kudos
Highlighted
Employee
24 Views

The C++ compiler issue was resolved. The latest composer XE 2013 update 5 contains the fix. The Fortran compiler issue has been fixed in the 14.0 compiler schedule to release in September (schedule subject to change).

-mark

0 Kudos
Highlighted
24 Views

Mark,

is it possible this was "partially fixed"?  I got to this thread via google but in my case I have structs of structs of structs.  (That add up to ~16GB.)

I'll work on a reproducer but I basically have a top-level struct with 4 instances of a second-level struct (which take ~4GB each).  I can cleanly access fields in the first instance.  All three of the other instances seg fault.

 

I am using ICC 14.0.1 20131008 on Linux (Ubuntu 13.10)

0 Kudos
Highlighted
24 Views

The following code fails on ICC 14.0.1 20131008 (Linux) depending on if the #include of stdlib.h is commented out or not.  (It seg faults if the include is commented out.)

Does that make any sense???  I only figured that out because GCC complained of "incompatible implicit declaration of built-in function 'malloc'"

Thanks.

 

#include <stdio.h>
//#include <stdlib.h>

typedef struct _bar
{
    char doug[4L*1024L*1024L*1024L];
}bar;

typedef struct _foo
{
    bar bob[4];
}foo;

int main()
{
    foo* foo0=(foo*)malloc(sizeof(foo));
    foo0->bob[0].doug[0]=1;
    foo0->bob[1].doug[0]=1;
    printf("Hello world\n");
}

0 Kudos
Highlighted
Employee
24 Views

Hello,

I've came across this thread and would like to have a quick look.

Matthias, could you please provide me the following information:

$ icpc -v     [lower case 'v']
...

$ icpc -V   [upper case 'V']
...

$ g++ -v    [lower case 'v']
...

...and the exact output of the failing compiler?

It's also important to know whether your system is 32 or 64 bit. I was running your case on a random 32 bit system with 15.0.4 (which I just had at hand; do not have 16.0 for IA32 currently) and I got a proper error:

$ icpc -mcmodel=large large.cpp icpc: command line warning #10148: option '-mcmodel=large' not supported
large.cpp(7): error: array is too large
  typedef float data[2];
                ^

large.cpp(8): error: array is too large
  typedef complex<float> data_complex[2];
                         ^

compilation aborted for large.cpp (code 2)

I need to understand your configuration before doing more analysis.

Thank you & best regards,

Georg Zitzlsberger

0 Kudos