Huge Memory Difference between 32 bit (5.6 MB) vs 64 bit (4 GB) Build

[chauvjo]

I am trying to understand why there is such a huge difference in my program's memory requirements between a 32 bit build vs a 64 bit build. I think something else is going on with the 64 bit compile since the memory requirement is the exact amount of RAM available on my system (4 GB). Here is a comparison between the two builds using the size command:

32 bit Build (using -m32 compiler option)

__TEXT __DATA __OBJC others dec hex

4096000 880640 0 942080 5918720 5a5000

64 bit Build

__TEXT __DATA __OBJC others dec hex

4423680 802816 0 4296024064 4301250560 1005fe000

Any ideas?

System Version: Mac OS X 10.6.2 (10C540)

Kernel Version: Darwin 10.2.0

Processor Name: Intel Core 2 Duo

Processor Speed: 3.06 GHz

Number Of Processors: 1

Total Number Of Cores: 2

L2 Cache: 6 MB

Memory: 4 GB

Bus Speed: 1.07 GHz

[Ron_Green · Accepted solution]

Quoting - chauvjo

I am trying to understand why there is such a huge difference in my program's memory requirements between a 32 bit build vs a 64 bit build. I think something else is going on with the 64 bit compile since the memory requirement is the exact amount of RAM available on my system (4 GB). Here is a comparison between the two builds using the size command:

32 bit Build (using -m32 compiler option)

__TEXT __DATA __OBJC others dec hex

4096000 880640 0 942080 5918720 5a5000

64 bit Build

__TEXT __DATA __OBJC others dec hex

4423680 802816 0 4296024064 4301250560 1005fe000

Any ideas?

System Version: Mac OS X 10.6.2 (10C540)

Kernel Version: Darwin 10.2.0

Processor Name: Intel Core 2 Duo

Processor Speed: 3.06 GHz

Number Of Processors: 1

Total Number Of Cores: 2

L2 Cache: 6 MB

Memory: 4 GB

Bus Speed: 1.07 GHz

Your TEXT and DATA are essentially the same, and that is all that is under the control of you as a programmer. I'm not sure what 'OTHERS' is, but I would guess just VM map space which would not matter.

Have you looked at the main program and manually added up the space needed by the large arrays declared in the main program?

ron

[chauvjo]

Quoting - Ronald W. Green (Intel)

Your TEXT and DATA are essentially the same, and that is all that is under the control of you as a programmer. I'm not sure what 'OTHERS' is, but I would guess just VM map space which would not matter.

Have you looked at the main program and manually added up the space needed by the large arrays declared in the main program?

ron

I am not sure what OTHERS is but it appears not to be a factor. I made some changes and recompiled and now the program runs both as 32 bit and 64 bit without errors but still crashes when I use the debugger:

iidb(55331) malloc: *** error for object 0x1010f4a00000000: pointer being freed was not allocated

I have noticed that a program can run without any errors but crashes if run within the debugger. This applies to very simple programs as well as complex.

[Ron_Green]

Quoting - chauvjo

I am not sure what OTHERS is but it appears not to be a factor. I made some changes and recompiled and now the program runs both as 32 bit and 64 bit without errors but still crashes when I use the debugger:

iidb(55331) malloc: *** error for object 0x1010f4a00000000: pointer being freed was not allocated

I have noticed that a program can run without any errors but crashes if run within the debugger. This applies to very simple programs as well as complex.

You'll want to use 'idb' and not 'iidb'. idb sets up various env vars needed before invoking iidb.

If this doesn't work we'll get someone from the debugger team to look into it.

[jimdempseyatthecove]

>>iidb(55331) malloc: *** error for object 0x1010f4a00000000: pointer being freed was not allocated

If this were to be a random programmer error relating tomalloc/free the odds of having 00000000 (one DWORD of 0's) in the lsb's of the pointer would be very small. This suspiciously looks like you passed a 64-bit pointer through a 32-bit calculation (including returning a 32-bit result into a 64-bit pointer).

Jim Dempsey

[chauvjo]

Quoting - jimdempseyatthecove

>>iidb(55331) malloc: *** error for object 0x1010f4a00000000: pointer being freed was not allocated

If this were to be a random programmer error relating tomalloc/free the odds of having 00000000 (one DWORD of 0's) in the lsb's of the pointer would be very small. This suspiciously looks like you passed a 64-bit pointer through a 32-bit calculation (including returning a 32-bit result into a 64-bit pointer).

Jim Dempsey

Here is the full debug session. I used idb as the debugger. This is a very old legacy code so what would I look for if a 64-pointer is passed through a 32-bit calculation and why would it run in normal mode and crash in debug mode?

[plain]idb /Users/chauvjo/Desktop/Tools/src/a.out
Intel Debugger for applications running on Intel 64, Version 11.1, Build [1.2097.2.319]
------------------ 
object file name: /Users/chauvjo/Desktop/Tools/src/a.out 
Reading symbols from /Users/chauvjo/Desktop/Tools/src/a.out...done.
(idb) break main
Breakpoint 1 at 0x1000014b3: file /Users/chauvjo/Desktop/Tools/src/MAIN.f, line 434.
(idb) run
Starting program: /Users/chauvjo/Desktop/Tools/src/a.out
iidb(4924) malloc: *** error for object 0x1010f4a00000000: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug

Intel Debugger for applications running on Intel 64, Version 11.1 caught signal "Abort trap" (6).
This is an unexpected condition and may indicate the presence of a defect.
If you wish to report this, please include the stack trace that follows.

terminate called after throwing an instance of 'FatalError'
Abort trap
[/plain]

[jimdempseyatthecove]

>>iidb(4924) malloc: *** error for object 0x1010f4a00000000: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug<<

Set a break point at malloc_error_break as suggested, re-run to get error and reach break point.

Walk back the stack looking at what is being returned. Once you know what the object was you can add some diagnostic code to aid in debugging. Common causes are:

a) returning an object more than once. This can be caused by not testing to see if the object pointer has been allocated. Generally a NULL is used for un-allocated pointer. If your code has this test then you might not be nulling out the pointer properly.Due to the00000000 in 0x1010f4a00000000 it suspiciously looks like you are nulling out only 32-bits of the 64-bit pointer.

b) Some error in your code causes this pointer to be modified. Usualy this will cause your program to error out, but in some cases the error will remain hidden.

Notes/Hints

Prior to any edits, no matter how trivial, make a copy of the code that you will be inserting your diagnostic code for debugging this problem. In the case of b) above, any edits might hide the problem. If you should add diagnostic code to try to trap the problem, and then you find the problem goes away, you must realize that the problem is NOT fixed. It is hiding elsewhere. Restructure your diagnostic code such that it does not hide the error. You must identify the cause and fix it while you have the opportunity tocorrect this error. Think of "iidb(4924) malloc: *** ..." as being good news.

Situation a) can occure when you declare a pointer to an array but do not nullify it in the declaration. Then run your code in your subroutine and taking a path that bypasses the allocation to the pointer thentake the exit path that cleans up allocations andwhich notices a non-null pointer (associated) and attempts to return it.

There are many causes for this type of problem. It will be your job to improvise a method to determine the cause of the problem.

Jim Dempsey

[Ron_Green]

and a few other points on the Mac compiler:

you can use -m32 and -m64 to control if you create 32bit or 64bit objects. HOWEVER, I really recommend EXPLICITLY sourcing the correct compiler with:

/opt/intel/Compiler/11.1/080/bin/ifortvars.sh ia32
or
/opt/intel/Compiler/11.1/080/bin/ifortvars.sh intel64

You can/should also explore compiling with

-gen-interfaces -warn interfaces

to get automatic checking of argument matching for your external procedures. And for runtime checks, add

-fp-stack-check

this will catch stack misalignment that may occur if your call/return is not correct.

Good hunting!

ron

[chauvjo]

Quoting - Ronald W. Green (Intel)

and a few other points on the Mac compiler:

you can use -m32 and -m64 to control if you create 32bit or 64bit objects. HOWEVER, I really recommend EXPLICITLY sourcing the correct compiler with:

/opt/intel/Compiler/11.1/080/bin/ifortvars.sh ia32
or
/opt/intel/Compiler/11.1/080/bin/ifortvars.sh intel64

You can/should also explore compiling with

-gen-interfaces -warn interfaces

to get automatic checking of argument matching for your external procedures. And for runtime checks, add

-fp-stack-check

this will catch stack misalignment that may occur if your call/return is not correct.

Good hunting!

ron

Ron I followed your recommendations and located a few problems that I fixed. The program now compiles without any errors using your suggested compiler options but still crashes in the debugger.

Jim This program is vintage Fortran 66. The code does not contain any pointer or allocate statements. The program does have lots of COMMON and some EQUIVALENCE statements.

As a reminder, this code runs in non-debug mode without any noticeable problems given the few test cases I have run. I decided to test the Intel debugger using this program just by chance.

I googled this error message and found many Apple developers have encountered this same error message which would seem to point the finger at something other than my code not that it does not have problems. I am no compiler/debugger expert but am struggling to understand how my code could cause the debugger to crash before reaching the first executable statement. What portion of my code is getting executed before the debugger stops at the first line? Base on the console traceback shown below, it appears the error is coming from iidb.

Thanks again for all your help....by the way notice the traceback from the console below. Is there a reason why the Intel compiler cannot provide this level of detail?

Here is what happens when I tried to set a break point at malloc_error_break as suggested:

[plain]idb /Users/chauvjo/Desktop/Tools/src/a.out
Intel Debugger for applications running on Intel 64, Version 11.1, Build [1.2097.2.319]
------------------ 
object file name: /Users/chauvjo/Desktop/Tools/src/a.out 
Reading symbols from /Users/chauvjo/Desktop/Tools/src/a.out...done.
(idb) break malloc_error_break
No symbol "malloc_error_break" in current context.
No symbol "malloc_error_break" in current context.
malloc_error_break has no valid breakpoint address
Make breakpoint pending on future shared library load? (y or ) y
Breakpoint 1 (malloc_error_break) pending
(idb) run
Starting program: /Users/chauvjo/Desktop/Tools/src/a.out
iidb(26414) malloc: *** error for object 0x1010f4a00000000: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug

Intel Debugger for applications running on Intel 64, Version 11.1 caught signal "Abort trap" (6).
This is an unexpected condition and may indicate the presence of a defect.
If you wish to report this, please include the stack trace that follows.
terminate called after throwing an instance of 'FatalError'
[/plain]

The following traceback was found in the console window which seems much more helpful:

[plain]Process:         iidb [26414]
Path:            /opt/intel/Compiler/11.1/080/bin/iidb
Identifier:      iidb
Version:         ??? (???)
Code Type:       X86-64 (Native)
Parent Process:  ksh [26397]

Date/Time:       2010-01-05 14:57:23.305 -0800
OS Version:      Mac OS X 10.6.2 (10C540)
Report Version:  6

Exception Type:  EXC_CRASH (SIGABRT)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Crashed Thread:  0  Dispatch queue: com.apple.main-thread

Application Specific Information:
abort() called

Thread 0 Crashed:  Dispatch queue: com.apple.main-thread
0   libSystem.B.dylib             	0x00007fff81b50fe6 __kill + 10
1   libSystem.B.dylib             	0x00007fff81bf1e32 abort + 83
2   libstdc++.6.dylib             	0x00007fff841ce5d2 __tcf_0 + 0
3   libstdc++.6.dylib             	0x00007fff841ccae1 __cxxabiv1::__terminate(void (*)()) + 11
4   libstdc++.6.dylib             	0x00007fff841ccb16 __cxxabiv1::__unexpected(void (*)()) + 0
5   libstdc++.6.dylib             	0x00007fff841ccbfc __gxx_exception_cleanup(_Unwind_Reason_Code, _Unwind_Exception*) + 0
6   iidb                          	0x00000001001c2a1a sigcatch(int, int, sigcontext*) + 332
7   libSystem.B.dylib             	0x00007fff81b62eaa _sigtramp + 26
8   ???                           	0x00007fff5fbfe560 0 + 140734799799648
9   libSystem.B.dylib             	0x00007fff81b09155 free + 128
10  iidb                          	0x000000010001229c ControlDefUxMac::isThisMessageAnEventForIDB(int, bool, int, int, bool*, bool*, LightweightWorkerMac**, int*) + 284
11  iidb                          	0x000000010001280b ControlDefUxMac::waitPidViaPort(int, int*, int) + 185
12  iidb                          	0x0000000100012c45 ControlDefUxMac::waitpidCall(char const*, int*, int, int*, int) + 197
13  iidb                          	0x0000000100014194 ControlDefUxMac::waitForCloneToStop(int) + 276
14  iidb                          	0x0000000100014604 ControlDefUxMac::waitForProcessToStop() + 164
15  iidb                          	0x00000001000056ee ControlDef::waitForEvent(ENGINENotification*&, bool) + 74
16  iidb                          	0x00000001000058bc ControlDef::waitForUserEvent(ENGINENotification*&) + 42
17  iidb                          	0x00000001000385d0 ProcessDef::go(bool, bool&) + 196
18  iidb                          	0x0000000100021799 SessionBase::RunUntilStopped(bool) + 383
19  iidb                          	0x000000010017a89d CmdRun::do_it(CmdExecutionContext&, BaseForCmd::CmdResult&) + 787
20  iidb                          	0x00000001001436d6 BaseForCmd::execute(bool) + 8106
21  iidb                          	0x00000001001bed87 processCommandsWkr(int&, char*&) + 1811
22  iidb                          	0x00000001001bee49 ProcessCommands() + 51
23  iidb                          	0x00000001001bf022 mainCommand() + 14
24  iidb                          	0x00000001001c2776 idbMain(int, char const**, char const**) + 484
25  iidb                          	0x00000001000e331b main + 273
26  iidb                          	0x0000000100001204 _start + 234
27  iidb                          	0x0000000100001119 start + 33

Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x0000000000000000  rbx: 0x00007fff70453298  rcx: 0x00007fff5fbfe058  rdx: 0x0000000000000000
  rdi: 0x000000000000672e  rsi: 0x0000000000000006  rbp: 0x00007fff5fbfe070  rsp: 0x00007fff5fbfe058
   r8: 0x0000000000000001   r9: 0x00000001010341d0  r10: 0x00007fff81b4d026  r11: 0x0000000000000206
  r12: 0x0000000100589859  r13: 0x00000001043677b0  r14: 0x00000001001c28ce  r15: 0x01010f4a00000000
  rip: 0x00007fff81b50fe6  rfl: 0x0000000000000206  cr2: 0x00007fff7080b940

Binary Images:
       0x100000000 -        0x100694fc7 +iidb ??? (???) <18693CF8-6C34-491A-8818-1B5785641C10> /usr/bin/iidb
    0x7fff5fc00000 -     0x7fff5fc3bdef  dyld 132.1 (???)  /usr/lib/dyld
    0x7fff81b02000 -     0x7fff81cc0ff7  libSystem.B.dylib ??? (???) <526DD3E5-2A8B-4512-ED97-01B832369959> /usr/lib/libSystem.B.dylib
    0x7fff84182000 -     0x7fff841fffef  libstdc++.6.dylib ??? (???) <35ECA411-2C08-FD7D-11B1-1B7A04921A5C> /usr/lib/libstdc++.6.dylib
    0x7fff8676c000 -     0x7fff86770ff7  libmathCommon.A.dylib ??? (???) <95718673-FEEE-B6ED-B127-BCDBDB60D4E5> /usr/lib/system/libmathCommon.A.dylib
    0x7fffffe00000 -     0x7fffffe01fff  libSystem.B.dylib ??? (???) <526DD3E5-2A8B-4512-ED97-01B832369959> /usr/lib/libSystem.B.dylib
[/plain]

[jimdempseyatthecove]

Does malloc_error_break show up in the linker map file?
If so you may be able to set the break point at the address indicated in the map file.

Another thing you might try is setup the debugger to step into the C runtime library startup code. You can find out where it dies on its way to the first FORTRAN statement.

Jim Dempsey

[chauvjo]

Quoting - jimdempseyatthecove

Does malloc_error_break show up in the linker map file?
If so you may be able to set the break point at the address indicated in the map file.

Another thing you might try is setup the debugger to step into the C runtime library startup code. You can find out where it dies on its way to the first FORTRAN statement.

Jim Dempsey

Jim....Thanks for the suggestion but I have no idea how to do what you recommend. I been running test cases all afternoon and encountered just a single runtime error. Unfortunately, I have yet to get any traceback to work with the Intel compiler so I spent hours inserting write statements and just found the problem. The error was minor and does not solve the issue discussed in this thread. Can someone explain why I cannot get a robust traceback? Have you confirmed that the current compiler and Mac OS can generate a useful traceback (without all the unknowns)?

[jimdempseyatthecove]

Chauvjo,

I am glad you found the problem. Now that you know what is was you can look for similar instances elsewhere in your code. I am a user of IVF and not an Intel support person so Icannot tell you why the error traceback is not working properly. From my personal experience, I seldom use the error traceback (due to the same issues relating to your experience). I typically use conditionaly compiled sanity checks and state logging. And I am not a Mac user so I cannot assist you in that area.

Jim

[Ron_Green]

Quoting - chauvjo

Jim....Thanks for the suggestion but I have no idea how to do what you recommend. I been running test cases all afternoon and encountered just a single runtime error. Unfortunately, I have yet to get any traceback to work with the Intel compiler so I spent hours inserting write statements and just found the problem. The error was minor and does not solve the issue discussed in this thread. Can someone explain why I cannot get a robust traceback? Have you confirmed that the current compiler and Mac OS can generate a useful traceback (without all the unknowns)?

There have been bugs in the traceback in the past. I'll run some tests to see if I can duplicate the problem.

Every Fortran program is actually started from a C main 'wrapper' that initializes the process, runtime, etc. before calling or branching to your Fortran main program. If something goes wrong in this startup there may not be any Fortran stack frames to unwind (traceback). This happens for both Mac OS and Linux: a linux process simply dies with a 'killed' statement and no stack frame.

Things that can kill the startup before the Fortran main program is entered include errors in data initializations such as DATA statement, constructors, and other static initializations of data before the first executable statement OR allocating more static data than your system has in RAM.

These are particularly hard problems, as there is no Fortran frames on the stack to traceback/unwind. And it's not obvious that our Fortran programs are wrapped by and started up and initialized by C code.

Another complication: even if your first statement is a WRITE or PRINT, that output is buffered by the Fortran runtime library and does not immediately get sent to the terminal. If the next statement or 2 cause a hard fault you may never see the output. To be safe, you can use a FLUSH statement after a WRITE or PRINT to cause the program to halt until that output is actually flushed to the output device.

Could you tell us what you found in the program that caused the bug? Was it an initialization of data in the declaration section?

[chauvjo]

Quoting - jimdempseyatthecove

Chauvjo,

I am glad you found the problem. Now that you know what is was you can look for similar instances elsewhere in your code. I am a user of IVF and not an Intel support person so Icannot tell you why the error traceback is not working properly. From my personal experience, I seldom use the error traceback (due to the same issues relating to your experience). I typically use conditionaly compiled sanity checks and state logging. And I am not a Mac user so I cannot assist you in that area.

Jim

Jim,

I must have not been clear. I did not find the problem. The program still crash when I attempt to debug. To be clear, the program runs with a normal compiler but crashes during debug.

[chauvjo]

Quoting - Ronald W. Green (Intel)

There have been bugs in the traceback in the past. I'll run some tests to see if I can duplicate the problem.

Every Fortran program is actually started from a C main 'wrapper' that initializes the process, runtime, etc. before calling or branching to your Fortran main program. If something goes wrong in this startup there may not be any Fortran stack frames to unwind (traceback). This happens for both Mac OS and Linux: a linux process simply dies with a 'killed' statement and no stack frame.

Things that can kill the startup before the Fortran main program is entered include errors in data initializations such as DATA statement, constructors, and other static initializations of data before the first executable statement OR allocating more static data than your system has in RAM.

These are particularly hard problems, as there is no Fortran frames on the stack to traceback/unwind. And it's not obvious that our Fortran programs are wrapped by and started up and initialized by C code.

Another complication: even if your first statement is a WRITE or PRINT, that output is buffered by the Fortran runtime library and does not immediately get sent to the terminal. If the next statement or 2 cause a hard fault you may never see the output. To be safe, you can use a FLUSH statement after a WRITE or PRINT to cause the program to halt until that output is actually flushed to the output device.

Could you tell us what you found in the program that caused the bug? Was it an initialization of data in the declaration section?

Ron,

As I mentioned above, the problem was not solved by the few minor bugs I located. I have run this code through several Fortran Lint tools and have not located anything that can explain this problem. The error says:

pointerbeingfreedwasnotallocated

My code does not allocate memory directly, the compiler code does so I would think that is the place to start. I really think Intel needs to create a special compiler option that keeps track of each and every statement execute so that it can generate a detailed traceback. I know this would be slow and have a lot of overhead but would be a time saver when trying to debug a tough problem.

At this point, I really do not know what else to try.