Memory Space Issue?

[pfwang]

I was able to compile and run a fortran code, which required a large memory space. After the compiled code started execution for some time, the code crashed. The following messages showed up on the debug screen:

Loaded 'ntdll.dll', no matching symbolic information found.
Loaded 'C:WINDOWSsystem32kernel32.dll', no matching symbolic information found.
Loaded 'C:PFWorkingSDBCH3DCH3D-SDBCH3D-SDBDebug etcdf.dll', no matching symbolic information found.
The thread 0xF64 has exited with code 0 (0x0).
The program 'C:PFWorkingSDBCH3DCH3D-SDBCH3D-SDBDebugjame_ch3d_transp_sinclair_FC_netCDF.exe' has exited with code 0

Can anyone provide any help on this? What doesthis mean? Thanks.

[Steven_L_Intel1]

Those messages don't mean anything important. They are all put out by the debugger and are informational.

The "Loaded" messages are informing you that a DLL is being used by the program but no debug information for that DLL is available.

The last two messages indicate that the program exited normally.

You should rerun it and this time use the "Start without debugging" menu item so that the console output window is not removed when the program finishes. Then you may see useful error messages in the console window.

[pfwang]

Hi Steve,

Thank you very much for the response. I will follow the instructions. I have two questions related to this one:

1) I have previously compiled my fortran code using Rebuild All (DVF 6.6). The code gets compiled with a warning, something like

"the code has 350,000,000 allocation exceeding the system (260,000,000). The executable file may not run".

I have a pentium 4 (dual core processor 3.6 GHz with 4 GB Ram). Does the warning mean that my program exceeds the memory limit? Is there anything I can do to (1) increase the 260,000,000 system limit or 2) to take advantage of the large memory of my PC (my colleague has 2 GB RAM and he gets the same warning on his PC).

2) I have DVF 6.6 and am wondering if I should buy IVF 9.1? What is the difference between this two products? Will the IVF 9.1 solve the problem in 1)?

Thanks.

PF Wang

[Steven_L_Intel1]

That warning you got is not relevant unless you are planning to run the program on Windows 95. The newer Microsoft linkers used by IVF don't put out this warning any more.

CVF 6.6 has not been maintained or developed in three years. Not only has IVF benefited from numerous new features and bug fixes, but programs built with IVF almost always run faster. IVF is a currently developed and supported product that works with current Microsoft development products. On the other hand, moving to IVF now means buying two products, IVF and a prerequisite version of Microsoft Visual Studio if you don't already have one.

If CVF is working for you and you're happy with it, there's certainly no requirement that you change. You don't actually have a problem in (1), so that's not an issue. But there's no longer any support for CVF so that should factor in to your decision.

[pfwang]

Thanks for the information. I have sent out a purchase of IVF9.1 Professional Edition (~$1300). I hope this is the right edition that you mentioned in your email. Also, I would still like to know with CVF 6.6, what does that error message mean (Code size exceeds system capacity) and why the same message appears even though the RAMS are twice as large (2 MB vs. 4 MB). Does that mean the code (CVF) has a memory capacity limit which does not depend on the RAMS of the computer? With IVF, how does this limit get alleviated? the bottom-line question is do I need to increase the RAMS of my computer in order to run the code on IVF?

Thanks, I am a new user of this system and your response has been extremely useful to me. Thanks again...

[Steven_L_Intel1]

If you want the IMSL library, then you need the Professional Edition, otherwise the Standard Edition will suffice and is much less expensive. You will also need one of the following:

• Microsoft Visual Studio 2005 (Standard Edition or above)
• Microsoft Visual Studio.NET 2002 or 2003 (any edition)
• Microosoft Visual C++.NET 2002 or 2003 Standard Edition

Note that Microsoft Visual C++ 2005 Express Edition is not sufficient if you want full functionality.

About the warning - older versions of Windows - Windows 95 and Windows NT 4 before SP3 - were unable to run an executable that had more than 256MB of code and static data combined. Windows 98 and later versions removed this limit. Microsoft's 5.0 and 6.0 generation linkers gave this warning to let you know that some versions of Windows would not run the program. As long as you don't care about Windows 95 or old NT4 systems, the warning could be ignored. Microsoft removed the warning in the 7.x and later linkers, which is why you don't get it with IVF. It has nothing to do with how much memory is on your system.

[pfwang]

I do have Microsoft Visual Studio (C++ and FORTRAN). So I guess I should be able to load IVF9.1.

Is ther any limit on the sizeof the code with IVF? My code uses fairly large static memory (declaring allocated space in include files) and are used to run on supercomputers. I have run this code for scanrios with smaller static memory with CVF6.6 before, but this time I have to run a scenario with much larger static memory. I just wonder if there is any limit on the memory of the code with IVF.

Thanks.

[Steven_L_Intel1]

If you have Microsoft Visual Studio 6.0 (or 98), which works with CVF, then that will NOT work for IVF. You need a newer version of Visual Studio - 2002, 2003 or 2005.

Neither CVF nor IVF have their own limits on memory usage - that is dependent on Windows. On 32-bit Windows, the theoretical limit is 2GB, though in practice it is somewhat less. On 64-bit Windows, the static code and data limit is still 2GB, but dynamically allocated data can be larger.

You may want to consider changing large static arrays to allocatable arrays. Especially if you move to the 64-bit platform, you'll want to do this.

[emersonone]

I am planning to order IVF too, will VStudio .NET Pro 2003 Win32 work for IVF9.1? What is the benefit of VStudio 2005 comparing with 2003?

Thanks

[Steven_L_Intel1]

VS.NET 2003 will work fine. To be honest, I'd say that it has advantages over VS2005 in that Microsoft dropped some functionality and broke others in VS2005 that are a pain for Fortran programmers. If you can still get VS.NET 2003 at a reasonable price, go for it.

[pfwang]

Mor equestion:

1) What is 64-bit Windows (vs. 32-bit Windows?) Is is machine dependent or OS dependent? Is Windows XP 64-bit? How can I get/install a 64-bit Windows?

Thanks.

[Steven_L_Intel1]

In the context of this discussion, 64-bit Windows is one of the "x64" editions of Windows, either Windows Server 2003 x64 or Windows XP Professional x64. It runs on systems based on Intel processors with Intel EM64T (Intel 64) or AMD processors with AMD64. On the Intel side, nearly all recent Intel Xeon, Pentium 4 and Pentium D processors support EM64T. The new Intel Core 2 processors do too. You need a system with a 64-bit capable processor, a BIOS which allows you to enable it, and the x64 Windows of your choice.

Regular Windows XP is not 64-bit.

Learn more here. I think that at this point, you can get it with a new PC if you ask for it, or from MSDN. I'm not aware that Microsoft sells it separately.

[nickelyu]

Hi, I am a rookie.

I have a similar problem when I want to transfer data from the main program to subroutines. In detail: in the main program I difine an integer N, and I want to use this constant in many subroutines to difine matrix size. So, I use it as aparameter. While if the N is a little big, such as 260, there is an error saying stack overflow. I can not even difine a matrix A(N,N) in the subroutine.The program can compile and run until the subroutine, and the subroutine does notrun even the first line giving the error message. While if N is small ,everything is OK. Isn't it strange?

At first ,I thought itis something aboutRAM if there is some limitfor any application using .Well ,I havea 2GB RAM,and after readingyour discussions above, it seems to be something else, notthe limituse of RAM.

My system is intel64,compiler IVF9.1, winXP pro, and using command line(I have no intel visual studil). While the same program can run in another workstation, amd opteron 64, compiler IVF9.0 Linux suse.

What on earth is the cause? Have I made myself clear? Would you apply some help? Thank you very much!

[TimP]

Look up the options /link /stack:nnnnnnnnn

[nickelyu]

Thank you very much! Anyway it works .

Is there any permanent way to change the amount of stack using command line? I just use the link command to change the amount every time like : link *.obj /stack:nnnnnnnnn for each program. Well, I know it is somehow stupid. I really don't know how to make the change permanent.

[Steven_L_Intel1]

If you always use ifort to do the link, you can do this.

Edit (with notepad) the file C:Program FilesIntelCompilerFortran9.1IA32Binifort.cfg and add the line:

-Qoption,link,/stack:nnnnnnnnn

This will be used for both command line and IDE builds.

[grg99]

Hi, here's a few hints:

Look at the size of your .exe file. If it is HUGE, like 100 megabytes or more, you might want to take a few steps to trim it down. If you change your array declarations so the big arrays are allocated at run-time, you often save many many megabytes.

Foir instance change:


real Biggy( 1000, 1000, 100 )


to:


real, allocatable Biggy( :, :, : )

ALLOCATE( Biggy( 1000, 1000, 100 ) )

That can save you hundreds of megabytes.

-----------------------------------------------------

Big use of stack space happens whenever you make a temporary array, as in a actual argument to a subroutine. Temp arrays are made for you automatically, which is a wonderful feature, but they do chew up stack space. For example:

Call Solver( Biggy(:,:,i), Biggy( :, :, i+1) )

.. can chew up many many megabytes of stack space.

If having to specify the /stack option irritates you, you could make the temporary arrays yourself, so they come out of the heap, and don't issue a stack message:

real, allocatable :: Temp1( 1000, 1000, 1 ) ); Temp1 = Biggy(:,:,i);
real, allocatable :: Temp2( 1000, 1000, 1 ) ); Temp2 = Biggy(:,:,i+1);
Call Solver( Temp1, Temp2 )
Deallocate( Temp1 ); Deallocate( Temp2 );

Of course if this is in a loop you can factor out the allocates/deallocates out of the loop.

Hope this helps.

[jeanvezina]

But a temporary is not always required in this case.

For the call:

Call Solver( Biggy(:,:,i), Biggy( :, :, i+1) )

temporary arraysare required only if Biggy(:,:,i) orBiggy( :, :, i+1) are not

contiguous in memory. The NAG compiler, even if it not considered a highly optimizing compiler, does runtime contiguity tests by examining the array descriptors and decides if a temporary is required or not. At the present time, the Intel compiler unfortunately doesn't do this and may create unneccessary copies.

Best regards,

Jean Vezina

[Steven_L_Intel1]

The Intel compiler definitely does do run-time contiguity tests when passing arguments and has for quite some time. There may be cases where it gets it wrong, and if so, we'd like to know about them.

[grg99]

Oh yeah, I suspect in many cases Fortran can build a descriptor that encompasses a slice, but is not even remotely contiguous in memory. There's a field in the descriptor "distance between elements" that can be fudged to make a non-continguous slice appear contiguous enough to the casual recipient.

And it's not just slices that might require temporary storage, many if not most array-returning functions need temporary space. Something as simple as SNGL( Array ) can and usually does require a full-size temp array.

.

[Steven_L_Intel1]

If the called routine declares the array argument as assumed-shape (:), and the caller sees an explicit interface so thst it knows that, then a descriptor is passed and a copy is never made. The only time the compiler might make a copy is if the called routine is expecting a simple contiguous array and the called routine believes that what is being passed is not contiguous.