topic Re: Array Capacity Question in Intel® Fortran Compiler

Array Capacity Question

mushroom89_ — Sun, 11 Sep 2005 07:07:47 GMT

Before I replace my PC, I was wondering about capacities of various PC platforms. For example, I run real*8 arrays for engineering analyses. I could dimension at 30e06 for DVF, I now run at 60e06 with IVF (it runs fine even with the "image may not run" warning). I use Windows-XP, a 3-Ghz Intel-P4 with 1-Gb of RAM. I'm not sure what actually limits array size, I assume it's the compiler. Can you speculate what the real*8 array capacity might be using an Itanium, IVF with a 64-bit OS? Any other PC or software recommendations if array size is the critical factor?

Re: Array Capacity Question

TimP — Sun, 11 Sep 2005 12:45:17 GMT

32-bit Windows has a fixed part of the address space which can be shared by arrays and program. It is larger with Windows 2000 (near 2GB) than with XPSP2. If you use all of this space, you will get better performance with 2GB RAM. XP, with the /3GB boot option, has another data address space of 1GB. As Steve noted in the next reply, that is available (with LARGEADDRESSAWARE) only to ALLOCATABLE arrays. On the Windows X64 OS for EM64T and AMD64, much larger ALLOCATABLE arrays should be possible, but static arrays and COMMON are limited to 2GB each. While ifort on Windows Server for Itanium, with enough RAM installed, would support much larger static arrays, if you read the news last week you would see that Microsoft officially left engineering analysis out of their plans for Itanium.

Message Edited by tim18 on 09-11-2005 06:13 AM

Re: Array Capacity Question

Steven_L_Intel1 — Sun, 11 Sep 2005 17:56:44 GMT

Both Windows 2000 and Windows XP have, by default, 2GB for user code and data. On some versions of Windows 2000 and on Windows XP, you can specialy configure the system to raise this to 3GB. But you must use dynamic allocation (ALLOCATABLE) to take advantage of any space more than 2GB. Note also that your application's code, including libraries, plus some OS overhead, means that you can't use the whole 2GB - 1.75GB is a practical limit.

On the 64-bit platforms, Itanium and Intel EM64T, running Windows Server 2003 or (for EM64T), the x64 editions of Windows, the virtual address space available is MUCH larger. How large depends on a combination of RAM and swapfile. But you're STILL limited to 2GB for static code and data (eg. COMMON or locally declared arrays,) due to the design of the Microsoft object language format. You can use dynamic allocation to use the increased space.

I would encourage Fortran programmers to move away from static allocation of large arrays for really big problems. ALLOCATABLE is easy to use, though in some contexts performance may suffer a bit.

Re: Array Capacity Question

mushroom89_ — Mon, 12 Sep 2005 09:40:27 GMT

Seve/Tim, I appreciate the interesting discussion on static array size. Most of my code uses "static" arrays from blank common. I don't like using virtual memory since the swapfile takes forever. I'd rather use "block solvers" which minimizes the disk crawl. My upgrade path would be to Xeon w/E7525 chipset and Win-XP Professional if I were to upgrade to EM64T now. However, I'm probably going to upgrade RAM from 1Gb to 2Gb first since that appears to be my current bottleneck. The Windows Task Manager seems to confirm that I have 1Gb of Physical Memory, and that 0.2Gb is used by Windows-XP, leaving 0.8Gb for the program. It looks like there is a definite step-function in static array size and cost going from PCs to servers.

Re: Array Capacity Question

Steven_L_Intel1 — Mon, 12 Sep 2005 19:55:01 GMT

You're using virtual memory anyway, whether it is in static arrays or dynanic arrays. The only issue is that with arrays in COMMON, it's not possible to represent in the EXE an array larger than 2GB.

Re: Array Capacity Question

mushroom89_ — Tue, 13 Sep 2005 06:34:42 GMT

Understood. However, I'd rather keep my programs as-is and buy a bigger 64b computer, than re-write them all. I can summarize some of my concerns: 64b systems can have up to 16 Tb of virtual memory, but only ~16 Gb of RAM. However, those same systems only have up to 2 Tb of usable disk space. I'm not sure what the IVF 64b array size limit is, 16Tb or 2Tb? Then we have the 2Gb EXE limit, unless we ALLOCATE more virtual array space. It seems to me that the 2Gb EXE limit matches-up well with the 2Gb RAM limit for IA32 Windows-XP for PCs, with the option to allocate more if needed. I'll let you know if I gain any more array size with another Gb of RAM.

Re: Array Capacity Question

Steven_L_Intel1 — Tue, 13 Sep 2005 08:12:12 GMT

The compiler has no inherent array size limit. Whatever the OS will let you do, the compiler will let you do.

Please keep in mind my note that even on the 64-bit versions of Windows, static code and data is limited to 2GB. So even on EM64T systems you cannot have a 4GB array in COMMON. This is a Windows design restriction, not a compiler limit.

Re: Array Capacity Question

mushroom89_ — Tue, 13 Sep 2005 09:39:08 GMT

For arguments sake, lets say that I can dynamically allocate the largest array of my programs effectively. Any idea of the "allocatable" array size limitation between IA32 and EM64T? Is the virtual memory limited by the OS as well? If so what is the ballpark difference between XP-IA32 and XP-Pro-EM64T?

Re: Array Capacity Question

Steven_L_Intel1 — Tue, 13 Sep 2005 20:14:45 GMT

It's determined by the OS and the size of the process virtual address space. On IA-32, that's 2GB, or 3GB with the /3GB boot option. These are theoretical limits, in practice you'll be cut off somewhat smaller.

On the 64-bit operating systems, you're wide open. I assume by "XP Pro-EM64T" you mean Windows XP Professional x64 Edition.

Re: Array Capacity Question

mushroom89_ — Wed, 14 Sep 2005 06:58:53 GMT

Thanks. I'll test the IA32 array size limits shortly. I'll try for ~1Gb of static and 2Gb of virtual space using an allocated array (w/3GB boot). With 1Gb of RAM the "image" run size is <2.15Gb and a common dimension of 220e06 runs. I'll add 1Gb of RAM and see if that changes. I was curious what the cut-off point was between IA32 and EM64T systems. If EM64T still has the 2Gb static limit, but virtual array space only limited by available disk capacity, 100Gb or more should be attainable. I'd also assume that the 64b system using that much virtual storage would be much slower than the in-core static array solutions, but with 50x the capacity. (capacity vs. speed) I know a lot more now than when I started this discussion, thanks again!

Re: Array Capacity Question

jim_dempsey — Tue, 20 Sep 2005 03:32:26 GMT

I would like share some ofmy exprience of conversion of an application using COMMON blocks to using allocatable arrays. The conversion is relatively trivial. In my case it involved ~700 program modules and 100's of common blocks.

You might want to try it first by converting only portions of your code at a time in the following manner:

1) create a file for use as a FORTRAN Module e.g.

Module_Common.f90

This file contains:

! Module_Common.f90

module Module_Common

real(8), pointer :: DVF(:)

end Module_Common

2) compile as module file

3) Add toyour subroutines and functions that reference DVF(i):

use Module_Common

4) in your initialization code add the allocation to the size you want

iSizeYouWant = int(60e6)

allocate(DVF(iSizeYouWant))

Re: Array Capacity Question

jim_dempsey — Tue, 20 Sep 2005 03:32:38 GMT

You might want to try it first by converting only portions of your code at a time in the following manner:

1) create a file for use as a FORTRAN Module e.g.

Module_Common.f90

This file contains:

! Module_Common.f90

module Module_Common

real(8), pointer :: DVF(:)

end Module_Common

2) compile as module file

3) Add toyour subroutines and functions that reference DVF(i):

use Module_Common

! (removing

4) in your initialization code add the allocation to the size you want

iSizeYouWant = int(60e6)

allocate(DVF(iSizeYouWant))

Re: Array Capacity Question

jim_dempsey — Tue, 20 Sep 2005 03:32:44 GMT

You might want to try it first by converting only portions of your code at a time in the following manner:

1) create a file for use as a FORTRAN Module e.g.

Module_Common.f90

This file contains:

! Module_Common.f90

module Module_Common

real(8), pointer :: DVF(:)

end Module_Common

2) compile as module file

3) Add toyour subroutines and functions that reference DVF(i):

use Module_Common

! (remove

4) in your initialization code add the allocation to the size you want

iSizeYouWant = int(60e6)

allocate(DVF(iSizeYouWant))

Re: Array Capacity Question

jim_dempsey — Tue, 20 Sep 2005 03:32:46 GMT

You might want to try it first by converting only portions of your code at a time in the following manner:

1) create a file for use as a FORTRAN Module e.g.

Module_Common.f90

This file contains:

! Module_Common.f90

module Module_Common

real(8), pointer :: DVF(:)

end Module_Common

2) compile as module file

3) Add toyour subroutines and functions that reference DVF(i):

use Module_Common

! (remove COMMON

4) in your initialization code add the allocation to the size you want

iSizeYouWant = int(60e6)

allocate(DVF(iSizeYouWant))

Re: Array Capacity Question

jim_dempsey — Tue, 20 Sep 2005 03:33:01 GMT

You might want to try it first by converting only portions of your code at a time in the following manner:

1) create a file for use as a FORTRAN Module e.g.

Module_Common.f90

This file contains:

! Module_Common.f90

module Module_Common

real(8), pointer :: DVF(:)

end Module_Common

2) compile as module file

3) Add toyour subroutines and functions that reference DVF(i):

use Module_Common

! Rember remove COMMON DVF()

4) in your initialization code add the allocation to the size you want

iSizeYouWant = int(60e6)

allocate(DVF(iSizeYouWant))

Re: Array Capacity Question

jim_dempsey — Tue, 20 Sep 2005 03:33:02 GMT

You might want to try it first by converting only portions of your code at a time in the following manner:

1) create a file for use as a FORTRAN Module e.g.

Module_Common.f90

This file contains:

! Module_Common.f90
module Module_Common
real(8), pointer :: DVF(:)
end Module_Common

2) compile as module file

3) Add toyour subroutines and functions that reference DVF(i):

use Module_Common

! Rember to remove COMMON DVF()

4) in your initialization code add the allocation to the size you want

iSizeYouWant = int(60e6)
allocate(DVF(iSizeYouWant))

Message Edited by sblionel on 09-19-2005 05:22 PM