- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Im using Win Server 2003 x64 (Athlon64) and newest Intel Fortran Compiler 9 for EM64T.
I allocate a single char-array of, say 2.5 GB. Allocation works, but when I access elements beyond ca. 2100 MB I get access violations. If I allocate two separate arrays of less than 2100MB it works. So it seems not to be the case, that the process is limited to 2GB, but single array seem to be...
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
This sounds like the index you are using to access the array is 32-bits. Check to see ifyour index variable is 64-bits. (As well as any variables used to produce the index.)
Note, F90 does not have "unsigned integer". Therefore any 32-bit expression exceeding 2,147,483,647 will be negative.
Jim Dempsey
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello.
Increasing the stack size using the option /stack, like for example "/link /stack:64000000" may help to enable verylarge arrays.
Lars Petter Endresen
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
----
Q2: How do I access larger memory models with the linker?
You can not do this on Windows (we do offer the ability to access larger memory models on Linux where the underlying tools/OS supports it - this is supported by the mcmodel switch).
----
This seems to be a direct contradiction to what Steve Lionel said on September 13:
----
"The compiler does not impose its own limit - on the 64-bit operating systems, you should be able to allocate arrays as large as the OS is willing to handle. I know we've tested up to 6GB at least."
----
Who is correct?
I must say, however, that the "compiler" did not impose the 2 GB limit but the linker did. Here's the output from my compile and link:
----
D:DesignCodesTestTest64>ifort test64.f90 /link /stack:64000000
Intel Fortran Compiler for Intel EM64T-based applications, Version 9.0 Build 20051130 Package ID: W_FC_C_9.0.028
Copyright (C) 1985-2005 Intel Corporation. All rights reserved.
Microsoft Incremental Linker Version 8.00.40310.39
Copyright (C) Microsoft Corporation. All rights reserved.
-out:test64.exe
-subsystem:console
-entry:mainCRTStartup
/stack:64000000
test64.obj
LINK : fatal error LNK1248: image size (BEC44000) exceeds maximum allowable size (80000000)
D:DesignCodesTestTest64>
----
Please note that the error is a linker error. The program had compiled sucessfully, but could not link. I was *not* dynamically allocating the 3 GB array, but set the array size on the declaration statement. Please also note that I tried to increase the stack as Lars suggested on December 12, but it did not help.
This has been an extremely frustrating experience for me. I've wasted thousands of dollars of the government's money for the server and tens of thousands for my time trying to build large-memory programs for Windows. If only someone or the documentation had just said it can't be done.
Marshall
Message Edited by Marshall-L-Buhl on 12-21-2005 08:30 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Windows, even the 64-bit versions, have a 2GB limit for static code and data. You can exceed this with allocatable arrays, but not static arrays (eg. COMMON.)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
OK, so I tried to dynamically allocate the array. Here's my test program:
----
program Test64
implicit none
integer(8), parameter :: arysize = Z'C0000000' ! this is 3 GB
integer(8) :: i
real(8), allocatable :: ary( : )
write(*,'(z12,i12)') arysize, arysize
allocate ( ary(arysize) )
end program Test64
----
Here's my build:
----
D:DesignCodesTestTest64>ifort test64d.f90 /link /stack:64000000
Intel Fortran Compiler for Intel EM64T-based applications, Version 9.0 Build 20051130 Package ID: W_FC_C_9.0.028
Copyright (C) 1985-2005 Intel Corporation. All rights reserved.
Microsoft Incremental Linker Version 8.00.40310.39
Copyright (C) Microsoft Corporation. All rights reserved.
-out:test64d.exe
-subsystem:console
-entry:mainCRTStartup
/stack:64000000
----
Here's my execution:
----
D:DesignCodesTestTest64>test64d
C0000000 3221225472
forrtl: severe (41): insufficient virtual memory
Image PC Routine Line Source
test64d.exe 000000000045576F Unknown Unknown Unknown
test64d.exe 0000000000452B29 Unknown Unknown Unknown
test64d.exe 000000000041411D Unknown Unknown Unknown
test64d.exe 00000000004074DF Unknown Unknown Unknown
test64d.exe 0000000000403D80 Unknown Unknown Unknown
test64d.exe 0000000000403C9B Unknown Unknown Unknown
test64d.exe 000000000040112E Unknown Unknown Unknown
test64d.exe 0000000000457B3C Unknown Unknown Unknown
test64d.exe 0000000000444E38 Unknown Unknown Unknown
kernel32.dll 0000000078D5965C Unknown Unknown Unknown
----
Normally, I wouldn't do an allocation without checking the status. When I tried that, I got a return status of 41, which I suspect means insufficient memory. My computer has 6 GB of RAM and another 6 GB of virtual memory--far more than is needed to hold a 3 GB array.
As you can see, I'm using the latest EM64T compiler. I downloaded the latest version of PSDK-amd64.exe yesterday. and applied it before installing the compiler.
So, Lionel, what am I doing wrong?
Marshall
Message Edited by Marshall-L-Buhl on 12-21-2005 09:51 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
This is a pretty simple program. Can you offer any advice? When I look at Task Manager, I'm told my Commit Charge is 232M / 11870M. I definitely have much more than 3 GB of available memory.
Does it sound as if the compiler is incorrectly installed?
Any hints as to how I can get past this roadblock would be greatly appreciated.
Marshall
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
It would be interesting to see if a call to the Win32 API routine VirtualAlloc for the same number of bytes succeeds.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
As you can see from the screen output quoted in my earlier posting, I am using the EM64T compiler.
I'll try the VirtualAlloc call just for grins if I can find documentation for it.
Thanks for helping,
Marshall
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks for your help! The real tip I needed was the bit about the stack size.
Marshall
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hey Marshall,
could you explain how the maths work and show your corrected code. I'm having the same problem ....
Also, I was wondering, if there is a tool around that keeps track of allocated memory. The task manager only seems to show, what is used, but not what is allocated only. Are there any other tools around that show how much memory is currently reserved for a certain process?
Thanks Chris
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Memory management under Windows (and indeed any OS with a rich process model) is a tricky business. There is no single right answer to the question of how much memory a process is using or has reserved.
In general, anything task manager can do process explorer, freefrom SystInternals (now owned by MSFT)can do better. But you need to frame the question more precisely -- in terms that match the real process model of the OS -- to get an answer. "Working set size" is what you want to compare against your physical memory resource. Be aware that windows slows WAY down when you start to cramp its disk cache, leave 15% or so for that.
But for non-insane amounts of memory on 64-bit windows the limit will be the unused swap space. If the swap file is expandable and you allocate something larger than it can hold you may get a temporary failure, at least on XP. That's a wrinkle you don't find in most other operating systems.
If you try to allocate a truly insane amount of virtual memory -- like say you have terabytes of SAN storage configured as your swap file -- it is conceivable you could hit some other limit, like say in the operating system's page table management code. You also have to have sufficient contiguous linear (process virtual) address space available but that won't be a problem in x64 unless you've deliberately fragmented your space.
At the margins, on classic Unix systems the swap space was the limit for system-wide total allocated virtual memory. On Windows and Linux the limit is physical memory - kernel overhead + swap space.
I'm not clear what "maths" you (Chris) are asking about. I'll watch for followup questions and try to provide less enigmatic answers next time.
Finally, if you're trying to force your allocated array to be resident in physical memory you're off into a whole other arena of pain. You're telling the operating system's memory management that you know better than it does how to manage the resource. You may be right but you can expect it to get huffy about it and not be terribly helpful.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
HI sir,can you help me to implement 2d array like below code
include<stdio.h>
#include<malloc.h>
int main()
{
int ch,*a,data,*old,n=0,i=0;
a=NULL;
printf("\n you want to add data");
scanf("%d",&ch);
while(ch)
{
old=a;
printf("\n enter");
scanf("%d",&data);
a=(int *)malloc(sizeof(int)*(n+1));
if(n>0)
{
for(i=0;i<n;i++)
{
a=old;
}
free(old);
}
a=data;
n++;
printf("\ndo you want to add another data");
scanf("%d",&ch);
}
printf("\n elements are");
for(i=0;i<n;i++)
{
printf("%d\n",a);
}
}
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page