- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have "inherited" and application that returns an istat of 41, error message is insufficient virtual memory when I am trying to open a file. This error is not repeatable, by that I mean that in opening say 10 files both std. text and binary. The first 4 may work on one run, failing on the fifth. Then the next time it fails on the second file open.
When it does succeed in opening a file, the contents appear to be valid, though the binary radom files may squawk about record size.
Some details:
Win 7 64 bit O/S building 32 bit applications
Intell 11.1 (/Qvc9)
MSVS Express 2008
Image size during execution ~38 MB , so not near the limit
I converted the source from F77 to F90 to pick up gen-interfaces, trying to clean things up.
Looking for ideas....
Regards
Carl
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
There aren't enough customers using VS Express for us to be in a position to test whether there may be issues peculiar to that combination.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
So that should increase available stack from 1 MB to 16 MB .... should I go more?
I thought that VS Express was Intel's recomended solution to get a linker/debugger. Is there any issue with using code generated from it too?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Is your application multi-threaded?
If so, you may need to specify a stack size value for use by your threading tool. OpenMP uses KMP_STACKSIZE and/or library function, other toolkits may have other function names.
Also note, if you use 16MB for stack, and create 100 threads, you now consumed 1.6GB of virtual memory (even though it has not been committed/used).
Similar problemif your file opens are specifying overly large buffers.
Similar problem if you are opening memory mapped files. All such files opened would have to be assigned to virtual addresses of your VM.
Last problem, you may have set/have a pagefile restriction.
What happens when you run your program in Debug mode, or place a PAUSE before and after the open, then check via Task Manager, the memory requirements of your process?
Jim Dempsey
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
If not requiring C/C++ capability, the VS2008 or 2010 Shell provided with ifort is recommended in preference to VS Express.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
With regards to the multi-threaded question .... the orginal app was two threads (used OpenGl) but I beleive I have sidestepped all the OpenGL stuff and just use simple print statements. I used process explorer to watch the exe and it seems to only report 1 thread.
I did just discover another "fly in the soup", this #$%* code wants to use namelist. Considering that the orginal data files came off of a HP workstation who know what flavor of namelist it is. My guess is that FORTRAN will try to allocate a structure corresponding to the namelist definition.... so that coud be another source of issues.
Now from the 11 manual .....run time errors... (yes I do read the manuals :) )
severe (41): Insufficient virtual memory
FOR$IOS_INSVIRMEM. The Intel Fortran RTL attempted to exceed its available virtual memory while dynamically allocating space. To overcome this problem, investigate increasing the data limit. Before you try to run this program again, wait until the new system resources take effect.
Note: This error can be returned by STAT in an ALLOCATE or a DEALLOCATE statement.
So I guess I am off to explorer this some more.... thingsI will try:
1) Drop back to Intel 9.1 and VS 2005 (pro edition)
2) Play games with namelist
Thanks for the suggestions
Carl
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
(IOW requiring swap of bytes in word sized units (4 or 8 bytes))
Jim Dempsey
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Can you reduce the program to one that just does the OPEN but still gets the problem?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Greetings!
I also converted some F77 to F90 code (removing named common blocks, adding modules, etc.), and I hit a similar issue. In debug mode, I see that the following call
open (unit = myunit, + status = 'scratch', + access = 'direct', + form = 'unformatted', + recl = myrecl, + iostat = ios)
returns with ios = 41. In release mode, the code crashes in libfcoremd.dll while writing to the scratch file. Here is part of the backtrace
[ 0] 0x00000000033fba34 for_lt_eq+00032948 at libifcoremd.dll+965172 (no debugging symbols found) [ 1] 0x0000000003365412 for_pause+00010162 at libifcoremd.dll+349202 (no debugging symbols found) [ 2] 0x000000000339f7b8 for_write_dir_xmit+00007256 at libifcoremd.dll+587704 (no debugging symbols found)
I would appreciate any suggestions on how to resolve or work around this issue. The compiler version is:
Version 12.0.3.175 Build 20110309
Thanks in advance!
-Franz
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
It is unlikely that the OPEN statement, by itself, causes memory to be exhausted. There was probably something that happened earlier in the run that consumed memory and did not release it in a timely manner.
What was the value of IOSTAT in the release mode run? Compile with /traceback and rerun to locate the error more precisely.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
As mecej4 said, the error most likely occurred earlier. If you haven't already done so then you should also compile with other debug options on - e.g. uninitialized variables, array bounds checking, argument mismatches etc. With a major re-write of code such as this a typo is easy to make.
Les
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks for your comments!
In release mode, the code just crashes while trying to write to the scratch file (see above).
I already compiled with /check:all, /warn:all and /traceback. It also tried the newest compiler (XE 2015) and it shows the same behavior. At the moment, I am puzzled where the real issue is.
-Franz
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
How big is the code, and is it feasible to post it (or a cut-down reproducer) here?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
What is the value of myrecl?
What is the type of myrecl?
Jim Dempsey
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Also, to aid in tracking down the issue, change the open from opening a 'scratch' file wherever temporary files are located, to opening a file named 'temp' in the current directory (where you have control of amount of disk space you can claim).
If this works, then the issue is with where 'scratch' is located.
If this fails, then it is some other issue (like goofy myrecl value or type).
Jim Dempsey
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks again for your comments and help!
@mecej4: Unfortunately, the code is quite large and at first I could not come up with a small reproduction example.
@Jim Dempsey: Your suggestion regarding the 'recl' specifier was very helpful. I compile with option /assume:byterecl on Win7 and currently use a record length of 512 bytes. The following code either returns directly with ios = 41 or the scratch file immediately gets really large (~1GB), although only a few records are written:
integer(4), pointer :: myrecl myrecl => global%vars%myrecl open (unit = myunit, + status = 'scratch', + access = 'direct', + form = 'unformatted', + recl = myrecl, + iostat = ios)
I checked that the pointer is associated and has value 512. Now, when I hard-code the record length, it seems to be working as expected:
open (unit = myunit, + status = 'scratch', + access = 'direct', + form = 'unformatted', + recl = 512, + iostat = ios)
Am I not allowed to use a pointer to specify 'recl'? Both versions work fine when I compile the code with the GNU Fortran compiler.
-Franz
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Interesting. I think it's a bug, and I'll report it (and copy the internal tracking id here once that done).
In the meantime there are a couple of workarounds. One is to hardcode the value, another is to declare myrecl as an integer not an integer,pointer and to do simple assignment.
Thanks for the small example -
--Lorri
PS: Internal tracking id is DPD200360789
![](/skins/images/B7BA37F9CE4B52FF7CAE1FB0388839C8/responsive_peak/images/icon_anonymous_message.png)
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page