- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have a gigantic array (several dozen megabytes) of a user-defined datatype. I would like to write this array out to a binary file using unformatted I/O for fastest operation.
Let's say that my array is X. This is my line of code:
TYPE(MYTYPE), ALLOCATABLE :: X
...
WRITE(3) X
However, when I execute this line I get "stack overflow." I can avoid the stack overflow by increasing the stack size, but this eventually leads to a stack size equal to the amount of system memory. I can also avoid it by writing out one element of the array at a time, but then I don't get the excellent performance of the unformatted write. Finally, I could write out the array in many smaller chunks, but this leads to a bunch of messy code.
Are there any ways to solve this problem short of increasing the stack size to a gigantic size?
Thank you,
Michael Carr
Let's say that my array is X. This is my line of code:
TYPE(MYTYPE), ALLOCATABLE :: X
...
WRITE(3) X
However, when I execute this line I get "stack overflow." I can avoid the stack overflow by increasing the stack size, but this eventually leads to a stack size equal to the amount of system memory. I can also avoid it by writing out one element of the array at a time, but then I don't get the excellent performance of the unformatted write. Finally, I could write out the array in many smaller chunks, but this leads to a bunch of messy code.
Are there any ways to solve this problem short of increasing the stack size to a gigantic size?
Thank you,
Michael Carr
Link Copied
3 Replies
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I can't reproduce this in 6.5. I suggest you send a small but complete example to us at vf-support@compaq.com and we'll take a look.
Steve
Steve
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Michael, your problem sounds like it's the same as if you tried to write
an array in a subroutine, like this:
SUBROUTINE SUBNAM(X,Y,Z)
DIMENSION X(*)
WRITE(IUNIT)X
........
The above does not work because "WRITE" has no way to know how many
elements the array X has.
In your case also, I would ask how the WRITE statement knows how big
the "ALLOCATABLE" array is, as you did not show a line of code that
allocated the size.
I don't know what the disk format is for unformatted write in this compiler,
but I can tell you that others I have used split large volume output into
blocks anyway, like 256 bytes, starting with a block count and containing
a CRC check. I you can find out from COMPAQ what the block size is,
then writing data in blocks that match the blocksize should be just as
efficient.
Regards, Paul Dent
an array in a subroutine, like this:
SUBROUTINE SUBNAM(X,Y,Z)
DIMENSION X(*)
WRITE(IUNIT)X
........
The above does not work because "WRITE" has no way to know how many
elements the array X has.
In your case also, I would ask how the WRITE statement knows how big
the "ALLOCATABLE" array is, as you did not show a line of code that
allocated the size.
I don't know what the disk format is for unformatted write in this compiler,
but I can tell you that others I have used split large volume output into
blocks anyway, like 256 bytes, starting with a block count and containing
a CRC check. I you can find out from COMPAQ what the block size is,
then writing data in blocks that match the blocksize should be just as
efficient.
Regards, Paul Dent
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
...btw, what happens when you try to WRITE into a binary file
a structure containing POINTERs to arrays or, still worse,
if it is a part of, say, linked list with POINTERs to another structures?
AFAIK, CVF stores POINTERs to arrays inside a structure in an
array descriptor. Are array elements stored somewhere in the file also
(as a naive user would expect), or, as I suspect, isn't the descriptor just
copied byte-by-byte to disk, thus losing its association with the target
when the file is reloaded on, say, a later run of the application? What the
(F2k draft) standard say should happen?
(Perhaps this belongs to another thread - but perhaps it has something
related with the problem of the original poster?)
Regards,
Jugoslav
a structure containing POINTERs to arrays or, still worse,
if it is a part of, say, linked list with POINTERs to another structures?
AFAIK, CVF stores POINTERs to arrays inside a structure in an
array descriptor. Are array elements stored somewhere in the file also
(as a naive user would expect), or, as I suspect, isn't the descriptor just
copied byte-by-byte to disk, thus losing its association with the target
when the file is reloaded on, say, a later run of the application? What the
(F2k draft) standard say should happen?
(Perhaps this belongs to another thread - but perhaps it has something
related with the problem of the original poster?)
Regards,
Jugoslav
Reply
Topic Options
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page