- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
My fortran program requires a large amount of memory and takes a long time to run (a simulation program). To manage memory usage, I used variable-sized arrays of link lists (the link lists are for quick popping in and out of data and I am sure there is no memory leak) and performed frequent memory allocation and deallocation inside several large loops (the loops are also multi-threaded). While I am able to maintain reasonable and steady memory usage, I also observed significant speed degradation over time (50%-100% run time increase from beginning to the end of the program run). An obvious symptom is the reduction of cpu utilization over time. The cpu utilization just continued to drop even through the computer is doing nothing but just to run this program.
My preliminary search for solution seems to indicate that this is caused by memory fragmentation due to my frequent memory allocation/deallocation and long run time. Perhaps there may be some other reasons for this that I am not aware of. Cpu utilization is not something I can control and I have no clue how to mitigatea situation like this.
I have not found any existing discussion helpful to this problem. Perhaps I have not read enough but any help would be greatly appreciated.
Calmag C
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
a pool of memory from which to allocate and deallocate meory chunks.
The idea is that you simply allocate one or more large chunks of memory at the start of the program.
During the run you assign small pieces of it and release them when you are done. It will probably
caused you some extra programming, but it may be worth investigating.
Regards,
Arjen
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have not tried it myself yet, but a technique that should work is something along
these lines:
For simplicity: suppose you have derived types with fixed components only
Then:
- Allocate an array of such types and a logical array of the same size to register whether
an element is in use.
- When youneed a chunk of memory, find the firstunused element and setit to"in use".
- You can then use a pointer to that element.
- When you are done, set the element to "free".
This is just a sketch of course. It should work with some modifications for assigning chunks
of integersor reals as well. Butbeware: if the amounts of memory vary a lot, you will find
yourself implementing an actual allocator yourself, complete withdefragmentation algorithms
and so on.
Whether that is worth your while - and how general you can make the implementation -
is an open question.
Regards,
Arjen
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
allocating these elements via the ALLOCATE statement, you "simply" use an element from a pre-allocated array. That way you only use contiguous memory.
Regards,
Arjen
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
COMMON /YouThreadPrivate/ type(yourFooType) freeFooListHead
!$OMP THREADPRIVATE(/YouThreadPrivate/)
...
// in your allocate routine
if(associated(freeListHead)) then
getFooPointer = freeFooListHead
freeFooListHead = freeFooListHead%next
return
endif
allocate(getFooPointer)
-------------
// in your deallocate routine
aFoo%next = freeFooListHead
freeFooListHead = aFoo
Jim Dempsey
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I'll attach the routine - might be that you could use part of this.
See thread http://software.intel.com/en-us/forums/showthread.php?t=80916&o=a&s=lr&wapkw=(aagaard)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You should look at the: Task Manager | Performance | Page File Usage
to observe if you have a similar problem lurking somewhere in your code.
Also, if these allocations are only required up to a point in your code, then if different allocations will be constantly reused later (and present a paging or out of memory issue), then consider adding a parallel region after use of these first allocations, that specifically deallocate these nodes.
It sounds like this is not needed, but look at the Page File Usage to assess the situation.
Jim Dempsey
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page