Virtual memory error

[fkrantz]

Hello,

In my code i'm trying to allocate an 2-dimensional array of integers of size 124036*900 but i obtain the following error :

forrtl: severe (41): insufficient virtual memory
Image PC Routine Line Source
libintlc.so.5 00007F042EACDB2D Unknown Unknown Unknown
libintlc.so.5 00007F042EACC635 Unknown Unknown Unknown
libifcore.so.5 00007F042F4D1985 Unknown Unknown Unknown
libifcore.so.5 00007F042F442FDD Unknown Unknown Unknown
libifcore.so.5 00007F042F490958 Unknown Unknown Unknown
libifcore.so.5 00007F042F4907B1 Unknown Unknown Unknown
riem2v 000000000043EA3F geom_mp_sort_edge 213 geom.f90
riem2v 0000000000492682 MAIN__ 82 main.f90
riem2v 000000000040139C Unknown Unknown Unknown
libc.so.6 00007F042E363B1D Unknown Unknown Unknown
riem2v 0000000000401299 Unknown Unknown Unknown

I'm working on an archlinux 64 with 8GB of ram, ifort 11.1.046.
The compiler options are : -O0 -g -check all -traceback -gen-interface -warn interface -check arg_temp_created -shared-intel -mcmodel=large -r8.

Perhaps my computer doesn't have enough memory, i don't know.

Thanks in advance for your help.

Fred.

[TimP]

Did you check whether you have stack overflow, at least raising your stack limit in your shell? As you haven't set -openmp or -parallel, you might try -heap-arrays.

[Steven_L_Intel1]

I would not expect this error message if stack was an issue. The "virtualsize" (I think) limit is relevant, as perhaps is the size of the pagefile.

You probably don't want -mcmodel=large, unless your code is larger than 2GB alone. Try -mcmodel=medium.

[Martyn_C_Intel]

Whilst I agree with Steve, I note that your array uses only about one twentieth of your physical memory, before getting on to any virtual memory. So either you have some other very large arrays pushing you close to the combined memorylimit, or something unexpected is going on. I suggest you type "free -m"in a separate window, to see your available swap space and monitor the memory usage of your application. Alternatively,you could just run "top", which updates automatically. (You should see two separate lines for physical memory and for swap space, that show the total available and the amount used.)

Until you sort out this memory issue, I would leave out -check all, just to keep things simpler.

[fkrantz]

Hi all,

Thanks for your answers.


Actually, i did a mistake : the size of the array that i would allocate is 124036*186504. In this case, clearly i'haven't enough memory. Right ?

[Ron_Green]

probably not 92.5GB single precision and 185GB double precision would be needed just for that array.

My rule of thumb is targeting 80% of physical memory on the server for all your arrays. You can find this info if you:

cat /proc/meminfo | grep MemTotal

This is an ideal target, since you are bound to have overhead for IO buffers, array temporaries, etc. The absolute upper bound on any server is MemTotal + disk swap space. swap is also in /proc/meminfo:

cat /proc/meminfo | grep SwapTotal

Keep in mind that once you exceed MemTotal your application will start to page, eventually as it grows things start to swap (even worse). Performance will absolutely tank once you start to page heavily. USUALLY when I run I keep an extra window open and run

vmstat 2

to keep a track on memory usage every 2 seconds or so. Some folks also use the command 'top' to monitor the state of the server.

ron

[fkrantz]

Hi all,

After a little more work, i modified my code, in fact, one of the data structures was bad.
Now everything works fine.
Thanks for your help, i have learned a lot here, especially on the stack memory and size.

Fred.