MALLOC failure detection

[Hashemi__Ali]

I am running the following code (a small chuck of a larger codebase) in a Linux Docker container (kernle 4.9.184) with 750MB of memory (equivalent of running the process in CGroups with limited memory).
Running the code shows that memory allocation was successful. But, later in the loop, when accessing the memory, program crashes. 

- Is this the right way to check status of MALLOC? How safe is it? 

- If this is the right way, how come it does not fail allocating 2GB when the available memory in the container is 750MB? 

- One might ask why using MALLOC and not ALLOCATE! :) 
  Good question!
  First, will ALLOCATE solve this issue, i.e. does it detect limited memory enforced by CGroups? 
  Second, how can I replace this code with ALLOCATE and still keep a pointer of the allocated data (which is used in other places in the code)? 

Thanks!

---------------------

INTEGER*4 NELEM
REAL*8 DATA(*)
POINTER (P, DATA)

NELEM = 262593830
SIZEINBYTES = NELEM*8
P = MALLOC( SIZEINBYTES)

IF ( P .EQ. 0 ) THEN
       print *, 'Memory allocation failed.'
       RETURN
ENDIF
print *, 'Memory allocation was successful.'

DO I=1, NELEM
 DATA(I) = 0.0D+00
ENDDO

[Steve_Lionel]

ALLOCATE uses malloc underneath (on Linux/Mac).

[jimdempseyatthecove · Accepted solution]

ALLOCATE and MALLOC do not acquire physical RAM nor physical page file pages upon invocation. Instead, they obtain Virtual Address space (from heap), and the actual allocation (for the process) occurs on first touch (first read or write of address within page) of the virtual pages within the Virtual Address space of the process from process start. A SegFault can occur on a first touch that fails to obtain a page file page or a page from physical RAM.

Jim Dempsey

[Hashemi__Ali]

Thank you Steve and Jim. Spot on, Jim!
(For future visitors of this thread, following Jim's answer, I found this page useful too https://www.etalabs.net/overcommit.html)

[MWind2]

https://serverfault.com/questions/141988/avoid-linux-out-of-memory-application-teardown

[jimdempseyatthecove]

Additional information.

A page from physical memory may (and often does) involve ejecting a page from a process (any process including your process, provided the page is not in the non-paged pool or not otherwise locked).

IMHO Windows implementation of paging needs a lot of improvement. As it stands now, a user application, with an appetite for memory can bring the system to a near lockup. I've experienced cases where it takes on the order of 30 minutes to bring up the Task Manager, and then several minutes more to kill the overzealous process.

Also on Linux (reference to #5 link), there are times that you want/need an application to consume more virtual memory than physical memory. In this case, you need to condition OOM Killer appropriately (as well as overcommit).

Comment 41 in #5 link "By default Linux has a somewhat brain-damaged concept of memory management..." it is not brain-damaged, it is simply the default behavior. In many cases it is necessary (see the replies to #41)

Jim Dempsey