COArray synchronization

[abhimodak]

In the program below, I am using synchronization as well as flush. Even then I sometimes get (by doing repeated runs) the echo of individual image AFTER the total number of images. Am I missing something or this is a compiler problem?

Abhi

--

[fortran]      Program Sample_COArray
   
         Use, Intrinsic :: ISO_Fortran_Env
         
         Implicit None
         
         Integer :: J

         Integer :: myImageNum
         
         Integer, Parameter :: MasterImage = 1        
         
         Write(OUTPUT_UNIT,"(A,I0)") "I am Image Number: ", THIS_IMAGE()
         Flush(OUTPUT_UNIT)
         SYNC ALL
         
         if (THIS_IMAGE() == MasterImage) then
            Write(OUTPUT_UNIT,"(A,I0)") "Number of Images: ", NUM_IMAGES()         
         endif
         
      End Program Sample_COArray[/fortran]

[jimdempseyatthecove]

This might be a documented behavior. In COArray environment the seperate images are typically run on seperate systems. Therfore your

Write(OUTPUT_UNIT,"(A,I0)")"IamImageNumber:",THIS_IMAGE()
Flush(OUTPUT_UNIT)

Will not proceed directly to the same file. What is likely happening is the file output (at least by the non-MasterImage) is being managed by a helper thread (on each system). My guess is the message delivery from the non-MasterImage to the MasterImage has not yet occured. Look in the advance API documentation to see if there is a file (message) sync function.

Jim Dempsey

[Steven_L_Intel1]

The FLUSH should not be necessary. This should "just work", and I've never seen a problem like you describe when running coarray apps. The SYNC ALL should cause all images to wait until they all get there.

I tried the program on my Nehalem system (8 threads) and always got the "number of images" print at the end. Can you show me the output of several runs showing the difference? I assume you are using the default shared memory environment.

[jimdempseyatthecove]

Steve,

What I was attempting to say (postulate) is:

SYNC ALL

may correctly synchronize the computational threads of each system
but it may be failing to synchronize the additional helper threads used for messaging I/O to the master system.

>>I tried the program on my Nehalem system (8 threads)

What is the user's configuration?
1P, nP SMP, cluster?

Jim Dempsey

[Steven_L_Intel1]

Jim,

There are no helper threads here.

[abhimodak]

Hi Steve

Attached snapshot shows results of two runs.

Abhi

[Steven_L_Intel1]

Well, I can't explain this, but also will comment that the Fortran standard doesn't specify the order in which things come out. It doesn't even require that the output streams be merged, though most implementations do so. Some strange timing thing, I guess.

[abhimodak]

Hi Steve

Did you try this with update 7 or 6? I am still using the latter.

Abhi

[abhimodak]

I see the same behavior with update 7. I am not 100% sure if the output is supposed to have this unpredictable behavior with flush and synchronization. A document by Numrich and Reid (ftp://ftp.nag.co.uk/sc22wg5/N1601-N1650/N1642.pdf page 10 and 11) talks about the required FLUSH statement. Since there is no clear mention/illustration how things should work, have I mis-interpreted it to mean the behavior I am expecting in this case?

Abhi

[Ron_Green]

Abhi,

I was able to reproduce what you are seeing. I could reproduce this on a 6-core Westmere server WITH hyperthreading turned on.

From your sample you are running 4 images. How many PHYSICAL cores do you have - what is your processor?

I suspect this is a bug due to running on hyperthreaded cores. And you are correct, the FLUSH followed by SYNC ALL should flush the IO buffers. So this is a bug, but again, I think it's hyperthreading related.

let me know if hyperthreading is turned on in your system. Also, what OS - Win7, ia32 or x64?

ron

[Steven_L_Intel1]

My opinion is that the behavior you want is not specified by the standard. The FLUSH simply tells the local image to flush its local buffers. What happens after that is unspecified. The order in which the merged output appears is unpredictable. SYNC ALL is properly synchronizing execution - it does not necessarily synchronize output.

[abhimodak]

Hi Ron

My system is:
(a) Win7 (64bit) Professional with Service Pack 1
(b) VS2010 Professional
(c) IVF 12 update 7
(d) Intel Xeon W3565 -- it has 4 cores. Hyperthreading is OFF. That is, if I open task manager I see 4 "cpu windows".

Steve, I think it is a bug in the Intel implementation as Ron has suggested. (Numrich/Reid document section 10 paragraph 4 makes a mention of using both FLUSH and SYNC in casessuch as when order is important or data from one image is to read by other.) But I may be wrong. Since you and Ron have posted two different opinions, I am confused.

Abhi

[Michael_R_Intel4]

Quoting Steve Lionel (Intel)

My opinion is that the behavior you want is not specified by the standard. The FLUSH simply tells the local image to flush its local buffers. What happens after that is unspecified. The order in which the merged output appears is unpredictable. SYNC ALL is properly synchronizing execution - it does not necessarily synchronize output.

Steve is correct - the order of the output is unpredictable, and not defined by the standard.