ID specifier in asynchronous I/O

[max_w_nimitz]

It is said that the value set for the ID= specifier by an asynchronous READ or WRITE statement must be the same value specified in the ID= specifier in the corresponding WAIT statement.
However, when I tried to print out the value of ID, it just remains constant for a long time and then increase by 1, then go on and on like this, its value starts from 0 and ends at 253, which is just the length of a (kind = 4) integer. Meanwhile, my loop of read has reached the number of 32753. I'm just wondering how the ID specifier really works, it seems that the variable gets the value from ID specifier while applying asynchronous I/O, then the value is used for wait statement to distinguish which I/O segment it's waiting, if this is the case, how come its value would remain constant for couple hundreds of read loops while i'm actually reading different chunks of data from the external file, how would wait statement distinguish between this different read?

[Steven_L_Intel1]

As I am not familiar with the implementation details of this, I have asked the engineer who implemented it to respond. When I hear from him, which may be a few days, I'll let you know.

[jimdempseyatthecove]

Steve,

I believe this is a documentation error. It appears that the definition (paragraph) describing id-var in WAIT is used as the definition of id-var in READ/WRITE. This is a case of blind use of cut and paste.

Using the definition (of the action imposed on the statement) for WAIT in the READ/WRITE incorrectly states a blocking action for the READ/WRITE. This is entirely contrary to the intent of the request (i.e. you intend to perform a non-blocking read/write).

Jim Dempsey

[jimdempseyatthecove]

Max,

The manner in which ID= is used is left to the implementation.

It would appear from your description that (as implemented in IVF of the version your have) the return value in the READ/WRITE is a "handle" to a completion status structure. In this case the "handle" appears to be an index into a table (array) of these structures or to an array of pointers to these structures. You are likely seeing the value remain at 0 because the I/O in your test program completes and WAIT with corresponding ID executes (and releases completion structure) prior to the issue of the subsequent asynchronous I/O. This causes the 0'th entry to get freed then re-used on the next I/O. When you did notice it going to 1 it would appear that you actually did have overlapped I/O. (all assumptions on my part)

Do not depend on the value to be meaningful, nor depend on the behavior to be consistent. The observed behavior not can change at the whim of the programmer.

Jim Dempsey

[jimdempseyatthecove]

Max,

I took a look at the F2003 specification

http://std.dkuug.dk/jtc1/sc22/open/n3661.pdf

You should note that you may use asychronous open/read/write but it is up to the implementation to perform asychronous or synchronous I/O.

Further, if asychronous I/O is supported, it is a requirement of the F2003 that the I/O be performed in the order in which the transfer (READ/WRITE) were issued. i.e. although the I/O may be asychronous it is required to be performed in a FIFO manner. There was no mention in the specification as to how (where) the implementation was to determination of which statement is first when competing threads issue (near) simulteaneous requests.

Due to the FIFO nature you could keep track of progress using a rollover counter or up/down counter/counters (with appropriate care for $ATOMIC or _Interlocked... usage). One tick on READ/WRITE, one tick on WAIT (plus possible additional code for exception handling).

Jim Dempsey

[Steven_L_Intel1]

Jim,

Thanks for pointing out the documentation error. I'll pass that along to our writers.

It is true that the contents of the ID variable are not intended to be examined by the user, but it is an interesting question nonetheless. If one wants more than one I/O "in flight", you need to use separate variables. I expect to get an explanation of our implementation sometime next week.

[jimdempseyatthecove]

>> If one wants more than one I/O "in flight", you need to use separate variables.

Yes, that is critical. The likely procedure to follow would be to perform the I/O using a local variable, test for error status, if error process accordingly, if no error, obtain a node or entry in a shared queue and store the contents of the returned ID variable plus any other stuff you though pertinent to pending i/o requests. Then presumable later or with a different thread dequeue (from shared queue) the next pending request and perform the WAIT or Inquiry or terminate or other operation on pending I/O. This would be done should you want many pending requests at a time.

A simplified method though would be for use in a double buffer i/o (or small number of buffered i/o). This could easily be used in a single threaded application as well as in a multi-threaded application.

Jim Dempsey

[Steven_L_Intel1]

What I had meant to say was different ID variable,s but that is incorrect. You can use the same ID variable for all the operations as long as you don't care about the status of any but the last.

What Jim says refers to the I/O list. There it is critical that you not reuse a variable being read or written until a previous operation with that variable completes.

As for ID, I heard back from the developer, but there's a part of his response I didn't understand so I need more clarification.

[jimdempseyatthecove]

Steve,

>>What Jim says refers to the I/O list. There it is critical that you not reuse a variable being read or written until a previous operation with that variable completes.

I meant what I said.

For you to have multiple concurrent I/O operations pending for the same unit, the returned IDs have to be different else WAIT would not be able to disambiguate the I/O completion operations.

Of course, you should not post multiple pending reads intothe samebuffer (if you intend to use the returned data). You might concievably queue multiple reads into the same buffer without regard to returned data if you were performing a performance test or if you wanted to discard data.

Jim Dempsey

[Steven_L_Intel1]

It matters only if you need to wait for a specific operation. If all you care about is that all the writes so far complete, you can wait for the last one.

[jimdempseyatthecove]

Steve,

If you want to wait for all pending I/O on a unit then

WAIT(UNIT=n)

If you want to wait until a specific I/O on a unit is complete

WAIT(UNIT=n, ID=SpecificID)

Where SpecificID was obtained on the asychronous write (for the specific I/O on which you desire to wait).

And yes you would likely want to add one or more of the other return arguments, in particular IOSTAT=iErr.

A lot will depend on how good the implementors wrote the code. The spec. says the asychronous writes occure in order issued (FIFO). I assume that they then complete in order issued. However the user might issue the WAIT on ID in a different order than read (e.g. LIFO). But this would not releave the user from issuing the WAIT for the other pending I/O (previously completed).

Jim

[Steven_L_Intel1]

The ID is an index into an array kept by the run-time library of asynchronous I/O requests, per unit. I was told that it is possible for IDs to get reused when the operation has completed, but in my testing I could not get it to do so. The ID incremented steadily from 1 and up without repeats.