Intel® Fortran Compiler
Build applications that can scale for the future with optimized code designed for Intel® Xeon® and compatible processors.
Announcements
This community is designed for sharing of public information. Please do not share Intel or third-party confidential information here.

Integer overflow on input

siyuliben
Beginner
558 Views

Hi, I have a source code that needs to open input files many times.

I used 'OPEN (NEWUNIT=unit31,FILE=cfilename,STATUS='old',iostat=io_status,ERR=900)' to open the files, and it is in a loop similar to the code below:

 

DO i = 1, 99999

     OPEN (NEWUNIT=unit31,FILE=cfilename,STATUS='old',iostat=io_status,ERR=900)

     READ (unit31,*)

     CLOSE(unit31)

 

When I run the compiled source code, it stopped when i = 1016.

The iostat I printed out equals 30 (io_status = 30). I searched online it said '30' means 'integer overflow on input'.

I also printed unit31, it equals 1 (unit31 = 1) when the error shows.

Does anyone know how to solve this problem?

Thank you very much.

Siyu

 

0 Kudos
16 Replies
Steve_Lionel
Black Belt Retired Employee
535 Views

An IOSTAT of 30 means "Open Failure".  Assuming you are on Windows, when you close a file, Windows doesn't release its resources right away, and sometimes you can run out of resources. 

My first suggestion would be to use REWIND rather than repeatedly closing and opening the file. What is the reason for doing that? There may be a better solution.

Another thing you can do is introduce a small delay - a half second usually works - after the CLOSE. Use "CALL SLEEPQQ(500)", with a "USE IFPORT" at the top of the routine to provide the declaration of SLEEPQQ.

siyuliben
Beginner
526 Views

Hi Steve,

Thank you very much for your reply. I run the program on a Linux system. 

The source code I am running is a complex program. It has many parts after the file is closed. when the loop comes back to the open file, it took some time already. I am not sure if REWIND would work since there are other parts outside of the loop using the files.

I tested the program with "CALL SLEEPQQ(500)", it still showed the same error in the same timestep of the loop.

Everything in the loop before i = 1016 went through. The error always came out when i = 1016. I am not sure what so special about this timestep. It is reading the same file.

The only thing I found different is the i/o unit. I printed out all 'unit31' before i = 1016. Their ranges were from -140 to -13335. Then when i = 1016. 'unit31' equals '0' or '1' (It equals 1 only once, now it is always 0 when the error comes out). Did the problem occur because the number changed from negative to positive?

I also know that if I change 'NEWUNIT=unit31' to '31' (assign an integer to the i/o unit directly). The program does not have the problem anymore. However, I need to use 'NEWUNIT' for my case.

Steve_Lionel
Black Belt Retired Employee
521 Views

I missed that you were using NEWUNIT. There may be a limit on how many unit numbers you can get this way. Can you just use NEWUNIT once, save the value, and use that in the future? Or are other parts of the program using NEWUNIT? I don't know how the Intel library implements NEWUNIT - I'd hope that it would pick from the set of unopened units rather than just starting with a pool and drawing from them.

siyuliben
Beginner
511 Views

Thank you Steve,

I will need to use NEWUNIT for parallel computing in the future. So I cannot save the value, I need the program to create i/o units itselve.

Do you know anyone who might be able to help me?

FortranFan
Honored Contributor II
491 Views
@siyuiliben wrote:
.. when i = 1016. 'unit31' equals '0' or '1' (It equals 1 only once, now it is always 0 when the error comes out). Did the problem occur because the number changed from negative to positive? ..

@siyuliben ,

This looks like a bug in Intel Fortran.  The Fortran standard requires NEWUNIT to be a negative number.  If you are able to able to, submit a support incident at Intel OSC and request Intel staff to review.

https://supporttickets.intel.com/servicecenter?lang=en-US

siyuliben
Beginner
462 Views

Thank you, I just sent a request. If you know anyone who can help me. Please let me know.

jimdempseyatthecove
Black Belt
446 Views

>>If you know anyone who can help me.

This is a hack, but should work...

Create threadprivate variables: unit1, unit2,...,unit31,...

Initialize them with unique unit numbers. You can do this in a critical section .OR. use an atomic capture.

.OR. initialize with say: 6+omp_get_thread_num()*numberOfUnits+yourUnitNumber

Then use these threadprivate variables in your open/read/write/close

 

Jim Dempsey

siyuliben
Beginner
443 Views

Thank you Jim,

So the program won't work without parallel computing?

Siyu

jimdempseyatthecove
Black Belt
415 Views

Threadprivate variables work just fine in a single threaded program. Depending on implementation, on Intel/AMD platforms, this involves changing the segment register indication in the instruction from DS to either FS or GS. This is done by including a segment override prefix to the instruction.

Jim Dempsey

siyuliben
Beginner
399 Views

Thank you very much Jim,

I will try your suggestion to see if this will work.

Best 

Siyu

Steve_Lionel
Black Belt Retired Employee
452 Views

I would not be so quick to call it a bug. If the OPEN fails, the value of the variable specified for NEWUNIT is undefined.

I did a test on Windows, and could open and close 5000 units, using NEWUNIT each time, without a problem. While getting a new number each time bothers me, in practical terms it shouldn't be an issue and it saves a lot of cycles otherwise spent in hunting for an unopened unit. I'll guess here that, on Linux, some resource is not getting released when the unit is closed.  I do think Intel should look into it.

siyuliben
Beginner
448 Views

I will post what Intel says here, if they reply me.

Arjen_Markus
Valued Contributor III
379 Views

Out of curiosity I tried opening and closing ten thousand (temporary) files on Linux (using Intel Fortran 19) and that worked without any run-time problem.

siyuliben
Beginner
301 Views

I am not sure which version of Intel Fortran my school has. I will ask about that.

My program is opening other files too. The ranges for the file that has problems were from -140 to -13335. So I believe the program opened more than ten thousand files (using NEWUNIT).

The program works fine if I assign the i/o units when opening files.

Steve_Lionel
Black Belt Retired Employee
420 Views

You say you want to use parallel computing in the future, with each thread wanting its own unit to open and close a file. Is there a need to have a given thread use a new unit every time it does the open? Why not just use NEWUNIT once, save that in a local variable, and then use that number in the thread?

siyuliben
Beginner
400 Views

Hi Steve,

I think I know what you mean. Please excuse me, I didn't understand what you mean very well in the beginning. I will try your suggestion.

Best

Siyu

Reply