Intel® Fortran Compiler
Build applications that can scale for the future with optimized code designed for Intel® Xeon® and compatible processors.

Integer overflow on input

siyuliben
Beginner
1,260 Views

Hi, I have a source code that needs to open input files many times.

I used 'OPEN (NEWUNIT=unit31,FILE=cfilename,STATUS='old',iostat=io_status,ERR=900)' to open the files, and it is in a loop similar to the code below:

 

DO i = 1, 99999

     OPEN (NEWUNIT=unit31,FILE=cfilename,STATUS='old',iostat=io_status,ERR=900)

     READ (unit31,*)

     CLOSE(unit31)

 

When I run the compiled source code, it stopped when i = 1016.

The iostat I printed out equals 30 (io_status = 30). I searched online it said '30' means 'integer overflow on input'.

I also printed unit31, it equals 1 (unit31 = 1) when the error shows.

Does anyone know how to solve this problem?

Thank you very much.

Siyu

 

0 Kudos
16 Replies
Steve_Lionel
Honored Contributor III
1,237 Views

An IOSTAT of 30 means "Open Failure".  Assuming you are on Windows, when you close a file, Windows doesn't release its resources right away, and sometimes you can run out of resources. 

My first suggestion would be to use REWIND rather than repeatedly closing and opening the file. What is the reason for doing that? There may be a better solution.

Another thing you can do is introduce a small delay - a half second usually works - after the CLOSE. Use "CALL SLEEPQQ(500)", with a "USE IFPORT" at the top of the routine to provide the declaration of SLEEPQQ.

0 Kudos
siyuliben
Beginner
1,228 Views

Hi Steve,

Thank you very much for your reply. I run the program on a Linux system. 

The source code I am running is a complex program. It has many parts after the file is closed. when the loop comes back to the open file, it took some time already. I am not sure if REWIND would work since there are other parts outside of the loop using the files.

I tested the program with "CALL SLEEPQQ(500)", it still showed the same error in the same timestep of the loop.

Everything in the loop before i = 1016 went through. The error always came out when i = 1016. I am not sure what so special about this timestep. It is reading the same file.

The only thing I found different is the i/o unit. I printed out all 'unit31' before i = 1016. Their ranges were from -140 to -13335. Then when i = 1016. 'unit31' equals '0' or '1' (It equals 1 only once, now it is always 0 when the error comes out). Did the problem occur because the number changed from negative to positive?

I also know that if I change 'NEWUNIT=unit31' to '31' (assign an integer to the i/o unit directly). The program does not have the problem anymore. However, I need to use 'NEWUNIT' for my case.

0 Kudos
Steve_Lionel
Honored Contributor III
1,223 Views

I missed that you were using NEWUNIT. There may be a limit on how many unit numbers you can get this way. Can you just use NEWUNIT once, save the value, and use that in the future? Or are other parts of the program using NEWUNIT? I don't know how the Intel library implements NEWUNIT - I'd hope that it would pick from the set of unopened units rather than just starting with a pool and drawing from them.

0 Kudos
siyuliben
Beginner
1,213 Views

Thank you Steve,

I will need to use NEWUNIT for parallel computing in the future. So I cannot save the value, I need the program to create i/o units itselve.

Do you know anyone who might be able to help me?

0 Kudos
FortranFan
Honored Contributor II
1,193 Views
@siyuiliben wrote:
.. when i = 1016. 'unit31' equals '0' or '1' (It equals 1 only once, now it is always 0 when the error comes out). Did the problem occur because the number changed from negative to positive? ..

@siyuliben ,

This looks like a bug in Intel Fortran.  The Fortran standard requires NEWUNIT to be a negative number.  If you are able to able to, submit a support incident at Intel OSC and request Intel staff to review.

https://supporttickets.intel.com/servicecenter?lang=en-US

0 Kudos
siyuliben
Beginner
1,164 Views

Thank you, I just sent a request. If you know anyone who can help me. Please let me know.

0 Kudos
jimdempseyatthecove
Honored Contributor III
1,148 Views

>>If you know anyone who can help me.

This is a hack, but should work...

Create threadprivate variables: unit1, unit2,...,unit31,...

Initialize them with unique unit numbers. You can do this in a critical section .OR. use an atomic capture.

.OR. initialize with say: 6+omp_get_thread_num()*numberOfUnits+yourUnitNumber

Then use these threadprivate variables in your open/read/write/close

 

Jim Dempsey

0 Kudos
siyuliben
Beginner
1,145 Views

Thank you Jim,

So the program won't work without parallel computing?

Siyu

0 Kudos
jimdempseyatthecove
Honored Contributor III
1,117 Views

Threadprivate variables work just fine in a single threaded program. Depending on implementation, on Intel/AMD platforms, this involves changing the segment register indication in the instruction from DS to either FS or GS. This is done by including a segment override prefix to the instruction.

Jim Dempsey

0 Kudos
siyuliben
Beginner
1,101 Views

Thank you very much Jim,

I will try your suggestion to see if this will work.

Best 

Siyu

0 Kudos
Steve_Lionel
Honored Contributor III
1,154 Views

I would not be so quick to call it a bug. If the OPEN fails, the value of the variable specified for NEWUNIT is undefined.

I did a test on Windows, and could open and close 5000 units, using NEWUNIT each time, without a problem. While getting a new number each time bothers me, in practical terms it shouldn't be an issue and it saves a lot of cycles otherwise spent in hunting for an unopened unit. I'll guess here that, on Linux, some resource is not getting released when the unit is closed.  I do think Intel should look into it.

0 Kudos
siyuliben
Beginner
1,150 Views

I will post what Intel says here, if they reply me.

0 Kudos
Arjen_Markus
Honored Contributor I
1,081 Views

Out of curiosity I tried opening and closing ten thousand (temporary) files on Linux (using Intel Fortran 19) and that worked without any run-time problem.

0 Kudos
siyuliben
Beginner
1,003 Views

I am not sure which version of Intel Fortran my school has. I will ask about that.

My program is opening other files too. The ranges for the file that has problems were from -140 to -13335. So I believe the program opened more than ten thousand files (using NEWUNIT).

The program works fine if I assign the i/o units when opening files.

0 Kudos
Steve_Lionel
Honored Contributor III
1,122 Views

You say you want to use parallel computing in the future, with each thread wanting its own unit to open and close a file. Is there a need to have a given thread use a new unit every time it does the open? Why not just use NEWUNIT once, save that in a local variable, and then use that number in the thread?

0 Kudos
siyuliben
Beginner
1,102 Views

Hi Steve,

I think I know what you mean. Please excuse me, I didn't understand what you mean very well in the beginning. I will try your suggestion.

Best

Siyu

0 Kudos
Reply