Question: Reading a large column (1D-array) and writing slices of this array as rows

[bhvj]

I am trying to read a large single column (which I am trying to read in as a 1-D array), and then to write slices of this array as rows until the end of the data. Please suggest me the best way to accomplish this task. Thanks in advance.

[mecej4]

A 1-D array is a 1-D array, i.e, there is no distinction between a 'column' and 'row', concepts that make sense only in a 2-D plane or the nearly flat surface of the Earth. 

To output a 2-D array in row-major format, you can use implied do lists or array slices. For example, 

REAL A(10,15)
...
READ (*,*) A ! read A by columns, i.e, Col-1 followed by Col-2 ...
...
WRITE(*,'(15E12.3)') ((A(i,j),j=1,15),i=1,10)
...

[bhvj]

Thank you for your response and the sample code for converting a 2-D array in row format. As I understand, in order to accomplish my task of  converting single column into rows, I cannot read it is an array. Can you please suggest me the best way to accomplish this?

Thank you again for your response.

[mecej4]

You can read numbers into a 2-D Fortran array in any order that you please, subject to the rules of the language. However, accessing memory with a stride other than the unit stride for the data type can result in poor performance.

If you will describe your situation in a bit more detail I could offer a more useful suggestion, but here is what I can say based on what you have written. If you wish to store a matrix on a storage device and read that matrix into a Fortran 2D array, that can be done efficiently if the storage contains the matrix of size m X n in the following order: (1,1), (2,1), ..., (m,1),    (1,2), (2,2), ..., (m,2),   ......,(1,n), (2,n), ..., (m,n), which is what is commonly named "column-major format" or "by columns".

You may consider whether to change the order in which the matrix is stored on mass storage (disk drive, SSD, USB drive, etc.) to make the Fortran program more efficient, if the matrix is not stored in the order that I described in the preceding paragraph. For instance, if the array is going to be read many times by the Fortran program, write a utility program to read the data file and rewrite the matrix in column-major order, and read the new file by columns in your Fortran program. 

[bhvj]

Thank you very much for your valuable and quick response. Just to let you know my situation in more detail:

I have 14975 values in a single column, which I am trying to read in and then write, in the  same order, as rows (each row containing 12 values) until the last value (it is ok if the last row has only a few values than 12). This read and write might have to be done several times by the FORTRAN program. Attached herewith are screenshots of the part-data in column and the part-data in row. Please let me know if you have further suggestions.

Thank you again for your valuable suggestions.

[jimdempseyatthecove]

Does the black text of rf_col.png represent the contents of the input file (red text is not part of file)?

Does the black text of rf_row.png represent the contents of the output file you desire?

Jim Dempsey

[bhvj]

Jim,

Yes, the black text of rf_col.png represent the contents of the input file and the red text are only row numbers, however, I am trying to read only the last column.

The black text of rf_row.png represent the contents of the desired output file.

[mecej4]

I now see that this is a simple data processing job involving reformatting some tabular data. For this, there is no need to think of matrices and 2D arrays.

The attached zip contains some made up data and a program to do the reformatting. Basically, what the program does is to read lines until either (i) the first column value (year?) changes or twelve values have been stored. In either of these cases, the appropriate output is done, counters reset, and the cycle is continued until the input file is exhausted.

Please test the program on your data, or attach your data file in your reply.

When you post program source or data, please do not use image formats such as PNG, but use plain text files, zipped up if appropriate.

[bhvj]

Thanks again for your response and suggestions. Attached herewith is the zip file (containing the source used, input, the desired output and error), with which I tested the program. I am using the intel64 FORTRAN compiler, not sure if the syntax errors are due to the differences between f90 and intel64.

Thank you again for your help.

[mecej4]

There was no need for you to convert the program code from free format to fixed format but, you did, and did not format the line with the label 100 correctly. Labels have to be in columns 1 to 5, and the statement must begin on or after column 7. Make that change, and the program will run.

[bhvj]

I appreciate it very much for all your help. The program runs to produce the desired output. I have the following questions, if I may ask: Can you please explain the purpose of the step: "flush stored values out". Is it to clear out the stored values in the memory? And the way to do that is to write an array which has null values in it, is that right (the way it is done here: "write (*,'(12F7.2)')v(1:nval)", where "nval"=1)?

Also, what do the terms free format & fixed format mean?

Thanks again for all your help.

[mecej4]

There are many ways to write a program to perform the desired actions. The version that I gave provides a small amount of memory to hold just one line of output. Input records are read and the information from the last column is stored into consecutive locations in that block of memory until all 12 locations are full. Then, the memory is "flushed" (only figuratively) by writing the 12 (or fewer, if appropriate) values to the output medium and resetting nval to 0.

The WRITE statement with I/O list v(1:nval) contains nval elements, when nval is > 0. If nval is zero, the list has no elements, but a WRITE statement will still output a blank line. If that is not desired, you can change the WRITE statement to "if (nval > 0) WRITE ...".

Fortran source code files may be in free form or fixed form. Fixed form is obsolete and can be troublesome, but many old codes are still in that form. See https://software.intel.com/en-us/blogs/2013/01/11/doctor-fortran-in-source-form-just-wants-to-be-free for details. Decades ago, one wrote Fortran programs on coding forms, which were sheets of paper with lines and columns printed in light color; for example, see this link .

[bhvj]

Thank you for the detailed explanation, and thanks again for your help.

[bhvj]

I have another follow-up question regarding reading and reformatting the data: If the number of data files to be read are large (for example, if there are 60 files), in FORTRAN is there an efficient way to do this, rather than numbering (or re-naming the files with suffixes as 1 to 60) and using "OPEN", with each specific file name in the program. (I would also want to write to 60 different output files). I will try to pick up from any expert suggestions you might have. Thank you very much once again.

[mecej4]

This is a frequently asked question. The attached program attempts to open files 'load10', 'load11',..., 'load15'. As you can see, only files 'load11' and 'load13' exist, and the program prints a message to indicate whether a file was opened or could not be found.

[bhvj]

So, just like the files 'load10', 'load11',..., 'load15' are opened using a do-loop, these files with suffixes 10 to 15 could read using a do-loop and also the corresponding output files with suffixes 10 to 15 could be written using a do-loop, is that correct?

This time I tried to compile the .f90 file without changing the format using  the intel-64 bit compiler, it did compile. But when the file with ".f90" extension was copied into a file with ".f" extension, it didn't get compiled, it gave errors. In order to get it to compile the file with ".f" extension, I had to change it to fixed format. What do the FORTRAN files with different extensions, such as ".f90" and ".f" mean?

Also, I have the following question regarding the executable: It gives a default name to the executable as "a.out", is there a way to give a specific name to the executable, with a ".exe" extension?

Thank you very much for all your valuable responses.

[mecej4]

Yes, you can open files for output in a loop in the same way that the sample program opened files for input. You have to change the value supplied with the STATUS= keyword option to 'NEW' or 'REPLACE', as appropriate.

Files with .f suffixes are taken as being in fixed format. Files with .f90 suffixes are taken to be in free format -- nothing is implied regarding which Fortran standard the contents of the file follow. Before Fortran 90, free format did not exist in public. You can write Fortran 2003 programs in fixed format in a file with the .f suffix, and you can write a Fortran II program in free format in a file with the suffix .f90 (leaving out some features of Fortran II that are 'deleted' in Fortran 90 and later). Some compilers accept suffixes such .f95, etc., but such suffixes are deprecated.

You should not name a file containing free format source a .f suffix, nor should you name a fixed format source file with the .f90 suffix. The compiler may provide options to override the implied file type based on the suffix, but at this stage you have no need for such features.

To give the executable a name other than the default 'a.out', use the -o option in the command line used for linking (or compiling and linking):

$ ifort file1.f file2.f90 ... -o myprog.exe

[bhvj]

Thank you very much once again. I have the following  questions regarding the sample program:

What does "intent(in)" mean in the variable declaration?

What does "I0" format mean when "fileno" is written to "filesno"?

Thanks again.

[mecej4]

For answers to these questions please consult the IFort manual (as installed on your machine, or at https://software.intel.com/en-us/compiler_15.0_ug_f ). The manual has a "search" feature which enables locating the section relevant to a question.

[bhvj]

Thank you very much once again.

[bhvj]

I need help with the following (Thank you very much in advance for any help/suggestions in this regard):

1. What would be the command to create a debug executable, so that it can be used in an intel debugger for debugging a code?

2. As can be seen in the attached program and files, I am trying to open 10 input files using a do-loop and then read them, followed by writing the corresponding 10 output files using a do-loop. The program appears to only open the 10 input files, but doesn't appear to go further. Is there a way to make sure that an input file with a particular unit number is being opened (rather than using "read (*,*)" ), read and also the corresponding output file opened and written.

Thank you very much in advance for any help/suggestions in this regard.