Tabs versus spaces

[Sudarshan_R_]

Hello,

I am working towards recompiling an old piece of code on a 32 bit platform. The old code was compiled on MS Fortran 5.1 and produced a 16 bit application. 

The input file for the application has formatted data. I saw that the old 16 bit application was not sensitive to tabs in the input file and probably just processed them as blank spaces. However, on recompiling the code to produce a 32 bit application, i get an input conversion error whenever a tab is encountered. If i replace the tab with a space, the program works fine. 

Why does this happen? Is there a way to just leave the tabs in the input file and make the application read them as blank spaces instead.

Regards,

Sudarshan

[Steven_L_Intel1]

I can't reproduce a problem based on your description. During formatted input, our implementation treats a tab as if it were a blank. Can you attach a ZIP of a test program and data file that shows the problem?

[andrew_4619]

I think things would be clearer if you added a sample line of fortran code showing how the data is read and what formatting is used.I presume it must be a text file you are reading? In a text file a tab is just a single character, it may be that if you set "use powerstation i/o format" option which was the successor to MS-Fortran it might work but I is probably better to just fix the code to work in a more standard way.

[Sudarshan_R_]

I will do that once I get back to office. Merry christmas :)

[Sudarshan_R_]

Hello,

I am attaching a zip containing TEST.FOR with sample code and a sample input file, 1.TXT. I noticed that when you insert tabs in certain places, it results in an input conversion error (for example 6th col, 7th col, 39th col etc) and in others the read is completed successfully (eg- 11th col, 12th col etc). This does not happen in the old 16 bit program. It looks like it reads tabs as though they were blank spaces. Can I make my program behave like the old version?

On a side note, if you build the test program, you would notice that the values read are not exactly equal to the values given. For example, 0.00020 and 0.0070 are read as 1.9999999E-04 and 7.0000002E-03 respectively. The program I have built produces results that differ in the 3rd decimal or on a relative magnitude of 10^-5 or 10^-6. I went through other posts on this issue and consequently used the /fp:Source floating point model. Yet the same differences persist. Is there any other way to exactly reproduce the results given by the old program?

Regards,

Sudarshan

[Sudarshan_R_]

Also, I forgot to mention that the tabs vs spaces problem persists even after enabling all the powerstation compatibility options.

[andrew_4619]

Many numbers cannot to represented in binary exactly, as you show in your examples, This was no different in MS Fortran. If you output the numbers to 5 decimal places in Fortran e,g with format F8.5  you will see exactly the same output as before.

You need to study the meaning of the Fortran format statement. If you are going to used fixed format reads(I5 grabs 5 characters and expects and integer), F10 grabs 10 characters and expects a real number)  then the data needs to conform to that format. Your program is reliant on a flavour of non-standard Fortran i/o from a compiler from many years ago. You need to analyse your input files format and use a standard and reliable method  of reading them. As Steve says, Ifort treats a tab as a single blank space.

[Steven_L_Intel1]

I don't see any evidence of a problem in the files you attached. Please attach a file with tabs that triggers an error. Might it be that the text editor you're using is replacing tabs with multiple blanks? That could cause a problem.

A couple of observations not directly relevant. First, the record in the file is shorter than what the format requires. In this case it doesn't cause problems, but I could see it perhaps doing so in other cases. Second, the I5 fields will be sensitive to the treatment of blanks if there are some after the digits. In old Fortrans, these were typically treated as if they were 0, so if you read the field "1 " with I2 you'd get 10. The current Fortran standard specifies that blanks are ignored, or "NULL" in the standard speak, so you'd get 1.

[John_Campbell]

You need to understand how the Fortran Standard indicates how tab characters are interpreted in a fortran read statement.
I think the answer is tabs are treated as spaces. How many spaces per tab character varies between Fortran compilers.

You might want to consider using a comma, insetad of a tab, as this is defined in the standard. The attached adaptation of your program might show some alternatives to consider.
For formatted text input, the comma is treated as a number field terminator.

You also might want to consider how "READ(5,*) KD,NG,X,FF,FF1,FIN" might work, as with a * format, blanks are used as a numeric field terminator, although all 6 number fields must be provided.

Tab characters are best avoided, as their screen display in IDE's and Fortran use varies between compilers, while you can use comma and spaces as defined in the standard.

John

[Steven_L_Intel1]

John, this program uses an explicit format, not list-directed. Tab is not addressed in the Fortran standard at all - it's not even part of the "Fortran character set". How an implementation chooses to interpret tabs is not specified by the standard, and there are differences. That said, in formatted input, tabs are almost always treated as if they were blanks.

[John_Campbell]

Steve,

I have seen tabs treated as a single blank, multiple blanks or a field delimiter with different Fortran compilers. Tabs are not described in the Fortran standard and should be avoided in data input with an explicit format.

I would recommend that Sudarshan consider using a comma in his data files as a number separator, as their use is described in the standard and provide some flexibility in data input.

John

[jimdempseyatthecove]

FWIW I've experienced IDE editors and other text editors converting tabs to spaces thus corrupting fixed format source files as well as formatted data files. Sometimes the conversion for multi-space to tab occurs too. When migrating old programs and data to newer systems you should be mindful of the space/tab issue and have in place a means to detect substitution errors. With WinDIff you can turn off Ignore Blanks and turn on Show Whitespace.

Jim Dempsey

[John_Campbell]

Jim,

Use of tabs in Fortran code is not a good idea at all. If you change the IDE you are using, it is likely the tab settings will also change. You only have to open with Notepad or Wordpad, code that uses tab characters and see how the layout becomes all messed up. It would be good to have an IDE option that colours any control characters as a big black mark, to discourage their use. Writing code that suits only one IDE is not very portable.There are quite a few code examples submitted to this forum that use tab characters for layout. Visual Studio must not discourage the use of tab characters.

As Steve points out, the tab character is not a valid character in the Fortran standard, so it's use in code is a compiler extension.

John

[andrew_4619]

Tab chars in code get a warning if /stand:fxx is used.

I have gone off comma separated value output a little after recent problems with Excel where the windows default separator is not defined as comma. This can occur in countries where a comma is used as a decimal separator. When writing a csv from Fortran  for use in Excel putting "SEP=," on line one (or whatever separation character you have used e.g SEP=; ) as this makes it more portable.

[Sudarshan_R_]

Hello,

Thanks for the insightful replies. I have a few limitations. My purpose is to basically just recompile the old program such that it is compatible with windows 7. That essentially just requires me to develop a 32 bit application of the old code. There are some standard input files which are modified and used for different test cases. On a broad level, I don't think changing the format of input would be acceptable. Hence, the explicitly formatted input is something that would have to be there. Furthermore, quantities that are left unspecified are supplied default values in the code. 

People might be receptive to an advisory of replacing tabs with spaces, but an idea of changing the input format would surely be shot down.

And also tabs make the layout look messed up only when they are positioned in between numbers. When they are placed at the end of lines, they often go unnoticed. 

As far as the test program goes, the attached text file gives me an input conversion error. Such an input line does not give any problem in the older 16 bit program. I am currently using ivf 10.1 

I would like to know if there is a way or any compiler options that allow me to have tabs in the file and still not give me any error? Or is it just best to issue an advisory against the usage of tabs in the input file? 

Regards,

Sudarshan

[Steven_L_Intel1]

The problem is that your file with tabs ADDED a tab to the existing blanks, it didn't replace a blank with a tab. This changes the placement of some of the values and, since you are using an explicit format, the values are in the wrong columns. This has nothing to do with blanks vs. tabs, you'd get the same problem if you added a blank. I have a difficult time believing that an older Fortran somehow ignored this error, though it's possible it simply read in the wrong values and you didn't notice.

When you use an explicit format, the character positions within the record of values is critical.

[Sudarshan_R_]

My mistake, that file is obviously wrong. Sorry about that. But the file I am attaching here also gives the same error.

[Steven_L_Intel1]

Thank you! I can now see the behavior you are describing. I had thought that we treated tabs as blanks in formatted input, but it seems we don't and I don't see anything in our documentation to suggest that we should. As discussed earlier, tabs don't belong in numeric input fields and there is no option I can find that would allow them.

[Sudarshan_R_]

Thanks for the clarity, Steve! I guess that settles the issue then :)