Need to suppress leading zero's.

[Lowell_S_]

Hey,

I'm migrating FORTRAN programs from HPUX itanium to Redhat Linux and I'm using the Intel ifort compiler.  Unfortunately, real numbers written to the unit 6 output file now have leading zero's that aren't in my HPUX output file.  The HPUX compile used the +io77 option to suppress the leading zero's.  I was unable to find a similar option for the ifort compiler.  This causes a couple problems.  First, my output files are almost 100% different than the HPUX files making validation of the migration process much more difficult.  I now have to exam every line visually vs. using diff.  Second, in many places, output tables only had a single space between columns of data which now contains a leading zero so the data all runs together with no separation between fields.  Obviously, I can spend a lot of time modify every format statement with floating point format descriptors to fix this second problem but I am hoping for a simple compiler fix I may have missed.  Any help for this simple fix would be greatly appreciated.

Sample Linux ifort output with the leading zero's are shown below.

   1.        0.  1.000.10000E+010.50000E-020.00000E+000.00000E+000.42900E+010.86200E+010.10000E+010.10000E+010.10000E+01
   1.   1000.  1.000.10000E+010.50000E-020.00000E+000.00000E+000.42900E+010.86200E+010.10000E+010.10000E+010.10000E+01
   1.   1001.  1.000.42000E+000.33000E-020.00000E+000.00000E+000.34000E+010.66900E+010.10000E+010.10000E+010.10000E+01
   1.   2500.  1.000.42000E+000.33000E-020.00000E+000.00000E+000.34000E+010.66900E+010.10000E+010.10000E+010.10000E+01
   1.   2501.  1.000.30000E+000.20000E-020.00000E+000.00000E+000.34200E+010.81700E+010.10000E+010.10000E+010.10000E+01

What I get from HPUX's f90 compiled executable is,

   1.        0.  1.00 .10000E+01 .50000E-02 .00000E+00 .00000E+00 .42900E+01 .86200E+01 .10000E+01 .10000E+01 .10000E+01
   1.   1000.  1.00 .10000E+01 .50000E-02 .00000E+00 .00000E+00 .42900E+01 .86200E+01 .10000E+01 .10000E+01 .10000E+01
   1.   1001.  1.00 .42000E+00 .33000E-02 .00000E+00 .00000E+00 .34000E+01 .66900E+01 .10000E+01 .10000E+01 .10000E+01
   1.   2500.  1.00 .42000E+00 .33000E-02 .00000E+00 .00000E+00 .34000E+01 .66900E+01 .10000E+01 .10000E+01 .10000E+01
   1.   2501.  1.00 .30000E+00 .20000E-02 .00000E+00 .00000E+00 .34200E+01 .81700E+01 .10000E+01 .10000E+01 .10000E+01

[Steven_L_Intel1]

Sorry, there is no simple fix for this. The standard makes the leading zero optional for the implementation and we chose differently than HP.

[jimdempseyatthecove]

I haven't tried this...

If the file is sequentially written, you might be able to use USEROPEN to open a pipe to another process (which your app launches). The launched app (on other end of pipe) edits the output stream.

Jim Dempsey

[Lowell_S_]

Steve,

Ouch!  This may just kill our decision to use the Intel compiler.  We have 100's of thousands of lines of code we are migrating.  Thanks for the response.

Jim,

I was looking for a simple fix.  If I am forced to go this route (using ifort), I will probably have to do something like you suggest though I probably would just add a reformatting program (which may have to be unique for each app) in the execution script. Thanks for the suggestion.

[mecej4]

Here is a simple fix: apply the following sed script to the output file(s). For example, if the Ifort output file has been put into file x.txt, the following command will produce file y.txt with the desired format:

sed -e 's/\(E...\)0\./\1 ./g' < x.txt > y.txt

Of course, you have to judge whether this fix meets your own criteria for "simple".

For output lines that have a different structure than the sample lines that you posted, you will have to change the script. If your output file contains more than one block with each block having a different structure, fixing the file will need more work.

[Lowell_S_]

mecej4,

I like it!  I might have to tweak it depending on the situation but it should make the migration validation check problem go away and the table readable.  Thanks for your response.

[mecej4]

In similar situations, where I had to compare the formatted output of a program when it is built with a number of competing compilers, I have found the following program very helpful: http://www.math.utah.edu/~beebe/software/ndiff . See also http://www.nongnu.org/numdiff/ .

[jimdempseyatthecove]

This looks like a fixed width formatted output file.

A simple Fortran program could read the lines as characters, then overstrike the 0's in the column gutters.

Jim Dempsey

[Lowell_S_]

Jim,

Yes that would work but has a couple of drawbacks over the sed command solution.  I have many programs that have varied output.  Even the file that contains the example in my original post has other types of tables, headers, and data in it so a reformat program would have to be smart enough to find the offending leading zeros in every case or have a reformat program for each.  Even if the code would be as simple as the sed command idea, it would still require the overhead of developing, testing, and maintaining another program. Each of the programs already has a csh or ksh run script to setup the run, make the run, and post process the run so adding the sed command after the run is the simplest solution so far and doesn't require yet another program or programs to maintain through the years.  btw, most of these programs were developed in the 60's and 70's and have been migrated several times at the whim of the architecture guys who like to keep changing the hardware. The HPUX compiler option saved us in the last migration.  If this sed command works, it will save us in this migration. Though not as nice as a compiler option it does save us from having to write reformatter programs. 

A perfect solution (in my mind) would be for Intel to offer a compiler option to support this in the future. There is an obvious need for it.
 

[Lowell_S_]

I tried the sed command today and it work for the most part except for the first column of numbers with (E...) because the sed command assumes an (E...) occurs to the left of the leading zero.  This is true for all subsequent columns but not the first occurrence.  I'm having trouble figuring out how to write a similar command for the first occurrence that will work in all cases since the number of decimal places that can occur will vary from one format statement to the next. Below is a sample of the output after invoking the sed command.  I might have to go with Jim's idea and write a program after all.

   1.        0.  1.000.10000E+01 .50000E-02 .00000E+00 .00000E+00 .42900E+01 .86200E+01 .10000E+01 .10000E+01 .10000E+01
   1.   1000.  1.000.10000E+01 .50000E-02 .00000E+00 .00000E+00 .42900E+01 .86200E+01 .10000E+01 .10000E+01 .10000E+01
   1.   1001.  1.000.42000E+00 .33000E-02 .00000E+00 .00000E+00 .34000E+01 .66900E+01 .10000E+01 .10000E+01 .10000E+01
   1.   2500.  1.000.42000E+00 .33000E-02 .00000E+00 .00000E+00 .34000E+01 .66900E+01 .10000E+01 .10000E+01 .10000E+01
   1.   2501.  1.000.30000E+00 .20000E-02 .00000E+00 .00000E+00 .34200E+01 .81700E+01 .10000E+01 .10000E+01 .10000E+01

[mecej4]

Whether you use sed or write your own code to do the transformation, you have to anticipate all the transformations that are needed and program accordingly, as you noted yourself in #6. As we well know, text substitutions have the following ways of not doing what we want: 

• the regular expression may not cover all the substitutions that we want to accomplish, but only a subset
• the regular expression may change text that we did not wish to change, because we did not realize that the R.E. covered more than what we wished to cover

Try sed -e 's/0\(\.[0-9]*E[+-]..\)/ \1/g' .

[Lowell_S_]

Thanks mecej4.  This newest sed command worked.  Yes, I am aware that there might be instances where I'll have to tweak the sed command as I work through all my programs.  I just need to get up to speed on these more complex expressions.  Why was the asterick required?

[mecej4]

Here is my understanding of the RegExp that I used in #11: 

Locate '0' (Zero), followed by a dot, followed by zero or more (that is what the '*' count suffix implies) decimal digits, followed by 'E', followed by '+' or '-', followed by the two digits of the exponent. Everything that comes after the leading '0' is labeled as sub-reg-exp 1, by using the '\(' and '\)' delimiters. The substitution consists of replacing the entire matched RegExp by a leading space, followed by the labeled sub-reg-exp 1, and the 'g' at the end says 'repeat the substitution as many times as possible for the entire input line'.

[Lowell_S_]

Thanks mecej4!

The following commands are what I've added to my run script so far to handle the leading zero issues I've found to date (file names have been changed to protect the innocent).  The last one handles floating point values without an E or D descriptor.  It may grow over time to cover more exceptions as I work through all of my programs.

# Suppress leading zero's left by the ifort compiler
 sed -e 's/0\(\.[0-9]*E[+-]..\)/ \1/g' < x.txt > y.txt
 mv y.txt x.txt
 sed -e 's/0\(\.[0-9]*D[+-]..\)/ \1/g' < x.txt > y.txt
 mv y.txt x.txt
 sed -e 's/ 0\(\.[0-9]\)/  \1/g' < x.txt > y.txt
 mv y.txt x.txt