Behavior of FPUTC() with newline

[jayb]

Howdy.  One more trip into Legacyland.  I have reduced my issue to a small test case that works with "ye olde" g77 compiler, but does not work with Intel Fortran, even if I "USE IFPORT" to invoke the portability version of FPUTC().

The following code when compiled with g77 (apologies for the ancient style and features like equivalencing) produces a file that consists of two lines:

hi
there

      PROGRAM OPENTEST

      INTEGER RESULT,IOSTAT

      INTEGER*1 INEWLINE

      CHARACTER*1 CNEWLINE

      EQUIVALENCE(INEWLINE,CNEWLINE)

      INEWLINE=10 

      OPEN(UNIT=14,STATUS='UNKNOWN',FILE='OUT.TXT',

     *ACCESS='SEQUENTIAL',FORM='FORMATTED',RECL=512,

     *BLANK='NULL',IOSTAT=IOSTAT)

      RESULT=FPUTC(14,'h')

      RESULT=FPUTC(14,'i')

      RESULT=FPUTC(14,CNEWLINE)

      RESULT=FPUTC(14,'t')

      RESULT=FPUTC(14,'h')

      RESULT=FPUTC(14,'e')

      RESULT=FPUTC(14,'r')

      RESULT=FPUTC(14,'e')

      RESULT=FPUTC(14,CNEWLINE)

      END

Yup, it should not be that hard, but I am trying to demonstrate how the legacy code works with a trivial test case.  Note the use of FPUTC, which I am told is a g77 intrinsic function and accepts a logical unit number as the first argument.  Also notice that for the newline character, I am storing 10 in a 1-byte integer, and equivalencing it with a character.  I plan to modernize this 40-year-old application, but I first need to get the program working on Intel Fortran to have a starting point for modernization.  (These are the steps that our sponsor finds most acceptable.)

Aside, not my issue:  The program as coded above would crash on Intel Fortran when FPUTC is invoked, presumably because it is invoking the C version, which expects a pointer to a file descriptor as the first argument.  But that is easily solved by using IFPORT:

      PROGRAM OPENTEST
      USE IFPORT

      INTEGER RESULT,IOSTAT

      INTEGER*1 INEWLINE

      CHARACTER*1 CNEWLINE

      EQUIVALENCE(INEWLINE,CNEWLINE)

      INEWLINE=10 

      OPEN(UNIT=14,STATUS='UNKNOWN',FILE='OUT.TXT',

     *ACCESS='SEQUENTIAL',FORM='FORMATTED',RECL=512,

     *BLANK='NULL',IOSTAT=IOSTAT)

      RESULT=FPUTC(14,'h')

      RESULT=FPUTC(14,'i')

      RESULT=FPUTC(14,CNEWLINE)

      RESULT=FPUTC(14,'t')

      RESULT=FPUTC(14,'h')

      RESULT=FPUTC(14,'e')

      RESULT=FPUTC(14,'r')

      RESULT=FPUTC(14,'e')

      RESULT=FPUTC(14,CNEWLINE)

      END

This program results in an empty file.  I also tested with a single record, without outputing CNEWLINE, and actually saw the characters in the output file.  So this code basically works, but writing ASCII 10 obliterates each line.

My compile line is:  ifort -c -debug -save -assume nounderscore

(The application requires -save because it assumes that local variables retain their values between calls; that's just the way it is.)

Basically, in a program like this, how should I start a new line?

JayB

[Steven_L_Intel1]

Interesting. The example in the manual works but yours doesn't. We'll look into this.

[jimdempseyatthecove]

In your second example you are not closing 14.

As a work around, experiment with inserting FLUSH(14) before and/or after the FPUTC(14,NEWLINE) in addition to adding the CLOSE at the end.

Jim Dempsey

[mecej4]

JayB, it would be of help if you detailed what the intent of your using FPUTC is. Do you do other WRITEs to the same unit number, interspersed with calls to FPUTC? Are there other I/O statements with that file unit, such as REWIND, BACKSPACE, etc.?

There is an inherent misalignment  between the C stream I/O model and the Fortran record I/O model, the most significant aspect of which in your example is how writing CR/LF causes the file length to become zero.

[Steven_L_Intel1]

The problem is that the FPUTC of the newline character doesn't actually write a newline, it empties the buffer. If you insert FLUSH(14) before each FPUTC of the newline, you'll see the other characters (but all on one line). I think this is a bug and will escalate it.

[Steven_L_Intel1]

Escalated as issue DPD200364616. If you're willing to rewrite the code, use ACCESS='STREAM' and ordinary WRITE statements to write the characters, one at a time. This is standard Fortran.

[jayb]

Hi, mecej4.  I cannot really speak to the developer's intent.  I am simply porting an existing code base that goes waaaay back to the days of the TOPS-20 operating system. I am, of course, willing to make small changes for portability sake.  I was given this task because of past experience porting legacy systems to modern platforms, and I think that the suggestion to switch to Intel Fortran was one of the best actions that I have taken.  Eventually, we will modernize the application.

One of the original authors assures me that there is no C i/o.  But in any case, I purposely reduced this issue to a small test case so that we would not need to deal with questions like intent, mixing of i/o approaches, etc.  The trivial application that I wrote demonstrates and issue, which appears, according to Steve, to be a bug.  Very glad to contribute to the improvement of a great product.

JayB

[jayb]

Thanks Steve.  How would I track this issue and know that a fix is available? 

I'd like to implement your suggestion, but there is just too much output code, for both binary and text files, that funnels through the few places where FPUTC() is invoked.  I'll ask the developer how he would go about this.  But I always risk getting an answer that reminds me that it "works" on g77 -- as if that is the standard we should be following.

JayB

[mecej4]

In your position, I should try GFortran on the old code, since that is currently maintained and is a logical successor to G77. Your small program works correctly in GFortran, after the USE IFport statement is removed.

[jayb]

gfortran is not able to handle many of the FORTRAN 66 usages of the larger application.  That's why the developer chose g77, not knowing that it was no longer supported, and that it requires glibc 3.4.6.  He did not know about Intel Fortran, which by and large supports all of the FORTRAN 66, VMSisms and other arcane features, with fewer compiler switches.

I do not intend to keep the application in this state forever.  But I do want to get off g77 as soon as possible; the further we advance in terms of Linux distributions, the more likely we will find a failure in g77 that no one is around to fix.  We have already hit a couple of bugs that required literally hundreds of small changes.  Contrast that with Intel Fortran, where the vendor has acknowledged a bug and plans to fix it.

Once I have a solid baseline under Intel Fortran -- and I'm almost there -- we will turn our attention to incremental, evolutionary modernization.  Maybe we will reach a point where gfortran becomes an option, but the cost of using Intel Fortran is minimal in my opinion.

JayB

[Steven_L_Intel1]

I will update this thread with news about the fix.

[jayb]

Steve,

Please feel free to delete this from the forum as soon as you read it.  I had no other way to contact you.

Generally, are Intel Fortran updates performed on a schedule?  The fputc() issue affects several of the applications, and although I appreciate the advice to standardize -- and intend to do so (that's the whole point of using Intel Fortran rather than g77) -- given the fact that the primary developer is rapidly adding features using RATFOR and g77 and has not yet linked up with our Git-based version control system, it is not yet practical.

My concerns are as much "political" as "technical"; I need to have a working baseline of the "current implementation", arcane as it is, and make the elimination of g77 a fait accompli.  So -- could I have a *rough* idea of when there might be an update?

Thanks for all your great support.

JayB

[Steven_L_Intel1]

Jay,

You should be able to use "Send Author a Message" to contact me privately.

We update the compiler about every 8 weeks, with a major release once a year.

[Steven_L_Intel1]

This bug is fixed in Update 2, which I expect to become available from the Intel Registration Center in the next few days.

[jayb]

I just downloaded Update 2 (it wasn't there this morning, so I must have a piping hot new copy).  It looks like this bug has definitely been fixed.

I am about to travel to an integration test of DoD models at which the Intel Fortran version of our application will be tested for the first time.  We will not need this feature for the test, and it's too late to re-build anyway.  But when I get back in a couple of weeks, I will be able to streamline our build process so that we do not depend on intermediate files generated by the g77 version.

Very happy with this product and the support.
Jay

[jayb]

I downloaded Update 2, and I'm afraid there is a new issue.  fputc() with ASCII 10 does, indeed, begin a new line.  But I also get an additional, unwanted new line every RECL characters (512 in this case) even if it is in the middle of one of my records.  g77, and apparently the old DEC/HP/OpenVMS FORTRAN compilers, consider the output of ASCII 10 to indicate the end of a record.

Such is the problem with small test cases:  They illustrate the problem, but do not cover the actual use case. 

I tried changing RECL to an arbitrarily high number, but beyond a certain value, nothing gets written, as before.

Leaving out RECL seems to default to an even lower value than 512, resulting in very chopped-up records.

The output files in question would contain anywhere from about a thousand to several hundred thousand characters. 

Can you suggest a way around this in the OPEN statement?  Or do I need to wait another update cycle?

Jay

[jayb]

Perhaps writing my last post triggered a solution that might work.  I tried UNFORMATTED output with no RECL.  So far, it might work.

If anyone has further thoughts, I would appreciated them.

Thanks
JayB

[jayb]

Got it.  Obviously, if the record is being physically recorded to disk, it will take significant time no matter what the code does.

I'll see how run-time performance is affected.  Fortunately, the main simulation / war game writes very little record-oriented data to disk.  Calling FLUSH() might be the solution.  In fact, there is an up-side to being able to point out that these ancient features incur a cost.

Thanks,
j

[jayb]

My apologies -- that last post was supposed to be an e-mail.  If it appears to be out of context, it is.

JayB

[Steven_L_Intel1]

The extra CRLF every 512 characters is fixed for a release later this year.