Strange behavior in Fortran

[Don_D_]

My program writes out many lines of formatted output to a named disk file. Often it works fine. In some cases, as it writes a new line to the file, the previous data vanish and all that gets there is the latest line. There is no REWIND... statement during this process. I can see no reason for this to happen. Please advise.

[mecej4]

Please show the code or attach a sample of the output file with "vanished" lines. If a file has been written without proper EOL (end-of-line) characters and you type out the file at the console, it may appear that lines have been lost. Make sure to attach the files (or a zip containing it) instead of using copy/paste, since the latter can change the file content, especially the non-printing characters in it.

First we have to see what you are seeing...

[Don_D_]

Here is what the file should contain:

RLE
ID TRIPLET START 64398
MERIT 0.813323E-01
LOG 64398
WAVL .6562700 .5875600 .4861300
APS 5
NOVIG
UNITS MM
OBB 0.0000000 9.2000000 18.1430000 -7.68199948077 0.0000000 0.0000000 18.1430000
0 AIR
1 RAD 58.1095164875656 TH 20.00000000
1 N1 1.87655327 N2 1.88299104 N3 1.89821418
1 CTE 0.670000E-05
1 GTB S 'N-LASF31A '
2 CV -1.6915684518980E-04 TH 9.67981241 AIR
3 RAD -127.3342396287016 TH 8.44629643
3 N1 1.77595306 N2 1.78471676 N3 1.80650665
3 CTE 0.852000E-05
3 GTB S 'N-SF11 '
4 CV 0.0264940066494 TH 3.00000000 AIR
5 CV 0.0000000000000 TH 34.14422609 AIR
6 RAD 157.3577514709504 TH 6.20000000
6 N1 1.60412736 N2 1.60737246 N3 1.61484769
6 CTE 0.600000E-05
6 GTB S 'N-SK2HT '
7 RAD -69.0539208684826 TH 104.10792165 AIR
7 CV -0.01448144
7 UMC -0.12500000
7 TH 104.10792165
7 YMT 0.00000000
8 CV 0.0000000000000 TH 0.00000000 AIR
END

Here is what shows up in some cases:

8 CV 0.0000000000000 TH 0.00000000 AIR
END

A typical write statement looks like this:

IF (IRP .EQ. 0 .AND. ITP .EQ. 0) WRITE(KOUT,7000,ERR=9999) J,ZCV,Z15B

The format statement is

7000 FORMAT(1X,I3,1X,'RAD',1X,A20,3X,'TH',1X,A15)

and the character variables have the correct text, as I can see in the debugger.  After it writes a few lines, the next write seems to rewind the output file first, which has KOUT = 2001.

[Don_D_]

I cannot attach the files, since they are of type .RLE.  But they are really just text file.  Your uploader rejects those files.

[mecej4]

Put the .RLE, etc., files into a Zip file (use Winzip, PKzip, etc.,) and attach the Zip file. Since we are dealing with a program that is not doing what you expect it to do, please include the source file, plus any include files and data input files).

[Don_D_]

Good idea.  The zipped files are attached.

The source file that does the writing is 2003 lines long, and I'm not sure you want to read so much.

But here's a clue:  The problem occurs only when I spawn other processes, and they do the writing.  The unit number is different for each one, so there is no collision there.  When the main process does it, the output is correct. Does that tell you anything?

[Steve_Lionel]

Oh! You are writing to the same file from multiple processes? That is not likely to work and can cause exactly this sort of symptom because the OS isn't tracking the end-of-file globally.

[Don_D_]

Not at all.  Each process creates a unique file name and unit number, for just this reason.

[mecej4]

Multiple processes writing to the same file is almost certain to cause problems without a protocol for file locking and file access. Fortran file unit numbers have no significance to the underlying Windows DLLs that do the actual I/O. Likewise, a Fortran EXE and other DLLs writing to the same unit number may be writing to different files.

I should still like to have the source file (and instructions for building and running/spawning).

[Don_D_]

The source file is about half a million lines long, although the portion in question is about 200K.  Since it works with process 0, I doubt there is anything in the source code at fault.

The clue seems to be a difference between process 0 and the others.  How can that be?

[Don_D_]

Here is the code that does the spawning:

IF (IS_MULTICORE .AND. ISFLAGS(206) .EQ. 0) THEN ! MAIN PROCESS SETS THINGS UP HERE (GSEARCH) ****************
CALL GET_COMMAND(CPATH, LEN, ISTAT) ! CPATH IS FULL PATH TO EXE FILE!

DO J = 1,LEN
IF (CPATH(J:J) .EQ. "\") CPATH(J:J) = "/" ! STUPID UNIX/DOS STUFF!
IF (CPATH(J:J) .EQ. '"') CPATH(J:J) = " " ! STUPID UNIX/DOS STUFF!
ENDDO

IF (CPATH(1:1) .EQ. ' ') THEN
DO J = 1,LEN-1
CPATH(J:J) = CPATH(J+1:J+1) ! STUPID UNIX/DOS STUFF!
ENDDO
K = 0
DO J = 1,LEN
IF (CPATH(J:J) .NE. " ") THEN ! STUPID UNIX/DOS STUFF!
K = K + 1
ENDIF
ENDDO
LEN = K
ENDIF

OPEN(198,NAME="GSEARCHDATAFILE",FORM="UNFORMATTED",STATUS="UNKNOWN",ERR=8011)
WRITE(198) JCAT,GNAMES,KSURFACES,NSURFACES,NNAMES,JTRIAL,KTOTAL,LIBGS,JDATSW,
$ JUSE,NEARNAMES,NEAREST,NEARCAT,OPTFILE,LCOMDF,JLCOM,JCOMDF,PAVAL,IS_SINGLE,PRICE,WDN,IS_QUICK,GSINDEX,
$ IS_SUBSET,JSUBSET

CLOSE(198)
OPEN(199,NAME="GSEARCHFLAGSFILE",FORM="UNFORMATTED",STATUS="UNKNOWN",ERR=8011)
JCSTART = 0 ! FOR ALL CORES
JCEND = KTOTAL - 1 ! STARTS AT ZERO
JSPAN = ISFLAGS(173) ! NUMBER OF CORES TO USE
ISFLAGS(207) = JCSTART ! EACH CORE STARTS HERE
ISFLAGS(208) = JCEND ! AND ENDS HERE
ISFLAGS(209) = JSPAN ! JUMP THIS NUMBER EACH CASE
WRITE(199) ISFLAGS,SFLAGS ! NEED OTHER CURRENT SETTINGS
CLOSE(199)
!

DO I = 1,ISFLAGS(173)-1
L = 600 + I
CLOSE(L)
OPEN(L,SHARE='DENYNONE',FORM="FORMATTED",STATUS="REPLACE")
WRITE(L,6002) I ! FLAG THIS CORE IS RUNNING
CLOSE(L)
K = ISFLAGS(213) ! NETWORK FLAG
CALL myspawn( I, 2, K, CPATH, LEN ) ! START NEW PROCESS
ENDDO
ENDIF

[Don_D_]

And here is the spawning routine:

void myspawn( int i, int j, int k, char* path )
{
CString imsg;
CString jmsg;
CString kmsg;
CString tmsg;
CString lmsg;
int l;
bool retlog;

/* task numbers in j:
1 DSEARCH Q
2 GSEARCH
3 MC
4 REDO DSEARCH Q
5 ZSEARCH Q
6 REDO ZSEARCH Q
7 AEI
8 AAA
9 ADA
10 SYNO | spawn
*/
// qnx_spawn_options.priority = 20; // this won't work

imsg.Format( "%d", i ); // is new core number
jmsg.Format( "%d", j ); // is task number
kmsg.Format( "%d", k ); // is network flag
tmsg.Format( "%d", frametop ); // data set by core 0
lmsg.Format( "%d", frameleft );

HANDLE hd;
l = _spawnl( _P_NOWAIT, path, "SYNOPSYS200v15.exe", imsg, jmsg, kmsg, tmsg, lmsg, NULL );
if ( l < 0 ) {
errno_t err = 0;

_get_errno( &err );
switch (err) {
case E2BIG:
AfxMessageBox( "TOO MANY ARGUMENTS" );
wndout( "TOO MANY ARGUMENTS\r\n" );
break;
case EINVAL:
AfxMessageBox( "INVALID MODE" );
wndout( "INVALID MODE\r\n" );
break;
case ENOENT:
AfxMessageBox( "EXECUTABLE FILE NOT FOUND" );
wndout( "EXECUTABLE FILE NOT FOUND\r\n" );
break;
case ENOEXEC:
AfxMessageBox( "NOT EXECUTABLE" );
wndout( "NOT EXECUTABLE\r\n" );
break;
case ENOMEM:
AfxMessageBox( "NOT ENOUGH MEMORY" );
wndout( "NOT ENOUGH MEMORY\r\n" );
break;
}
}
corehandle[i-1] = l;

//hd = (HANDLE) corehandle[i-1]; // don't use; need for killspawn()
//retlog = CloseHandle(hd);
//if ( !retlog ) {
// flasherror();
//}
}

[Don_D_]

Here's another clue:

While trying to solve an unrelated problem (program runs fine but Windows is corrupted afterwards) a colleague sent me info on a product called Intel Inspector.

So I installed it, but did not figure out how to launch it.  However, the instructions had me change some of the properties of the compiler.  I remember I changed very little, but maybe that was the cause.  Here is the command line for Fortran:

/nologo /debug:full /MP /Od /heap-arrays0 /I"Debug/" /reentrancy:none /extend_source:132 /Qopenmp /Qopenmp-report1 /warn:unused /warn:truncated_source /Qauto /align:rec4byte /align:commons /assume:byterecl /Qtrapuv /Qzero /fpe:1 /fp:strict /Qfp_port /fpconstant /Qftz /Qfp-stack-check /iface:cvf /module:"Debug/" /object:"Debug/" /traceback /check:all /libs:dll /threads /winapp /c

and this is for C++:

/I"C:\Program Files\Microsoft Platform SDK\Include" /ZI /nologo /W1 /WX- /MP /Od /Ot /Oy- /D "WIN32" /D "_DEBUG" /D "_WINDOWS" /D "_VC80_UPGRADE=0x0600" /D "_MBCS" /Gm /EHsc /MTd /GS /Gy /fp:strict /Zc:wchar_t /Zc:forScope /GR /openmp /Fp".\Debug\SYNOPSYS200.pch" /Fa".\Debug\" /Fo".\Debug\" /Fd".\Debug\" /FR".\Debug\" /Gd /analyze- /errorReport:queue

And this is for the linker:

/OUT:".\Debug\SYNOPSYS200v15.exe" /VERSION:"15.0" /INCREMENTAL:NO /NOLOGO /LIBPATH:"C:\Program Files (x86)\Intel\Composer XE 2011 SP1\\compiler\lib\ia32" /LIBPATH:"C:\Projects\U105136\OpenGL\freeglut 2.4.0 (compiled)" /LIBPATH:"C:\Projects\U105136\Libraries\Static\MT" /LIBPATH:"C:\Program Files\Microsoft Platform SDK\Lib" "windowscodecs.lib" "mpr.lib" "SentinelKeys.lib" "wsock32.lib" "freeglut.lib" "Debug\SYNOPSYS200_lib_.lib" "VCOMPD.LIB" "Htmlhelp.lib" "kfunc32MT.lib" "NETAPI32.LIB" /NODEFAULTLIB:"LIBCMT.lib" /NODEFAULTLIB:"msvcrtd.lib" /NODEFAULTLIB:"msvcrt.lib" /MANIFEST /ManifestFile:".\Debug\SYNOPSYS200v15.exe.intermediate.manifest" /ALLOWISOLATION /MANIFESTUAC:"level='asInvoker' uiAccess='false'" /DEBUG /PDB:"C:\SYNOPSYSV15\Debug\SYNOPSYS200v15.pdb" /MAP /SUBSYSTEM:WINDOWS /STACK:"3000000" /PGD:"C:\SYNOPSYSV15\Debug\SYNOPSYS200v15.pgd" /TLBID:1 /DYNAMICBASE /NXCOMPAT /MACHINE:X86 /ERRORREPORT:QUEUE

Is there anything there that should not be?

[Steve_Lionel]

That's a lot of, to me unnecessary, options. None of these would be related to different processes writing to different files. Are you sure that's what's happening?

If it were me, I'd throw out at least half of those options, but I don't think any are particularly harmful except perhaps /reentrancy:none.

[Don_D_ · Accepted solution]

My bad.

I restored an earlier version that worked, and then replaced routines one at a time to see where it fails.

That worked.  I was rewinding the output text file, but the bug was the unit number was that of the saved file, which was of course rewound.

Sorry for the trouble.