Inconsistent behavior with null-terminated strings

dboggs · ‎10-20-2011

After frustration developing a routine for use in a giant application (involving interfacing to an API routine--another story), I finally pinned down the trouble source with a simple demo program. Consider the following:

PROGRAM TRYIT
CHARACTER(8) :: TESTS(3)
TESTS(1) = 'test1'
TESTS(2) = 'test2'
TESTS(3) = 'test3'
print *, 'These strings are not null terminated'
print *, TESTS

TESTS(1) = TRIM (TESTS(1)) // ''C
TESTS(2) = TRIM (TESTS(2)) // ''C
TESTS(3) = TRIM (TESTS(3)) // ''C
print *, 'These strings are null terminated'
print *, TESTS

PRINT *, 'Detailed analysis of first string follows'
DO I = 1, 8
PRINT *, 'Posn', I, TESTS(1)(I:I), ICHAR(TESTS(1)(I:I))
END DO

END PROGRAM TRYIT

If I build and run this program as a Console application, I get the following:

These strings are not null terminated
test1 test2 test3
These strings are null terminated
test1 test2 test3
Detailed analysis of first string follows
Posn 1 t 116
Posn 2e 101
Posn 3 s 115
Posn 4 t 116
Posn 5 1 49
Posn 6 32
Posn 7 32
Posn 8 32

This is what I expect. But if I build and run as a Quickwin app, I get

These strings are not null terminated
test1 test2 test3
These strings are null terminated
test1 Detailed analysis of first string follows
Posn 1 t 116
Posn 2 e 101
Posn 3 s 115
Posn 4 t 116
Posn 5 1 49
Posn 6 Posn 7 32
Posn 8 32

Whis is not at all what I want. Apparently the null characters terminate the formatting, causing everything thereafter to be lost, including the CR/LF.

This is wreaking havoc with my formatting efforts. Since CHAR(0) is not an official member of the F90 character set, I suppose I can't complain about unexpected behavior when I try to use it, but couldn't it at least be the same in console and quickwin applications?

Is there any other quirky behavior of nulls in strings that I need to know about before getting too far down this road?

Steven_L_Intel1 · ‎10-20-2011

Which compiler version are you using? Your console results are NOT what I'd expect and not what I see, which is this:

[plain] These strings are not null terminated
 test1   test2   test3
 These strings are null terminated
 test1   test2   test3
 Detailed analysis of first string follows
 Posn           1 t         116
 Posn           2 e         101
 Posn           3 s         115
 Posn           4 t         116
 Posn           5 1          49
 Posn           6             0
 Posn           7            32
 Posn           8            32[/plain]

Note in particular that position 6 has a value of 0, for the NUL character. You showed 32, which is just wrong. I do see the results you describe for QuickWin.

I understand what is happening - the console application uses WriteFile which specifies a length. The NUL character is just another character and the console window ignores it (or displays it as a blank, which is what I think it does.) QuickWin is rendering fonts, and the Windows APIs for that use nul to indicate the end of the string.

Let me comment that your program is properly inserting the NUL (at least when I try it - your console results suggest otherwise), and this should be fine for APIs. Trying to do Fortran formatted I/O of strings with embedded NULs is likely to be troublesome.

dboggs · ‎10-20-2011

Thanks for the quick reply and the explanation. I'm using XE 2011. Sorry about what I submitted--I had to do a partial hand transcription and I spaced out that all-important line for Posn 6. Indeed I had 0 there as you do.

I think the bottom line is your statement "Trying to do Fortran formatted I/O of strings with embedded NULs is likely to be troublesome." I understand why there is troublebut itis unfortunate that the trouble is different between console and quickwin (or windows?) applications. I'm attempting to write a library (or module) general-use program to allow easy access to certain API functions, and I want it to be useful in both console and quickwin applications. So this is going to be far more difficult than I had imagined, just because of the formatted string I/O.

I think what I need to do is make sure my program removes all null terminating characters and replaces them with spaces before returning the strings to the calling routines. If this does not work, I wonder--is there some way my program can determine whether it was built for the console or for quickwin, so it can use the appropriate null character processing, or do I have to write two separate programs, one to be called by console apps and a different one to be called by quickwin apps?

Steven_L_Intel1 · ‎10-20-2011

Typically you would return strings up to the trailing NUL, and let Fortran pad the rest with blanks. Do you envision having strings with embedded NULs and non-blank characters afterward? I usually use INDEX to get the NUL position and substring the value to omit the NUL. Fortran doesn't have any convenient routines along the lines of TRIM to take care of this for you, but you could certainly write your own NULL_TRIM or some such.

IanH · ‎10-20-2011

Make the arguments to the procedures in the library/module deferred length character. Then you can return a string that has the same length as the string that comes back from the API call (you truncate the string immediately before the first null, or in some cases at whatever position the API call says is valid, before returning it).

Most Windows API calls have some way of querying the length of a string that will be returned prior to actually returning the string, so you can make your library safe against buffer overruns or similar. Then you won't have users swearing at you in ten years time because you hard coded some arbitrary maximum length for a filename or similar.