Legacy programs and Read / Write functions

[abhisinghal]

Hi

I have 3-4 legacy FORTRAN programs which are essentially command line exes.
Each program takes in an input file and produces 2-3formatted text files for the next program

I have the source code of these programs and so I intend to convert them to dll to be called from my VB .NET program.

The programs heavily use READ and WRITE functions upon file handles opened/closed by OPEN/CLOSE functions.

Intended Flow is as follows:

VB .NET ---string1, string2, string 3 --> PROG 1 ---- string1, string2, string3 ---> VB.NET
VB .NET ---string1, string2, string 3 --> PROG2 ---- string1, string2, string3 ---> VB.NET
and so on.

1) Issue is whether the same READ/WRITE statements can still be used ... i am looking for least modification to the original source code.
2) What sort of variables should be passed in this case.

Thanks
Abhishek

[anthonyrichards]

Do I gather correctly that 'string1, string2, string3' are the original command line string arguments that you now wish to pass FROM VB .NET TO Fortran DLL routines, and do you also want to pass different 'string1, string2, string3' FROM Fortran BACK to VB .NET to be passed onto another Fortran DLL routine?

The passing of string arrays to Fortran using SafeArrays has been addressed in a very recent thread here.
Otherwise you could try passing the command line string as one string, which should be easier to arrange.

Clearly, once you have successfully passed a character string to Fortran (VB .NET calls Fortran), you could then use an internal READ in teh Fortran to access the seperate string elements (if they were comma delimited within the complete command line string). Similarly formatted internal WRITE could be used to create a command line string to be passed back to VB .NET using a seperate routine (Fortran calls VB .NET)

[abhisinghal]

So actually the original programs were all exes which worked through files.
but that is slowing things down so i am looking to pass this as variables.

i will try to pass string as argument and declare the variable as character*(*) inside the subroutine.
will this take care of the fact that the input / output string can be of any length ?

other problem is that there are lot of Common variables used in the individual FORTRAN exes [which will now all becomes subroutines in a common dll]
since FORTRAN functions are static, if different threads call them, there might be goof up.
So what is the best way to solve this ?
Whether it is possible to create FORTRAN class which can be instantiated instead of using static methods ?

Thanks

[mecej4]

Regarding your question "I will try to pass string as argument and declare the variable as character*(*) inside the subroutine.
will this take care of the fact that the input / output string can be of any length ?":

Definitely not.

When a dummy argument is declared character*(*), you are telling the compiler that the length of the character variable is not known at compile time, but is to be passed in by the caller as a hidden argument. Unless you use null-terminated strings and possibly other attributes/features, there is nothing in the contents of a character variable that provides information about the length of a string. That is why the hidden argument(s) are needed. There is no escaping the fact that the caller has to know the length of the character argument, if you are using standard Fortran character variables.

There is more than one way of passing the hidden arguments and, if you want to call Fortran from another language, you need to ascertain which of these ways your Fortran compiler uses, and the caller has to correctly pass the length(s) of the character argument(s) to the subroutine in your DLL.

[abhisinghal]

OK
So for input it is possible for caller to pass length as hidden parameter.

regarding output string(s), i intend to use WRITE functions to write to the variable and it is impossible to compute the length of the stringin the beginning.

how to declare such a string variable ?

Thanks

[Jugoslav_Dujic]

Quoting abhisinghal

The programs heavily use READ and WRITE functions upon file handles opened/closed by OPEN/CLOSE functions.

Intended Flow is as follows:

VB .NET ---string1, string2, string 3 --> PROG 1 ---- string1, string2, string3 ---> VB.NET
VB .NET ---string1, string2, string 3 --> PROG2 ---- string1, string2, string3 ---> VB.NET
and so on.

1) Issue is whether the same READ/WRITE statements can still be used ... i am looking for least modification to the original source code.
2) What sort of variables should be passed in this case.

For least modification, you could use memory-mapped files Windows functionality. The idea is to

1) create a memory-mapped file in VB, (sample)
2) write to that file in the exact format that Fortran requires
3) in Fortran, attach to it using OPEN(USEROPEN=...) extension (sample) rathern than OPEN(FILE=...). Within the useropen routine, use MapViewOfFile and/or OpenFileMapping APIs to attach.
4) , i.e. READ from the unit as usual.

Repeat the procedure for another file, in reverse direction (WRITE from Fortran, read from VB).

This is not as efficient as passing arguments (because you will still lose time on I/O formatting in VB and Fortran, just not on the disk, but in memory). For somewhat more efficient approach (and less hassle to properly format the input/output in VB), you can get rid of formatted I/O and use access="STREAM" (formerly BINARY) files, i.e. just drop FORMAT strings on Fortran side.

Disclaimer: Haven't done all of this this myself, obviously. In particular, I'm not sure if .NET-handled memory-mapped files use the same underlying mechanism as Windows API (CreateFileMapping/MapViewOfFile). I think it's worth a shot though, especially if the data structure is complex and mutually dependent (e.g. read a number of things, then depending on their contents read another type and number of things, etc.)

[mecej4]

"...it is impossible to compute the length of the stringin the beginning"

True, but you only need to know the maximum possible string length, and pass that length as a hidden argument.

In Fortran, when a shorter string is assigned to the variable, padding to full length is done with trailing blanks. If you want, you can terminate your string with a null.

[abhisinghal]

Thanks

I have got some useful insights. Let me try.

Thanks again
Abhishek

[anthonyrichards]

Can you clarify one thing for me that is not quite clear in your posts.
Please correct me where I misunderstand your requirements:

You have several Fortran programs that require command line arguments.
The arguments on the command line are names of input (or perhaps also output?) data files.
Each Fortran program generates more data files whose names are required as command line arguments for another Fortran program, and so on.

Now you wish to control the execution of your Fortran programs froma VB .NET controller.
The route you want to take is to convert your Fortran executable Main programs into Subroutines in a DLL and call each main subroutine in sequence, each time supplying the required list of command line arguments as a character string argument to the subroutine.
When the main code has run to completion, the subroutine is required to return a new character string to VB .NET caller containing the names of several data files that are required by the next Fortran program to be called from VB .NET. The string passing back and forth can be easily accomplished. If you wish I can supply VB .NET and Fortran code that does this.

However, things might be more complicated if some of the data files are generated within the VB .NET program. So my query is, are all the required command line data files generated external to the VB .NET controller or not?

[abhisinghal]

Hi

Your understanding is correct. However I dont wish to supply the file names as arguments but the file contents itself as argument. This will avoid costly disk I/O.

Initial input is generated by the VB .NET program and subsequent are all generated by the FORTRAN subroutines. However flow is intended to still be VB .NET --> subroutine 1 --> VB .NET ---> subroutine 2 --> VB .NET and so on.

main problem coming here is that the READ function which is used heavily to read the disk file, when it is used with an internal file (string argument passed from VB .NET) does not maintain its position and is always positioned at the beginning of the string argument. This is leading to the failure of subsequent READ statements after the 1st READ statement.

Any solutions ?

Thanks
Abhishek

[anthonyrichards]

So instead of writing data to files, whose names you have until now given as arguments on the command lines used to start the main prgrams, you want to actually 'pass' the data to the Fortran and then, when the Fortran program has signalled VB NET it has completed (using a function return value perhaps) you want to the VB NET program to grab data back from the Fortran when it has finished executing?

In which case, what type of data is written to the intermediate data files? Reals, integers, characters, arrays?

[abhisinghal]

Yes I want to pass data and recieve back data.

Data is basically formatted string which is read / written using many sequential formatted read and write statements.

[anthonyrichards]

By Calling your data a "Formatted string" requiring many READs and WRITEs, you do not clarify what data TYPEs you are writing to a data file.
A formatted file consists of RECORDS, each of which could contain a combination of INTEGER, REAL and CHARACTER data types.
If your data file consists of many such RECORDS, which at present have to be READ sequentially, it would not be efficient to pass the data record-by-record from VB .NET to Fortran. Better to pass references to where your data is stored in memory along with data that describes the size, type and structure of the data stored there. Then the FORTRAN can access it and use it randomly as required.

Clearly if you then need to give VB .NET access to data that the Fortran subsequently computes (which until now you write out to data files), it needs to be stored in memory in a structure that VB .NET can easily access, when the Fortran signals to VB .NET that it has finished computations and that the computed data is now ready for processing.

The situation described above requies that VB .NET set up storage, as needed, on its side for the initial data, sending its address(es) to Fortran so that Fortran can access it (and perhaps even change it?) and Fortran set up storage on its side, again as needed, for subsequently computed data.

If you know the exact size and structure of the data that the Fortan program generates, you might consider setting that storage up on the VB .NET side as well and giving Fortran access to its location and structure information. That way, all the storage will be available to VB .NET to manage.

To repeat, just supposing your mooted VB .NET controlling program were replaced by a FORTRAN controlling program, what Types of data would your Fortran program have to specify in your case and how would you pass the data to the Fortran subprograms?

[abhisinghal]

Well I intend to pass all the "records" as one string ... so the data exchange between VB and FORTRAN happens only in the beginning and at the end ofeach program.

Yes I agree sequential READ and WRITE statements may not be efficient. In reality FORTRAN programs require arrays of REAL data as input and outputs arrays of REAL data and a formatted text output (sort of a report)

There are two reasons for taking the approach as above:

1. No fundamental change has to be done to FORTRAN source code. I intend to just replace the I/O unit in read/write statements. Decrease in performance is acceptable as it is still much more efficient than going via disk.

2. To produce the formatted text output, the formatted WRITE statement is required anyway and we need to find a way to do it without going to a disk file.

[abhisinghal]

Any solution to the internal READ/WRITE issue ?

[Jugoslav_Dujic]

I don't think that there is one. Internal I/O is simply not record-oriented -- in other words, every string is treated as a single record -- and I don't think you can make them parsed as multiple records.

Have you checked out my suggestion from page 1 with memory-mapped files?

[Arjen_Markus]

Actually you can write to multiple records with an internal write:

! write_internal.f90 --

! Write to multiple lines via internal write

!

program write_internal

implicit none

integer :: i

character(80), dimension(2) :: line

write( line, '(a,/,a)' ) 'Hello', 'World'

write( *, '(a)' ) ( trim(line(i)) ,i=1,2)

end program write_internal

You need to use an array of character strings - that is the closest to a set of records you can get

Regards,

Arjen

[anthonyrichards]

This seems perverse. A REAL(8) number uses 8 bytes of storage. -1.2345678912345e-39 needs 20 bytes of storage as characters.
Your use of output files implies that the data you need to transfer between programs is too much to store in program memory and transfer by means of subrutine calls. Is that correct?

[abhisinghal]

I saw your suggestion and found it very interesting. Right now it seems the only way to have the least modification in the program (which is my primary goal)

However, I am not very familiar on the FORTRAN side and the link that you wrote regarding the USEROPEN function seems complex. Is there a simpler way to accomplish the same ? Some small working example will surely help.

BTW I found this code on this forum:

FUNCTION GetEntireReadMap(MapID, strCallingProg, hMapObject)

USE KERNEL32, only: OpenFileMapping, MapViewOfFile, FILE_MAP_READ

! Return as handle to Map View of entire FileMapping

! ..also return hMapObject which is the File Mapping Handle

INTEGER*4 GetEntireReadMap, MapView

INTEGER*4 hMapObject

CHARACTER*(*) MapID

CHARACTER*(*) strCallingProg

GetEntireReadMap=0 !Default return unsuccessful...

hMapObject = OpenFileMapping (FILE_MAP_READ,FALSE, MapID)

if (hMapObject==0) then

write(*,*) strCallingProg // ': OpenFileMapping failed'

else !Try to create a map view...

MapView = MapViewOfFile (hMapObject, FILE_MAP_READ, 0, 0, 0)

if (MapView==0) then

write(*,*) strCallingProg // ': MapViewOfFile failed'

else

GetEntireReadMap=MapView

endif

endif

return

END FUNCTION

Thanks

[abhisinghal]

I tried this suggestion.

I created a memory mapped file with CREATEFILEMAPPING passed with INVALID_HANDLE_VALUE inside VB.

I then called the FORTRAN subroutine which used the OPEN function with the USEROPEN function specified.
Inside the USEROPEN I tried to use the OPENFILEMAPPING andMAPVIEWOFFILE functions.
OPENFILEMAPPING succeeds and returns a handle and all seems to be fine.

However, when a READ statement is executed over the I/O unit used in the OPEN Function as above, following error is thrown:

"An invalid handle was specified"

I guess READ statement expects a file handle and the handle returned by CREATEFILEMAPPING/OPENFILEMAPPING is not the same.

Please suggest

Thanks
Abhishek

[psantos]

abhisinghal you are right the function OPENFILEMAPPING returns a window HANDLE. However, the read function on the Fortran side expects an integer file descriptor. I have never attempted to convert between these two but I will probably start trying to invoke the c function _open_osfhandle on the fortran side to attempt a conversion. This is just a guess. I have never tried this. There is also the possibility that the integer file descriptor returned by _open_osfhandle be not valid in Fortran. I hope this helps.

Best Regards,

Pedro