topic 1. is there a better way to in Intel® Fortran Compiler

Within Fortran, Call Matlab

Jia__Dun — Tue, 25 Aug 2015 19:09:50 GMT

Hi,

I have a question about how to call Matlab script within Fortran, which performs Matlab parallel computing "parfor" syntax. Basically, I am solving a problem in Fortran which is mainly a Do-Loop. For each iteration, I write out a series from Fortran for a Matlab script to read in, process through "parfor", and write back to Fortran. Fortran iteration continues until convergence.

In my Fortran codes, I use call system to let the shell return to Linux commands and call matlab, run matlab script, exit matlab and return to Fortran.

CALL system(" matlab -nodesktop -nojvm -r 'Matlab_script; exit' ")

The codes look like the above. ifort compiles the entire program with no problem. Also, the entire Fortran program runs fine and the results are correct.

However, the problem is that within Matlab_script.m, a matlab parfor loop should be executed. Rather, the loop was executed in plain "for, ..., end" loop. It takes so long for Fortran to get back the matlab processed results. I tested the Matlab_script.m without Fortran call system() and the parfor loop goes fine.

Does anyone know

1. is there a better way to let Fortran handle over to Matlab to process, wait and get back the results besides Call System() ?

2. Why call system does not implement the matlab parfor function?

Please advise. Thank you so much!

Calvin

1. is there a better way to

rudi-gaelzer — Wed, 26 Aug 2015 17:09:21 GMT

1. is there a better way to let Fortran handle over to Matlab to process, wait and get back the results besides Call System() ?

I guess that you're using an extension to the standard. Fortran 2008 has the new intrinsic subroutine:

execute_command_line(command [, wait] [, exitstat] [, cmdstat] [, cmdmsg])

So, your Matlab script could be executed by

call execute_command_line(" matlab -nodesktop -nojvm -r 'Matlab_script; exit' ")

and by default (wait = .true.) the program would wait for the Matlab script to finish the execution before resuming.

However, apart from changing from an intel-specific extension to a standard instruction, you would be doing exactly the same thing...

2. Why call system does not implement the matlab parfor function?

I don't know why you Matlab code would take longer to execute when called from fortran than calling it directly. Moreover, I think that the answer to your question would be the lack of interoperability between fortran and Matlab. So far, the standard only provides interoperability between fortran and c/c++ via iso_c_binding.

I had a similar problem. I wanted not only to execute a python script from fortran, but also transfer the result from python to F. The simplest solution was something similar to your approach: call execute_command_line to run the python script, write the result to a known file and then read the file in F. After a lot of reading and tweaking I managed to access a python script via c (F <-> C/C++ <-> Python). The interface between C and python is provided by the Py/C API.

So, if you know that a Matlab script can be run from C/C++, then you could do the same...

Matlab has an "engine" API

IanH — Thu, 27 Aug 2015 01:45:37 GMT

Matlab has an "engine" API that you can use to invoke it from within another application.

See http://mathworks.com/help/matlab/calling-matlab-engine-from-c-c-and-fortran-programs.html

I have no idea whether this will alter in anyway what you are seeing with parfor though.

Hi Rudi,

Jia__Dun — Thu, 27 Aug 2015 03:17:57 GMT

Hi Rudi,

Thank you so so much for giving me the insights. I tried the execute_command_line with the wait option. This works fine and well reaches my goal. Therefore I guess this one is an equivalent to call system() syntax.

About bullet 2, I actually figured out why matlab parfor was not called. In order to reduce the matlab system time, I used the option -nojvm to turn off the Java functionality. It turned out that this can prevent Matlab to create parpool. So instead, I used

matlab -nosplash

in my call system () syntax.

I think what you said in bullet 2 is really helpful. All these problems arise from the limited interactive API among these applications. But it seems we at least find two shortcuts to achieve the goals as long as the interactions are limited.

Thank you so much!

Calvin

rudi-gaelzer wrote:

1. is there a better way to let Fortran handle over to Matlab to process, wait and get back the results besides Call System() ?

I guess that you're using an extension to the standard. Fortran 2008 has the new intrinsic subroutine:
execute_command_line(command [, wait] [, exitstat] [, cmdstat] [, cmdmsg])
So, your Matlab script could be executed by
call execute_command_line(" matlab -nodesktop -nojvm -r 'Matlab_script; exit' ")
and by default (wait = .true.) the program would wait for the Matlab script to finish the execution before resuming.

However, apart from changing from an intel-specific extension to a standard instruction, you would be doing exactly the same thing...

2. Why call system does not implement the matlab parfor function?

I don't know why you Matlab code would take longer to execute when called from fortran than calling it directly. Moreover, I think that the answer to your question would be the lack of interoperability between fortran and Matlab. So far, the standard only provides interoperability between fortran and c/c++ via iso_c_binding.

I had a similar problem. I wanted not only to execute a python script from fortran, but also transfer the result from python to F. The simplest solution was something similar to your approach: call execute_command_line to run the python script, write the result to a known file and then read the file in F. After a lot of reading and tweaking I managed to access a python script via c (F <-> C/C++ <-> Python). The interface between C and python is provided by the Py/C API.

So, if you know that a Matlab script can be run from C/C++, then you could do the same...

Thank you very much Ian, I

Jia__Dun — Thu, 27 Aug 2015 03:20:51 GMT

Thank you very much Ian, I noticed that this API engine is available but some the learning curve can be steep. Plus, I read about compiling using Matlab mex function which I hate the most somehow. I will definitely look into these some time later.

Best

Calvin

ianh wrote:

Matlab has an "engine" API that you can use to invoke it from within another application.

See http://mathworks.com/help/matlab/calling-matlab-engine-from-c-c-and-fort...

I have no idea whether this will alter in anyway what you are seeing with parfor though.

It has been a few years now,

IanH — Thu, 27 Aug 2015 03:38:30 GMT

It has been a few years now, but when I was interfacing with Matlab I had (and still have) a very strong distaste for their approaches they provided for interfacing Fortran and Matlab. I wrote my own wrappers, using the C interop facilities of Fortran 2003, against the specification of their API in their documentation and provided C header files. After the initial investment I found this approach far superior.

As far as I can tell, my wrappers didn't cover their engine API (which surprises me, because I recall using it, but perhaps I wasn't calling it from Fortran). But that API is not particularly large - writing F2003 wrappers for it shouldn't take very long. If you do go down the path of using the API, I suggest you consider this wrapper approach.