topic Ed, in Intel® MPI Library

MPI_Comm_spawn strips empty strings from argv

Ed_F_1 — Wed, 12 Aug 2015 10:44:57 GMT

Hi. I'm using Intel MPI 5 on Windows and have the following problem. I have a C++ application that spawns worker processes using MPI_Comm_spawn() and passing parameters via the argv argument. I have an array of strings that is passed this way - however, one of these string is empty. When the worker process receives the argv array, the empty string has been removed from the array.

f argv is an array of 3 strings, then in the worker process, I would expect argc = 4 (3 + 1 for the worker exe name). However, if one of the strings is empty (note, empty but not NULL), then argc = 3 in the worker process. This seems to me to be a bug. Can someone confirm? Is there a workaround?

Ed,

Steve_H_Intel1 — Fri, 14 Aug 2015 22:23:00 GMT

Ed,

What does the "MPI_Comm_spawn" call look like within your C++ application?

If you were to insert debug semantics just before the call to "MPI_Comm_spawn" within the C++ application that spawns the worker process, and the semantics look something like:

int i;

printf("master - argc = %-d\n", argc);

for (i = 1; i < argc; i++)

printf("%s%c", argv, (i < argc - 1) ? ' ' : '\n');

what does master application generate for the "argv" array? Within the source of the worker application, if you insert debug semantics that look something like:

int i;

printf("worker - argc = %-d\n", argc);

for (i = 1; i < argc; i++)

printf("%s%c", argv, (i < argc - 1) ? ' ' : '\n');

what does the worker application generate?

Thank you,

-Steve

So:

Ed_F_1 — Mon, 17 Aug 2015 09:09:45 GMT

So:

I tweaked the printing to add quotes around the argument: printf("\"%s\"%c", argv, (i < argc - 1) ? ' ' : '\n');

Here is the argv array that I construct and which is passed to MPI_Comm_spawn

master - argc = 5
"f:\tmp\gsa" "pinene" "simpinene" "" "other_info"

worker - argc = 5
"f:\tmp\gsa" "pinene" "simpinene" "other_info"

As you see the argument between "simpinene" and "other_info" has been removed by the time the worker receives the argv array.

I do have a workaround - effectively never submit an empty string but the behaviour still looks like a bug to me.

Steve_H_Intel1 — Mon, 17 Aug 2015 23:32:09 GMT

Ed,

Could you please provide the command-line that you are using for the "master" application? The following example does not have any MPI semantics within it:

#include <stdio.h>

int main(int argc, char *argv[]) {

int i;

printf("argc: %d\n", argc);

printf("argv[0]: %s\n", argv[0]);

if (argc == 1) {

printf("No arguments were passed from the command-line.\n");

}

else {

printf("Arguments from the command-line:\n");

for (i = 1; i < argc; ++i) {

printf(" %d. %s\n", i, argv);

}

return 0;

}

When the compiled executable is run with "mpiexec.smpd" or "mpiexec.hydra" on Windows OS:

mpiexec.hydra -n 1 test_arguments.exe first second third "" fourth fifth

The following output is generated:

    argc:     6
    argv[0]: test_arguments.exe
    Arguments from the command-line:
      1. first
      2. second
      3. third
      4. fourth
      5. fifth

When the compiled executable is run without "mpiexec.smpd" or "mpiexec.hydra":

test_arguments.exe first second third "" fourth fifth

The following output is generated:

    argc:     7
    argv[0]: test_arguments.exe
    Arguments from the command-line:
     1. first
     2. second
     3. third
     4.
     5. fourth
      6. fifth

This problem does not appear on Linux OS.

Thank you,

-Steve

Steve,

Ed_F_1 — Tue, 18 Aug 2015 06:52:08 GMT

Steve,

I am running on Windows 7. My command line is effectively: mpiexec.smpd.exe -n 1 ..\bin\prog.exe

(Although the same happens with mpiexec.hydra.exe)

In prog.exe I effectively do the following:

static const in NumArgs=3;

char const* m_argv[NumArgs+1];

m_argv[0] = "arg0";

m_argv[1] = "";

m_argv[2] = "arg2";

m_argv[NumArgs] = NULL;

I then call MPI_Comm_spawn passing m_argv.

Your example shows the exact problem.

Like I said earlier, I now ensure that none of my arguments are empty (by prepending option flags: e.g. -p:) - this is a more robust approach for me anyway so I'll not trip up over this behaviour. However, any feedback on the underlying cause would be welcome.

Cheers, Ed

Ed,

Steve_H_Intel1 — Fri, 25 Sep 2015 21:21:12 GMT

Ed,

The Intel MPI Library development team has determined that this is Microsoft Windows specific issue. There is no way to pass an empty string as one of the command line parameters when you create a new process. Windows just ignores an empty string argument. You can refer to the MSDN article titled, "CreateProcess function" where the details can be found at the Microsoft URL:

https://msdn.microsoft.com/en-us/library/windows/desktop/ms682425(v=vs.85).aspx

In essence, when you invoke your program as:

    mpiexec -n 1 mpicc_argument.exe a "" b

Intel MPI Library receives the correct command-line information:

    mpicc_argument.exe a "" b

and passes it to the CreateProcess function, which proceeds to create your process instance, but with the following command-line:

    mpicc_argument.exe a b

The same happens when using mpiexec.smpd.exe, also.

Thank you,

-Steve