OpenMP with multiple processes

[schwitrs]

I have an computational application which makes heavy use of fork to generate subprocesses which carry out (mostly) independent calculations.

With Intel C++ 14.0.3 20140422, I find that the application gives incorrect results for OMP_NUM_THREADS>1 if I have called fork(), but works otherwise. Is there some way to fix this?

Could I, for instance, explicitly shut down OpenMP before the fork and then restart it?

thank you--

Charles

[jimdempseyatthecove]

When using fork, you must do so prior to issuing any OpenMP directives or omp_... library calls.

Making any such #pragmas or library calls may initialize the OpenMP state within the program. An then these states will be copied to the child process. IOW the child process won't see the OpenMP uninitialized state, and thus assume it is initialize (which it won't be for the child process).

Jim Dempsey

[schwitrs]

Thank you very much for the reply. That covers about 95% of my use case. Do you know whether the __kmpc_end, or something like it can be used to shut down OpenMP threads before a fork?

thanks again--

Charles

[jimdempseyatthecove]

Think about using CreateProcess to run the new processes. Pass any startup context via the command line arguments (e.g. ID number, and/or list of initial args or file name containing args).

Note, you can run the program "Start" followed by option switches then your program name then command line arguments. You can use the program Start to manipulate the CMD window.

Jim Dempsey

[jimdempseyatthecove]

Another hack you can do, I will sketch it for you, and let you figure out how to implement it.

At your very first start (pre-OpenMP) do any global initialization for your application.

Next, enter the main thread loop that does no work for your application other than

a) perform an initial fork to start the actual processing, then
b) run a loop till done performing fork's as a proxy service for the process started by a) or subsequently forked processes

You will have to figure out how to signal the main (initial) process when it is time to perform a fork.

Jim Dempsey

[schwitrs]

Thank you for the reply. I'll have to ask: what is createProcess- is this a windows call?

[jimdempseyatthecove]

CreateProcess is a Windows API. On Linux, consider using posix_spawn http://linux.die.net/man/3/posix_spawn

I found this simple example on stackoverflow.com:

#include <stdlib.h>
#include <stdio.h>
#include <string.h>

#include <unistd.h>
#include <spawn.h>
#include <sys/wait.h>

extern char **environ;

void test_fork_exec(void);
void test_posix_spawn(void);

int main(void) {
  test_fork_exec();
  test_posix_spawn();
  return EXIT_SUCCESS;
}

void test_fork_exec(void) {
  pid_t pid;
  int status;
  puts("Testing fork/exec");
  fflush(NULL);
  pid = fork();
  switch (pid) {
  case -1:
    perror("fork");
    break;
  case 0:
    execl("/bin/ls", "ls", (char *) 0);
    perror("exec");
    break;
  default:
    printf("Child id: %i\n", pid);
    fflush(NULL);
    if (waitpid(pid, &status, 0) != -1) {
      printf("Child exited with status %i\n", status);
    } else {
      perror("waitpid");
    }
    break;
  }
}

void test_posix_spawn(void) {
  pid_t pid;
  char *argv[] = {"ls", (char *) 0};
  int status;
  puts("Testing posix_spawn");
  fflush(NULL);
  status = posix_spawn(&pid, "/bin/ls", NULL, NULL, argv, environ);
  if (status == 0) {
    printf("Child id: %i\n", pid);
    fflush(NULL);
    if (waitpid(pid, &status, 0) != -1) {
      printf("Child exited with status %i\n", status);
    } else {
      perror("waitpid");
    }
  } else {
    printf("posix_spawn: %s\n", strerror(status));
  }
}

Modify it to remove the fork test and insert an OpenMP parallel region before the call the test_posix_spawn.

Please report back if this fixes your issue.

Jim Dempsey