mpiexec hangs for 5 seconds when launching notepad

[adambruss]

I'm running Intel(R) MPI Library for Windows* OS, Version 4.0 Update 3 Build 8/24/2011 3:07:12 PM on Windows 8 64bit. Mpiexec takes 5 seconds to launch notepad. I'm trying to figure out where the time is being spent. The computer this happens on is an Intel Core i7 3.20 GHz with 16 GB of RAM. This test runs mpiexec when minimal CPU resources are being used.

The command line: mpiexec -verbose -localonly notepad

I've marked below where the hang happens in the log.

.......moving 'launch' command to the wait_list.
....../smpd_state_writing_cmd
...../smpd_handle_op_write
.....sock_waiting for the next event.
.....\SMPDU_Sock_wait

HANGS FOR 5 SECONDS FOR NO APPARENT REASON.

notepad finally launches and closing it shuts down mpiexec

I have attached the entire verbose log and marked it clearly where the hang happens. We have other Windows 8 machines on our domain where mpiexec runs fine without the hang. I've tried reinstalling multiple times and restarting the smpd service. Nothing has helped yet. I've tried disabling firewall and Windows 8 defender to no avail. Please see the attached log. THe hang happens in the same spot every time according to the log.

[James_T_Intel]

Hi Adam,

First, keep in mind that the Intel® MPI Library does not yet officially support Windows* 8.

Have you seen this behavior with command-line only programs?

Sincerely
James Tullos
Technical Consulting Engineer
Intel® Cluster Tools

[adambruss]

Yes I see the behavior with command-line only programs such as svn and dir.

[James_T_Intel]

Hi Adam,

Please try running an MPI program (even a simple hello world) with

[plain]-verbose -genv I_MPI_DEBUG 5[/plain]

and send me the output.

Sincerely,
James Tullos
Technical Consulting Engineer
Intel® Cluster Tools

[adambruss]

Command line: mpiexec -verbose -genv I_MPI_DEBUG 5 -localonly c:\Users\abruss\Desktop\helloworld.exe > c:\temp\mpiout.txt

helloworld.exe writes out a hello world messages and returns.

The output is attached.

[James_T_Intel]

Hi Adam,

Does your helloworld.exe make any MPI calls?  Was it compiled and linked with the Intel® MPI Library?  The verbose output is shown, but there is still no debug output.  It appears you are only using the runtime libraries.  If needed, I can provide an executable to test.

Sincerely,
James Tullos
Technical Consulting Engineer
Intel® Cluster Tools

[adambruss]

No it doens't make MPI calls. It's just a bare bones console application. It's not linked with the Intel MPI library. I'm just trying to get mpiexec to run a simple app at this point. If mpiexec can't properly run notepad or a helloworld app then why would I care if it can properly do the MPI stuff? That's like worrying about air conditioning in a car when the motor is faulty.

If it would help the debugging process, I can use your exe and provide the debugging output. 

[James_T_Intel]

Hi Adam,

It will provide additional information.  I don't know if it will help or not, but at this point, I will need as much information as possible to submit to our developers.  The debug information is something we typically use, and it is only available with an MPI program.

I am attaching a zip file containing the executable.  Please run the following two commands and attach them in separate output files.

[plain]mpiexec -n 2 -verbose -genv I_MPI_DEBUG 5 test.exe[/plain]
[plain]mpiexec -n 2 -verbose -genv I_MPI_DEBUG 1000 test.exe[/plain]

The first is standard, the second will give information that is normally only useful in specialized cases.  Normally, this is unneeded, but it will be easier to go ahead and have it just in case.

Sincerely,
James Tullos
Technical Consulting Engineer
Intel® Cluster Tools

[adambruss]

I ran the two commands. The output files are attached.

[James_T_Intel]

Hi Adam,

There's still something odd.  Namely, there is no debug output.  Let's try something else.  Please send the output from the following commands:

[plain]where mpiexec[/plain]

[plain]echo %I_MPI_ROOT%[/plain]

[plain]mpiexec -V[/plain]

Maybe one of these will shed some light on what's happening.

Sincerely,
James Tullos
Technical Consulting Engineer
Intel® Cluster Tools

[adambruss]

James, I ran the three commands and put the output in the attachment.

[James_T_Intel]

Hi Adam,

Everything here looks normal.  Is it possible for you to try using the most recent version (4.1)?

Sincerely,
James Tullos
Technical Consulting Engineer
Intel® Cluster Tools

[adambruss]

Sure, could you point me to it for download?

[James_T_Intel]

Hi Adam,

If you have registered the runtime environment previously, you can go to the Intel® Registration Center and download it from there.  If not, the best way is to go to http://www.intel.com/go/mpi and choose the Runtime for Windows* link on the right.

Sincerely,
James Tullos
Technical Consulting Engineer
Intel® Cluster Tools

[adambruss]

I got my colleague who is registered to download version 4.1. I installed it and tested things. There is still a delay when starting a program like notepad but my internal tests are passing so I'm able to move forward. The internal tests consist of using mpiexe to run a program that actually uses mpi.

[James_T_Intel]

Hi Adam,

That's more in line with what we've seen thus far.  We have seen a problem loading graphical applications with mpiexec in Windows* 8.  Did notepad actually start?  The behavior we have seen is that graphical applications do not launch at all.

Sincerely,
James Tullos
Technical Consulting Engineer
Intel® Cluster Tools

[adambruss]

mpiexec -localonly -np 2 notepad

The first instance of notepad starts after about 10 seconds. The second instance starts about 5 seconds after the first instance started.

C:\Users\abruss\Desktop>mpiexec -localonly -np 2 helloworld.exe
######### adam bruss - Hello World!!! - adam bruss ##########
######### adam bruss - Hello World!!! - adam bruss ##########

For the console app above it takes 20 some seconds to start them and they both start at the same time. 

And I see the same type of delay when running our actual app which uses MPI. This is all on Windows 8. At least things work now. There's just a delay before the programs fire up.

[James_T_Intel]

Hi Adam,

What happens if you run without -localonly?

Sincerely,
James Tullos
Technical Consulting Engineer
Intel® Cluster Tools

[adambruss]

What happens if you run without -localonly?

For the console application the same thing, a delay.

For notepad it never shows.

[James_T_Intel]

Do you have the IMPI runtime environment installed, or are you running without installing it?

James.

[adambruss]

I'm running with Intel MPI Library Runtime Environment 4.1.0.028.