OK one of my settings is wrong. I can not figure out which one it is. When I enter: "mpirun -hosts mic0 -np 128 ./a.out" I get an error /packages/intel/compilers_and_libraries_2016.3.210/linux/mpi/intel64/bin/pmi_proxy: No such file or directory.
I have the following settings on the host:
env | grep MPI
Thank you Artem,
Difficulty cutting and pasting into these forum editing boxes (but you saw that I had problem editing in the first box submission). I have things working now. My environment settings were wrong. Notice in my original posting when I entered echo $I_MPI_MIC I got the correct result - "enable" but also notice that when I entered "env | grep -i mpi" I_MPI_MIC was not part of the environment settings! My script setting this was configured wrong. When I corrected the script setting my environment and then enter "env | grep -i mpi" I_MP_MIC is part of the environment settings (and other things). Things are working correctly for me now thank you.