How to compile for the Phi from a remote host?

[CStac]

Greetings,

My Phi just got in earlier this week. So far things have gone well and I have been able to answer all my questions from the documentation/forums. However, I have hit a bit of a snag and my searches are not turning up anything of use to solve the problem (just a lot of things that don't work).

I have the Phi set up in a cluster with a scheduler. On the cluster, I offer several desktop nodes for the users to log into. From there they can run the Intel Cluster Studio 2015 (just updated it yesterday to the latest) and have access to all of the tools (well, I have VTune installed on those hosts and the Phi node only; not the whole cluster while the other tools are availible on every host). Typically the users compile and test locally before submitting to the scheduler queue. However, I can't seem to compile for the Phi from these hosts.
<!--break-->

I have chased several other rabbit trails and I am getitng stumped as to why I can't seem to compile for the Phi on a remote host. Here are what I believe to be some of the more relevant details on things I have tried:

#include <stdio.h>
#include <unistd.h>
int main(){
	printf("Message from the host! I have %ld logical cores.\n",
	sysconf(_SC_NPROCESSORS_ONLN ));
#pragma offload target(mic)
	{
		printf("Message from the Phi! I have %ld logical cores.\n",
		sysconf(_SC_NPROCESSORS_ONLN ));
		fflush(0);
	}
}

On the Phi host, if I compile this and submit it to the scheduler I get this:
$ module load compilers/intel_64
$ icc -o both_host_and_phi_a.out hello.c
$ echo 'module load compilers/intel_64;/home/user/code/Intel_Phi/both_host_and_phi_a.out' | qsub -q phi.q -N PhiTest01
Your job 44478 ("PhiTest01") has been submitted
$ cat ~/PhiTest01.*
Message from the Phi! I have 244 logical cores.
Message from the host! I have 20 logical cores.

It works. Great. Now if I jump over to one of the compile hosts.
$ module load compilers/intel_64
$ icc -o both_host_and_phi_a.out hello.c
icc: warning #10362: Environment configuration problem encountered.  Please check for proper MPSS installation and environment setup.
hello.c(1): catastrophic error: *MIC* cannot open source file "stdio.h"
  #include <stdio.h>
                    ^
compilation aborted for hello.c (code 4)

Errr....OK. Maybe I have to specify '-mmic'. I am still confused on when I am supposed to use '-mmic' and when I am not suppoed to. While on the Phi node, the Intel compiler tends to get angry at me if I use '-mmic' on any of the test code with pragmas...but maybe since I am not on the Phi node, I need to specify it??

$ icc -mmic -o both_host_and_phi_a.out hello.c
icc: warning #10362: Environment configuration problem encountered.  Please check for proper MPSS installation and environment setup.
hello.c(1): catastrophic error: cannot open source file "stdio.h"
  #include <stdio.h>
                    ^
compilation aborted for hello.c (code 4)

Nope. Well, it complains about the environment configuration. Maybe I screwed up when I wrote the environment modules (I am usually very careful but who knows...I have goofed up before). Lets dump those and try with the Intel provided source script.
$ module unload compilers/intel_64
$ . /opt/intel/composer_xe_2015.0.090/bin/compilervars.sh intel64
$ icc -o both_host_and_phi_a.out hello.c
icc: warning #10362: Environment configuration problem encountered.  Please check for proper MPSS installation and environment setup.
hello.c(1): catastrophic error: *MIC* cannot open source file "stdio.h"
  #include <stdio.h>
                    ^
compilation aborted for hello.c (code 4)
$ icc -mmic -o both_host_and_phi_a.out hello.c
icc: warning #10362: Environment configuration problem encountered.  Please check for proper MPSS installation and environment setup.
hello.c(1): catastrophic error: cannot open source file "stdio.h"
  #include <stdio.h>
                    ^
compilation aborted for hello.c (code 4)

Still not working. OK, well it says I need to have MPSS installed.
$ cat ~/mpss-3.3/docs
[snip]
2.1 Requirements
[snip]
  3) Supported hardware platform with at least one Intel(R) Xeon Phi(TM) coprocessor installed
[snip]

Well I don't have a Phi installed in the desktop compile nodes...And I see nothing in the documentation that leads me to believe I need the kernel modules installed. Maybe there is just a library I need from MPSS? But which one of these MPSS rpm's do I need to install to just compile?

Also, I really thought that when I installed Intel 2015, it set up a bunch of environment variables for the Phi. Well, lets check.
$ env | grep -i mic
MANPATH=[snip]:/opt/intel/composer_xe_2015.0.090/debugger/gdb/intel64_mic/share/man/:[snip]
GDBSERVER_MIC=/opt/intel/composer_xe_2015.0.090/debugger/gdb/target/mic/bin/gdbserver
MIC_LD_LIBRARY_PATH=/opt/intel/composer_xe_2015.0.090/compiler/lib/mic:/opt/intel/composer_xe_2015.0.090/mpirt/lib/mic:/opt/intel/composer_xe_2015.0.090/compiler/lib/mic:/opt/intel/composer_xe_2015.0.090/mkl/lib/mic:/opt/intel/composer_xe_2015.0.090/tbb/lib/mic:/opt/intel/composer_xe_2015.0.090/compiler/lib/mic
MIC_LIBRARY_PATH=/opt/intel/composer_xe_2015.0.090/compiler/lib/mic:/opt/intel/composer_xe_2015.0.090/mpirt/lib/mic:/opt/intel/composer_xe_2015.0.090/tbb/lib/mic:/opt/intel/composer_xe_2015.0.090/tbb/lib/mic
NLSPATH=[snip]:/opt/intel/composer_xe_2015.0.090/debugger/gdb/intel64_mic/share/locale/%l_%t/%N:[snip]
PATH=[snip]:/opt/intel/composer_xe_2015.0.090/debugger/gdb/intel64_mic/bin:[snip]
GDB_CROSS=/opt/intel/composer_xe_2015.0.090/debugger/gdb/intel64_mic/bin/gdb-mic
INFOPATH=[snip]:/opt/intel/composer_xe_2015.0.090/debugger/gdb/intel64_mic/share/info/:/opt/intel/composer_xe_2015.0.090/debugger/gdb/intel64_mic/share/info

Yup! They are there alright.
Anyone know what I am missing?
Thanks!

[Kevin_D_Intel]

Your remote host is missing the k1om binutils (at least) that install as part of MPSS and are needed for compilation targeting the Xeon Phi™.

I have not tried this with MPSS 3.3 yet myself, but try only installing the MPSS 3.3 mpss-sdk-k1om-3.3-1.x86_64.rpm on your remote host since you note having the latest Intel Cluster Studio XE 2015 and then retry the compilation.

That single RPM above should work for ICS XE 2015. If you have/use 14.0 or earlier compilers then you need to install the additional intel-composerxe-compat-k1om-3.3-1.x86_64.rpm on the remote host. This RPM supports the 14.0 and earlier compilers from the Composer XE 2013 SP1 (and Intel Cluster Studio XE 2013).

[CStac]

Greetings,

Thank you so much! I installed both of those RPM's as I have users who use both. Everything compiled for my test code!

Thanks again!

[Evan_P_Intel]

Chris S. wrote:

Maybe there is just a library I need from MPSS? But which one of these MPSS rpm's do I need to install to just compile?

Successful cross-compilation (or offload compilation) for Xeon Phi using an Intel compiler requires that you have installed, at minimum, these two RPMs from the MPSS installation package:

• mpss-sdk-k1om-3.3-1.x86_64.rpm
  • When compiling for x86_64, the non-Intel-compiler components you need—system libraries and their headers (e.g. glibc), an assembler, a linker—are provided by your Linux distribution vendor (e.g. they are part of RHEL, SLE, etc.); when building for Xeon Phi, these components are provided by this RPM.
• intel-composerxe-compat-k1om-3.3-1.x86_64.rpm
  • MPSS 3.x is much more disciplined when creating its cross-compilation environment than MPSS 2.x, and as a side-effect the installation locations of the files that comprise it have changed. In order to retain compatibility with previous versions of Intel Composer XE, which look for the files in their MPSS 2.x locations, this RPM installs a series of symlinks to paper over the differences in MPSS 3.x. (I'm not aware of any released versions of Intel Composer XE yet assume the new locations, so for the moment you may need to install this RPM regardless of the version you have.)

You may need to install more than the minimum, depending on what exactly you'll be compiling. Likely suspects are x86_64 RPMs with "-dev-" (headers, static libraries, shared objects) and "-doc-" (documentation) in the filename.

-- EDIT: Too slow to beat Kevin. Also changed wording to reflect my lack of familiarity with the latest Intel Composer XE.