host: service56 host: service54 ================================================================================================== mpiexec options: ---------------- Base path: /ichec/home/packages/intel-cluster-studio/2013-sp1-u1/impi/4.1.2.040/intel64/bin/ Launcher: ssh Debug level: 1 Enable X: -1 Global environment: ------------------- I_MPI_PERHOST=allcores LD_LIBRARY_PATH=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/impi/4.1.2.040/intel64/lib:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/tbb/lib/intel64:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/mkl/lib/intel64:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/itac/8.1.3.037/itac/slib_impi4:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/ipp/lib/intel64:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/debugger/lib/intel64:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/debugger/gui/intel64:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/compiler/lib/intel64:/ichec/packages/intel-cluster-studio/2013-sp1-u1/impi/4.1.2.040/intel64/lib:/ichec/packages/intel-cluster-studio/2013-sp1-u1/impi/4.1.2.040/mic/lib I_MPI_HYDRA_DEBUG=1 MKLROOT=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/mkl CPLUS_INCLUDE_PATH=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/impi/4.1.2.040/intel64/include MODULE_VERSION_STACK=3.1.6 LESSKEY=/etc/lesskey.bin NNTPSERVER=news INFODIR=/usr/local/info:/usr/share/info:/usr/info MANPATH=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/impi/4.1.2.040/man:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/itac/8.1.3.037/man:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/man/en_US:/usr/local/man:/usr/share/man:/opt/intel/mic/share/man:/opt/c3/man:/opt/sgi/share/man VT_MPI=impi4 HOSTNAME=service56 XKEYSYMDB=/usr/share/X11/XKeysymDB PBS_VERSION=TORQUE-4.2.5 IPPROOT=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/ipp INTEL_LICENSE_FILE=28518@service0 _MODULESBEGINENV_=/ichec/home/staff/alin/.modulesbeginenv HOST=service56 SHELL=/bin/bash TERM=xterm PROFILEREAD=true HISTSIZE=1000 TMPDIR=/ichec/work/scratch/tmp/30202.service1.cb3.ichec.ie PBS_JOBNAME=STDIN LIBRARY_PATH=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/impi/4.1.2.040/intel64/lib:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/tbb/lib/intel64:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/mkl/lib/intel64:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/ipp/lib/intel64:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/compiler/lib/intel64 FPATH=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/impi/4.1.2.040/intel64/include:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/mkl/include MORE=-sl PBS_ENVIRONMENT=PBS_INTERACTIVE OLDPWD=/ichec/home/staff/alin/acratus/XeonPhi/hello/fortran/mpiNative PBS_O_WORKDIR=/ichec/home/staff/alin PBS_TASKNUM=1 USER=alin VTROOT=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/itac/8.1.3.037 LS_COLORS=no=00:fi=00:di=01;34:ln=00;36:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=41;33;01:ex=00;32:*.cmd=00;32:*.exe=01;32:*.com=01;32:*.bat=01;32:*.btm=01;32:*.dll=01;32:*.tar=00;31:*.tbz=00;31:*.tgz=00;31:*.rpm=00;31:*.deb=00;31:*.arj=00;31:*.taz=00;31:*.lzh=00;31:*.lzma=00;31:*.zip=00;31:*.zoo=00;31:*.z=00;31:*.Z=00;31:*.gz=00;31:*.bz2=00;31:*.tb2=00;31:*.tz2=00;31:*.tbz2=00;31:*.avi=01;35:*.bmp=01;35:*.fli=01;35:*.gif=01;35:*.jpg=01;35:*.jpeg=01;35:*.mng=01;35:*.mov=01;35:*.mpg=01;35:*.pcx=01;35:*.pbm=01;35:*.pgm=01;35:*.png=01;35:*.ppm=01;35:*.tga=01;35:*.tif=01;35:*.xbm=01;35:*.xpm=01;35:*.dl=01;35:*.gl=01;35:*.wmv=01;35:*.aiff=00;32:*.au=00;32:*.mid=00;32:*.mp3=00;32:*.ogg=00;32:*.voc=00;32:*.wav=00;32:*.xyz=00;35:*.inp=00;33:*.vmd=00;32:*.restart=00;33: PBS_O_HOME=/ichec/home/staff/alin XNLSPATH=/usr/share/X11/nls ENV=/etc/bash.bashrc CPATH=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/impi/4.1.2.040/intel64/include:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/tbb/include:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/mkl/include:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/ipp/include OSCAR_HOME=/opt/oscar HOSTTYPE=x86_64 PBS_WALLTIME=1800 FROM_HEADER= PBS_GPUFILE=/var/spool/torque/aux//30202.service1.cb3.ichec.iegpu PBS_MOMPORT=15003 PAGER=less CSHEDIT=emacs XDG_CONFIG_DIRS=/etc/xdg PBS_O_QUEUE=PhiQ NLSPATH=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/mkl/lib/intel64/locale/en_US/%N:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/ipp/lib/intel64/locale/en_US/%N:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/debugger/intel64/locale/en_US/%N:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/compiler/lib/intel64/locale/en_US/%N MINICOM=-c on VT_ADD_LIBS=-ldwarf -lelf -lvtunwind -lnsl -lm -ldl -lpthread MODULE_VERSION=3.1.6 MAIL=/var/spool/mail/alin PATH=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/impi/4.1.2.040/intel64/bin:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/itac/8.1.3.037/bin:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/bin/intel64:/ichec/home/staff/alin/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/opt/c3/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:.:/opt/sgi/sbin:/opt/sgi/bin PBS_O_LOGNAME=alin CPU=x86_64 PBS_O_LANG=en_IE.UTF-8 MKL_NUM_THREADS=1 PBS_JOBCOOKIE=885F487FC4EC93DAD79EAF1B84823E5B C_INCLUDE_PATH=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/impi/4.1.2.040/intel64/include TBBROOT=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/tbb INPUTRC=/etc/inputrc PWD=/ichec/home/staff/alin _LMFILES_=/ichec/home/staff/alin/modules/intel/2013-sp1-1 IDB_HOME=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/bin/intel64 LANG=en_US.UTF-8 PBS_NODENUM=0 PYTHONSTARTUP=/etc/pythonstart MODULEPATH=/ichec/home/staff/alin/modules:/ichec/work/tiche041a/modules:/ichec/work/tcphy039b/modules:/ichec/modulefiles/base VT_LIB_DIR=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/itac/8.1.3.037/itac/lib_impi4 LOADEDMODULES=intel/2013-sp1-1 PBS_NUM_NODES=2 C3_RSH=ssh -oConnectTimeout=10 -oForwardX11=no PS1=\[\033[0;32m\][\u@\h:\[\033[0m\]\[\033[1;33m\]\w\[\033[0m\]\[\033[0;32m\]]:\[\033[0m\] PBS_O_SHELL=/bin/bash PBS_JOBID=30202.service1.cb3.ichec.ie HISTCONTROL=ignorespace QT_SYSTEM_DIR=/usr/share/desktop-data SHLVL=1 HOME=/ichec/home/staff/alin LESS_ADVANCED_PREPROCESSOR=no OSTYPE=linux LS_OPTIONS=-N --color=tty -T 0 XCURSOR_THEME=crystalwhite PBS_O_HOST=fionn3.cb3.ichec.ie VT_SLIB_DIR=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/itac/8.1.3.037/itac/slib_impi4 WINDOWMANAGER=/usr/bin/gnome BASH_ENV=/etc/profile PBS_VNODENUM=0 G_FILENAME_ENCODING=@locale,UTF-8,ISO-8859-15,CP1252 LESS=-M -I MACHTYPE=x86_64-suse-linux LOGNAME=alin CVS_RSH=ssh CLASSPATH=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/itac/8.1.3.037/itac/lib_impi4 XDG_DATA_DIRS=/usr/share:/etc/opt/kde3/share:/opt/kde3/share PBS_QUEUE=PhiQ MODULESHOME=/usr/share/modules LESSOPEN=lessopen.sh %s PBS_O_MAIL=/var/mail/alin PBS_MICFILE=/var/spool/torque/aux//30202.service1.cb3.ichec.iemic INTEL_MPI_PATH=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/impi/4.1.2.040 tfiles=*.fdf *psf *vps *HST *DM bulk* *KPIN *HM INFOPATH=/usr/local/info:/usr/share/info:/usr/info PBS_NP=40 PBS_NUM_PPN=20 PBS_O_SERVER=service1 INCLUDE=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/mkl/include:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/ipp/include LESSCLOSE=lessclose.sh %s %s HISTTIMEFORMAT=%F %T G_BROKEN_FILENAMES=1 PBS_NODEFILE=/var/spool/torque/aux//30202.service1.cb3.ichec.ie I_MPI_ROOT=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/impi/4.1.2.040 COLORTERM=1 PBS_O_PATH=/ichec/home/staff/alin/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/opt/c3/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:/opt/mam/bin:/opt/moab/bin:/opt/moab/sbin:.:/opt/sgi/sbin:/opt/sgi/bin module=() { eval `/usr/bin/modulecmd bash $*` } _=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/impi/4.1.2.040/intel64/bin/mpiexec.hydra Hydra internal environment: --------------------------- MPICH_ENABLE_CKPOINT=1 GFORTRAN_UNBUFFERED_PRECONNECTED=y Proxy information: ********************* [1] proxy: service56 (20 cores) Exec list: ./hello.X (4 processes); ================================================================================================== [mpiexec@service56] Timeout set to -1 (-1 means infinite) [mpiexec@service56] Got a control port string of service56:52352 Proxy launch args: /ichec/home/packages/intel-cluster-studio/2013-sp1-u1/impi/4.1.2.040/intel64/bin/pmi_proxy --control-port service56:52352 --debug --pmi-connect lazy-cache --pmi-aggregate -s 0 --rmk pbs --launcher ssh --demux poll --pgid 0 --enable-stdin 1 --retries 10 --control-code 803524668 --proxy-id [mpiexec@service56] PMI FD: (null); PMI PORT: (null); PMI ID/RANK: -1 Arguments being passed to proxy 0: --version 1.4.1p1 --iface-ip-env-name MPICH_INTERFACE_HOSTNAME --hostname service56 --global-core-map 0,20,0 --filler-process-map 0,20,0 --global-process-count 4 --auto-cleanup 1 --pmi-rank -1 --pmi-kvsname kvs_38714_0 --pmi-process-mapping (vector,(0,2,20)) --topolib ipl --ckpointlib blcr --ckpoint-prefix /tmp --ckpoint-preserve 1 --ckpoint off --ckpoint-num -1 --global-inherited-env 117 'I_MPI_PERHOST=allcores' 'LD_LIBRARY_PATH=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/impi/4.1.2.040/intel64/lib:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/tbb/lib/intel64:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/mkl/lib/intel64:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/itac/8.1.3.037/itac/slib_impi4:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/ipp/lib/intel64:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/debugger/lib/intel64:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/debugger/gui/intel64:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/compiler/lib/intel64:/ichec/packages/intel-cluster-studio/2013-sp1-u1/impi/4.1.2.040/intel64/lib:/ichec/packages/intel-cluster-studio/2013-sp1-u1/impi/4.1.2.040/mic/lib' 'I_MPI_HYDRA_DEBUG=1' 'MKLROOT=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/mkl' 'CPLUS_INCLUDE_PATH=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/impi/4.1.2.040/intel64/include' 'MODULE_VERSION_STACK=3.1.6' 'LESSKEY=/etc/lesskey.bin' 'NNTPSERVER=news' 'INFODIR=/usr/local/info:/usr/share/info:/usr/info' 'MANPATH=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/impi/4.1.2.040/man:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/itac/8.1.3.037/man:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/man/en_US:/usr/local/man:/usr/share/man:/opt/intel/mic/share/man:/opt/c3/man:/opt/sgi/share/man' 'VT_MPI=impi4' 'HOSTNAME=service56' 'XKEYSYMDB=/usr/share/X11/XKeysymDB' 'PBS_VERSION=TORQUE-4.2.5' 'IPPROOT=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/ipp' 'INTEL_LICENSE_FILE=28518@service0' '_MODULESBEGINENV_=/ichec/home/staff/alin/.modulesbeginenv' 'HOST=service56' 'SHELL=/bin/bash' 'TERM=xterm' 'PROFILEREAD=true' 'HISTSIZE=1000' 'TMPDIR=/ichec/work/scratch/tmp/30202.service1.cb3.ichec.ie' 'PBS_JOBNAME=STDIN' 'LIBRARY_PATH=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/impi/4.1.2.040/intel64/lib:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/tbb/lib/intel64:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/mkl/lib/intel64:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/ipp/lib/intel64:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/compiler/lib/intel64' 'FPATH=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/impi/4.1.2.040/intel64/include:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/mkl/include' 'MORE=-sl' 'PBS_ENVIRONMENT=PBS_INTERACTIVE' 'OLDPWD=/ichec/home/staff/alin/acratus/XeonPhi/hello/fortran/mpiNative' 'PBS_O_WORKDIR=/ichec/home/staff/alin' 'PBS_TASKNUM=1' 'USER=alin' 'VTROOT=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/itac/8.1.3.037' 'LS_COLORS=no=00:fi=00:di=01;34:ln=00;36:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=41;33;01:ex=00;32:*.cmd=00;32:*.exe=01;32:*.com=01;32:*.bat=01;32:*.btm=01;32:*.dll=01;32:*.tar=00;31:*.tbz=00;31:*.tgz=00;31:*.rpm=00;31:*.deb=00;31:*.arj=00;31:*.taz=00;31:*.lzh=00;31:*.lzma=00;31:*.zip=00;31:*.zoo=00;31:*.z=00;31:*.Z=00;31:*.gz=00;31:*.bz2=00;31:*.tb2=00;31:*.tz2=00;31:*.tbz2=00;31:*.avi=01;35:*.bmp=01;35:*.fli=01;35:*.gif=01;35:*.jpg=01;35:*.jpeg=01;35:*.mng=01;35:*.mov=01;35:*.mpg=01;35:*.pcx=01;35:*.pbm=01;35:*.pgm=01;35:*.png=01;35:*.ppm=01;35:*.tga=01;35:*.tif=01;35:*.xbm=01;35:*.xpm=01;35:*.dl=01;35:*.gl=01;35:*.wmv=01;35:*.aiff=00;32:*.au=00;32:*.mid=00;32:*.mp3=00;32:*.ogg=00;32:*.voc=00;32:*.wav=00;32:*.xyz=00;35:*.inp=00;33:*.vmd=00;32:*.restart=00;33:' 'PBS_O_HOME=/ichec/home/staff/alin' 'XNLSPATH=/usr/share/X11/nls' 'ENV=/etc/bash.bashrc' 'CPATH=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/impi/4.1.2.040/intel64/include:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/tbb/include:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/mkl/include:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/ipp/include' 'OSCAR_HOME=/opt/oscar' 'HOSTTYPE=x86_64' 'PBS_WALLTIME=1800' 'FROM_HEADER=' 'PBS_GPUFILE=/var/spool/torque/aux//30202.service1.cb3.ichec.iegpu' 'PBS_MOMPORT=15003' 'PAGER=less' 'CSHEDIT=emacs' 'XDG_CONFIG_DIRS=/etc/xdg' 'PBS_O_QUEUE=PhiQ' 'NLSPATH=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/mkl/lib/intel64/locale/en_US/%N:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/ipp/lib/intel64/locale/en_US/%N:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/debugger/intel64/locale/en_US/%N:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/compiler/lib/intel64/locale/en_US/%N' 'MINICOM=-c on' 'VT_ADD_LIBS=-ldwarf -lelf -lvtunwind -lnsl -lm -ldl -lpthread' 'MODULE_VERSION=3.1.6' 'MAIL=/var/spool/mail/alin' 'PATH=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/impi/4.1.2.040/intel64/bin:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/itac/8.1.3.037/bin:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/bin/intel64:/ichec/home/staff/alin/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/opt/c3/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:.:/opt/sgi/sbin:/opt/sgi/bin' 'PBS_O_LOGNAME=alin' 'CPU=x86_64' 'PBS_O_LANG=en_IE.UTF-8' 'MKL_NUM_THREADS=1' 'PBS_JOBCOOKIE=885F487FC4EC93DAD79EAF1B84823E5B' 'C_INCLUDE_PATH=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/impi/4.1.2.040/intel64/include' 'TBBROOT=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/tbb' 'INPUTRC=/etc/inputrc' 'PWD=/ichec/home/staff/alin' '_LMFILES_=/ichec/home/staff/alin/modules/intel/2013-sp1-1' 'IDB_HOME=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/bin/intel64' 'LANG=en_US.UTF-8' 'PBS_NODENUM=0' 'PYTHONSTARTUP=/etc/pythonstart' 'MODULEPATH=/ichec/home/staff/alin/modules:/ichec/work/tiche041a/modules:/ichec/work/tcphy039b/modules:/ichec/modulefiles/base' 'VT_LIB_DIR=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/itac/8.1.3.037/itac/lib_impi4' 'LOADEDMODULES=intel/2013-sp1-1' 'PBS_NUM_NODES=2' 'C3_RSH=ssh -oConnectTimeout=10 -oForwardX11=no' 'PS1=\[\033[0;32m\][\u@\h:\[\033[0m\]\[\033[1;33m\]\w\[\033[0m\]\[\033[0;32m\]]:\[\033[0m\] ' 'PBS_O_SHELL=/bin/bash' 'PBS_JOBID=30202.service1.cb3.ichec.ie' 'HISTCONTROL=ignorespace' 'QT_SYSTEM_DIR=/usr/share/desktop-data' 'SHLVL=1' 'HOME=/ichec/home/staff/alin' 'LESS_ADVANCED_PREPROCESSOR=no' 'OSTYPE=linux' 'LS_OPTIONS=-N --color=tty -T 0' 'XCURSOR_THEME=crystalwhite' 'PBS_O_HOST=fionn3.cb3.ichec.ie' 'VT_SLIB_DIR=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/itac/8.1.3.037/itac/slib_impi4' 'WINDOWMANAGER=/usr/bin/gnome' 'BASH_ENV=/etc/profile' 'PBS_VNODENUM=0' 'G_FILENAME_ENCODING=@locale,UTF-8,ISO-8859-15,CP1252' 'LESS=-M -I' 'MACHTYPE=x86_64-suse-linux' 'LOGNAME=alin' 'CVS_RSH=ssh' 'CLASSPATH=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/itac/8.1.3.037/itac/lib_impi4' 'XDG_DATA_DIRS=/usr/share:/etc/opt/kde3/share:/opt/kde3/share' 'PBS_QUEUE=PhiQ' 'MODULESHOME=/usr/share/modules' 'LESSOPEN=lessopen.sh %s' 'PBS_O_MAIL=/var/mail/alin' 'PBS_MICFILE=/var/spool/torque/aux//30202.service1.cb3.ichec.iemic' 'INTEL_MPI_PATH=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/impi/4.1.2.040' 'tfiles=*.fdf *psf *vps *HST *DM bulk* *KPIN *HM' 'INFOPATH=/usr/local/info:/usr/share/info:/usr/info' 'PBS_NP=40' 'PBS_NUM_PPN=20' 'PBS_O_SERVER=service1' 'INCLUDE=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/mkl/include:/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/composer_xe_2013_sp1.1.106/ipp/include' 'LESSCLOSE=lessclose.sh %s %s' 'HISTTIMEFORMAT=%F %T ' 'G_BROKEN_FILENAMES=1' 'PBS_NODEFILE=/var/spool/torque/aux//30202.service1.cb3.ichec.ie' 'I_MPI_ROOT=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/impi/4.1.2.040' 'COLORTERM=1' 'PBS_O_PATH=/ichec/home/staff/alin/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/opt/c3/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:/opt/mam/bin:/opt/moab/bin:/opt/moab/sbin:.:/opt/sgi/sbin:/opt/sgi/bin' 'module=() { eval `/usr/bin/modulecmd bash $*` }' '_=/ichec/home/packages/intel-cluster-studio/2013-sp1-u1/impi/4.1.2.040/intel64/bin/mpiexec.hydra' --global-user-env 0 --global-system-env 2 'MPICH_ENABLE_CKPOINT=1' 'GFORTRAN_UNBUFFERED_PRECONNECTED=y' --proxy-core-count 20 --exec --exec-appnum 0 --exec-proc-count 4 --exec-local-env 0 --exec-wdir /ichec/home/staff/alin --exec-args 1 ./hello.X [mpiexec@service56] Launch arguments: /ichec/home/packages/intel-cluster-studio/2013-sp1-u1/impi/4.1.2.040/intel64/bin/pmi_proxy --control-port service56:52352 --debug --pmi-connect lazy-cache --pmi-aggregate -s 0 --rmk pbs --launcher ssh --demux poll --pgid 0 --enable-stdin 1 --retries 10 --control-code 803524668 --proxy-id 0 [mpiexec@service56] STDIN will be redirected to 1 fd(s): 9 [proxy:0:0@service56] Start PMI_proxy 0 [proxy:0:0@service56] STDIN will be redirected to 1 fd(s): 15 [proxy:0:0@service56] got pmi command (from 10): init pmi_version=1 pmi_subversion=1 [proxy:0:0@service56] PMI response: cmd=response_to_init pmi_version=1 pmi_subversion=1 rc=0 [proxy:0:0@service56] got pmi command (from 10): get_maxes [proxy:0:0@service56] PMI response: cmd=maxes kvsname_max=256 keylen_max=64 vallen_max=1024 [proxy:0:0@service56] got pmi command (from 10): barrier_in [proxy:0:0@service56] got pmi command (from 12): init pmi_version=1 pmi_subversion=1 [proxy:0:0@service56] PMI response: cmd=response_to_init pmi_version=1 pmi_subversion=1 rc=0 [proxy:0:0@service56] got pmi command (from 12): get_maxes [proxy:0:0@service56] PMI response: cmd=maxes kvsname_max=256 keylen_max=64 vallen_max=1024 [proxy:0:0@service56] got pmi command (from 12): barrier_in [proxy:0:0@service56] got pmi command (from 19): init pmi_version=1 pmi_subversion=1 [proxy:0:0@service56] PMI response: cmd=response_to_init pmi_version=1 pmi_subversion=1 rc=0 [proxy:0:0@service56] got pmi command (from 19): get_maxes [proxy:0:0@service56] PMI response: cmd=maxes kvsname_max=256 keylen_max=64 vallen_max=1024 [proxy:0:0@service56] got pmi command (from 19): barrier_in [proxy:0:0@service56] got pmi command (from 14): init pmi_version=1 pmi_subversion=1 [proxy:0:0@service56] PMI response: cmd=response_to_init pmi_version=1 pmi_subversion=1 rc=0 [proxy:0:0@service56] got pmi command (from 14): get_maxes [proxy:0:0@service56] PMI response: cmd=maxes kvsname_max=256 keylen_max=64 vallen_max=1024 [proxy:0:0@service56] got pmi command (from 14): barrier_in [proxy:0:0@service56] forwarding command (cmd=barrier_in) upstream [mpiexec@service56] [pgid: 0] got PMI command: cmd=barrier_in [mpiexec@service56] PMI response to fd 6 pid 14: cmd=barrier_out [proxy:0:0@service56] PMI response: cmd=barrier_out [proxy:0:0@service56] PMI response: cmd=barrier_out [proxy:0:0@service56] PMI response: cmd=barrier_out [proxy:0:0@service56] PMI response: cmd=barrier_out [proxy:0:0@service56] got pmi command (from 10): get_ranks2hosts [proxy:0:0@service56] PMI response: put_ranks2hosts 23 1 9 service56 0,1,2,3, [proxy:0:0@service56] got pmi command (from 12): get_ranks2hosts [proxy:0:0@service56] PMI response: put_ranks2hosts 23 1 9 service56 0,1,2,3, [proxy:0:0@service56] got pmi command (from 14): get_ranks2hosts [proxy:0:0@service56] PMI response: put_ranks2hosts 23 1 9 service56 0,1,2,3, [proxy:0:0@service56] got pmi command (from 19): get_ranks2hosts [proxy:0:0@service56] PMI response: put_ranks2hosts 23 1 9 service56 0,1,2,3, [proxy:0:0@service56] got pmi command (from 10): get_appnum [proxy:0:0@service56] PMI response: cmd=appnum appnum=0 [proxy:0:0@service56] got pmi command (from 12): get_appnum [proxy:0:0@service56] PMI response: cmd=appnum appnum=0 [proxy:0:0@service56] got pmi command (from 14): get_appnum [proxy:0:0@service56] PMI response: cmd=appnum appnum=0 [proxy:0:0@service56] got pmi command (from 19): get_appnum [proxy:0:0@service56] PMI response: cmd=appnum appnum=0 [proxy:0:0@service56] got pmi command (from 10): get_my_kvsname [proxy:0:0@service56] PMI response: cmd=my_kvsname kvsname=kvs_38714_0 [proxy:0:0@service56] got pmi command (from 14): get_my_kvsname [proxy:0:0@service56] PMI response: cmd=my_kvsname kvsname=kvs_38714_0 [proxy:0:0@service56] got pmi command (from 19): get_my_kvsname [proxy:0:0@service56] PMI response: cmd=my_kvsname kvsname=kvs_38714_0 [proxy:0:0@service56] got pmi command (from 12): get_my_kvsname [proxy:0:0@service56] PMI response: cmd=my_kvsname kvsname=kvs_38714_0 [proxy:0:0@service56] got pmi command (from 10): get_my_kvsname [proxy:0:0@service56] PMI response: cmd=my_kvsname kvsname=kvs_38714_0 [proxy:0:0@service56] got pmi command (from 19): get_my_kvsname [proxy:0:0@service56] PMI response: cmd=my_kvsname kvsname=kvs_38714_0 [proxy:0:0@service56] got pmi command (from 12): get_my_kvsname [proxy:0:0@service56] PMI response: cmd=my_kvsname kvsname=kvs_38714_0 [proxy:0:0@service56] got pmi command (from 14): get_my_kvsname [proxy:0:0@service56] PMI response: cmd=my_kvsname kvsname=kvs_38714_0 [proxy:0:0@service56] got pmi command (from 19): barrier_in [proxy:0:0@service56] got pmi command (from 10): put kvsname=kvs_38714_0 key=sharedFilename[0] value=/dev/shm/Intel_MPI_T1k1h4 [proxy:0:0@service56] forwarding command (cmd=put kvsname=kvs_38714_0 key=sharedFilename[0] value=/dev/shm/Intel_MPI_T1k1h4) upstream [mpiexec@service56] [pgid: 0] got PMI command: cmd=put kvsname=kvs_38714_0 key=sharedFilename[0] value=/dev/shm/Intel_MPI_T1k1h4 [mpiexec@service56] PMI response to fd 6 pid 10: cmd=put_result rc=0 msg=success [proxy:0:0@service56] got pmi command (from 12): barrier_in [proxy:0:0@service56] got pmi command (from 14): barrier_in [proxy:0:0@service56] we don't understand the response put_result; forwarding downstream [proxy:0:0@service56] got pmi command (from 10): barrier_in [proxy:0:0@service56] forwarding command (cmd=barrier_in) upstream [mpiexec@service56] [pgid: 0] got PMI command: cmd=barrier_in [mpiexec@service56] PMI response to fd 6 pid 10: cmd=barrier_out [proxy:0:0@service56] PMI response: cmd=barrier_out [proxy:0:0@service56] PMI response: cmd=barrier_out [proxy:0:0@service56] PMI response: cmd=barrier_out [proxy:0:0@service56] PMI response: cmd=barrier_out [proxy:0:0@service56] got pmi command (from 12): get kvsname=kvs_38714_0 key=sharedFilename[0] [proxy:0:0@service56] PMI response: cmd=get_result rc=0 msg=success value=/dev/shm/Intel_MPI_T1k1h4 [proxy:0:0@service56] got pmi command (from 14): get kvsname=kvs_38714_0 key=sharedFilename[0] [proxy:0:0@service56] PMI response: cmd=get_result rc=0 msg=success value=/dev/shm/Intel_MPI_T1k1h4 [proxy:0:0@service56] got pmi command (from 19): get kvsname=kvs_38714_0 key=sharedFilename[0] [proxy:0:0@service56] PMI response: cmd=get_result rc=0 msg=success value=/dev/shm/Intel_MPI_T1k1h4 [proxy:0:0@service56] got pmi command (from 10): put kvsname=kvs_38714_0 key=P0-businesscard-0 value=fabrics_list#shm$ [proxy:0:0@service56] got pmi command (from 12): put kvsname=kvs_38714_0 key=P1-businesscard-0 value=fabrics_list#shm$ [proxy:0:0@service56] got pmi command (from 19): put kvsname=kvs_38714_0 key=P3-businesscard-0 value=fabrics_list#shm$ [proxy:0:0@service56] got pmi command (from 14): put kvsname=kvs_38714_0 key=P2-businesscard-0 value=fabrics_list#shm$ [mpiexec@service56] [pgid: 0] got aggregated PMI command (part of it): cmd=put kvsname=kvs_38714_0 key=P0-businesscard-0 value=fabrics_list#shm$ [mpiexec@service56] reply: cmd=put_result rc=0 msg=success [mpiexec@service56] [pgid: 0] got aggregated PMI command (part of it): cmd=put kvsname=kvs_38714_0 key=P1-businesscard-0 value=fabrics_list#shm$ [mpiexec@service56] reply: cmd=put_result rc=0 msg=success [mpiexec@service56] [pgid: 0] got aggregated PMI command (part of it): cmd=put kvsname=kvs_38714_0 key=P3-businesscard-0 value=fabrics_list#shm$ [mpiexec@service56] reply: cmd=put_result rc=0 msg=success [mpiexec@service56] [pgid: 0] got aggregated PMI command (part of it): cmd=put kvsname=kvs_38714_0 key=P2-businesscard-0 value=fabrics_list#shm$ [mpiexec@service56] reply: cmd=put_result rc=0 msg=success [proxy:0:0@service56] got pmi command (from 10): barrier_in [proxy:0:0@service56] got pmi command (from 19): barrier_in [proxy:0:0@service56] got pmi command (from 12): barrier_in [proxy:0:0@service56] got pmi command (from 14): barrier_in [proxy:0:0@service56] forwarding command (cmd=barrier_in) upstream [mpiexec@service56] [pgid: 0] got PMI command: cmd=barrier_in [mpiexec@service56] PMI response to fd 6 pid 14: cmd=barrier_out [proxy:0:0@service56] PMI response: cmd=barrier_out [proxy:0:0@service56] PMI response: cmd=barrier_out [proxy:0:0@service56] PMI response: cmd=barrier_out [proxy:0:0@service56] PMI response: cmd=barrier_out I am process 0 out of 4 running on service56 with MPI version 2.2 I am process 1 out of 4 running on service56 with MPI version 2.2 I am process 3 out of 4 running on service56 with MPI version 2.2 I am process 2 out of 4 running on service56 with MPI version 2.2 [proxy:0:0@service56] got pmi command (from 10): finalize [proxy:0:0@service56] PMI response: cmd=finalize_ack [proxy:0:0@service56] got pmi command (from 19): finalize [proxy:0:0@service56] PMI response: cmd=finalize_ack [proxy:0:0@service56] got pmi command (from 12): finalize [proxy:0:0@service56] PMI response: cmd=finalize_ack [proxy:0:0@service56] got pmi command (from 14): finalize [proxy:0:0@service56] PMI response: cmd=finalize_ack