https://community.intel.com/t5/Intel-MPI-Library/Problem-with-IntelIB-Basic-Intel-MPI-and-I-MPI-FABRICS-tmi/m-p/1059310#M4497 <P>If I set <CODE class="functions">export</CODE> <CODE class="plain">I_MPI_FABRICS=tmi:tmi</CODE> the problem appear too.</P> <P>If I set <CODE class="functions">export</CODE> <CODE class="plain">I_MPI_FABRICS=shm:tmi</CODE> the problem appear.</P> <P>If I set <CODE class="functions">export</CODE> <CODE class="plain">I_MPI_FABRICS=dapl:tmi</CODE> the problem does not appear.</P> <P>Any ideas?</P> Wed, 19 Nov 2014 14:26:00 GMT Guillaume_De_Nayer 2014-11-19T14:26:00Z https://community.intel.com/t5/Intel-MPI-Library/Problem-with-IntelIB-Basic-Intel-MPI-and-I-MPI-FABRICS-tmi/m-p/1059309#M4496 <P>Hi,</P> <P>We have a small cluster (head node + 4 nodes with 16 cores) using Intel infiniband. This cluster is under CentOS 6.6 (with the kernel of CentOS 6.5).</P> <P>On this cluster Intel Parallel Studio XE 2015 is installed. I_MPI_FABRICS is set per default to "tmi" only.</P> <P>When I start a job (using torque+maui) on several nodes, for example this one:</P> <PRE class="brush:bash;">#!/bin/bash #PBS -N IMB-MPI1_intelmpi #PBS -l walltime=2:00:00 #PBS -l nodes=3:ppn=4 cd $PBS_O_WORKDIR export I_MPI_FABRICS=tmi mpirun IMB-MPI1</PRE> <P><BR /> The job is running fine without any problem.</P> <P>Now I start a job on a node:</P> <PRE class="brush:bash;">#!/bin/bash #PBS -N IMB-MPI1_intelmpi #PBS -l walltime=2:00:00 #PBS -l nodes=1:ppn=16 cd $PBS_O_WORKDIR export I_MPI_FABRICS=tmi mpirun IMB-MPI1</PRE> <P>This job does not start and I get this message:</P> <PRE class="brush:plain;">can't open /dev/ipath, network down tmi fabric is not available and fallback fabric is not enabled</PRE> <P>Is it normal?</P> <P>If I set as default I_MPI_FABRICS=dapl, I don't have this problem at all.</P> <P>How can I solve that?</P> <P>Best regards,</P> <P>Guillaume</P> Wed, 19 Nov 2014 07:54:24 GMT https://community.intel.com/t5/Intel-MPI-Library/Problem-with-IntelIB-Basic-Intel-MPI-and-I-MPI-FABRICS-tmi/m-p/1059309#M4496 Guillaume_De_Nayer 2014-11-19T07:54:24Z https://community.intel.com/t5/Intel-MPI-Library/Problem-with-IntelIB-Basic-Intel-MPI-and-I-MPI-FABRICS-tmi/m-p/1059310#M4497 <P>If I set <CODE class="functions">export</CODE> <CODE class="plain">I_MPI_FABRICS=tmi:tmi</CODE> the problem appear too.</P> <P>If I set <CODE class="functions">export</CODE> <CODE class="plain">I_MPI_FABRICS=shm:tmi</CODE> the problem appear.</P> <P>If I set <CODE class="functions">export</CODE> <CODE class="plain">I_MPI_FABRICS=dapl:tmi</CODE> the problem does not appear.</P> <P>Any ideas?</P> Wed, 19 Nov 2014 14:26:00 GMT https://community.intel.com/t5/Intel-MPI-Library/Problem-with-IntelIB-Basic-Intel-MPI-and-I-MPI-FABRICS-tmi/m-p/1059310#M4497 Guillaume_De_Nayer 2014-11-19T14:26:00Z