NFS export problem on MPSS 3.1

[Jyh-Shyong_H_]

Hi,

I installed MPSS 3.1 on one of workstations with CentOS 6.4,  the installation process went well before I configured the host NFS export.

Before I set up the export file system (/home and /work), I can ssh to mic0 and mic normally, but after that, I am not able to log on either of the cards.

Here are the network information of these two cards:

mic0      Link encap:Ethernet  HWaddr 4C:79:BA:32:12:2B  
          inet addr:172.31.1.1  Bcast:172.31.1.255  Mask:255.255.255.0
          inet6 addr: fe80::4e79:baff:fe32:122b/64 Scope:Link
          UP BROADCAST RUNNING  MTU:64512  Metric:1
          RX packets:36 errors:0 dropped:0 overruns:0 frame:0
          TX packets:10 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1728 (1.6 KiB)  TX bytes:636 (636.0 b)

mic1      Link encap:Ethernet  HWaddr 4C:79:BA:32:12:63  
          inet addr:172.31.1.2  Bcast:172.31.1.255  Mask:255.255.255.0
          inet6 addr: fe80::4e79:baff:fe32:1263/64 Scope:Link
          UP BROADCAST RUNNING  MTU:64512  Metric:1
          RX packets:36 errors:0 dropped:0 overruns:0 frame:0
          TX packets:10 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1728 (1.6 KiB)  TX bytes:636 (636.0 b)

I can still use command  "ssh  mic0", with no error, but I am still in the host (from the host hpctest-2)

[root@hpctest-2 ~]# who
root     pts/0        2013-11-12 16:51 (hpctest-0)  
[root@hpctest-2 ~]# ssh mic0
Last login: Tue Nov 12 16:51:42 2013 from hpctest-0
[root@hpctest-2 ~]# who
root     pts/0        2013-11-12 16:51 (hpctest-0)
root     pts/1        2013-11-12 16:53 (hpctest-2-mic0.hpctest)

It seems that I log back on to the host from mic0 automatically.  I tried again with command "ssh mic0 -v" and got the following message:

[root@hpctest-2 ~]# ssh mic0 -v
OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug1: Connecting to mic0 [172.31.1.1] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
debug1: identity file /root/.ssh/identity type -1
debug1: identity file /root/.ssh/id_rsa type 1
debug1: identity file /root/.ssh/id_dsa type 2
debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3
debug1: match: OpenSSH_5.3 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.3
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5 none
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Host 'mic0' is known and matches the RSA host key.
debug1: Found key in /root/.ssh/known_hosts:6
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug1: Next authentication method: gssapi-keyex
debug1: No valid Key exchange context
debug1: Next authentication method: gssapi-with-mic
debug1: Unspecified GSS failure.  Minor code may provide more information
Credentials cache file '/tmp/krb5cc_0' not found

debug1: Unspecified GSS failure.  Minor code may provide more information
Credentials cache file '/tmp/krb5cc_0' not found

debug1: Unspecified GSS failure.  Minor code may provide more information


debug1: Unspecified GSS failure.  Minor code may provide more information
Credentials cache file '/tmp/krb5cc_0' not found

debug1: Next authentication method: publickey
debug1: Trying private key: /root/.ssh/identity
debug1: Offering public key: /root/.ssh/id_rsa
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug1: Offering public key: /root/.ssh/id_dsa
debug1: Server accepts key: pkalg ssh-dss blen 433
debug1: read PEM private key done: type DSA
debug1: Authentication succeeded (publickey).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: Sending environment.
debug1: Sending env LANG = en_US.UTF-8
Last login: Tue Nov 12 16:56:53 2013 from hpctest-0

I am returning back to the palce where I started.  It is confusing, and I hope someone can provide me some help.

Thanks

Jyh-Shyong Ho, Ph.D.

Research Scientist

National Center for High Performance Computing

Hsinchu, Taiwan, ROC
 

[Frances_R_Intel]

You were able to ssh into mic0 before you set up NFS?

I don't see anything unusual in the network information you have provided. The first thing that surprised me was the version information in the ssh -v output. It says "remote software version OpenSSH_5.3". But I believe the version that is included with MPSS 3.1 is OpenSSH_5.9. Then it goes on to say that the accepted forms of authentication are "publickey,gssapi-keyex,gssapi-with-mic,password" but I don't think the ssh on the coprocessor card is set up for anything other than publickey, password and keyboard-interactive. It's as if you are never connecting to the coprocessor at all but are talking to the processor from the start. I could be wrong - networking was never my strong suit - but that is what it looks like to me.

Based on the addresses for the coprocessors, I take it that you have an internal bridge set up on the host. When you look at the ifcfg-br0 and ifcfg-mic0 file, do they look ok? Some people have noticed weird things happening in the ifcfg-mic0 file, which has been reported to the developers.

[Jyh-Shyong_H_]

Dear Dr. Roth,

Thanks for your reply.  Finally I found the mistake that I made in my configuration of the mic0/mic1 network. I manually modified IPADDR of ifcfg-mic0 and ifcfg-mic1, so they were 172.31.1.1 and 172.31.1.2, respectively, this caused the problem of log on mic cards from the host. The default settings (172.31.1.1 and 172.31.2.1) work just fine.  

Jyh-Shyong Ho