A problem in setting up external bridge with NetBits=16

[T__C__L_]

I'm using the latest MPSS - mpss_gold_update_3-2.1.6720-15-rhel-6.4. And I would like to use the external bridge with NetBits=16.

The following commands were issued for setting up the external bridge and network configuration

        micctrl --addbridge=br0 --type=external --ip=10.16.33.221 --netbits=16 --mtu=9000

        micctrl --network=static --bridge=br0 --ip=10.16.33.223

The command "micctrl --config" shows configurations as follows.

mic0:
=============================================================
Linux Kernel: /lib/firmware/mic/uos.img
BootOnStart: Enabled
Shutdowntimeout: 300 seconds

......

Network: Static bridge br0
MIC IP: 10.16.33.223
Host IP: 10.16.33.221
Net Bits: 16
NetMask: 255.255.0.0
MtuSize: 9000
Hostname: PhiServer
MIC MAC: xx:xx:xx:xx:xx:xx
Host MAC: xx:xx:xx:xx:xx:xx

However, launching "ifconfig" on MIC card derives the following output:

mic0 Link encap:Ethernet HWaddr F2:B4:D7:08:25:15
inet addr:10.16.33.223 Bcast:0.0.0.0 Mask:255.255.255.0
inet6 addr: fe80::48c:f0ff:fe44:7208/64 Scope:Link
UP BROADCAST RUNNING MTU:64512 Metric:1
RX packets:47151 errors:0 dropped:1352 overruns:0 frame:0
TX packets:75 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3948092 (3.7 MiB) TX bytes:11736 (11.4 KiB)

As a result, Mask, Bcast, and MTU are inconsistent to the configuration I set. Namely, Mask should be 255.255.0.0 as the NetBits=16 and Bcast should hence be 10.16.255.255, and MTU should be 9000 instead of 64k.

Could any one tell me what's wrong with my steps to configure external bridge????

[T__C__L_]

I got some progress in this issue by myself. It's weird that I have to manually add netmask configuration to "ifcfg-mic0.conf" even though NetBits=16 is explicitly given when launching "micctrl --addbridge ..." command. I'm now able to configure the netmask to 255.255.0.0 but the Bcast is still 0.0.0.0 and MTU is still 64k instead of 9000. 

[Frances_R_Intel]

There is a known problem with using a netbits of 8 but 16 should have worked. You appear to have done all the right things in your 'micctrl --addbridge' and 'micctrl --network' commands; did you also run 'micctrl --resetconfig'? That should force the changes you made out to all the Linux config files for the coprocessor.

As far as the Bcast of 0.0.0.0, others have seen that as well. That is one of the questions I have been meaning to track down an explanation for - I'll go ask that right now.

[T__C__L_]

Thank you for helping me with confirming my procedure.

I did run 'micctrl --resetconfig' after 'micctrl --addbridge' and 'micctrl --network' commands had been done. I also checked '/etc/sysconfig/network/ifcfg-mic0' after then. However, the network settings contained merely IPADDR, HWADDR, GATEWAY, and MTU except NetMask. I, therefore, manually added a netbits of 16 to the file.

[SDT]

I have similar problem, but I am using NetBits=23 and mpss_gold_update_3-2.1.6720-15-rhel-6.2 .
I've set up network by executing following commands:

micctrl --addbridge=br0 --type=External --ip=172.16.28.40 --netbits=23 --mtu=9000 mic0 mic1
micctrl --network=static --bridge=br0 --ip=172.16.28.41:172.16.28.42 mic0 mic1

"micctrl --config" shows configurations:

mic0:

=============================================================
Linux Kernel: /lib/firmware/mic/uos.img
BootOnStart: Enabled
Shutdowntimeout: 300 seconds

...

Network: Static bridge br0

MIC IP: 172.16.28.41
Host IP: 172.16.28.40
Net Bits: 23
NetMask: 255.255.254.0
MtuSize: 9000
Hostname: phi-mic0
MIC MAC: xx:xx:xx:xx:xx:xx
Host MAC: xx:xx:xx:xx:xx:xx

but ifconfig on mic0 and mic1 was showing me wrong value of MTU and NETMASK.

After that I added info about MTU to ifcfg-mic0:
echo 'MTU=9000' >> /usr/local/mic/filesystem/mic0/etc/sysconfig/network/ifcfg-mic0

what solved the problem. Unfortunatelly the same trick doesn't work with NETMASK.

After login to mic0 or mic1 ifconfig shows me that NETMASK is set to 255.255.255.0, which is not correct (proper value should be 255.255.254.0).

Even if I set this value manually on mic0 and mic1 by executing this command:

[root@phi-mic0 /]# ifconfig mic0 netmask 255.255.254.0; ifconfig mic0 down; ifconfig mic0 up

everything works fine at the begining, but after a while Xeon Phi card hangs,

e.g. after execution command df -h - it shows local filesystems, but hungs when it is trying to show info about network filesystem.

It also hangs when I am trying copy files from network filesystem, what clearly mean that there is something wrong with network configuration.
Could you please help me to resolve this issue?

[T__C__L_]

The following command could be of help.

echo 'NETMASKBITS=23' >> /usr/local/mic/filesystem/mic0/etc/sysconfig/network/ifcfg-mic0

[SDT]

I used: "NETMASKBITS=23" instead of: "NETMASK=255.255.254.0" and it seems that network is now set up properly. Ifconfig on mic0 and mic1 shows the proper values. Thank you very much for this advice.

Unfortunately mic0 and mic1 still seems to hang again. It seems to work for a 10-20 seconds after first log in, but as soon as I start accessing files on network file system is hangs and all subsequent logins also hang immediately on trying to cd to network file system.

Do you have any ideas? Do you think that I should open a new topic since it is no longer related to "Setting up external bridge with NetBits"?

[T__C__L_]

I'm curious about your networking configurations. Could you please attach 'ifconfig' output on the host and the mic cards???

[SDT]

[root@phi ~]# ifconfig
br0 Link encap:Ethernet HWaddr xx:xx:xx:xx:xx:xx 
inet addr:172.16.28.40 Bcast:172.16.29.255 Mask:255.255.254.0
inet6 addr: fe80::225:90ff:fe96:248c/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:9000 Metric:1
RX packets:4477863 errors:0 dropped:0 overruns:0 frame:0
TX packets:1118405 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:707396290 (674.6 MiB) TX bytes:8060051641 (7.5 GiB)

eth0 Link encap:Ethernet HWaddr xx:xx:xx:xx:xx:xx
inet6 addr: fe80::225:90ff:fe96:248c/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:9000 Metric:1
RX packets:4810454 errors:0 dropped:0 overruns:0 frame:0
TX packets:5819300 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1096282693 (1.0 GiB) TX bytes:7956089741 (7.4 GiB)
Memory:fbe20000-fbe40000

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:489597 errors:0 dropped:0 overruns:0 frame:0
TX packets:489597 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:7533911903 (7.0 GiB) TX bytes:7533911903 (7.0 GiB)

mic0 Link encap:Ethernet HWaddr xx:xx:xx:xx:xx:xx
inet6 addr: fe80::4e79:baff:fe33:ad9/64 Scope:Link
UP BROADCAST RUNNING MTU:64512 Metric:1
RX packets:92 errors:0 dropped:0 overruns:0 frame:0
TX packets:38502 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:10969 (10.7 KiB) TX bytes:5400729 (5.1 MiB)

mic1 Link encap:Ethernet HWaddr xx:xx:xx:xx:xx:xx
inet6 addr: fe80::4e79:baff:fe33:aeb/64 Scope:Link
UP BROADCAST RUNNING MTU:64512 Metric:1
RX packets:6637 errors:0 dropped:0 overruns:0 frame:0
TX packets:50733 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:422311 (412.4 KiB) TX bytes:116081991 (110.7 MiB)

[root@phi ~]# ssh mic0
[root@phi-mic0 /root]# ifconfig
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

mic0 Link encap:Ethernet HWaddr xx:xx:xx:xx:xx:xx
inet addr:172.16.28.41 Bcast:0.0.0.0 Mask:255.255.254.0
inet6 addr: fe80::4e79:baff:fe33:ad8/64 Scope:Link
UP BROADCAST RUNNING MTU:9000 Metric:1
RX packets:40341 errors:0 dropped:521 overruns:0 frame:0
TX packets:118 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:5647817 (5.3 MiB) TX bytes:15254 (14.8 KiB)

[T__C__L_]

I don't see any weird setting in the above configurations. I think you'd better creating a new topic for your problem.

[Frances_R_Intel]

There are been a number of changes in the way micctrl configures network bridges since this issue was first raised. Anyone having similar problems should consider moving to the latest version of the MPSS.