Port shows as NO-CARRIER, state down though the cable is plugged in, Please suggest

[PASOK2]

Port shows as NO-CARRIER, state down though the cable is plugged in, Please suggest

$ lspci | grep Omni

0c:00.0 Fabric controller: Intel Corporation Omni-Path HFI Silicon 100 Series [discrete] (rev 11)

1b:00.0 Fabric controller: Intel Corporation Omni-Path HFI Silicon 100 Series [discrete] (rev 11)

58:00.0 Fabric controller: Intel Corporation Omni-Path HFI Silicon 100 Series [discrete] (rev 11)

5e:00.0 Fabric controller: Intel Corporation Omni-Path HFI Silicon 100 Series [discrete] (rev 11)

$

5: ib0: mtu 2044 qdisc pfifo_fast state UP qlen 256

link/infiniband 80:00:00:02:fe:80:00:00:00:00:00:00:00:11:75:01:01:79:00:1c brd 00:ff:ff:ff:ff:12:40:1b:80:01:00:00:00:00:00:00:ff:ff:ff:ff

inet 10.14.0.3/8 brd 10.255.255.255 scope global ib0

valid_lft forever preferred_lft forever

inet6 fe80::211:7501:179:1c/64 scope link

valid_lft forever preferred_lft forever

6: ib1: mtu 2044 qdisc pfifo_fast state UP qlen 256

link/infiniband 80:00:00:02:fe:80:00:00:00:00:00:00:00:11:75:01:01:78:ce:d9 brd 00:ff:ff:ff:ff:12:40:1b:80:01:00:00:00:00:00:00:ff:ff:ff:ff

inet 10.14.1.3/8 brd 10.255.255.255 scope global ib1

valid_lft forever preferred_lft forever

inet6 fe80::211:7501:178:ced9/64 scope link

valid_lft forever preferred_lft forever

7: ib2: mtu 2044 qdisc pfifo_fast state UP qlen 256

link/infiniband 80:00:00:02:fe:80:00:00:00:00:00:00:00:11:75:01:01:78:ce:d7 brd 00:ff:ff:ff:ff:12:40:1b:80:01:00:00:00:00:00:00:ff:ff:ff:ff

inet 10.14.2.3/8 brd 10.255.255.255 scope global ib2

valid_lft forever preferred_lft forever

inet6 fe80::211:7501:178:ced7/64 scope link

valid_lft forever preferred_lft forever

8: ib3: mtu 2044 qdisc pfifo_fast state DOWN qlen 256

link/infiniband 80:00:00:02:fe:80:00:00:00:00:00:00:00:11:75:81:01:79:00:1c brd 00:ff:ff:ff:ff:12:40:1b:80:01:00:00:00:00:00:00:ff:ff:ff:ff

inet 10.14.3.3/8 brd 10.255.255.255 scope global ib3

valid_lft forever preferred_lft forever

$

after rebooting the node. it came up.

8: ib3: mtu 2044 qdisc pfifo_fast state UP qlen 256

link/infiniband 80:00:00:02:fe:80:00:00:00:00:00:00:00:11:75:81:01:79:00:1c brd 00:ff:ff:ff:ff:12:40:1b:80:01:00:00:00:00:00:00:ff:ff:ff:ff

inet 10.14.3.3/8 brd 10.255.255.255 scope global ib3

valid_lft forever preferred_lft forever

inet6 fe80::211:7581:179:1c/64 scope link

valid_lft forever preferred_lft forever

may I know what could be the reason for such behavior. is it possible to recover without reboot for such cases