FPGA, SoC, And CPLD Boards And Kits
FPGA Evaluation and Development Kits
6188 Discussions

how to test FPGA PAC N3000 network port

fzhan51
New Contributor I
4,258 Views

FPGA PAC N3000, configed in 8x10G mode, attached to the Xeron serverA with pcie x16 。we got 40GbE servrB/ 10GbE serverC , QSFP+40G SR-L optical module and 12core mo3 fiber。test case:1) serverA-->one port QSFP and fiber--> serverB: leds out 2)serverA(B)-->one port to another port-->serverA(B);leds on

(a).should we puchase E40GQSFPSR to displace QSFP+40G SR-L? (b)give some advices.

by the way,I didn't enable Vx-Tx Vx-d(not found in the bios),may it help? thanks a lot

0 Kudos
30 Replies
JonWay_C_Intel
Employee
3,000 Views

Hi @fzhan51​  First of all, what is the objective of your test? A simple functionality test? If yes, you can refer to:

Test by DPDK

https://www.intel.com/content/www/us/en/programmable/documentation/xgz1560360700260.html#hnb1561577285390

 

Test by fpgadiag:

https://www.intel.com/content/www/us/en/programmable/documentation/xgz1560360700260.html#hll1561577008230

0 Kudos
fzhan51
New Contributor I
3,000 Views

thank you. we want a simple test fot the connection between n3000 qsfp port and the port of other 40GbE networkcard , a ping test etc. beside,

today,I got empty when entered lspci | grep 0d58 or lspci | grep 0cf8 ,what may cause that?

 

0 Kudos
JonWay_C_Intel
Employee
3,000 Views

Is the card alive? Did you connect the AUX cable? Can this recover by reset? Is it overheated?

any status logs e.g. fpgainfo fme? fpgainfo bmc? etc

...i can help more if there is more info.

0 Kudos
fzhan51
New Contributor I
3,000 Views

the cable is 12core mo3 fiber; we are purchasing X4DACBL3 to try; reset not fix;

I just found that the Image may be upgraded to Production Version for trying the connection, how can I revert it?

we bought a E40GQSFPSR optic mode to test with 12core mo3 fiber and other optic mode,seems no help.

 

for now,the 40G serverB is not available, i connect the netport QSFPa(QSFP+40G SR-L) to netportQSFPb(QSFP+40G SR-L) with 12core mo3 fiber

-----------------------------------------------

[fpga@localhost ~]$sudo fpgainfo fme

Board Management Controller, MAX10 NIOS FW version D.2.0.19

Board Management Controller, MAX10 Build version D.2.0.6

//****** FME ******//

Object Id           : 0xF500000

PCIe s:b:d.f         : 0000:07:00.0

Device Id           : 0x0b30

Numa Node           : 0

Ports Num           : 01

Bitstream Id         : 0x23000010010309

Bitstream Version       : 0.2.3

Pr Interface Id        : 901dd697-ca79-4b05-b843-8138cefa2846

Boot Page           : user

----------------------------------------------

[fpga@localhost ~]$ sudo fpgainfo phy

[sudo] password for fpga: 

Board Management Controller, MAX10 NIOS FW version D.2.0.19

Board Management Controller, MAX10 Build version D.2.0.6

//****** PHY ******//

Object Id           : 0xF500000

PCIe s:b:d.f         : 0000:07:00.0

Device Id           : 0x0b30

Numa Node           : 0

Ports Num           : 01

Bitstream Id         : 0x23000010010309

Bitstream Version       : 0.2.3

Pr Interface Id        : 901dd697-ca79-4b05-b843-8138cefa2846

//****** PHY GROUP 0 ******//

Direction           : Line side

Speed             : 10 Gbps

Number of PHYs        : 8

//****** PHY GROUP 1 ******//

Direction           : Host side

Speed             : 10 Gbps

Number of PHYs        : 8

//****** Intel C827 Retimer ******//

Port0 10G           : Up

Port1 10G           : Up

Port2 10G           : Up

Port3 10G           : Up

Port4 10G           : Up

Port5 10G           : Up

Port6 10G           : Up

Port7 10G           : Up

Retimer A Version       : 101c.1064

Retimer B Version       : 101c.1064

--------------------------------------------

[fpga@localhost ~]$ sudo fpgainfo bmc

Board Management Controller, MAX10 NIOS FW version D.2.0.19

Board Management Controller, MAX10 Build version D.2.0.6

//****** BMC SENSORS ******//

Object Id           : 0xF500000

PCIe s:b:d.f         : 0000:07:00.0

Device Id           : 0x0b30

Numa Node           : 0

Ports Num           : 01

Bitstream Id         : 0x23000010010309

Bitstream Version       : 0.2.3

Pr Interface Id        : 901dd697-ca79-4b05-b843-8138cefa2846

( 1) Board Power       : 69.56 Watts

( 2) 12V Backplane Current  : 3.20 Amps

( 3) 12V Backplane Voltage  : 12.15 Volts

( 4) 1.2V Voltage       : 1.18 Volts

( 6) 1.8V Voltage       : 1.80 Volts

( 8) 3.3V Voltage       : 3.25 Volts

(10) FPGA Core Voltage    : 0.90 Volts

(11) FPGA Core Current    : 15.07 Amps

(12) FPGA Die Temperature   : 69.00 Celsius

(13) Board Temperature    : 43.50 Celsius

(14) QSFP0 Supply Voltage   : 3.27 Volts

(15) QSFP0 Temperature    : 51.00 Celsius

(24) 12V AUX Current     : 2.53 Amps

(25) 12V AUX Voltage     : 12.11 Volts

(37) QSFP1 Supply Voltage   : 3.27 Volts

(38) QSFP1 Temperature    : 54.00 Celsius

(44) PKVL0 Core Temperature  : 68.00 Celsius

(45) PKVL0 SerDes Temperature : 68.50 Celsius

(46) PKVL1 Core Temperature  : 69.00 Celsius

(47) PKVL1 SerDes Temperature : 69.50 Celsius

---------------------------------------------------------------------------

how to test the connection in this case ,and how to check tx and rx ?

0 Kudos
fzhan51
New Contributor I
3,000 Views

for above connection ,if disconnect the fiber

[fpga@localhost ~]$ sudo fpgainfo phy

Board Management Controller, MAX10 NIOS FW version D.2.0.19

Board Management Controller, MAX10 Build version D.2.0.6

//****** PHY ******//

Object Id           : 0xF500000

PCIe s:b:d.f         : 0000:07:00.0

Device Id           : 0x0b30

Numa Node           : 0

Ports Num           : 01

Bitstream Id         : 0x23000010010309

Bitstream Version       : 0.2.3

Pr Interface Id        : 901dd697-ca79-4b05-b843-8138cefa2846

//****** PHY GROUP 0 ******//

Direction           : Line side

Speed             : 10 Gbps

Number of PHYs        : 8

//****** PHY GROUP 1 ******//

Direction           : Host side

Speed             : 10 Gbps

Number of PHYs        : 8

//****** Intel C827 Retimer ******//

Port0 10G           : Down

Port1 10G           : Down

Port2 10G           : Down

Port3 10G           : Down

Port4 10G           : Down

Port5 10G           : Up

Port6 10G           : Up

Port7 10G           : Down

Retimer A Version       : 101c.1064

Retimer B Version       : 101c.1064

-------------------------

why Port5/Port6 always Up ?

 

0 Kudos
JonWay_C_Intel
Employee
3,000 Views

Hi @fzhan51​  Lets take a step back. So i suppose you are able to see output from lspci | grep ocf8, right?

Let me get to your last question... If you have removed all the cable, i would expect port 5/6 to be "Down"...

Could you try "rsu bmcimg <BDF>" and run fpgainfo phy again to see if that recovers the card?

0 Kudos
fzhan51
New Contributor I
3,000 Views

@JwChin​ :   lspci | grep ocf8 ,the output is ocf8 before my upgradation ,but now, i can't see ;

"rsu bmcimg <BDF>" is good for port 5/6 up problem. now all show "Down". one thing to mention, I forggot to do "sudo rm -rf /usr/share/opae/n3000/one-time-update/*/"

0 Kudos
JonWay_C_Intel
Employee
3,000 Views

So, you still cant see 0CF8?

Then, lets fix this first.

 

I know you have provided some info, but could you fill up some basic info as below (if too long, then put in a text file and attach it here). Thanks.

 

Operating System Vendor and Version

Kernel Version

Server Vendor and Model

Intel Acceleration Stack release i.e. Beta, Production

Ethernet Port Configuration i.e 2x2x25G, 4x25G, 8x10G

I40e and iavf driver version

I40e and iavf firmware version

 

log or screenshot of fpgainfo fme, fpgainfo bmc, fpgainfo phy, fpgainfo errors

dmesg log file

lspci -vvv log file

Run "lspci -vt"

and also lspci | grep Ethernet

0 Kudos
fzhan51
New Contributor I
3,000 Views

Dear JwChin:

[3days ago,the system installed by others]

Operating System Vendor and Version:  linux-4.19.8

Kernel Version:     4.19.8

rpm -qa | grep kernel-header: kernel-headers-3.10.0-1062.18.1.el7.x86_64

Server Vendor and Model: HUAWEI/H22H-03

Intel Acceleration Stack release i.e. Beta, Production: IntelAccelerationStack release.jpg Alpha-1 D.1.0.13

Ethernet Port Configuration i.e 2x2x25G, 4x25G, 8x10G: 8x10G

I40e and iavf driver version: i40e-2.9.21

I40e and iavf firmware version: iavf-3.7.53 

 

[now]

Operating System Vendor and Version: CentOS-7-x86_64-Everything-1611.iso(maybe)

Kernel Version:         3.10.0-1062.18.1.el7.x86_64

rpm -qa | grep kernel-header: kernel-headers-3.10.0-1062.18.1.el7.x86_64

Server Vendor and Model:     HUAWEI/H22H-03

Intel Acceleration Stack release i.e. Beta, Production: Production D.2.0.6

Ethernet Port Configuration i.e 2x2x25G, 4x25G, 8x10G: 8x10G

I40e and iavf driver version:    i40e-2.10.19.30(i40e-2.9.21 compiled failed)

I40e and iavf firmware version: iavf-3.7.61.20(when install,it says invalid private key)

 

maybe there were some mistakes when I did OTSU/RSU

0 Kudos
JonWay_C_Intel
Employee
3,000 Views

hi @fzhan51​  It doesnt seems right that 0CF8 DID is not showing.

From experience, it could be the kernel (im using 3.10.0-957.el7.x86_64). Second, it could be the OTSU/RSU.

If it were me, I would probably check the correct OS and kernel version and to perform the OTSU/RSU again....might be a long-winded process but at least you can watch for any error and foolproof it.

 

0 Kudos
fzhan51
New Contributor I
2,947 Views

Dear JwChin: reinstall Centos 7.6 does help,0cf8 shows correctly.thank you very much .

0 Kudos
JonWay_C_Intel
Employee
2,947 Views

Hi @fzhan51​  Thank you for your update. So, it might be something wrong with the kernel, and it resolved over re-installing. This is great input for others watching the forum, thanks.

0 Kudos
fzhan51
New Contributor I
3,000 Views

Dear JwChin I would try it.note that, Retimer A/B Version are zero after "rsu bmcimg <BDF>", and port0-7 all Down now even connect the fiber. if it become Porduct version,it can't revert back to Alpha Version,isn't it ? if no, it means that I have to sign the image first

0 Kudos
fzhan51
New Contributor I
3,000 Views

Dear JwChin I would try it . note that Retimer A/B Version become 0000.0000, port0-port7 always Down after "rsu bmcimg <BDF>". if it become product version,cant it revet to Alpha version ? last time i try the $ sudo super-rsu /usr/share/opae/n3000/one-time-update/<25G or 10G>/\

rollback-*.json --with-rsu in section "B.1.2. Root Entry Hash Not Programmed",if failed. that means I do have to prepare signed image?

0 Kudos
JonWay_C_Intel
Employee
3,000 Views

Hi @fzhan51​  Unfortunately, no you cannot revert from Production to Alpha.

0 Kudos
fzhan51
New Contributor I
3,000 Views

Dear JwChin,I switch the kernel to 4.19.8; and tried the section "B.1.2.",attached the log.

I remember that I didn't get any signed image,and the last operation i follow to do is the section "B.1.2."

seemed that it went through,then the version become Product version.

I just read the ug-pac-security-n3000.pdf,it says "This operation is permanent and irreversible."on page 21.Does that mean root entry hash bitstream is programmed,I even don't know any keys(root key/csks),then the card is dead for me? could you help me please.

0 Kudos
JonWay_C_Intel
Employee
3,000 Views

Hi @fzhan51​  If you did not create + program any root hash into the device, then it should be fine. You can still program unsigned images.

0 Kudos
fzhan51
New Contributor I
3,000 Views

would you help to login our n3000 server to check what's going wrong? much appreciated if there is your email to send teamviewer id /psw. i just send it to supportreplies@intel.com i don't know whether you can received it or not.

i Update the Intel XL710 Firmware,and after rsu bmcimg <FPGA PCIe B:D.F> the server hang.attached is the sreenshot

 

0 Kudos
JonWay_C_Intel
Employee
3,000 Views

Hi @fzhan51​  I see you have created another thread https://forums.intel.com/s/question/0D50P00004dmzbn/pac-n3000-8x10g-xl710-update-failed?language=en_US&t=1586513388811 on the same issue.

But i probably continue to help from here, and close the duplicated.

 

Could you run lspci -s BDF_OfTheEthernetCtrller -x

 

Expected to see something like:

00: 86 80 f8 0c xx xx xx

 

I see that your version is up-to-date, and if the lspci command as above is ok...then probably you can try to move on

0 Kudos
fzhan51
New Contributor I
3,000 Views

so glade to get your answer.

[root@localhost Factory_Image]# lspci -s 07:00.0 -x

07:00.0 Processing accelerators: Intel Corporation Device 0b30

00: 86 80 30 0b 46 05 10 00 00 00 00 12 08 00 00 00

10: 0c 00 f0 ff 3f 02 00 00 0c 00 e0 ff 3f 02 00 00

20: 00 00 00 00 00 00 00 00 00 00 00 00 86 80 00 00

30: 00 00 00 00 68 00 00 00 00 00 00 00 0a 01 00 00

0 Kudos
Reply