Intel BD-NVV-N3000-2 issue

[samir_bhansali]

After installing new FPGA card on PCIe x16 Slot, when we try to enter in BIOS server is halting in BIOS with error "Nmi activated - system halted".

Note : It’s booting to OS without any issue.

Server Product Code : LWF2208IR848505
Server S/N : BQF900200624
FPGA Model : BD-NVV-N3000-2
FPGA S/N : 644C36123B68
Intel BIOS : 02.01.0012

OS : CentOS 7.7

Please find attached lspci output.

Need your help to resolve this issue, can you please escalate this issue to Intel core development team. We are assuming this is not an hardware failure issue, it could be due to compatibility issue or new modified/beta BIOS can resolve this issue.

[JonWay_C_Intel]

Hi @samir_bhansali 

I dont see the card in your PCIe Bus. 

What is the state of LED at the card back panel? Is it blinking 1sec interval? If yes, you do not have enough airflow to cool the card

OR

You do not have enough power for the card. You need to connect the AUX power to the card.

[samir_bhansali]

Hi Jonway,

LED is blinking 1sec.

Aux power cable is connected to FPGA card which was delivered with Intel server.

Server is installed in datacenter and there is sufficient cooling.

[JonWay_C_Intel]

Hi @samir_bhansali 

When the FPGA Die Temperature hits 100C, it will shutdown the card. I believe the card does not shutdown instantly upon power up, you may take awhile for the card to heat up. While it is still operational, run  "watch -d -n 1 fpgainfo bmc"...monitor the card...if the FPGA Die temperature goes into the 90-100C range - you do not have enough cooling.  

[samir_bhansali]

Hi Jonway,

Can you please share the link to download the below tool/package?

Looks command is not available.

O/P:-

------------------------

sh: fpgainfo: command not found

-----------------------

[JonWay_C_Intel]

You will need to install the Acceleration Stack  (Either the Runtime or Development)

https://www.intel.com/content/www/us/en/programmable/products/boards_and_kits/dev-kits/altera/intel-fpga-pac-n3000/getting-started.htmlhttps://www.intel.com/content/www/us/en/programmable/products/boards_and_kits/dev-kits/altera/intel-fpga-pac-n3000/getting-started.html#n3000

You might want to go with the Runtime first. The filesize is smaller.

[samir_bhansali]

After installing the software. Got below message.

Every 1.0s: fpgainfo bmc

No FPGA resources found.

Attached snapshot.

[samir_bhansali]

Hi Jonway,

There can be power drawing issue from motherboard which could not be sufficient power for FPGA card ?

I have connected cable on 4 pin power connector on motherboard which is Optional 12v.

Cable Connected : H38461-001 AST

Kindly find attached output of “lspci command” from 2nd server in which also FPGA card is installed and motherboard connector pictures for your reference.

[JonWay_C_Intel]

If the LED are blinking, you will get "No FPGA resources found" because the card is shutdown. You will not see the card on the PCIe bus. To rule out the Motherboard is not providing sufficient power, you may want to connect the card to an external high power power supply.

[samir_bhansali]

Hi Jonway,

Currently i have connected single H38461-001 AST cable on 4 pin power connector on motherboard which is Optional 12 V. As per N3000 data sheet it is mentioned to connect both Optional 12 V power connector on motherboard. To connect two H38461-001 AST cable from motherboard to card i will require converter to connect both cable to one cable to card.

Can you please confirm converter should be connected to (two 8 pin to one 6 pin) or (two 6 pin to one 6 pin) cable. Also can you share cable converter part number.

Please find attached cable picture and N3000 data sheet.

Refer 2.2.5 from data sheet.

[samir_bhansali]

Hi Jonway,

Waiting for your update.

[JonWay_C_Intel]

The connector on N3000 is a standard 6pin ATX connector.

Pin 1,2,3 = 12V

Pin 5 = SNS (NC due to not used)

Pin 4,6  = GND

[samir_bhansali]

Yes, N3000 has standard 6 Pin connector and we have already received H38461-001 AST cable with Intel server.

But the challenge is we have to connect two cables from motherboard on Optional 12V connector to card.

On card there is only single 6 Pin connector, hence both cable cannot be connected at card end.

We will require splitter to connect at card end or another cable which has single 6 Pin to connect at card end and other end should have dual 4 Pin connector which can connect on motherboard on both Optional 12V.

Can you please suggest cable part number.

[samir_bhansali]

Hi Jonway,

As per data sheet two H38461-001 cables need to connect on 12V motherboard port.

On FPGA card there is only single 6 pin port available to power up card.

Now we require to converge two 6 pin cable to one 6 pin to connect on FPGA card.

Can you please help with cable part number. 

[JonWay_C_Intel]

Hi @samir_bhansali 

AUX cable is not provided with the card. Thus,  I do not have a cable PN to share.
Could you check with your cable vendor?

[samir_bhansali]

I am unable to get converger cable part number from cable vendor.

Just to double check, you could have referred data sheet snapshot of power requirement. Power should be given from both 12V port on motherboard correct?

[samir_bhansali]

Hi Jonway,

We have observed after booting into OS the led status of 1,2,4 port is blinking yellow and 3rd is blinking green on card. Can you please confirm Intel LWF2208IR514206 server is compatible with Intel BD-NVV-N3000-2 FPGA card. Yet we haven't got any solution, please help to resolve this issue.

[samir_bhansali]

Server Prod Code : Intel LWF2208IR848505

Motherboard Model : S2600WF

CPU : Intel Xeon Gold 6248

Slot : PCIe x16

Aux Cable : H38461-001

[JonWay_C_Intel]

Hi,

The LED blinking combination you mentioned is not logical. Attaching the screenshot of the LED (you can find this info in the datasheet too), I would expect that since you are using  BD-NVV-N3000-2, this is a 25bE card. Thus, connectivity LED should be GREEN when you connected the cable to an active link partner. I am not expecting any yellow (which is for 10Gb). Can you specify which is LED 1,2,3,4 based on the screenshot provided?

Did you reimage the FPGA to a 10Gb? 

Could you paste the output for "fpgainfo fme" "fpgainfo phy" "fpgainfo port" "fpgainfo bmc" here?

Can you try a different card?

Thanks.

[samir_bhansali]

Hi Jonway,

As per attached snapshot, counting led from left to right. So the 1,2,4 is showing blinking yellow and 3 is blinking green. This is 25GbE card and there are no QSFP cables connected. We have installed each card in 2 servers and both are not detecting in OS.

Kindly find attached command output.

[JonWay_C_Intel]

Hi, I have sent you a private message.