FPGA, SoC, And CPLD Boards And Kits
FPGA Evaluation and Development Kits
5892 Discussions

AN881 reference design need to be upgraded for device 1SM21CH.

CHung
Novice
2,710 Views

Hi,

The following reference design is based on 1SM21BH device.

https://fpgacloud.intel.com/devstore/platform/19.1.0/Pro/an881-pcie-avmm-dma-gen3x16-ddr4-and-hbm2/

We use Stratix 10 MX 1SM21CH device, so we cannot synthesize this reference design.

I guess the difference is HBM controller interface because of the size.

Could you please provide a version for 1SM21CH device?

This reference is important to us, or we don't know how to access HBM2/DDR4 by PCIe driver.

​Thanks.

0 Kudos
54 Replies
sstrell
Honored Contributor III
1,106 Views

Example and reference designs are starting points. This design is specifically for the Stratix 10 MX dev kit. If you take this design, you can change the target device and regenerate the IP. Since you're probably building your own board, you would probably need to make assignment changes, such as pin locations, anyway.

 

#iwork4intel

0 Kudos
BoonT_Intel
Moderator
1,106 Views

Yes, thanks @sstrell​ 

Hi @CHung​ as mentioned by sstrell, you can use the reference design as reference, eg, which component to instantiate in Qsys, and how its connect. Some modification on the top level to suit your board connection. Then make pin assignment change.

0 Kudos
CHung
Novice
1,106 Views

Got it.

Thanks for your answers.

0 Kudos
CHung
Novice
1,106 Views

Hi Intel support,

​I've tried it.

I modified the device and upgrade all IP.

The sof could be programmed to the FPGA successfully.

Besides, I install the Linux driver of this reference design.

However, it doesn't work.

Linux cannot ​recognize this FPGA board which has been connected to motherboard PCIe port.

We followed the steps on this user guide:

https://www.intel.com/content/www/us/en/programmable/documentation/sox1520633403002.html#lmm1520633289264

Linux didn't find the FPGA PCIe device when we entered the following command:

​lspci -d 1172:000 -v | grep intel_fpga_pcie_drv

We have no idea what's wrong.

Perhaps Linux driver needs to be update for the new device? ​

Or the pin assignment of the reference design is wrong?

Or sof file has some problems?

Please help to check this sof and Linux driver on your board.

For your reference, our project files is as follows.

https://drive.google.com/file/d/1mJJYheW6pBN08tSwS4-lgF3XSJfcn-pz/view?usp=sharing

Thanks.​

0 Kudos
BoonT_Intel
Moderator
1,106 Views

Hi do you run the $ lsmod | grep intel_fpga_pcie_drv to ensure the driver is install successful?

0 Kudos
CHung
Novice
1,106 Views

Hi,

Yes, we did, and Linux showed message as expected.

By the way,

as we know, Linux and Window OS could find PCIe devices even the driver is not installed.

If the driver is not installed successfully, it should list an unknown device with ID information.

However, after we entered "lspci" command, it didn't show any device new.

So we think the hardware must have something wrong.

(e.g. SOF file is not correct for some reason)

We have asked Arrow's AE in Taiwan.

But they and Intel's AE in asia-pacific area don't have this FPGA development kit at hand.

If it's possible, please find someone who has this board to try the reference design.

It seems not that straightforward to upgrade reference design from 1SMBHU device to 1SMCHU device.

 

Thanks.

0 Kudos
CHung
Novice
1,106 Views

Hi,

We have tried BTS, and BTS PCIe function is OK.

But BTS works by JTAG port, not PCIe connect.

So we still don't know how to let Linux communicate with FPGA through PCIe connector.

0 Kudos
BoonT_Intel
Moderator
1,106 Views

Hi I agree that even driver is not install successful, it still able to detect by the host as unknown device. So now we can rule out the driver issue.

Suggestion:

  1. is it possible to borrow the BHU variant board from field team and test it with original AN881 design using the same linux host? The reason is to rule out is issue is due to variant or the particular host actually unable to detect any FPGA gen3 card.
  2. What is the slot PCIe capability? Have you tried go into the bios and fix the PCIe slot setting to Gen3x16? In some host, it will create some problem to detect when the setting is AUTO.
0 Kudos
CHung
Novice
1,106 Views

Hi,

1. We'll try it.

2. After confirmation, the slot PCIe capability is gen3 x16.

We do the experiments on different computers with different CPUs, motherboards, and OS (CentOS 7.0 and Ubuntu 16).

The result is the same.

Besides, we have made sure the PCIe slot works well by changing PCIe cards.

When we plugged-in other PCIe cards like nVidia graphic card,

"lspci" command could show PCIe device.

 

0 Kudos
CHung
Novice
1,106 Views

Hi,

Another question, is the pin assignment of 1SM21BHU the same with 1SM21CHU device?

​Currently all the user guides and documents are based on 1SM21BHU.

We don't know whether there is difference except HBM2 size.

0 Kudos
CHung
Novice
1,061 Views

Hi,

We have borrowed a BHU variant board today.

After programming SOF of AN881 reference design (originally based on 1SM21BHU) and BTS PCIe image,

Linux still couldn't recognize it by lspci command.

We also tried 2 hosts, one of them is CentOS 7.0 and the other is Ubuntu 16, both got the same result.

Besides, we tried different PCIe slots on the motherboard.

​If on your side, Linux could recognize the BHU board, could you provide your Linux version (include kernel version), installation files, and your sof?

I heard that some Linux kernels don't accept PCI devices which is not certificated. Perhaps lspci command won't list the unacceptable devices. But I didn't know the detail.

Is that possible it's Linux version related issue?

 

Thanks.

0 Kudos
CChen361
Beginner
1,065 Views

Hi Intel,

Did you ever test your so called reference design before release?

S10 Hard IP+ for PCIE, gen 3 x 16 still can't work on DK-DEV-1SMC-H-A(HBM 16GB version) after 4 weeks hard working.

For your information, AN881 can't work on DK-DEV-1SMX-H-A(HBM 8GB version) either, no matter compiled with v19.1 or v19.3 tested by agent.

Please clarify in what conditions Intel Dev Board + Intel Ref Design can work !

We had good experience with CV, but we are un-happy to be Intel Test Engineer now.

Or, maybe I should return the kit back to Intel and get my money back.

0 Kudos
BoonT_Intel
Moderator
1,106 Views

Hi Yes, they should same. I check all the design from the dev kit package, all the pin location are same. As per my understanding, the different between both board is just the HBM density.

0 Kudos
BoonT_Intel
Moderator
1,106 Views

They are the same, I check the example design in edv kit package. All pin location is compatible.

This this moment, maybe you should signalap the PCIe signal and understand where the problem come from. Example of signal tap signals as below:

stp.png

0 Kudos
CHung
Novice
1,106 Views
0 Kudos
CHung
Novice
1,106 Views

Hi

We have dumped PCIe signalTap waveforms (.jpg and .vcd), please download here:

http://gofile.me/4v7Zs/5QrPiwKwN

  1. pcie_reset_trigger
  2. pcie_reset_release
  3. avmm_bridge_512_0_rst_n_release
  4. pcie_idle

1, 2, 3 are the waveform around system reset.

4 is the waveform after reset.

 

Because we are not familiar with PCIe protocol and signals inside IP,

could you please help to analyze them?

It seems clocks and reset signals are OK. But Linux could not recognize it.

 

Thanks.

 

0 Kudos
BoonT_Intel
Moderator
1,106 Views

Hi I am unable to get the waveform due to security reason. Can you please attach the screenshot here.

What you need to do is to capture the ltssmstate and lane_act signal to understand the Pcie issue

https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/ug/ug_s10_pcie_avmm.pdf#page=80

 

0 Kudos
CHung
Novice
1,106 Views

Hi,

1. when reset is asserted:

pcie_reset_trigger.jpg

2. reset is de-asserted

pcie_reset_release.jpg

3. avmm_bridge_512_0 reset is de-asserted

avmm_bridge_512_0_rst_n_release.jpg

​4. idle pcie_idle.jpg

0 Kudos
BoonT_Intel
Moderator
1,106 Views

Sorry, one dump question. After you program the SOF, I believe you did restart the host right?

 

Let me upgrade the design to CHU at my side and send you the SOF to retry. In case the process to upgrade the design from BHU to CHU have problem at your side.

0 Kudos
Reply