Can't get PCIe (avalon-MM) reference design to work on Arria 10 Dev Kit.

[Altera_Forum]

Hi all, 

 Our research lab recently purchased an Arria 10 GX FPGA Dev Kit. We would like conduct some studies which require transferring data between the host computer and the FPGA via PCIe, so we started with the reference design found on Altera Wiki (http://www.alterawiki.com/wiki/Reference_Design:_Gen3x8_AVMM_DMA_-_Arria_10), to be sure we are doing the right thing, we followed the exact steps instructed in this Altera Tutorial video on Youtube (https://www.youtube.com/watch?v=cgs2q3028ss).   We could successfully program the ref design onto the FPGA board. Since we have only one computer (referred to as machine#1) available, we use it as both the programming machine and the host machine. We first power off machine#1 -> detach the board from the machine#1 -> power on machine#1 and use machine#1 to program the FPGA -> power off machine#1 while keep the power of FPGA on -> plug in FPGA to machine#1 -> power on machine#1.   After doing the above, we can see that the Altera device show up in the output of  lspci | grep altera When we typed lsmod, we can also see that altera_dma is loaded.   Unfortunately, when we install the software part of the ref design by sudo ./install, it always complain that chmod: /dev/altera_dma: no such directory of file. Executing ./run also failed because the device couldn't be found.  I found another thread on this forum (https://www.alteraforum.com/forum/showthread.php?t=54886) talking about a similar situation (the difference is that his board was a StratixV), but following the workaround there didn't help in my case.   At this point, I am not sure what to do to make this reference design work. Could anyone please give me some help? Any suggestion or hint is much appreciated!  Thank you very much!

[Altera_Forum]

Are you using the JTAG to program the FPGA or programming the EPQ flash? If you're using the JTAG, the FPGA will loose its configuration once it is powered off. How are you able to keep the PICE dev kit powered on when you power off the PC? Are you using a separate power supply for the FPGA Kit?  

 Why not try to program the EPQ flash in the kit and then do then insert it into the system when its powered off. The issue could also lie due to incorrect PCIe enumeration , coz you've not power-cycled the PCIe card after programming it.   I would suggest you use another system for testing out the FPGA Dev kit and one for programming it. Not sure if using the same system like this will help / work out.

[Altera_Forum]

--- Quote Start ---  I would suggest you use another system for testing out the FPGA Dev kit and one for programming it. Not sure if using the same system like this will help / work out.  --- Quote End ---    Hi @eapenabrm,  Thank you so much for the suggestion. Now I am actually trying to use two separate computers to complete the task. Following the convention stated in this Altera document (https://www.altera.com/content/dam/altera-www/global/en_us/pdfs/literature/an/an456.pdf), we refer the one where the FPGA's PCIe connector connects to as computer# 1, and the one for programming the FPGA as computer# 2.  The steps we are taking now are (1) Power off computer# 1 -> plug FPGA to computer# 1's PCIe and ATX power supply (2) Connect one end of the USB blaster download cable to computer# 2, and the other end to the FPGA (3) Power on computer# 1 -> interrupt the booting of computer# 1 and make it enters BIOS screen (4) Use computer# 2 to program the reference design's bitstream (output_file/top.sof) into the FPGA. The progress bar shows "100% (successful)" (5) Exit the BIOS on computer# 1 -> press control+alt+del to soft reboot computer# 1 (6) After computer# 1 is booted, go to the software part of the reference design, execute sudo ./install  At this point I am still getting the error chmod: /dev/altera_dma: no such directory of file. executing ./run also failed because the device is still not found. May I ask if there's anything wrong in this sequence of steps? I tried to follow the official document as close as possible, but am still failing. Also, you mentioned in your reply that    --- Quote Start ---   The issue could also lie due to incorrect PCIe enumeration , coz you've not power-cycled the PCIe card after programming it.    --- Quote End ---   May I ask what it means? How should I see if computer# 1 has a correct PCIe enumeration?  Any suggestion is much appreciated!

[Altera_Forum]

Hi eapenabrm, 

 thank you for the suggestions!  Now I've switched to the "two-machine" method, as stated on page 13 in this document (https://www.altera.com/content/dam/altera-www/global/en_us/pdfs/literature/an/an456.pdf), to program the FPGA. I can see the Altera device in the output of lspci | grep Altera , and when I execute lsmod | grep altera, I can see that the module altera_dma is loaded.   However, the device file /dev/altera_dma never really shows up. There was only once I could get the the device /dev/altera_dma to appear under /dev/, but in that one case, the DMA requests always timed out. In all the other attempts, I never be able to make /dev/altera_dma to show up again (meaning I will see "chmod: cannot access /dev/altera_dma: No such file or directory" when executing "sudo ./install" of the ref design software; I am not able to execute the ./install and ./run of the reference software successfully).  dmesg shows the following relevant messages, but I am not sure what went wrong:  altera_dma: module verification failed: signature and/or required key missing - tainting kernel Altera DMA: altera_dma_init(), Nov 7 2017 10:46:02 Altera DMA 0000:01:00.0: Refused to change power state, currently in D3 Altera DMA 0000:01:00.0: can't enable device: BAR 0 not claimed Altera DMA 0000:01:00.0: pci_enable_device() failed Altera DMA 0000:01:00.0: goto err_enable Altera DMA 0000:01:00.0: goto err_initchrdev Altera DMA 0000:01:00.0: goto err_bk_alloc Altera DMA: probe of 0000:01:00.0 failed with error -22 Altera DMA: altera_dma_init(), Nov 7 2017 10:46:05 Altera DMA 0000:01:00.0: Refused to change power state, currently in D3 Altera DMA 0000:01:00.0: can't enable device: BAR 0 not claimed Altera DMA 0000:01:00.0: pci_enable_device() failed Altera DMA 0000:01:00.0: goto err_enable Altera DMA 0000:01:00.0: goto err_initchrdev Altera DMA 0000:01:00.0: goto err_bk_alloc Altera DMA: probe of 0000:01:00.0 failed with error -22   Any suggestion is much appreciated!!!!

[Altera_Forum]

Out of curiosity, what do you get when you execute 'uname -r'?

[Altera_Forum]

the Linux uname command returns the name and information about the current Kernel.  

 It will return things like:  -System type (name).  -Host name (Nodename).  -Software release.  -Software date.  -Machine type.  -Domainname.