Linux device driver for Altera PCI Express Chaining DMA reference design

[Altera_Forum]

I have released a Linux device driver for Altera PCI Express Chaining DMA reference design. I am currently trying to get this in newer kernel releases. 

 The patch can be found here:  http://marc.info/?l=linux-kernel&m=122813921631142&w=2  It has been tested on x86 PC and PowerPC platforms.  Regards,  Leon Woestenberg.

[Altera_Forum]

I recently started getting involved using an Altera Aria IIGX for an embedded design. I am using the PCIe with chaining DMA on the devel board to test out my Linux driver. I use the same techniques as the Linux driver posted here, but with a difference. We use high end server class PC's with lots of memory (8G or more), so we use the PAE kernel. This means that it is very possible that buffers needed for DMA exist above 4GB, so different calls must be used to assemble the SG list, specifically pci_dac_page_to_dma(), which provides the pages as a 64 bit entity (dma64_addr_t). All this works great except when programming a descriptor with any non-zero value in the upper address field of the descriptor table. The resulting DMA from the DMA engine should issue a 4DW read request providing a 64 bit address, however it does not, it still issues the 32 bit address 3DW read request instead. I have PCIe protocol analyzer traces to prove it. I have submitted this to Altera, but have not received an answer yet. This same type of problem would exist with a 64 bit kernel.

[Altera_Forum]

I had a similar issue and found that the reference design DMA hardware has a parameter set that causes it to assume 32-bit addressing, and thus, always generates 3DW headers. I worked around the issue by changing the parameter RC_64BITS_ADDR in altpcierd_example_app_chaining.v from 0 to 1. After rebuilding the h/w in Quartus, I observed 4DW headers where required. 

 I hope this helps.

[Altera_Forum]

BrettFavre, 

 Thanks! This indeed did the trick. I now see it properly addressing up past the 4G barrier.

[Altera_Forum]

OK, so I now have it properly addressing with 64 bit addresses, but there is a problem with actually verifying data transfer. I can DMA a block of data to it, for example the following is part of a PCIe analyzer trace for a 256byte transfer. The first two TLP's are reads from the RC and show the correct data which is a simple incrementing pattern.. 

 TLP 1  05550000 05550001 05550002 05550003 05550004 05550005 05550006 05550007 05550008 05550009 0555000A 0555000B 0555000C 0555000D TLP 2  0555000E 0555000F 05550010 05550011 05550012 05550013 05550014 05550015 05550016 05550017 05550018 05550019 0555001A 0555001B 0555001C 0555001D 0555001E 0555001F   But when I read it back using DMA, the data is corrupted..   05550000 05550001 05550002 05550003 05550004 05550005 05550006 05550007 05550008 05550009 0555000E 0555000F 05550010 05550011 05550012 05550013 05550014 05550015 05550016 05550017 05550018 05550019 0555001A 0555001B 0555001C 0555001D 0555001E 0555001F 00000000 00000000 00000000 00000000  The ones highlighted in RED show where the errors start. Basically the 11th value which should be 0555000A is actually 0555000E. 0555000A - 0555000D are missing. It almost looks like the data starting at 0555000E overwrote it at the wrong spot as if it had a bug incrementing the internal memory address.  

[Altera_Forum]

from where i can download the pcie driver for linux? 

can you provide any link for that?

[Altera_Forum]

soni1877 

 See the first post in this thread for a publicly released driver.  My driver is proprietary company property.

[Altera_Forum]

rpoffen, 

the link in first post is a patch not the base code. i downloaded one code from  git (as its my second post i cant paste links) but that doesn't seems to be correct as per the patch you are telling  is it the exact driver code for pci express?

[Altera_Forum]

Hi soni1877, 

 It is the complete driver code, it appears to be provided as a patch to a Linux distribution. If you want to make standalone files, it should not be hard to extract the code into a plain .c file. If nothing else, just copy the text of the patch into the respective file (altpciechdma.c) and edit it to remove the '+' characters at the front of the lines and the stuff before the actual C code.

[Altera_Forum]

--- Quote Start ---  OK, so I now have it properly addressing with 64 bit addresses, but there is a problem with actually verifying data transfer. I can DMA a block of data to it, for example the following is part of a PCIe analyzer trace for a 256byte transfer. The first two TLP's are reads from the RC and show the correct data which is a simple incrementing pattern..  TLP 1  05550000 05550001 05550002 05550003 05550004 05550005 05550006 05550007 05550008 05550009 0555000A 0555000B 0555000C 0555000D TLP 2  0555000E 0555000F 05550010 05550011 05550012 05550013 05550014 05550015 05550016 05550017 05550018 05550019 0555001A 0555001B 0555001C 0555001D 0555001E 0555001F   But when I read it back using DMA, the data is corrupted..   05550000 05550001 05550002 05550003 05550004 05550005 05550006 05550007 05550008 05550009 0555000E 0555000F 05550010 05550011 05550012 05550013 05550014 05550015 05550016 05550017 05550018 05550019 0555001A 0555001B 0555001C 0555001D 0555001E 0555001F 00000000 00000000 00000000 00000000  The ones highlighted in RED show where the errors start. Basically the 11th value which should be 0555000A is actually 0555000E. 0555000A - 0555000D are missing. It almost looks like the data starting at 0555000E overwrote it at the wrong spot as if it had a bug incrementing the internal memory address.    --- Quote End ---    I am having a similar problem with the 64 bit PCIe ref design on a Stratix IV GT FPGA. I wonder if you have had a solution for it. It seems to me that the problem is related to the Avalon-ST interface between the PCIe hard IP core and the application on endpoint.

[Altera_Forum]

Hi hw342, 

 I have a host PC running linux (Ubuntu 10.04 LTS 64) and connected to an DE4 Stratix IV GX over PCIe. I explore the use of the FPGA as co-processor, but i get same problem of yours.  Do you find same solution?  Best Regards

[Altera_Forum]

Hi,  

 I haven't used yet the driver, but, perhaps that is non-posted transaction?  Thanks.