pcie for dummies

[Altera_Forum]

Hello to all. 

I would need a text on the fundamentals of the PCIe. I would like to study the concepts of Endpoint, Root, and Protocol.  I'm using the example hip_cvgt_g2_x4_avmm_dma128_1602.qar, downloaded from the altera's site, but I do not understand what should I change in order to read and write on the PCIe my custom data (from other logic FPGA).  Can you help me?  Regards, Luca

[Altera_Forum]

This book was very useful for me: 

https://www.mindshare.com/learn/pci_express/books

[Altera_Forum]

--- Quote Start ---  This book was very useful for me: https://www.mindshare.com/learn/pci_express/books  --- Quote End ---    Hi Tricky! can you tell me if in a configuration consisting of : Computer (linux) -> PCIe connector-> C5GTdevboard, the read and write commands are made only from the Computer? The PCIe is similar to a master slave configuration?

[Altera_Forum]

Its been a long time since I worked with PCIe (and I didnt do it in much depth). But yes, iirc, the PC shouldnt receive commands from the dev board, but the dev board does need to respond appropriately to any reads etc.

[Altera_Forum]

PCIe is not master/slave. As long as the system is set up correctly the endpoint can master the bus and read/write the PC's memory space, such as main memory.

[Altera_Forum]

--- Quote Start ---  PCIe is not master/slave. As long as the system is set up correctly the endpoint can master the bus and read/write the PC's memory space, such as main memory.  --- Quote End ---    I thought that was the case - but I only used it in a simple set up with the dev card only acting as a slave - as a port to a dual port ram between PC and another processor. What was difficult was having only an avalon streaming port as the stand alone hard core (not using Qsys) would only provide a streaming output. Here I had to write logic to decode and reply to the packets. Hence the reason for the book.

[Altera_Forum]

Yeah, with the streaming interface you have to deal at the TLP level. We're currently using the Avalon-MM flavor of the hard IP block (gen3x4 endpoint) in an Arria 10 GX dev kit plugged into an Intel motherboard running Linux. The Avalon-MM interface relieves you of the TLP details but we're finding that the latency of the MM interface is costing us throughput. We have the endpoint mastering the bus to stream images from a high-speed image sensor into system memory on the PC. We will likely end up going to the streaming interface to try to eliminate a couple dead cycles per burst on the MM interface. Still have a ton to learn about PCIe. We have a PCIe bus analyzer coming in a week or two and that should be very interesting to play with.

[Altera_Forum]

I did learn that windows sends all sorts of wierd and wonderful packets to the core. And it really doesnt like it if you dont respond :) (in the same blue way as holding an interrupt high on PCI)

[Altera_Forum]

--- Quote Start ---  Yeah, with the streaming interface you have to deal at the TLP level. We're currently using the Avalon-MM flavor of the hard IP block (gen3x4 endpoint) in an Arria 10 GX dev kit plugged into an Intel motherboard running Linux. The Avalon-MM interface relieves you of the TLP details but we're finding that the latency of the MM interface is costing us throughput. We have the endpoint mastering the bus to stream images from a high-speed image sensor into system memory on the PC. We will likely end up going to the streaming interface to try to eliminate a couple dead cycles per burst on the MM interface. Still have a ton to learn about PCIe. We have a PCIe bus analyzer coming in a week or two and that should be very interesting to play with.  --- Quote End ---     Hi rsefton. You know where I can find some examples where the dev board (endpoint) writes and reads the ram the computer? I use cyclonevgt dev board and PCIe gen2 x4.

[Altera_Forum]

--- Quote Start ---  Hi rsefton. You know where I can find some examples where the dev board (endpoint) writes and reads the ram the computer? I use cyclonevgt dev board and PCIe gen2 x4.  --- Quote End ---    Altera has a reference design that does this:  https://www.altera.com/content/dam/altera-www/global/en_us/pdfs/literature/an/an456.pdf  If you look at page 12 of this document there is a list of links to reference designs for different families. There is a Cyclone V GT design in the list.  Good luck.

[Altera_Forum]

I am desperate. 

I can not carry out my project. I have to make a PCIe link from cyclone gt dev board to pc linux. The board must generate an interrupt ever 100mSEC ,and linux pc must react to interrupt with a DMA request to the board that sends about 64kB data to PC SDRAM. CAn you help me?

[Altera_Forum]

I've only worked with PCIe on Arria 10 but I assume the PCIe hard IP core is set up the same on Cyclone V. The easiest way to generate interrupts is to use the legacy interrupt capability. The MSI stuff is complicated. To enable legacy interrupts go into the Avalon-MM settings tab for the PCIe core in Qsys and check the box for "Export MSI/MSI-X conduit interfaces". Doing that exposes some new ports on the PCIe block. The signals you're interested in are intx_req and intx_ack. To generate an interrupt you will assert the intx_req line. The core then asserts intx_ack when the interrupt packet has been sent to the root port (PC). Software (on the PC) also has to change a bit in a register in the endpoint configuration space to enable the legacy interrupt generation. If intx_ack is not asserted after you assert intx_req then you don't have interrupts enabled. You can read about all of this in the user guide for the PCIe hard IP block. Just search for "legacy interrupts" in the pdf. 

 I assume you have PCIe transfers working between the end point and PC and just need to get interrupts working. If you don't even have PCIe transfers working then you need to get technical help from Altera or someone else.

[Altera_Forum]

Hi rsefton. I'm not skill with pcie. 

Can you explain me if i must use the avalonst or avalonmm examples? And can you explain me if for my project is better to use a internal memory example or a external sdram  example? The idea is: 1) the board generates 64kB of data 2) the board send with dma 64kB to sdram of PC 3)the board checks if. Dma is complete (is possible?) 4)the board send interrupt to pc. 5)the pc read 64kB of data from pc sdram. 6) the pc executes a dma write to board for inform it to send another 64kB of data