FPGA Intellectual Property
PCI Express*, Networking and Connectivity, Memory Interfaces, DSP IP, and Video IP
6343 Discussions

PCI Express SOPC flow hardware ready example with README

Altera_Forum
Honored Contributor II
1,857 Views

When I tried PCI E SOPC flow for the first time, I got stuck a little bit. 

So I put some tips for the SOPC flow in the README file. 

Hope the combination of README and example will help someone going through the same flow.
0 Kudos
18 Replies
Altera_Forum
Honored Contributor II
478 Views

Hi, 

 

This is a good little refdesign. Thanks! I hope you dont mind but I have taken your design and ported it across to Arria GX and the Altera Arria Gx PCI Express Dev Kit. 

 

The design doesnt completely meet timing (yet) but I have tested it on the Altera Arria Gx PCI Express Development board using the GUI attached in your original zip file using the SIIGX drivers and it appears to work fine. 

 

I havent documented the LED, Switch, or CLK pin mapping that I chose or the pin placement that I selected but you should be able to read this out of the pin planner, Assignment editor, or QSF file and map against the schematics supplied with the kit. 

 

I hope this is helpful.
0 Kudos
Altera_Forum
Honored Contributor II
478 Views

too difficult pro at this one?

0 Kudos
Altera_Forum
Honored Contributor II
478 Views

Thank you for the example code. I've been doing something similar using an Arria GX device and I have a couple of questions. First, I was quite unable to make the DMA controller work with memory without adding several pipeline delays in between the memory and the DMA component. I read about this fix in Altera's knowledge base. However, I notice that the Stratix reference design doesn't seem to need the pipeline delays and I wondered why. 

 

My second question is about the PC software side. I'm trying to set up a DMA such that data are copied from my Arria memory and into the PC's memory. Using Jungo's driver software I can get kernel-mode physical memory pointers and I can set up a DMA transaction within the DMA controller which completes OK but nothing turns up in the PC's memory. Presumably the example program PCIe.exe must do all this but Altera won't release the source code so I can't figure out what I'm doing wrong. Any help greatly appreciated. 

 

Kind regards
0 Kudos
Altera_Forum
Honored Contributor II
478 Views

Andy, how do you know the dma transfer occurred correctly? Did you check the dma status register? 

 

Here are a couple of tips from my experience with getting the dma working (only doing single pages so far): 

 

1. Make sure your PCIe simulation works correctly. Reference the control signals used in the simulation testbench (inside altpcietb_bfm_driver.vhd) to see which addresses the DMA control registers are mapped to .. I found that they differed from the address map for the PCIe user guide example. 

 

2. Base your software off the Jungo driver code provided in the WinDriver PCI User's Manual. I'm not sure, but I think using the windriver API calls for word writes are better for writing to the dma control registers rather than using the kernel-mode physical memory pointers you mentioned. I suspect caching can cause problems when attempting to write to a memory mapped hardware peripheral register. On a similar note, be sure to syncCpu and syncIo as shown in the sample code. 

 

3. There is some jungo sample code also available under the altera directory. I've found the code there to be incorrect for the pcie example, but it serves as a useful guide for how to write your own. 

 

Good luck!
0 Kudos
Altera_Forum
Honored Contributor II
478 Views

Thank you. There are some very helpful points here that I shall give careful consideration to. I was basing my notion that DMA had completed successfully on the final state of the status register.  

 

I have made one significant discovery which may be helpful to novices like me, however, since posting the note. When locking memory for DMA within Windriver, the call returns three pointers (more for scatter/gather DMA.) I was mistaking the kernel mode pointer for the physical address pointer. Because the context-sensitive auto-complete element of my programming environment was hiding the physical memory pointer array, I didn't see it at first. The physical pointers appear in an array at the end of the DMA structure.  

 

Kind regards
0 Kudos
Altera_Forum
Honored Contributor II
478 Views

Andy ! Can you help me with some small example for build PC software in Jungo Win driver. I was create SOPC system and can start and stop DMA, read status register e.t.s. Create for Borland Builder small application for Sample Contiguous Buffer DMA Implementation from WinDriver™ PCI/ISA/CardBus User’s Manual Version 9.21 . But I can not see data from on board memory in DMA write cycle.  

Thanks a lot !!!! 

Vladimir
0 Kudos
Altera_Forum
Honored Contributor II
478 Views

Dear Vladimir, 

 

while I doubt that I could find it again, if you dig deep enough into the DMA data sheets you'll find that it doesn't, in fact, work. It turns out that you have to add some pipe line delays (at least, that's how I solved this - there may be better ways.) 

 

I've tried to attach a jpeg of the final architecture around the DMA controller that I ended up with. I hope this helps you because this was a very frustrating journey for me. 

 

While this workd, in my application I managed in the end to avoid the DMA controller by using the NIOS processor to read/write directly into the PCIe address space. 

 

Kind regards 

 

Andy
0 Kudos
Altera_Forum
Honored Contributor II
478 Views

Dear Andy ! 

thank you for your answer ! 

At present I do not understand why my project SOPC PCIexpress for Stratix II GX board work with Altera driver and simple_dma.exe or pcie.exe, but not work with my WinXP application with I create with help jungo win driver library

I have create my Altera Stratix progect following AN532, complile, make pof, sof files, programm flash memory on board. After then use windriver_811 progect from Altera arhive (AN532_PCIe_sopc_s2gx_x4). And all work good ! 

But my own application - is in state of nonoperability. 

I want look small small small example witch work with board in contig.DMA mode.  

thank very much !!!! 

Vladimir
0 Kudos
Altera_Forum
Honored Contributor II
478 Views

Dear Vladimir, 

 

I seem to recall that either Jungo or SOPC builder with PCIe (can't remember which) does not support legacy system interrupts. As a result, you have to use message signalled interrupts (msi.) 

 

Sadly, only Vista and W7 support msi so, if you're relying on message signalled interrupts and using XP, it won't work. The simplest way to see if this is the problem is to try your application on a Vista machine. Like you, I found Altera's application worked but mine didn't at first and this was one of the principal reasons. 

 

Kind regards 

 

Andy
0 Kudos
Altera_Forum
Honored Contributor II
478 Views

thank you for your answer ! 

But if I have a lot of answers then I have a lot of misunderstanding :confused:. 

First - PCIe the principally use MSI ??? interrupt. or INT emulation.  

But how Altera's progect work in example AN532? With out interrupts :confused:. 

 

Thanks !
0 Kudos
Altera_Forum
Honored Contributor II
478 Views

Any PCIe example working on XP must support legacy interrupts. The PCIe core that is used in Altera's example presumably does; if I get time I'll have a look at the application note. You need to check your SOPC PCIe configuaration builder settings to see if this capability exists and, if it does, make sure its selected. The core that I used supported only msi so I had no choice but to move to Vista and W7 for development. 

 

You can, of course, manually control the DMA registers and poll them to see if DMA is actually working in your design, without using/needing interrupts. 

 

Kind regards 

 

Andy
0 Kudos
Altera_Forum
Honored Contributor II
478 Views

Hi, 

 

I have two question, I'm working on the basic SOPC builder Design example ( chapter 8 in the PCIe complier User guide). 

I'm using Arria GX kit dev - simulation running fine, while I was synthesis the design I got a critical warning of crossing between clocks if one of the clocks are from transceiver - and the constrains at the pcie_complier.sdc file are ignored - I try to use set false path between those clocks -no critical warning. 

I use simple Linux application that write and read packet to/from host to local RAM, low rate of packet everything work fine. 

I moved to higher rate (not real high 34MByte/s) and after 10,000 packets the DMA "stuck" und don't send back the Interrupt, I add signal tap on the Avalon and on the 512 test signal, and when this occur the error signal are asserted on error sequence number at the Data link layer, my question are :  

1. Constrains- what should I do with this crossing clock critical warning if not ignore it? 

2. Avalon MM seems to be problematic in case of error from the transaction layer (PCIe) -no signal that indicate error on this transaction? 

 

Thank you, 

Tal.
0 Kudos
Altera_Forum
Honored Contributor II
478 Views

Hi, 

 

Can any one suggest how we can simulate PCIE SOPC design example 

using SystemVerilog/VMM based environemt under Synopsys VCS compiler? 

 

I assume the follwoing: 

1) Endpoint as RTL (DUT) 

2) Root Port (RP and VC) as BFM 

3) How can driver be used thru an interfcae to the BFM. 

 

Thanks. 

 

Regards, 

 

Muhammad
0 Kudos
Altera_Forum
Honored Contributor II
478 Views

Hi Mohammad, 

 

I will take one step back and try to use the Avalon verification suite (Quartus 9.1). 

you can add to your SOPC system Avalon BFMs and Avalon Monitors. 

I believe that this will be a good start point - verifying you Avalon MM components. 

and for this you don't need the PCIe core. 

later when you sure that your component are well covers you try using the PCIe VMM. 

Hope that I help a bit? 

 

Regards, 

Tal.
0 Kudos
Altera_Forum
Honored Contributor II
478 Views

Has anyone had issues with PC hanging and require power cycle when using SOPC based PCIe Hard IP core? Our own PCIe driver installs but if we try to disable the driver, the PC hangs. The same driver doesn't hang when disabling/enabling/uninstall for Altera's PCIe_hiperf_a2gx builds. However, Altera didn't use SOPC for their hiperf demos. I am using the Arria II GX 125k dev board. Also, Jungo created .inf files hang the PC during install with SOPC based PCIe, but exhibit no problems with Altera's hiperf demos. We can read and write to regs in our SOPC version with Jungos GUI and all seems to work fine except when trying to change the driver.

0 Kudos
Altera_Forum
Honored Contributor II
478 Views

Andy, 

Did you do any speed tests when you did get the DMA working? Our setup is working but the speed is only 16Mbyte/s to 65Mbyte/s depending on the host chipset. We are using 9.1sp2 and x1 lane with SOPC flow.
0 Kudos
Altera_Forum
Honored Contributor II
478 Views

hello: 

I downloaded your files and did some simulation in modelsim-altera6.5.First I built an sopc system including DMA Control,On Chip memory(RAM) and PCI Express.My quartus version is 9.1.Then I run the testbentch provided in the project folder.The result is in the attachement,I think the result is not correct. 

I don't understand why the value of DMA signal "length" is always"000000",and the simulation result is also run out of control,Can anyone figure out the reason?
0 Kudos
Altera_Forum
Honored Contributor II
478 Views

btesar, 

Your PC hanging When you uninstall PCIE driver. This case was described in "MegaCore IP Library Release Notes and Errata". It said "When the host programs the PMCSR register at address 0x1f to the D3 state, Hard IP endpoint variants created in SOPC Builder might hang while performing the power management message handshaking protocol with the root port." 

 

The solution is to set test_in[7] = 1 to disable all low state power negotiations. And this was fixed in version 10.1 of the PCI Express MegaCore function.
0 Kudos
Reply