Community
cancel
Showing results for 
Search instead for 
Did you mean: 

Cyclone V PCIe Avalon-MM Multifunction

Cyclone V PCIe Avalon-MM Multifunction


Cyclone V – PCIe MultiFunction with Qsys Avalon-MM

This is an example design which provides Avalon-MM based read and write access to separate functions behind a single PCIe Hard IP endpoint in Cyclone V. A fully productized version of the Avalon-MM core featuring multifunction is not currently planned. This example design heavily leverages the Stratix V Config Bypass design (in the Quartus installation directory under ip/altera/altera_pcie/altera_pcie_hip_ast_ed/altera_pcie_cfgbp_ed/ ).

This design implements 256 kB BARs** for each of the two functions from a single Gen1 x1 endpoint. The overall hierarchy of the design is shown below.

14.1 Example Design Download: c/c4/Cyclonev_pcie_mfc_rev9_v14.1_13Apr15.zip

13.1 Example Design Download: 3/36/Cyclonev_pcie_mfc_rev5_7April2014.zip

April 2014 Updates

- The design was updated to Quartus v13.1

- The design now enables multiple MSI interrupts. The simulation was edited to incorporate this, details below.

Hierarchy

5/50/Cv_pcie_mf_0.png

The functions of these individual blocks are detailed in their respective sections.

top

This is the top level instance for the design and connects altpcierd_tl_cfg_sample, altpcied_cfbp_64b_control_edit_sj, cyclonev_qsys_pcie and dual_function_ram together.

altpcierd_tl_cfg_sample

The function of this block is to decode the Transaction Layer Configuration Space Signals to provide access decodes for the individual functions. Details on these signals can be found in the Cyclone V Hard IP for PCI Express Handbook, pg. 126-127. The decode signal from this block, cfg_prmcsr, is fed to altpcied_cfbp_64b_control and is the primary decode for the individual functions.

altpcied_cfbp_64b_control_edit_sj

The function of this block is to decode the Avalon-ST backend of the PCIe Hard IP in conjunction with function decodes provided by the altpcierd_tl_cfg_sample block.

cyclone_qsys_pcie Qsys System

This Qsys block contains an Avalon-ST implementation of the Cyclone V Hard IP for PCI Express block. The current configuration of this block for the example design is a PCIe Gen1 x1 endpoint with two functions, Func 0 and Func 1. Each of these functions has been allocated a single 32-bit non pre-fetchable 256 kB BAR. Also included in this block is the Altera PCIe Reconfig Driver and Transceiver Reconfiguration controller. These blocks handle the reconfiguration of the PCIe transceivers for things like offset cancellation, etc. The Avalon-ST ports are exported and connected at the top level.

dual_function_ram

This is another Qsys system that can be customized by the end user. In this example design it consists of two On-Chip Memory blocks (4 kB each) that export separate Avalon-MM Slave interfaces. These interfaces are then connected to the Avalon-MM interface from the altpcied_cfbp_64b_control_edit_sj block at the top level

9/91/Cv_pcie_mf_1.png

Customizing BAR Depth

Currently both the Qsys system and this block are setup to have a BAR depth of 256 kB (address width 18 bits). You can change the BAR depth by changing both the Qsys system and the parameter at the top of altpcied_cfbp_64b_control_edit_sj.v file.

To change the BAR Depth first open the Qsys system and Edit the Cyclone V PCIe Hard IP

3/33/Cv_pcie_mf_2.png

Change Func 0 BAR0 and Func 1 BAR0 to another value besides 256 kB. For simplicity sake both BAR spaces should match.

d/d0/Cv_pcie_mf_3.png

Once this is set Generate the Qsys system and when this is complete run the “replace_defaults.bat” file in the main project directory. Next open the altpcied_cfbp_64b_control_edit_sj.v block and change the AVALON_WADDR parameter to match the address width specified in Qsys.

5/5e/Cv_pcie_mf_4.png

Customization Guidelines

Anytime the cyclonev_qsys_pcie system is edited it is recommended that you run the replace_defaults.bat file in the main project directory. This copies crucial source code (maintained in the .\default_files folder) into the proper locations. Below is a list of the files and their destinations.

Simulation folder - .\cyclonev_qsys_pcie\testbench\cyclonev_qsys_pcie_tb\simulation

The setup_sim.tcl script is using a number of customized files for this simulation that will be regenerated by Qsys. You can see these files in the TCL script as they are in the "default_files" folder.

Synthesis folder - .\cyclonev_qsys_pcie\synthesis\submodules

cyclonev_qsys_pcie_tb.v – simulation folder altpcietb_bfm_top_rp.v – simulation folder altpcietb_bfm_shmem.v – simulation folder altpcietb_bfm_driver_chaining.v – simulation folder altpcietb_bfm_cfbp.v – simulation folder altpcietb_ltssm_mon.v – simulation folder

altpcied_cfbp_64b_control_edit_sj.v – synthesis and simulation folder altpcierd_tl_cfg_sample_edit.v – synthesis and simulation folder

Simulation Hierarchy

The “main” section of the simulation is in the altpcietb_bfm_driver_chaining.v file which is in the cyclonev_qsys_pcie\testbench\cyclonev_qsys_pcie_tb\simulation\submodules directory. “main” starts on L1570, performs configuration and finally tests each function (L1732 for Func 0 and L1748 for Func 1). The “config_bypass_test2” task is contained in the altpcieb_bfm_cfbp.v file which is in the same directory. Otherwise the overall hierarchy and the recommended compile order of the files is documented in the setup_sim.do file.

Running the Simulation

  1. Extract the contents of the downloaded ZIP file
  2. The simulation script for QuestaSim or ModelSim is located in .\cyclonev_qsys_pcie\testbench\mentor
  3. Change to this directory and edit setup_sim.do. You may need to change the default location of the Altera Quartus installation. This is specified on L41 and 43, e.g. “set QUARTUS_INSTALL_DIR "C:/altera/quartus" “
  4. Type “source setup_sim.tcl”
  5. After this completes type “ld_debug”
  6. Once that completes type “do wave.do”
  7. Finally to run the simulation “run -all"

When the simulation completes you should see the following in the Transcript window

c/cd/Cv_pcie_mf_5.png

This is a screenshot of the first write to Function 0

8/80/Cv_pcie_mf_6.png

In terms of performance (roughly the same for both functions), for the first two writes the time between them is 208 ns. The time between the second and third write is 80 ns. The times between the first and second read, and the second to third read are both 1.68 us.

** The BAR depth is adjustable, refer to the “Customizing BAR Depth” section

Adding MSI Capabilties to the Simulation

The altpcietb_bfm_driver_chaining.v is the main simulation task for simulating this design. A number of configuration writes, reads and manual assertion of the MSI Interrupt signals (pg. 195-197 http://www.altera.com/literature/ug/ug_c5_pcie.pdf) were added to this file. These changes are from L1015-1113 in this file. Please also familiarize yourself with pg. 157, Table 8–4. MSI Capability Structure shown below.

5/53/Cv_pcie_mf_msi_3.JPG

Here is the process that was followed to create these example MSI interrupts:

  1. Write to the MSI Capability Register using the msi_cfg task. The MSI Address is set (MSI upper address if 64-bit), the MSI Data, enable multi messaging and finally enable MSI.
  2. (optional) The configuration space was read to validate the settings using the cfg_rd task, this is done for Function 0 and Function 1.
  3. The int_app_msi_tc, int_app_msi_num and int_app_msi_req are properly set and we wait for int_app_msi_req_ack to return
  4. On the root port side we read the shared memory space, this space is defined as the scr_mem variable. In the real world this is the MSI address allocated space. For the simulation this is a scratch memory that is shared between the root port and the altpcietb_bfm_driver_chaining tasks. The shared memory is polled until the MSI is received.
  5. In this simulation we wrote to int_app_msi_num 0, 1, 3 and 7 all in Function 0 and we wrote to int_app_msi_num 0, 1, 3 and 7 in Function 1.

Here is the screen capture of the messages from the simulation and the waveform view.


9/90/Sim_Transcript.PNG

5/5f/Sim_Waves.PNG

Version history
Revision #:
1 of 1
Last update:
‎06-25-2019 04:26 PM
Updated by:
 
Contributors