- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi everyone,
Operating system: CentOS Linux release 7.4.1708
Base board: ASUS P8Z77-V LK
FPGA: 1SG280LN2F43E2VG
I have problem with pcie gen3x16 x2 (two functions). FPGA card inserted in slot PCIEX16_1 (blue), slot PCIEX16_2 is available (not used).
Base board provides maximum speed and width (gen3x16), all functions are displayed in the status (lspci):
01:00.0 Unassigned class [ff00]: Altera Corporation Device e10a (rev 01)
Subsystem: Device a106:2804
Flags: bus master, fast devsel, latency 0, IRQ 16
Memory at f0000000 (32-bit, non-prefetchable) [size=32M]
Memory at f2000000 (32-bit, non-prefetchable) [size=2M]
Memory at f22c0000 (32-bit, non-prefetchable) [size=64K]
Memory at f2280000 (32-bit, non-prefetchable) [size=256K]
Capabilities: [40] Power Management version 3
Capabilities: [50] MSI: Enable- Count=1/16 Maskable+ 64bit-
Capabilities: [70] Express Endpoint, MSI 00
Capabilities: [100] Advanced Error Reporting
Capabilities: [148] Virtual Channel
Capabilities: [178] Alternative Routing-ID Interpretation (ARI)
Capabilities: [188] #19
Capabilities: [b80] Vendor Specific Information: ID=1172 Rev=0 Len=05c <?>
Kernel driver in use: dd_board
01:00.1 Unassigned class [ff00]: Adobe Systems, Inc Device e10b (rev 01)
Subsystem: Device a106:2804
Flags: bus master, fast devsel, latency 0, IRQ 16
Memory at f2240000 (32-bit, non-prefetchable) [size=256K]
Memory at f2200000 (32-bit, non-prefetchable) [size=256K]
Capabilities: [40] Power Management version 3
Capabilities: [50] MSI: Enable- Count=1/16 Maskable+ 64bit-
Capabilities: [70] Express Endpoint, MSI 00
Capabilities: [100] Advanced Error Reporting
Capabilities: [178] Alternative Routing-ID Interpretation (ARI)
Kernel driver in use: dd_board2
No errors in drivers.
The error occurs when addressing the second function - function 1 is always ignored. The problem is in the decoding signal from HIP (rx_st_func_num_o), the signal value is always 0.
In the SignalTap, I can see that the packages for the first function are coming correct: header and data. The headers contain the bar addresses for the first function but rx_st_func_num_o = 0.
Because of this, all requests go to function 0. The project was built in two different software versions (18.0pro, 18.1pro).
The following pcie configurations work without errors (bar settings, drivers (and test soft) and slot (PCIEX16_1) are the same): gen3x16 with one function, gen3x8 with two functions or one function,
gen2x8 with two functions or one function.
Has anyone encountered such a problem?
Intel pcie HIP documentation:
https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/ug/ug_s10_pcie_avst.pdf
Base board documentation (page 33):
Best regards,
Alex
- Tags:
- Intel® Stratix®
- PCIe
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi SK Lim,
It's true. I have two physical functions and tlp requests always come in zero function. There are two drivers for working with the device:
dd_board and dd_board2. Driver dd_board only works with ph0 (01:00.0 - e10a), dd_board2 with ph1 (01:00.1 - e10b). In my tests, I only change
the speed and width. BAR settings, drivers (and test soft) are the same. I wrote about this in detail.
Best regards,
Alex
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I did new tests on the new PC (host1). First PC - host0.
Host0:
Operating system: CentOS Linux release 7.4.1708
CPU: i7-3770K (host0_cpu file)
Base board: ASUS P8Z77-V LK
FPGA: 1SG280LN2F43E2VG
Host1 (new):
Operating system: CentOS Linux release 7.6.1810 (Core)
CPU: i7 950 (host1_cpu file)
Base board: ASUS P6X58D-E
FPGA: 1SG280LN2F43E2VG
The problems are the same. Test for host1 (slot PCIEX16_1(blue)): Gen3x16 (two functions) - failure. Gen2x16 (two functions), Gen3x8 (two functions) - OK.
New test for host0: Card in PCIEX16_1: Gen2x16 (two functions) - OK, card in PCIEX16_2: Gen3x16 (two functions) - failure, Gen2x16 (two functions) - OK.
I also attach the full output of pcie statuses for host0 and host1.
Host1 Base board documentation (page 40):
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
HI Sir,
What Quartus version you are using? Could you please try to use the latest version v18.1.1? Are the design can pass PCIe Gen3x8 with two Physical Function?
Regards -SK
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi SK Lim,
The PCIE Gen3x8 with two functions passed the test (18.0). In the previous messages listed the tested settings.
Explanations for my messages: 18.0 - 18.0.0 Build 219, 18.1 - 18.1.0 Build 222. These versions I use.
Only one configuration does not pass tests - gen3x16 two functions (Avalon-ST 512). Project was compiled in two versions: 18.0, 18.1.
Are you looked the hip settings? It's are correct settings?
I will try to compile in the new version (v18.1.1).
Best regards,
Alex
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi SK Lim,
I found the cause of the error. Parameter enable_sriov_hwtcl must be equal to 1 (hip settings in hip_1811 file). I compiled the project with the corrected parameter in the latest version (v18.1.1).
Gen3 x16 configuration passed tests.
The reason is in the adapter.
Path: .../hip/altera_pcie_s10_gen3x16_adapter_inst/u_rx_st_if
Module: u_rx_st_if (altera_pcie_s10_gen3x16_rx_st_if)
module altera_pcie_s10_gen3x16_rx_st_if
#(
parameter pld_rx_parity_ena = "enable",
parameter enable_sriov_hwtcl = 0
)(
...
//avst rx interface
...
input logic rx_st_func_num_i,
...
output logic [1:0] rx_st_func_num_o,
...
);
localparam par_sriov = (pld_rx_parity_ena == "enable" && enable_sriov_hwtcl)? 1 : 0;
localparam only_sriov = (pld_rx_parity_ena != "enable" && enable_sriov_hwtcl)? 1 : 0;
localparam only_par = (pld_rx_parity_ena == "enable" && !enable_sriov_hwtcl)? 1 : 0;
...
//async FIFO read interface
always @ (posedge clk250)..
begin
...
if (rst_clk250)
begin
...
rx_st_func_num_o <= '0;
...
end
else
begin
...
rx_st_func_num_o <= (par_sriov) ? (rx_fifo_rdreq_q? {rx_fifo_rddata_hi[304], rx_fifo_rddata_lo[304]} : '0) : (only_sriov) ? (rx_fifo_rdreq_q? {rx_fifo_rddata_hi[272], rx_fifo_rddata_lo[272]} : '0) : '0 ;
...
end
end // always @ (posedge clk250)
...
endmodule
If enable_sriov_hwtcl = 0 then par_sriov = 0, only_sriov = 0 and rx_st_func_num_o is always zero.
I have two questions.
1. Is this a wrong description in the documentation?
From documentation:
4.2. Multifunction and SR-IOV System Settings
Table 13. Multifunction and SR-IOV System Settings
...
Enable SR-IOV Support On/Off When On, the variant supports multiple VFs. When Off, supports PFs only. SR-IOV is only available in H-Tile devices.
...
My device is L-Tile. For non-Gen3x16 configurations, this parameter can be omitted (tested in 17.1.0, 18.0.0, 18.1.0, 18.1.1). For Gen3x16 physical function 1 is not available (enable_sriov_hwtcl = 0).
My system is built using composition scripts. Adding an HIP instance with 2 functions:
Before (non-Gen3x16 configurations):
add_instance HIP altera_pcie_s10_hip_ast
...
set_instance_parameter_value HIP enable_multi_func_hwtcl 1
set_instance_parameter_value HIP total_pf_count_hwtcl 2
...
After:
add_instance HIP altera_pcie_s10_hip_ast
...
set_instance_parameter_value HIP enable_multi_func_hwtcl 1
set_instance_parameter_value HIP enable_sriov_hwtcl 1 <-- FIX Gen3x16
set_instance_parameter_value HIP total_pf_count_hwtcl 2
...
2. I use the parameter enable_multi_func_hwtcl, its was in 17.1:
4 Parameters
...
Table 10. System Settings
...
Enable multiple physical functions On/Off When On, you can enable multiple physical functions (PFs) on the Multifunction and SR-IOV System Settings tab. You can also also enable TLP Processing
Hints (TPH) and Address Translation Services (ATS) on the TPH/ATS Capabilities tab
...
4.2 Multifunction and SR-IOV System Settings
...
Enable SR-IOV Support On/Off When On, the variant supports multiple VFs. When Off, supports PFs only.
...
This parameter is missing in the latest documentation, but I can assign it in the scripts. Do I need to use it?
The description of SR-IOV is also different. In the latest edition there is a clarification about the tile. Is that the correct description?
Best regards,
Alex
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Sir,
Thank for this update. You are actually using L tile device, and the "multiple physical functions" is actually not available for the L tile device. If you open PCIe GUI for H-tile, you will see there is an option for "enable multiple physical functions". Since this feature is not officially available for L-tile, you probably can hack the lower level file to achieve the same feature, but this is not guarantee that the functional behavior is work as expected.
Regards -SK
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page