PCIe SRIOV VF capabilities

[agula]

Hi,

When using virtual functions on the Arria 10 device , I notice that the virtual functions do not have the same maximum read size and maximum payload size as the physical function. Is there a way to change this so that they match. Since they all share the same physical hardware, I would imagine that they would have the same MSS and MPS in configuration space. Not sure if this is the case of not.

Thank you!

[Deshi_Intel]

HI,

Based on what I read from the user guide doc, by right the virtual function register (PCI Express Device Capabilities Register) should be the same between physical function (PF) and virtual function (VF)

• https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/ug/ug_a10_pcie_sriov.pdf
• page 101 - PCI Express Device Capabilities Register. The VF Device Capabilities Register supports the same fields as the PF Device Capabilities Register.

So, let's take max payload setting for example

• page 85 - reg (0x084) - Maximum Payload Size supported by the Function. Can be configured as 000 (128 bytes) or 001 (256 bytes)
• The setting should follow the max payload setting set in PCIe IP
• page 24 - the only requirement in max payload setting is just to set setting > 128 if used more than 2 PF

May I know where do you see the setting difference in PF vs VF ?

Thanks.

Regards,

dlim

[agula]

I received this information through lspci -d 1172: -vv

DevCtl under the PF capabilities shows a MPS of 256 and read request size of 4K.

DevCtl under the VF capabilities shows a MPS of 128 and read request size of 128.

I'm more worried about the read request size being that small. Since I need high bandwidth on all virtual functions for network interfaces. 

Thanks

[Deshi_Intel]

Hi,

Have you review your PCIe software driver to confirm the actual MPS and read request size setting for both PF and VF in PCI Express Device Capabilities Register space ?

Also, what's the MPS setting set in FPGA PCIe hard IP ?

Thanks.

Regards,

dlim

[agula]

Yes, I have verified that both MPS and maximum read request size are both 0 at address 0x88 of the virtual function driver. The MPS setting of the FPGA PCIe hard IP is set to 2K, which is correctly reflected in device capabilities for both PF and VF.

[Deshi_Intel]

Hi,

According to user guide page 85 (table 58)

• https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/ug/ug_a10_pcie_sriov.pdf

PCI Express Device Control and Status Register - 0x088

• Value setting of 0 for both MPS bit [7:5] and RRS bit [14:12] is indeed = 128 bytes
• So did you modify your driver design to change MPS and RRS setting to other larger value beside 0 ?

Another thing to take note is PCI Express Device Capabilities Register -0x084

• bit [2:0] that control Maximum Payload Size supported by the Function
• What's the value setting here ?

Thanks.

Regards,

dlim

[agula]

Hello,

I tried to program them but they did not update their values. It seems they are set to read only for the Virtual Function. The MPS value at 0x084 is set to 'b100 reflecting a 2K MPS, which is what it is set in the PCIe IP editor.

[agula]

Is 128B just the maximum read request possibly supported by the SR-IOV specification. I just want to know if this is the case.

Thanks

[Deshi_Intel]

Hi,

So far based on all the article that I read so far there is no special control that limit device control register access for VF.

But one thing that I find out is for VF device control register in user guide doc is pointed to 0x048, not 0x088.

• Just wonder have you try to change reg value at 0x048 ?

Thanks.

Regards,

dlim

[agula]

Changing 0x48 also does not work. Seems to be set read only. I am now using the Stratix 10. Is the register mapping the same ? The Stratix 10 SRIOV documentation does not seem to specify. 

[Deshi_Intel]

HI,

Arria 10 and Stratix 10 are having different PCIe reg address space

For A10 : chapter 6.7, figure 48

• https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/ug/ug_a10_pcie_sriov.pdf
• device capability = 0x084
• device control = 0x088

For S10 : chapter 8.1.3, figure 66

• https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/ug/ug_s10_pcie_avst.pdf
• device capability = 0x074
• device control = 0x078

For you case, I guess you should be changing device control which is 0x078 for S10.

Thanks.

Regards,

dlim

[agula]

writing to address 0x78 on Stratix 10 has no effect. It seems they are RO parameters that cannot be changed.

I am assuming that 128B is maximum read request size and max payload size allowed by spec. However, I have not been able to get confirmation that this is the case. The information does not seem available at all.

[Deshi_Intel]

HI,

I don't have any other suggestion for you except following.

I forgot to ask you whether you changed the value in address 0x078 of PF or VF.

• I assume is VF ?

Device control register should be RW register and not RO register. Only device capability register is RO.

• Unless the VF register value is carry forward from PF and can't be edited directly
• Can you try to change the PF 0x078 to see if it bring any changes to VF ?
• If it still doesn't work then likely this is capability limit of VF

Regards,

dlim

[Deshi_Intel]

HI,

After some digging, I managed to find out about PCIe SRIOV spec as per attached pic

[Deshi_Intel]

HI,

Based on the attached spec pic in previous post, looks like there is no reg control for VF. It will just follow PF reg setting automatically.

In fact, user can't control VF reg

Thanks.

Regards,

dlim