FPGA Intellectual Property
PCI Express*, Networking and Connectivity, Memory Interfaces, DSP IP, and Video IP
6343 Discussions

PCIe SRIOV VF capabilities

agula
New Contributor I
1,780 Views

Hi,

When using virtual functions on the Arria 10 device , I notice that the virtual functions do not have the same maximum read size and maximum payload size as the physical function. Is there a way to change this so that they match. Since they all share the same physical hardware, I would imagine that they would have the same MSS and MPS in configuration space. Not sure if this is the case of not.

Thank you!

0 Kudos
14 Replies
Deshi_Intel
Moderator
1,764 Views

HI,


Based on what I read from the user guide doc, by right the virtual function register (PCI Express Device Capabilities Register) should be the same between physical function (PF) and virtual function (VF)


So, let's take max payload setting for example


  • page 85 - reg (0x084) - Maximum Payload Size supported by the Function. Can be configured as 000 (128 bytes) or 001 (256 bytes)
  • The setting should follow the max payload setting set in PCIe IP
  • page 24 - the only requirement in max payload setting is just to set setting > 128 if used more than 2 PF


May I know where do you see the setting difference in PF vs VF ?


Thanks.


Regards,

dlim


0 Kudos
agula
New Contributor I
1,757 Views

I received this information through lspci -d 1172: -vv

DevCtl under the PF capabilities shows a MPS of 256 and read request size of 4K.

DevCtl under the VF capabilities shows a MPS of 128 and read request size of 128.

 

I'm more worried about the read request size being that small. Since I need high bandwidth on all virtual functions for network interfaces. 

 

Thanks

0 Kudos
Deshi_Intel
Moderator
1,724 Views

Hi,


Have you review your PCIe software driver to confirm the actual MPS and read request size setting for both PF and VF in PCI Express Device Capabilities Register space ?


Also, what's the MPS setting set in FPGA PCIe hard IP ?


Thanks.


Regards,

dlim


0 Kudos
agula
New Contributor I
1,719 Views

Yes, I have verified that both MPS and maximum read request size are both 0 at address 0x88 of the virtual function driver. The MPS setting of the FPGA PCIe hard IP is set to 2K, which is correctly reflected in device capabilities for both PF and VF.

 

0 Kudos
Deshi_Intel
Moderator
1,691 Views

Hi,


According to user guide page 85 (table 58)


PCI Express Device Control and Status Register - 0x088

  • Value setting of 0 for both MPS bit [7:5] and RRS bit [14:12] is indeed = 128 bytes
  • So did you modify your driver design to change MPS and RRS setting to other larger value beside 0 ?


Another thing to take note is PCI Express Device Capabilities Register -0x084

  • bit [2:0] that control Maximum Payload Size supported by the Function
  • What's the value setting here ?


Thanks.


Regards,

dlim


0 Kudos
agula
New Contributor I
1,683 Views

Hello,

 

I tried to program them but they did not update their values. It seems they are set to read only for the Virtual Function. The MPS value at 0x084 is set to 'b100 reflecting a 2K MPS, which is what it is set in the PCIe IP editor.

 

 

0 Kudos
agula
New Contributor I
1,632 Views

Is 128B just the maximum read request possibly supported by the SR-IOV specification. I just want to know if this is the case.

 

Thanks

0 Kudos
Deshi_Intel
Moderator
1,620 Views

Hi,


So far based on all the article that I read so far there is no special control that limit device control register access for VF.


But one thing that I find out is for VF device control register in user guide doc is pointed to 0x048, not 0x088.

  • Just wonder have you try to change reg value at 0x048 ?


Thanks.


Regards,

dlim


0 Kudos
agula
New Contributor I
1,612 Views

Changing 0x48 also does not work. Seems to be set read only. I am now using the Stratix 10. Is the register mapping the same ? The Stratix 10 SRIOV documentation does not seem to specify. 

0 Kudos
Deshi_Intel
Moderator
1,597 Views

HI,


Arria 10 and Stratix 10 are having different PCIe reg address space


For A10 : chapter 6.7, figure 48


For S10 : chapter 8.1.3, figure 66


For you case, I guess you should be changing device control which is 0x078 for S10.


Thanks.


Regards,

dlim


0 Kudos
agula
New Contributor I
1,569 Views

writing to address 0x78 on Stratix 10 has no effect. It seems they are RO parameters that cannot be changed.

I am assuming that 128B is maximum read request size and max payload size allowed by spec. However, I have not been able to get confirmation that this is the case. The information does not seem available at all.

0 Kudos
Deshi_Intel
Moderator
1,547 Views

HI,


I don't have any other suggestion for you except following.


I forgot to ask you whether you changed the value in address 0x078 of PF or VF.

  • I assume is VF ?


Device control register should be RW register and not RO register. Only device capability register is RO.

  • Unless the VF register value is carry forward from PF and can't be edited directly
  • Can you try to change the PF 0x078 to see if it bring any changes to VF ?
  • If it still doesn't work then likely this is capability limit of VF


Regards,

dlim




0 Kudos
Deshi_Intel
Moderator
1,531 Views

HI,

 

After some digging, I managed to find out about PCIe SRIOV spec as per attached pic

0 Kudos
Deshi_Intel
Moderator
1,531 Views

HI,


Based on the attached spec pic in previous post, looks like there is no reg control for VF. It will just follow PF reg setting automatically.


In fact, user can't control VF reg


Thanks.


Regards,

dlim


0 Kudos
Reply