Community
cancel
Showing results for 
Search instead for 
Did you mean: 
jliu85
Beginner
2,218 Views

XL710 VF driver can reply to ARP request but cannot reply to ICMP's

We try to write a XL710 VF driver simply,and test it by using ping command form PF to VF.

Now the VF driver can receive the ARP request and can reply it successfully, and the VF can also receive the ICMP request,and the reply data seems to put into the transmit buffer.

But in the PF, we cannot catch any reply data.

The ARP request can be reply that is to say the VF driver works normally, does it correctly? At least, it shows that the receive part and transmit part are correct.

Things that I want to know is what registers are relative to this issues, or any ideas about this issues?

Thanks & regards!

Animy.

0 Kudos
24 Replies
SYeo3
Valued Contributor I
221 Views

Dear animy,

Thank you for contacting Intel.

We would like to request for the operating system and driver version that you are working on.

We look forward to your reply.

Sincerely,

Sandy

jliu85
Beginner
221 Views

Hi,Sandy

Thanks for your reply.

The driver i used is based on Linux-kernel 3.17.6 and this problem have been solved by ignoring the protocol differences in transmit.

And there is a new problem occurred that shows as follows:

1. after transmit be called 4 times that process 4 transmit packets. (solved. we have fixed it to one transmit,one process tx packet).

2. process 4 receive pakets only after 4 tx packets processing happened. (not solved)

i don't kown where or what register makes the 4 receive packets be processed together after processing 4 tx packets.

Could you give me a help?

Regards!

Animy

SYeo3
Valued Contributor I
221 Views

Hi Animy,

Thanks for providing the information.

I'll check on this internally and will get back to you with my findings.

Sincerely,

Sandy

SYeo3
Valued Contributor I
221 Views

Hi Animy,

In addition to the Linux kernel version, please provide the driver filename you used so we can understand your environment. Thanks.

Sincerely,

Sandy

jliu85
Beginner
221 Views

Hi Sandy,

Thanks for replying.

The driver i referred is i40evf in kernel 3.17.6. Now i used it in iPXE(single process),and the driver works in polling mode.

Now the VF diver can ping successfully, but the process of receive packets is always after 4 packets received,

and the the register is rx_desc->wb.qword1.status_error_len in i40evf code or DD in datasheet 8.3.2.2 Receive Descriptor - Write Back Format, section 8.3.2.2.1 16 Byte Receive Descriptors WB Format, or section 8.3.2.2.2 32 Byte Receive Descriptors WB Format (for 32 byte mode) correspondingly.

In linux code, we find if one packet received,the DD bit of rx_desc->wb.qword1.status_error_len is setted,but in our iPXE code, we find that only after 4 packets received, the register bit will be set.

The things i want to know is what factors make this register be setted (or need some configuration by the VF driver ? ), is it relative to the QRX_TAIL set or it is only done by hardware?

I look forward your reply sincerely.

Thanks,

Animy

SYeo3
Valued Contributor I
221 Views

Hi Animy,

Thanks for providing the details. We'll check on this and update you accordingly.

Sincerely,

Sandy

jliu85
Beginner
221 Views

Hi Sandy,

Thanks for replying. We look forward your further reply as soon as possible.

Best Regards,

Animy

jliu85
Beginner
221 Views

Hi Sandy,

How is going on?

I checked i40evf codes based on Linux 3.17.6, and i found that the value of

num_rx_qp and num_tx_qp be parsed to 1

in i40e_common.c i40e_vf_parse_hw_config

hw->dev_caps.num_rx_qp = msg->num_queue_pairs;

hw->dev_caps.num_tx_qp = msg->num_queue_pairs;

and i forced set these parameters to 1,and the adapter->vsi_res->num_queue_pairs is still 4.

then i the adapter->vsi_res->num_queue_pairs to 1,the divers could not work normally.(both in Linux 14.04. PF driver(work on real machine) and VF driver(work on QEMU) based on Linux-kernel-3.17.6.

I dont't konw the receive latency is relative to the number of queue pairs. If it is yes,how to fix this problem.

I look forword to your reply.

Best,

Animy

SYeo3
Valued Contributor I
221 Views

Hi animy,

Thanks for providing the additional details. I'll check this.

Sincerely,

Sandy

jliu85
Beginner
221 Views

Hi Sandy,

I'm so sorry. In former label line 3

animy 撰写:

Hi Sandy,

How is going on?

I checked i40evf codes based on Linux 3.17.6, and i found that the value of

num_rx_qp and num_tx_qp be parsed to 1

in i40e_common.c i40e_vf_parse_hw_config

hw->dev_caps.num_rx_qp = msg->num_queue_pairs;

hw->dev_caps.num_tx_qp = msg->num_queue_pairs;

and i forced set these parameters to 1,and the adapter->vsi_res->num_queue_pairs is still 4.

then i the adapter->vsi_res->num_queue_pairs to 1,the divers could not work normally.(both in Linux 14.04. PF driver(work on real machine) and VF driver(work on QEMU) based on Linux-kernel-3.17.6.

I dont't konw the receive latency is relative to the number of queue pairs. If it is yes,how to fix this problem.

I look forword to your reply.

Best,

Animy

the num_rx_qp and num_tx_qp be parsed to 4 from HW not to 1.

Best,

Animy

SYeo3
Valued Contributor I
221 Views

Hi Animy,

Thanks for the additional info. I'm still checking it.

I'll get back to you once I have any updates.

Sincerely,

Sandy

SYeo3
Valued Contributor I
221 Views

Hi Animy,

It seems the issue is with RX descriptor write back and why it only operates 4 at a time. This is default operation but can be changed. If you look at the DPDK driver on dpdk.org you can see how this can be done as that is poll mode driver.

Look at functions i40e_vsi_queues_bind_intr and i40e_vsi_enable_queues_intr.

Looking at the i40e and i40vf won't help as they don't work in poll mode.

Hope this helps.

Sincerely,

Sandy

HXie3
Novice
221 Views

Hi Sandy,

My problem is same with Animy. And the PF used Linux driver, not dpdk PMD. So I think i40e_vsi_queues_bind_intr can not be called. because it is in DPDK pmd, not in Linux driver.

I check the code of vf, it will call I40E_VIRTCHNL_OP_CONFIG_VSI_QUEUES, when Linux driver receive this message from VF, it will do some configuration, in i40e_config_vsi_rx_queue function. but I don't know which option is that you mentioned above.

jliu85
Beginner
221 Views

Hi Sandy,

I'm sorry to reply so late.

  1. I have already read the DPDK driver simply, and I found the DPDK driver set interval for irq in i40e_vsi_queues_bind_intr and i40e_vsi_enable_queues_intr. I do the same work in Linux-kernel 3.17.6 that is not effective.
# define I40E_GLINT_CTL

0x0003F800

in i40e_vsi_queues_bind_intr( DPDK ) is not clear to me, and there is no description in XL710 Datasheet.

3. i40e_vsi_queues_bind_intr and i40e_vsi_enable_queues_intr this two function are mentioned above both work in i40e PF,and our work only about XL710 driver. Is there any way to fix this problem only by modifying the XL710 VF driver?

Sincerely,

Animy

SYeo3
Valued Contributor I
221 Views

Thanks for your posts and updates, animy and GroundSea.

 

Let me check on this.

Sincerely,

Sandy

jliu85
Beginner
221 Views

Hi Sandy,

the detail information i use XL710 PF(Linux kernel 3.17.6 PF driver) ping XL710 VF (iPXE driver,polling mode) as follows:

PING 192.168.57.81 (192.168.57.81) 56(84) bytes of data.

64 bytes from 192.168.57.81: icmp_seq=4 ttl=64 time=3039 ms

64 bytes from 192.168.57.81: icmp_seq=5 ttl=64 time=2034 ms

64 bytes from 192.168.57.81: icmp_seq=6 ttl=64 time=1032 ms

64 bytes from 192.168.57.81: icmp_seq=7 ttl=64 time=35.0 ms

64 bytes from 192.168.57.81: icmp_seq=8 ttl=64 time=3045 ms

64 bytes from 192.168.57.81: icmp_seq=9 ttl=64 time=2039 ms

64 bytes from 192.168.57.81: icmp_seq=10 ttl=64 time=1038 ms

64 bytes from 192.168.57.81: icmp_seq=11 ttl=64 time=42.1 ms

64 bytes from 192.168.57.81: icmp_seq=12 ttl=64 time=3047 ms

64 bytes from 192.168.57.81: icmp_seq=13 ttl=64 time=2042 ms

64 bytes from 192.168.57.81: icmp_seq=14 ttl=64 time=1035 ms

64 bytes from 192.168.57.81: icmp_seq=15 ttl=64 time=36.7 ms

FYI

Best regards,

Animy

SYeo3
Valued Contributor I
221 Views

Hi GroundSea,

The flow in that section is done by the DPDK driver not the Linux driver. This might help explain what you are seeing:

See section 8.3.3.1.4.2 Receive descriptor reporting (descriptor write back):

http://www.intel.com/content/dam/www/public/us/en/documents/datasheets/xl710-10-40-controller-datash... Intel® Ethernet Controller XL710 Datasheet

Sincerely,

Sandy

HXie3
Novice
221 Views

Hi Sandy,

Thank you for your information, this is a newer version datasheet, in the previous version, I didn't see such description.

I'll try based on its description. Thank you very much.

Regards,

Haifeng

HXie3
Novice
221 Views

Hi Sandy,

The datasheet said "The receive descriptors could be reported instantly for each packet by setting the ITR_INDX to NoITR. Another option is enabling some coalescing of reported descriptors by using the ITR option. This option could be used for better PCIe and host memory bandwidth use at the expense of some possible latency of reporting the completed descriptor."

So I have modified the RTE_LIBRTE_I40E_ITR_INTERVAL macro to 6 in VF's code, then in the operation I40E_VIRTCHNL_OP_CONFIG_IRQ_MAP, rxitr_idx equals to 3(NoITR). and I added some print in Linux PF driver( in function i40e_config_irq_link_list), it showed the itr_idx had already been set to 3. But the problem still exists. I don't know the reason. is there anything else neet to be set?

Thank you!

Regards,

Haifeng

SYeo3
Valued Contributor I
111 Views

Hi animy, Thank you for posting the ping results. Does it mean the VF is able to reply to ping already?

Hi Haifeng, I'll check on this.

Sincerely,

Sandy

Reply