RoCEv2 Specification

[Derek64]

I am trying to send data from an FPGA to an E810 via RoCE with a point-to-point link. I have formated the packets so that Wireshark recognizes the Ethernet/IP/UDP block as a valid RoCE packet. Now I am trying to set the BTH and the ICRC so that data is written to CPU memory.

I have not been able to find a copy of a recent version of the RoCEv2 specification, and I haven't heard back yet from the trade association on how to purchase one. Does anyone have any suggestions for finding the correct format of the BTH?

Because I am using a point-to-point link and the unreliable datagram, it would be particularly valuable if there were wildcard settings which would be universally accepted on the PC side. Also, I wonder whether it might be possible to use an invalid CRC-32 value? This would simplify the FPGA design tremendously.

Thank you,

Derek

[Sazirah]

Hi Derek64,

Thank you for posting in Intel Community Forum.

Regarding this issue, we require some additional information from your end:

1) Kindly share the product name of your Ethernet E810 product.

2) Kindly share the system that you are using with this adapter.

3) Have you purchased this adapter together with the system or separately?

Meanwhile, you may want to refer to documentation below for the enquiry that you are looking for:

Intel® Ethernet Controller E810 Feature Support Matrix:

https://www.intel.com/content/www/us/en/content-details/630155/intel-ethernet-controller-e810-feature-support-matrix.html?DocID=630155

Intel® Ethernet Controller E810 Datasheet:

https://www.intel.com/content/www/us/en/content-details/613875/intel-ethernet-controller-e810-datasheet.html?DocID=613875

Regards,

Sazzy_Intel

Intel Customer Support Technician

[Derek64]

Thank you for your response.

I am using the E810-CQDA2 in an Ubuntu 24.04.3 PC. I purchased the adapter separately from the system.

The adapter is working exactly as it is supposed to, I just hoped to find the documentation which would allow me to send properly formatted RoCEv2 packets to the adapter.

Thank you,

Derek

[Amina_Sadiya]

Hi Derek64,

Thank you for your response. We can see that the information you are looking for is in the link below under the section 11.6.3. Please let us know if this is the information you were looking for, or if there is anything else we can assist you with.

Link: https://www.intel.com/content/www/us/en/content-details/613875/intel-ethernet-controller-e810-datasheet.html?DocID=613875

Best regards,

Amina

Intel Customer Support Technician

[Sazirah]

Hi Derek64,

Good day,

This is related to your enquiries above, I would like to check if the issue has resolved?

If you have any questions or clarifications, please let me know.

Regards,

Sazzy_Intel

Intel Customer Support Technician

[Derek64]

The manual page you indicated didn't answer my question. The information on the page appears to define the memory for the network adapter for storing a queue pair's parameters. I am looking for the definition of the elements of the RoCE BTH header and the invariant CRC header for packets sent to the network adapter.

There is a very old copy of the InfiniBand and RoCE specification on line, and I am working with that and Wireshark to try to create packets on my FPGA which are accepted as valid.

It is not clear from the Intel documentation whether the E810 will accept a multicast unreliable datagram. It is also not clear from any source I have found how you implement a multicast datagram. Is there a universal Queue Pair / Partition Key for a packet which allows it to be accepted by several nodes at once?

Also, I have inferred from Wireshark's output and the Intel documentation that it may be possible for the adapter to perform RDMA with a packet which has a bad CRC. It would be ideal for my application if I could pass zeros for the CRC block and still have the packet accepted by the adapter.

Thank you for any insight you can offer.

Derek

[TejasMohan]

Hello Derek64,

 

Greetings from Intel!

Thank you for writing back and share the details .

We have sent an email to you requesting some information . 

Kindly check and revert to the email for further assistance .

Thank you for using Intel products and services.

Best regards,

Tejas

Intel Customer Support Technician

[Amina_Sadiya]

Hello Derek, 

Thank you for your response. I see that you need assistance with FPGA products. I recommend posting your query on the FPGAs Forum for further assistance.

Kindly refer to the below link to post your query.

Link: https://community.intel.com/

Best regards,

Amina

Intel Customer Support Technician

[Derek64]

I am not using an Intel FPGA.

I don't need help with the FPGA code or the Network Card performance. I need help understanding the proper format of RoCE packets.

[Sazirah]

Hi Derek64,

Greetings.

I would like to follow up with you regarding this case. Kindly share the SSU logs along with the Wireshark logs so we can analyze these internally and provide our findings.

You can refer to the link below for the steps to collect SSU logs:

Intel® System Support Utility for the Linux* Operating System:

https://www.intel.com/content/www/us/en/download/18895/intel-system-support-utility-for-the-linux-operating-system.html

Also, please provide clear front and back pictures of the adapter, ensuring that the labels on it are clearly visible.

Regards,

Sazzy_Intel

Intel Customer Support Technician

[Derek64]

Here is the SSU log.

I cannot open the PC at this time to get pictures of the adapter.

Thank you,

Derek

[Sreelakshmi1]

Hi Derek64,

Greetings!

Thank you for sharing the logs.

Regarding the NIC images, if those are not available, please check whether you can provide the product markings (primary label) from the outside of the box. This will help us investigate the issue further.

Thank you for understanding.

Best Regards,

Sreelakshmi

Intel Customer Support Technician

[Sazirah]

Hi Derek64,

Greetings.

I would like to follow up regarding this case.

If you have any questions or clarifications regarding this case, please let me know.

Regards,

Sazzy_Intel

Intel Customer Support Technician

[Derek64]

Thank you for following up.

I am going to purchase the latest RoCE specification, so that should address many of my questions.

I wonder whether you could answer the following questions:

• I am looking at the Ethernet Controller E810 datasheet for ways to change how it reacts to improperly formatted packets. Is there documentation on the ice driver interface on how to communication with the network card? So far I have only found examples using ethtools.
• Is there any example code or tutorials on programming the E810 chip other than that contained within the datasheet?

Thank you,

Derek

[Sazirah]

Hi Derek64,

Greetings.

Thank you for patiently waiting for us. Regarding this issue, you may refer information shared below:

1. RoCE specification need to be downloaded from IBTA. 

2. Ethernet frames with CRC errors are dropped by the card/driver.

3. You may find the list of documents available in below link:

https://www.intel.com/content/www/us/en/secure/design/confidential/products-and-solutions/networking-and-io/ethernet-controller-e810/technical-library.html?grouping=rdc%20Content%20Types&sort=title:asc  

4. Please also refer section 1.6 of datasheet which has the list of Standards Supported by the E810 and the link to respective documents. 

Additonally, in below question, are you referring to invalid CRC? Could you give us more insight on what details are you looking?

"I am looking at the Ethernet Controller E810 datasheet for ways to change how it reacts to improperly formatted packets. Is there documentation on the ice driver interface on how to communication with the network card? So far I have only found examples using ethtools."

We look forward for your response.

Regards,

Sazzy_Intel

Intel Customer Support Technician

[Sazirah]

Hi Derek64,

This is related to your enquiries above, I would like to check if the issue has resolved?

If you have any questions or clarifications, please let me know.

Best Regards,

Sazzy_Intel

Intel Customer Support Technician

[Derek64]

Yes, thank you.

Derek

[Dineshbabu]

Hi Derek64,

Greetings!

Thank you for your response. Since your issue has been resolved, we will now close this request. Please don’t hesitate to reach out if you have any further questions. You’re welcome to start a new conversation at any time, as this thread will no longer be monitored.

Regards,

Dinesh

Intel Customer Support Technician