I want to use DSA, but can't set CONFIG_INTEL_IOMMU_DEFAULT_ON

[HaoqiLu]

My CPU is Intel(R) Xeon(R) Gold 6438N, and I want to use Intel® Data Streaming Accelerator (DSA).

I set my configuration by User Guide. One of the steps requires setting  CONFIG_INTEL_IOMMU_DEFAULT_ON or the CONFIG_INTEL_IOMMU_SCALABLE_MODE_DEFAULT_ON.

I use command "cat /boot/config-$(uname -r) | grep CONFIG_INTEL_IOMMU ", which shows:

 so I checked the information and did the following operations：

1. sudo vim /etc/default/grub

and I added intel_iommu=on,sm_on into the end of line GRUB_CMDLINE_LINUX_DEFAULT

2. sudo update-grub

3. now I use command:  cat /proc/cmdline

it shows:
BOOT_IMAGE=/boot/vmlinuz-6.5.0-27-generic root=UUID=4fa8d2bb-7f5b-4d02-90d6-1a40cfed0e84 ro quiet splash intel_iommu=on,sm_on vt.handoff=7

4. I reboot the machine
Now the problem is: the result of cat /boot/config-$(uname -r) | grep CONFIG_INTEL_IOMMU doesn't change, it's still the same.
What can I do to set the CONFIG_INTEL_IOMMU_DEFAULT_ON and CONFIG_INTEL_IOMMU_SCALABLE_MODE_DEFAULT_ON equal y as the User Guide said.

I hope to get a reply as soon as possible, I will be grateful !

[Azeem_Intel]

Hi HaoqiLu,

Greetings for the day!

Thank you for your quick response. Please allow us some time to review the details and we will get back with an update shortly.

Best Regards,

Azeem_Intel

[Azeem_Intel]

Hi OwenG05,

Greetings for the day!

Thank you for sharing the information. We've checked with our internal team. As mentioned, Ubuntu 22.04 may require loading the IDXD drivers onto the Kernel.

Ubuntu 22.04 uses Kernel version 5.15, and the IDXD driver was introduced around Kernel version 6.2, if I'm not mistaken. Please refer to the OS vendor documentation for confirmation.

Please also check the BIOS options to enable IDXD with the board manufacturer.

Regarding the IDXD repo, please check the following links:

GitHub - intel/idxd-config: Accel-config / libaccel-config

Release accel-config: release 4.1.6 · intel/idxd-config · GitHub

The output below clearly indicates that the DSA is present on the system. However, due to driver issues (IDXD), it is not possible to turn on SVA and continue with the DSA configuration 

❯ sudo dmesg | grep idxd

[ 817.158224] idxd 0000:75:01.0: Unable to turn on user SVA feature.

[ 817.165738] idxd 0000:75:01.0: Intel(R) Accelerator Device (v100)

[ 817.165936] idxd 0000:f2:01.0: Unable to turn on user SVA feature.

[ 817.168988] idxd 0000:f2:01.0: Intel(R) Accelerator Device (v100)

Best Regards,

Azeem_Intel

[HaoqiLu]

Hi Azeem_Intel,

Greetings for the day!

It was too late yesterday and I went to rest. I didn't reply in time, sorry.
Based on your reply, do I just need to go to that link  to download and install accel-config and the project will help me install IDXD?

I have now successfully installed the idxd-config project according to the project introduction, what should I do next?
After I restarted the machine, I found that the output was still：

❯ sudo dmesg | grep idxd
[ 68.776021] idxd 0000:75:01.0: Unable to turn on user SVA feature.
[ 68.787714] idxd 0000:75:01.0: Intel(R) Accelerator Device (v100)
[ 68.787880] idxd 0000:f2:01.0: Unable to turn on user SVA feature.
[ 68.789908] idxd 0000:f2:01.0: Intel(R) Accelerator Device (v100)

Looking forward to your reply!

[Azeem_Intel]

Hi HaoqiLu,

Greetings for the day!

Thank you for the update. We are currently checking with our internal team. Please allow us some time, and we will get back to you shortly.

Best Regards,

Azeem_Intel

[Azeem_Intel]

Hi HaoqiLu,

Greetings for the day!

Hope you are doing well. We're currently investigating this issue internally. It seems to be related to the OS configuration in the BIOS, so we're putting our best effort into resolving it. It's important to note that Intel doesn't provide support for the OS, so we recommend you reaching out to the OS vendor for further assistance. Thank you for your understanding."

Best Regards,

Azeem_Intel

[Azeem_Intel]

Hi HaoqiLu,

Greetings for the day!

We appreciate your participation. Upon reviewing the kernel logs, it appears that all DSA is available, but the feature cannot be enabled due to OS configuration issues related to IDXD and SVA. Could you please confirm that the following options are enabled in the BIOS of the system?

Best Regards,

Azeem_Intel

[HaoqiLu]

Hi Azeem_Intel,

Greetings for the day!

I checked these BIOS settings. After updating the new version of bios, I confirmed that these settings are turned on.

The result of the command is:

❯ sudo dmesg | grep idxd
[ 825.345963] idxd 0000:75:01.0: Unable to turn on user SVA feature.
[ 825.369938] idxd 0000:75:01.0: Intel(R) Accelerator Device (v100)
[ 825.370060] idxd 0000:f2:01.0: Unable to turn on user SVA feature.
[ 825.393042] idxd 0000:f2:01.0: Intel(R) Accelerator Device (v100)

What else could be the cause of this problem?

Here is the picture:

 

 

  

[Pintu]

Hi Lu Haoqi,

Greetings for the Day!

As per the issue, with enabling the User SVA feature for the Intel Accelerator Device, start by reviewing the BIOS settings to ensure that all relevant options, especially those related to Shared Virtual Addressing (SVA), are properly configured. Next, ensure that you have the latest drivers installed for the Intel Accelerator Device by visiting the Intel website or your system manufacturer's support page. Additionally, confirm hardware compatibility by consulting documentation provided by Intel or your system manufacturer to verify support for the User SVA feature. Check for any resource conflicts or limitations that may be preventing the feature from being enabled, and examine system logs and diagnostic tools for relevant errors or warnings. Lastly, consider updating the firmware of the Intel Accelerator Device, as firmware updates can often address compatibility issues and improve functionality

Please don't hesitate to ask any further questions in the future. 

Thank you for using Intel products and services.

Regards,

Manoranjan.