- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
My CPU is Intel(R) Xeon(R) Gold 6438N, and I want to use Intel® Data Streaming Accelerator (DSA).
I set my configuration by User Guide. One of the steps requires setting CONFIG_INTEL_IOMMU_DEFAULT_ON or the CONFIG_INTEL_IOMMU_SCALABLE_MODE_DEFAULT_ON.
I use command "cat /boot/config-$(uname -r) | grep CONFIG_INTEL_IOMMU ", which shows:
so I checked the information and did the following operations:
1. sudo vim /etc/default/grub
and I added intel_iommu=on,sm_on into the end of line GRUB_CMDLINE_LINUX_DEFAULT
2. sudo update-grub
3. now I use command: cat /proc/cmdline
it shows:
BOOT_IMAGE=/boot/vmlinuz-6.5.0-27-generic root=UUID=4fa8d2bb-7f5b-4d02-90d6-1a40cfed0e84 ro quiet splash intel_iommu=on,sm_on vt.handoff=7
4. I reboot the machine
Now the problem is: the result of cat /boot/config-$(uname -r) | grep CONFIG_INTEL_IOMMU doesn't change, it's still the same.
What can I do to set the CONFIG_INTEL_IOMMU_DEFAULT_ON and CONFIG_INTEL_IOMMU_SCALABLE_MODE_DEFAULT_ON equal y as the User Guide said.
I hope to get a reply as soon as possible, I will be grateful !
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
help! Anyone knows how to solve this problem? Or how to get relevant Intel employees to notice this post?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello HaoqiLu,
Thank you for posting in our communities.
We have a forum category for specific issues related to your product, which is the Xeon Processor. This category is monitored by our experts, who can provide you with the best solution. To help you get your answer more quickly, we will now move your post to that category.
Thank you, and have a great day ahead!
Best regards,
Carmona A.
Intel Customer Support Technician
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi HaoqiLu,
Greetings for the day!
Hope you're well. Could you please specify the Linux distribution you're running? Also, is this running on a virtual machine? Additionally, could you share the output of the command: dmesg | grep idxd?
You can find information about checking the Intel® Data Streaming Accelerator on the OS in the https://www.intel.com/content/www/us/en/content-details/759709/intel-data-streaming-accelerator-user-guide.html?DocID=759709 Please check and let us know.
Best Regards,
Azeem_Intel
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks for your reply. Here are the information:
❯ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04.3 LTS
Release: 22.04
Codename: jammy
❯ sudo dmesg | grep idxd
[ 817.158224] idxd 0000:75:01.0: Unable to turn on user SVA feature.
[ 817.165738] idxd 0000:75:01.0: Intel(R) Accelerator Device (v100)
[ 817.165936] idxd 0000:f2:01.0: Unable to turn on user SVA feature.
[ 817.168988] idxd 0000:f2:01.0: Intel(R) Accelerator Device (v100)
I am configuring it on my physical machine now, but I also hope to be able to use DSA in containers in k8s in the future.
Looking forward to your next reply!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi HaoqiLu,
Greetings!
We don't have enough information about working DSA with Kubernetes. As per the user guide, it can be used in the Linux kernel.
Regards,
Dinesh
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
OK, sir. But my problem now is that I can't use it in host, just with the error show before.
I followed the user manual, but something unexpected happened.
How can I solve this?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi HaouqiLu,
Greetings!
Please allow us some time to review the details and we will get back with an update shortly.
Regards,
Dinesh
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thank you very much! Looking forward to your reply!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi HaoqiLu,
Since you are using Ubuntu 22.04.3 LTS Release: 22.04. I would like to know if you have checked the compatibility of the OS with the Intel DSA accelerator. Please refer to page 13 and Table 2-1 of this link https://www.intel.com/content/www/us/en/content-details/759709/intel-data-streaming-accelerator-user-guide.html?DocID=759709
The DSA is compatible with SUSE Linux Enterprise Server SLES 15 SP4, please let us know if you have further questions.
Regards,
Irwan_Intel
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Sorry sir, did you mean that only SUSE Linux Enterprise Server SLES 15 SP4 has complete IDXD driver and i can't use DSA on Ubuntu 22.04.3 LTS? Is there any other way to instasll this driver on Ubuntu or other support? After all this is a very popular linux distribution.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello HaoqiLu,
Thank you for your reply!
May I ask you to confirm Which Linux Distribution are you running?
Is it a virtual machine?
Also May I ask you to run command : Run dmesg | grep idxd
and share the output with us?
You can also check the DSA on the OS using the steps given in this link : https://www.intel.com/content/www/us/en/content-details/759709/intel-data-streaming-accelerator-user-guide.html?DocID=759709
Regards,
Sachin KS
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi HaoqiLu,
Greetings for the day!
This is the first follow-up regarding the issue (I want to use DSA, but can't set CONFIG_INTEL_IOMMU_DEFAULT_ON) you reported to us.
We wanted to inquire whether you had the opportunity to review the plan of action (POA) we provided.
Best Regards,
Azeem_Intel
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello sir,
I am not sure what the action plan you are referring to is. I have responded to all the above replies one by one and given the required information, such as:
i'm using Ubuntu 22.04.3 LTS, and the result of sudo dmesg | grep idxd is:
[ 817.158224] idxd 0000:75:01.0: Unable to turn on user SVA feature.
[ 817.165738] idxd 0000:75:01.0: Intel(R) Accelerator Device (v100)
[ 817.165936] idxd 0000:f2:01.0: Unable to turn on user SVA feature.
[ 817.168988] idxd 0000:f2:01.0: Intel(R) Accelerator Device (v100)
you said the DSA is compatible with SUSE Linux Enterprise Server SLES 15 SP4. However, I cannot change the system due to limited conditions. My current problem is how to use DSA on my Ubuntu22.04 system.
And according to the documentation on Intel’s official website:
Technology Guide | Intel® Data Streaming Accelerator (DSA) - Calico VPP with Intel® DSA on 5th Gen Intel® Xeon® Scalable
Processor
It also uses Ubuntu22.04 system. I want to know why I can’t use it.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi HaoqiLu,
Greetings for the day!
Thank you for your response. Kindly grant us some time as we need to check with our internal team. We will provide you with a resolution as soon as possible.
Best Regards,
Azeem_Intel
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi HaoqiLu,
Greetings for the day!
1 – There is a misunderstanding on the compatibility of DSA Accelerator and Operational Systems. On page 13 and Table 2-1 of this link https://www.intel.com/content/www/us/en/content-details/759709/intel-data-streaming-accelerator-user-guide.html?DocID=759709, mentions: “Distribution kernel versions with complete IDXD driver support are shown in Table 2-1.” Which shows: “SUSE Linux Enterprise Server 15 SP4”.
That means the only Kernel including all IDXD driver support is “SLES 15 SP4”. Other OSs (like Ubuntu 22.04 and others) can be used with DSA, however, the Kernel needs to be updated including IDXD drivers.
For this enablement on Ubuntu 22.04, please check with the OS vendor. We do not provide on Operational systems.
Is good to highlight also that Intel® Xeon® Gold 6438N is a 4th Generation of Xeon and per Operating System Compatibility for Xeon the compatibility for SUE is only “SLES 15 SP4”.
Again, that does not mean that the feature will not work, but the kernel maybe does not have the full drivers. For not included Drivers on the OS, needs to contact OS vendor.
2 – The mention “Technology Guide | Intel® Data Streaming Accelerator (DSA) - Calico VPP with Intel® DSA on 5th Gen Intel® Xeon® Scalable Processor It also uses Ubuntu22.04 system.” This is valid for 5th Gen Xeon (not 4th Gen Xeon) as the customer is using in this case.
3 – The error the you reporting is: “0000:f2:01.0: Unable to turn on user SVA feature”. This error could be related to missing drivers parameters on the OS.
You needs to analyze the information from DSA and Linux commands together and look for inconsistencies or missing elements related to VT-d and PRICtl functionalities.
Kernel Boot Parameters:
- Run grep -i vtd /etc/default/grub and grep -i vtx /etc/default/grub
- Look for options like intel_vtd.active=off or intel_vtd_cmt_mode=off that might disable VT-d functionalities.
VT-d Status:
- Run grep -i vtd /proc/cpuinfo and grep -i vtx /proc/cpuinfo
- Look for entries like "VT-d: active" or "VT-d: disabled" to confirm VT-d activation.
Driver Status:
- Run dmesg | grep -i prictl or journalctl | grep -i prictl
- Look for any error messages or information about PRICtl driver loading or initialization issues. If VT-d is disabled, investigate the cause through BIOS settings or kernel boot parameters.
BIOS
On the BIOS settings, please make sure the following options are correct:
enabled "Opt-Out Illegal MSI Mitigation", enabled "PRS Capability for PCIe" and disabled "Limit CPU PA to 46 bits".
Note: Please also consult with the system or motherboard documentation for specific BIOS settings related to VT-d and VT-x and make sure no additional config is needed on BIOS level.
On the Intel® Data Streaming Accelerator User Guide Page 15, there is a description about checking the DSA on the OS:
- Run lspci -vvv -s 6a:01.0
The output should be like the following:
- Capabilities: [220 v1] Address Translation Service (ATS)
- ATSCap: Invalidate Queue Depth: 00
- ATSCtl: Enable+, Smallest Translation Unit: 00
- Run dmesg | grep -i idxd
- If you see “Unable to turn on SVA feature”, VT-d scalable mode may not be enabled by default, reboot with “intel_iommu=on,sm_on” added to the kernel command line to enable
VT-d scalable mode.
- Run dmesg | grep idxd
- Received: Unable to turn on SVA
- Add this grub config command in /etc/default/grub file:
- GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on,sm_on iommu=on"
- Run update-grub
- Reboot the system.
If the commands above do not show the issue, please share OS Logs, and below Logs
- Authorization log. Location: /var/log/auth.log.
- Daemon Log. Location: /var/log/daemon.log.
- Debug log. Location: /var/log/debug.
- Kernel log. Location: /var/log/kern.log.
- System log. Location: /var/log/syslog.
I hope this helps.
Best Regards,
Azeem_Intel
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Azeem_intel:
Thank you very much for your thoughtful reply!
First of all I want to apologize, the example I gave is indeed about 5th Gen Xeon (not 4th Gen Xeon) . But there are still many examples using 4th Gen Xeon. Such as:
"Intel® Data Streaming Accelerator (DSA)DPDK-DMA Packet Copying with IntelDSA on 4th Gen Intel® Xeon® ScalableProcessors"
"Intel® Data Streaming Accelerator (DSA) –Offloading Memory Copying in VPP by Shared Memory Packet Interface (memif) Plugin with Intel® DSA on 4th Gen Intel® Xeon® Scalable Processors"
"Intel® Data Streaming Accelerator (Intel® DSA) - Calico VPP Multinet with Intel® DSA on 4th Gen Intel® Xeon® Scalable Processor"
According to the System Configurations, the first two used Ubuntu-Server 22.04 and the third one used Ubuntu 22.04.1 LTS.
So I think Ubuntu 22.04 can also use DSA smoothly.
You said I needs to contact OS vendor. But I don't know how to do that, because I am using a free distribution on the Internet, I don’t know how to get technical support.
I think the problem is that I don't have idxd driver as needed.
I found intel/idxd-driver on github, but it said this project will no longer be maintained by Intel.
So I want to ask if this project can be used to install the driver.
And tou give me many command to check the problem, here is the result:
1- Run grep -i vtd /etc/default/grub and grep -i vtx /etc/default/grub
2- Run grep -i vtd /proc/cpuinfo and grep -i vtx /proc/cpuinfo
3- Run dmesg | grep -i prictl or journalctl | grep -i prictl
The result of these three commands is that there is no reply. That is, there is no explicit command indicating that VT-d is disabled.
4- Run lspci -vvv -s 6a:01.0
There is no reply, too.
5- Run dmesg | grep -i idxd
it shows:
[ 817.158224] idxd 0000:75:01.0: Unable to turn on user SVA feature.
[ 817.165738] idxd 0000:75:01.0: Intel(R) Accelerator Device (v100)
[ 817.165936] idxd 0000:f2:01.0: Unable to turn on user SVA feature.
[ 817.168988] idxd 0000:f2:01.0: Intel(R) Accelerator Device (v100)
I reboot with “intel_iommu=on,sm_on” added to the kernel command line to enable, the reuslt of "cat /etc/default/grub" is:
❯ cat /etc/default/grub
# If you change this file, run 'update-grub' afterwards to update
# /boot/grub/grub.cfg.
# For full documentation of the options in this file, see:
# info -f grub -n 'Simple configuration'
GRUB_DEFAULT=0
GRUB_TIMEOUT_STYLE=hidden
GRUB_TIMEOUT=0
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash intel_iommu=on,sm_on iommu=on"
GRUB_CMDLINE_LINUX=""
# Uncomment to enable BadRAM filtering, modify to suit your needs
# This works with Linux (no patch required) and with any kernel that obtains
# the memory map information from GRUB (GNU Mach, kernel of FreeBSD ...)
#GRUB_BADRAM="0x01234567,0xfefefefe,0x89abcdef,0xefefefef"
# Uncomment to disable graphical terminal (grub-pc only)
#GRUB_TERMINAL=console
# The resolution used on graphical terminal
# note that you can use only modes which your graphic card supports via VBE
# you can see them in real GRUB with the command `vbeinfo'
#GRUB_GFXMODE=640x480
# Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to Linux
#GRUB_DISABLE_LINUX_UUID=true
# Uncomment to disable generation of recovery mode menu entries
#GRUB_DISABLE_RECOVERY="true"
# Uncomment to get a beep at grub start
#GRUB_INIT_TUNE="480 440 1"
So the method(update /etc/default/grub file and reboot) doesn't solve the issue.
In general, I still want to ask about how to complete DSA configuration on Ubuntu22.04 system, including information about updating the idxd driver.
I have basically tried the above commands, but they seem to have no results.
It is difficult to filter out the relevant information from the log files you provided earlier. From my general inspection, I feel that there is no content related to idxd.
Thank you again for your help, and I hope you can continue to follow up, thank you very much!
Best Regards.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi HaoqiLu,
Greeting for the Day!
Please allow us some time to review the details and we will get back with an update shortly.
Regards,
Manoranjan Das.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi HaoqiLu,
Greetings for the day!
Thank you for providing the update. While we review the information, could you please share the OS logs with us? This will help us proceed further with resolving the issue.
- Authorization log. Location: /var/log/auth.log.
- Daemon Log. Location: /var/log/daemon.log.
- Debug log. Location: /var/log/debug.
- Kernel log. Location: /var/log/kern.log.
- System log. Location: /var/log/syslog.
Best Regards,
Azeem_Intel
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello HaoqiLu,
Greetings!
We are checking in to see if you were able to find the information we requested. Could you please let us know, so we can proceed with finding a resolution? We look forward to hearing back from you.
Best regards,
Dinesh
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Surely!
I found there are no
- Daemon Log. Location: /var/log/daemon.log.
- Debug log. Location: /var/log/debug.
And I copy other logs you need in /var/log,hope that can help you to find a resolution!
https://drive.google.com/drive/folders/1fxP3gzv0VB9cnK7A62E1icTKSh69oI86?usp=drive_link
The above are some logs I found that you may need. Note that because I am also working on this machine (using k8s), there may be irrelevant information interference, so please try your best to filter it out.
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page