Server Products
Data Center Products including boards, integrated systems, Intel® Xeon® Processors, RAID Storage, and Intel® Xeon® Processors
4825 Discussions

I want to use DSA, but can't set CONFIG_INTEL_IOMMU_DEFAULT_ON

HaoqiLu
Beginner
2,708 Views

My CPU is Intel(R) Xeon(R) Gold 6438N, and I want to use Intel® Data Streaming Accelerator (DSA).

I set my configuration by User Guide. One of the steps requires setting  CONFIG_INTEL_IOMMU_DEFAULT_ON or the CONFIG_INTEL_IOMMU_SCALABLE_MODE_DEFAULT_ON.

I use command "cat /boot/config-$(uname -r) | grep CONFIG_INTEL_IOMMU ", which shows:

 

1.png

 so I checked the information and did the following operations:

1. sudo vim /etc/default/grub

and I added intel_iommu=on,sm_on into the end of line GRUB_CMDLINE_LINUX_DEFAULT

2. sudo update-grub

3. now I use command:  cat /proc/cmdline

it shows:
BOOT_IMAGE=/boot/vmlinuz-6.5.0-27-generic root=UUID=4fa8d2bb-7f5b-4d02-90d6-1a40cfed0e84 ro quiet splash intel_iommu=on,sm_on vt.handoff=7

4. I reboot the machine
Now the problem is: the result of cat /boot/config-$(uname -r) | grep CONFIG_INTEL_IOMMU doesn't change, it's still the same.
What can I do to set the CONFIG_INTEL_IOMMU_DEFAULT_ON and CONFIG_INTEL_IOMMU_SCALABLE_MODE_DEFAULT_ON equal y as the User Guide said.

I hope to get a reply as soon as possible, I will be grateful !

Labels (1)
0 Kudos
40 Replies
Azeem_Intel
Employee
882 Views

Hi HaoqiLu,



Greetings for the day!


Thank you for your quick response. Please allow us some time to review the details and we will get back with an update shortly.



Best Regards,

Azeem_Intel


0 Kudos
Azeem_Intel
Employee
881 Views

Hi OwenG05,



Greetings for the day!



Thank you for sharing the information. We've checked with our internal team. As mentioned, Ubuntu 22.04 may require loading the IDXD drivers onto the Kernel.


Ubuntu 22.04 uses Kernel version 5.15, and the IDXD driver was introduced around Kernel version 6.2, if I'm not mistaken. Please refer to the OS vendor documentation for confirmation.


Please also check the BIOS options to enable IDXD with the board manufacturer.


Regarding the IDXD repo, please check the following links:


GitHub - intel/idxd-config: Accel-config / libaccel-config

Release accel-config: release 4.1.6 · intel/idxd-config · GitHub


The output below clearly indicates that the DSA is present on the system. However, due to driver issues (IDXD), it is not possible to turn on SVA and continue with the DSA configuration 


❯ sudo dmesg | grep idxd

[ 817.158224] idxd 0000:75:01.0: Unable to turn on user SVA feature.

[ 817.165738] idxd 0000:75:01.0: Intel(R) Accelerator Device (v100)

[ 817.165936] idxd 0000:f2:01.0: Unable to turn on user SVA feature.

[ 817.168988] idxd 0000:f2:01.0: Intel(R) Accelerator Device (v100)

 



Best Regards,


Azeem_Intel



0 Kudos
HaoqiLu
Beginner
853 Views

Hi Azeem_Intel,

Greetings for the day!

It was too late yesterday and I went to rest. I didn't reply in time, sorry.
Based on your reply, do I just need to go to that link  to download and install accel-config and the project will help me install IDXD?

I have now successfully installed the idxd-config project according to the project introduction, what should I do next?
After I restarted the machine, I found that the output was still:

❯ sudo dmesg | grep idxd
[ 68.776021] idxd 0000:75:01.0: Unable to turn on user SVA feature.
[ 68.787714] idxd 0000:75:01.0: Intel(R) Accelerator Device (v100)
[ 68.787880] idxd 0000:f2:01.0: Unable to turn on user SVA feature.
[ 68.789908] idxd 0000:f2:01.0: Intel(R) Accelerator Device (v100)

 

Looking forward to your reply!

0 Kudos
Azeem_Intel
Employee
833 Views

Hi HaoqiLu,



Greetings for the day!


Thank you for the update. We are currently checking with our internal team. Please allow us some time, and we will get back to you shortly.



Best Regards,

Azeem_Intel


0 Kudos
Azeem_Intel
Employee
819 Views

Hi HaoqiLu,



Greetings for the day!



Hope you are doing well. We're currently investigating this issue internally. It seems to be related to the OS configuration in the BIOS, so we're putting our best effort into resolving it. It's important to note that Intel doesn't provide support for the OS, so we recommend you reaching out to the OS vendor for further assistance. Thank you for your understanding."



Best Regards,


Azeem_Intel


0 Kudos
Azeem_Intel
Employee
754 Views

Hi HaoqiLu,

 

Greetings for the day!

 

We appreciate your participation. Upon reviewing the kernel logs, it appears that all DSA is available, but the feature cannot be enabled due to OS configuration issues related to IDXD and SVA. Could you please confirm that the following options are enabled in the BIOS of the system?

 

 

Best Regards,

 

Azeem_Intel

0 Kudos
HaoqiLu
Beginner
718 Views

Hi Azeem_Intel,

Greetings for the day!

I checked these BIOS settings. After updating the new version of bios, I confirmed that these settings are turned on.

The result of the command is:

❯ sudo dmesg | grep idxd
[ 825.345963] idxd 0000:75:01.0: Unable to turn on user SVA feature.
[ 825.369938] idxd 0000:75:01.0: Intel(R) Accelerator Device (v100)
[ 825.370060] idxd 0000:f2:01.0: Unable to turn on user SVA feature.
[ 825.393042] idxd 0000:f2:01.0: Intel(R) Accelerator Device (v100)

What else could be the cause of this problem?

Here is the picture:

2.png

3.png

4.png

5.png

  

 

0 Kudos
Pintu
Employee
702 Views

Hi Lu Haoqi,


Greetings for the Day!


As per the issue, with enabling the User SVA feature for the Intel Accelerator Device, start by reviewing the BIOS settings to ensure that all relevant options, especially those related to Shared Virtual Addressing (SVA), are properly configured. Next, ensure that you have the latest drivers installed for the Intel Accelerator Device by visiting the Intel website or your system manufacturer's support page. Additionally, confirm hardware compatibility by consulting documentation provided by Intel or your system manufacturer to verify support for the User SVA feature. Check for any resource conflicts or limitations that may be preventing the feature from being enabled, and examine system logs and diagnostic tools for relevant errors or warnings. Lastly, consider updating the firmware of the Intel Accelerator Device, as firmware updates can often address compatibility issues and improve functionality


Please don't hesitate to ask any further questions in the future. 


Thank you for using Intel products and services.


Regards,

Manoranjan.


0 Kudos
Azeem_Intel
Employee
611 Views

Hi HaoqiLu,

 

Greetings for the day!

 

My apologies for the delay. Please allow us some time to review the details, and we will get back to you with an update shortly.

 

 

Best Regards,

 

Azeem_Intel


0 Kudos
Azeem_Intel
Employee
599 Views

Hi HaoqiLu,



Greetings for the day!


Working internally, the error is pointing to something related to the BIOS and/or OS compatibility.


We can see the hardware DSA is assigned on the system per the logs:


```

May 8 22:59:43 node13 kernel: [  68.212591] idxd 0000:75:01.0: Intel(R) Accelerator Device (v100)

```


Can you please share the following information:


1. Have you tried running the default kernel "5.15" to see if the issue reproduces?

2. Please share the output of the command: `lspci -vvvv -s 0x75:1.0`

3. Please check page 38 of the Doc#:759709 Intel® Data Streaming Accelerator User Guide for steps to enable DSAA on your project for troubleshooting.

4. As this CPU is Tray, can you confirm us if the CPU was repurposed from another system?



Best regards,

Azeem_Intel


0 Kudos
HaoqiLu
Beginner
331 Views

Hi Azeem_Intel,

Sorry it took so long to reply。

1. I currently have a small number of machines using this CPU, and all of them have tasks to run, so it is not convenient to reinstall the system. In addition, my system kernel version is 6.5.0-35-generic. Do you think it is a problem with the updated system?

2. the result is:
❯ sudo lspci -vvvv -s 0x75:1.0
75:01.0 System peripheral: Intel Corporation Device 0b25
Subsystem: Device 1f24:6000
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
NUMA node: 0
IOMMU group: 1
Region 0: Memory at d6ffff20000 (64-bit, prefetchable) [size=64K]
Region 2: Memory at d6ffff00000 (64-bit, prefetchable) [size=128K]
Capabilities: [40] Express (v2) Root Complex Integrated Endpoint, MSI 00
DevCap: MaxPayload 512 bytes, PhantFunc 0
ExtTag+ RBE+ FLReset+
DevCtl: CorrErr+ NonFatalErr+ FatalErr+ UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+ FLReset-
MaxPayload 512 bytes, MaxReadReq 4096 bytes
DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
DevCap2: Completion Timeout: Not Supported, TimeoutDis+ NROPrPrP- LTR+
10BitTagComp+ 10BitTagReq+ OBFF Not Supported, ExtFmt+ EETLPPrefix+, MaxEETLPPrefixes 1
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis+ LTR- OBFF Disabled,
AtomicOpsCtl: ReqEn-
Capabilities: [80] MSI-X: Enable+ Count=9 Masked-
Vector table: BAR=0 offset=00002000
PBA: BAR=0 offset=00003000
Capabilities: [90] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [100 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP+ FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
UESvrt: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
AERCap: First Error Pointer: 00, ECRCGenCap- ECRCGenEn- ECRCChkCap- ECRCChkEn-
MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
HeaderLog: 00000000 00000000 00000000 00000000
Capabilities: [150 v1] Latency Tolerance Reporting
Max snoop latency: 0ns
Max no snoop latency: 0ns
Capabilities: [160 v1] Transaction Processing Hints
Device specific mode supported
Steering table in TPH capability structure
Capabilities: [170 v1] Virtual Channel
Caps: LPEVC=1 RefClk=100ns PATEntryBits=1
Arb: Fixed+ WRR32- WRR64- WRR128-
Ctrl: ArbSelect=Fixed
Status: InProgress-
VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
Arb: Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=fd
Status: NegoPending- InProgress-
VC1: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
Arb: Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
Ctrl: Enable+ ID=1 ArbSelect=Fixed TC/VC=02
Status: NegoPending- InProgress-
Capabilities: [200 v1] Designated Vendor-Specific: Vendor=8086 ID=0005 Rev=0 Len=24 <?>
Capabilities: [220 v1] Address Translation Service (ATS)
ATSCap: Invalidate Queue Depth: 00
ATSCtl: Enable+, Smallest Translation Unit: 00
Capabilities: [230 v1] Process Address Space ID (PASID)
PASIDCap: Exec- Priv+, Max PASID Width: 14
PASIDCtl: Enable+ Exec- Priv+
Capabilities: [240 v1] Page Request Interface (PRI)
PRICtl: Enable- Reset-
PRISta: RF- UPRGI- Stopped+
Page Request Capacity: 00000200, Page Request Allocation: 00000200
Kernel driver in use: idxd
Kernel modules: idxd

3. as typing the command:
❯ sudo dmesg -t | grep ACPI | grep DMAR
ACPI: DMAR 0x00000000773E9000 000390 (v01 XF EGLSTRM 00000001 ACPI 00040000)
ACPI: Reserving DMAR table memory at [mem 0x773e9000-0x773e938f]

❯ sudo dmesg | grep -i dmar
[ 0.010658] ACPI: DMAR 0x00000000773E9000 000390 (v01 XF EGLSTRM 00000001 ACPI 00040000)
[ 0.010712] ACPI: Reserving DMAR table memory at [mem 0x773e9000-0x773e938f]
[ 0.963825] DMAR: IOMMU enabled
[ 0.963826] DMAR: Enable scalable mode if hardware supports
[ 2.015163] DMAR: Host address width 46
[ 2.015165] DMAR: DRHD base: 0x000000dc7fc000 flags: 0x0
[ 2.015172] DMAR: dmar0: reg_base_addr dc7fc000 ver 6:0 cap 9ed008c40780466 ecap 3ee9e86f050df
[ 2.015175] DMAR: DRHD base: 0x000000e43fc000 flags: 0x0
[ 2.015180] DMAR: dmar1: reg_base_addr e43fc000 ver 6:0 cap 9ed008c40780466 ecap 3ee9e86f050df
[ 2.015182] DMAR: DRHD base: 0x000000ebffc000 flags: 0x0
[ 2.015186] DMAR: dmar2: reg_base_addr ebffc000 ver 6:0 cap 9ed008c40780466 ecap 3ee9e86f050df
[ 2.015188] DMAR: DRHD base: 0x000000f37fc000 flags: 0x0
[ 2.015199] DMAR: dmar3: reg_base_addr f37fc000 ver 6:0 cap 9ed008c40780466 ecap 3ee9e86f050df
[ 2.015201] DMAR: DRHD base: 0x000000faffc000 flags: 0x0
[ 2.015205] DMAR: dmar4: reg_base_addr faffc000 ver 6:0 cap 9ed008c40780466 ecap 3ee9e86f050df
[ 2.015207] DMAR: DRHD base: 0x000000d4bfc000 flags: 0x0
[ 2.015212] DMAR: dmar5: reg_base_addr d4bfc000 ver 6:0 cap 9ed008c40780466 ecap 3ee9e86f050df
[ 2.015214] DMAR: DRHD base: 0x000000fb7fc000 flags: 0x0
[ 2.015219] DMAR: dmar6: reg_base_addr fb7fc000 ver 6:0 cap 9ed008c40780466 ecap 3ef9ea6f050df
[ 2.015220] DMAR: DRHD base: 0x000000a73fc000 flags: 0x0
[ 2.015224] DMAR: dmar7: reg_base_addr a73fc000 ver 6:0 cap 9ed008c40780466 ecap 3ee9e86f050df
[ 2.015226] DMAR: DRHD base: 0x000000b07fc000 flags: 0x0
[ 2.015230] DMAR: dmar8: reg_base_addr b07fc000 ver 6:0 cap 9ed008c40780466 ecap 3ee9e86f050df
[ 2.015232] DMAR: DRHD base: 0x000000b9bfc000 flags: 0x0
[ 2.015236] DMAR: dmar9: reg_base_addr b9bfc000 ver 6:0 cap 9ed008c40780466 ecap 3ee9e86f050df
[ 2.015238] DMAR: DRHD base: 0x000000c2ffc000 flags: 0x0
[ 2.015242] DMAR: dmar10: reg_base_addr c2ffc000 ver 6:0 cap 9ed008c40780466 ecap 3ee9e86f050df
[ 2.015244] DMAR: DRHD base: 0x000000cbffc000 flags: 0x0
[ 2.015248] DMAR: dmar11: reg_base_addr cbffc000 ver 6:0 cap 9ed008c40780466 ecap 3ee9e86f050df
[ 2.015250] DMAR: DRHD base: 0x000000cc7fc000 flags: 0x0
[ 2.015254] DMAR: dmar12: reg_base_addr cc7fc000 ver 6:0 cap 9ed008c40780466 ecap 3ef9ea6f050df
[ 2.015256] DMAR: DRHD base: 0x0000009dffc000 flags: 0x1
[ 2.015259] DMAR: dmar13: reg_base_addr 9dffc000 ver 6:0 cap 9ed008c40780466 ecap 3ee9e86f050df
[ 2.015261] DMAR: RMRR base: 0x0000006e3e0000 end: 0x0000006e3e2fff
[ 2.015263] DMAR: ATSR flags: 0x0
[ 2.015265] DMAR: RHSA base: 0x0000009dffc000 proximity domain: 0x0
[ 2.015267] DMAR: RHSA base: 0x000000a73fc000 proximity domain: 0x0
[ 2.015268] DMAR: RHSA base: 0x000000b07fc000 proximity domain: 0x0
[ 2.015269] DMAR: RHSA base: 0x000000b9bfc000 proximity domain: 0x0
[ 2.015270] DMAR: RHSA base: 0x000000c2ffc000 proximity domain: 0x0
[ 2.015271] DMAR: RHSA base: 0x000000cbffc000 proximity domain: 0x0
[ 2.015272] DMAR: RHSA base: 0x000000cc7fc000 proximity domain: 0x0
[ 2.015273] DMAR: RHSA base: 0x000000dc7fc000 proximity domain: 0x1
[ 2.015274] DMAR: RHSA base: 0x000000e43fc000 proximity domain: 0x1
[ 2.015276] DMAR: RHSA base: 0x000000ebffc000 proximity domain: 0x1
[ 2.015277] DMAR: RHSA base: 0x000000f37fc000 proximity domain: 0x1
[ 2.015278] DMAR: RHSA base: 0x000000faffc000 proximity domain: 0x1
[ 2.015278] DMAR: RHSA base: 0x000000d4bfc000 proximity domain: 0x1
[ 2.015279] DMAR: RHSA base: 0x000000fb7fc000 proximity domain: 0x1
[ 2.015280] DMAR: SATC flags: 0x0
[ 2.015284] DMAR-IR: IOAPIC id 8 under DRHD base 0x9dffc000 IOMMU 13
[ 2.015285] DMAR-IR: HPET id 0 under DRHD base 0x9dffc000
[ 2.015287] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[ 2.018736] DMAR-IR: Enabled IRQ remapping in x2apic mode
[ 3.886313] DMAR: IOMMU feature pasid inconsistent
[ 3.886315] DMAR: IOMMU feature prs inconsistent
[ 3.886316] DMAR: IOMMU feature pasid inconsistent
[ 3.886317] DMAR: IOMMU feature prs inconsistent
[ 3.886318] DMAR: IOMMU feature pasid inconsistent
[ 3.886319] DMAR: IOMMU feature prs inconsistent
[ 3.886320] DMAR: IOMMU feature pasid inconsistent
[ 3.886320] DMAR: IOMMU feature prs inconsistent
[ 3.886321] DMAR: IOMMU feature pasid inconsistent
[ 3.886322] DMAR: IOMMU feature prs inconsistent
[ 3.886323] DMAR: IOMMU feature pasid inconsistent
[ 3.886324] DMAR: IOMMU feature prs inconsistent
[ 3.886324] DMAR: IOMMU feature pasid inconsistent
[ 3.886325] DMAR: IOMMU feature prs inconsistent
[ 3.886326] DMAR: IOMMU feature pasid inconsistent
[ 3.886326] DMAR: IOMMU feature prs inconsistent
[ 3.886327] DMAR: IOMMU feature pasid inconsistent
[ 3.886328] DMAR: IOMMU feature prs inconsistent
[ 3.886328] DMAR: IOMMU feature pasid inconsistent
[ 3.886329] DMAR: IOMMU feature prs inconsistent
[ 3.886330] DMAR: IOMMU feature pasid inconsistent
[ 3.886330] DMAR: IOMMU feature prs inconsistent
[ 3.886331] DMAR: IOMMU feature pasid inconsistent
[ 3.886332] DMAR: IOMMU feature prs inconsistent
[ 3.886333] DMAR: dmar12: Using Queued invalidation
[ 3.886340] dmar12 SVM disabled, incompatible paging mode
[ 3.886345] DMAR: dmar11: Using Queued invalidation
[ 3.886347] DMAR: dmar10: Using Queued invalidation
[ 3.886349] DMAR: dmar9: Using Queued invalidation
[ 3.886351] DMAR: dmar8: Using Queued invalidation
[ 3.886358] DMAR: dmar7: Using Queued invalidation
[ 3.886360] DMAR: dmar6: Using Queued invalidation
[ 3.886362] dmar6 SVM disabled, incompatible paging mode
[ 3.886364] DMAR: dmar5: Using Queued invalidation
[ 3.886367] DMAR: dmar4: Using Queued invalidation
[ 3.886372] DMAR: dmar3: Using Queued invalidation
[ 3.886374] DMAR: dmar2: Using Queued invalidation
[ 3.886376] DMAR: dmar1: Using Queued invalidation
[ 3.886377] DMAR: dmar0: Using Queued invalidation
[ 3.886382] DMAR: dmar13: Using Queued invalidation
[ 4.104308] DMAR: Intel(R) Virtualization Technology for Directed I/O

❯ sudo dmesg | grep -i idxd
[ 825.345963] idxd 0000:75:01.0: Unable to turn on user SVA feature.
[ 825.369938] idxd 0000:75:01.0: Intel(R) Accelerator Device (v100)
[ 825.370060] idxd 0000:f2:01.0: Unable to turn on user SVA feature.
[ 825.393042] idxd 0000:f2:01.0: Intel(R) Accelerator Device (v100)

Just to be clear, I have already set “intel_iommu=on,sm_on”.

4. No, these CPUs are purchased from suppliers.

Thank you for your support and look forward to your reply!

 

 

0 Kudos
Azeem_Intel
Employee
523 Views

Hi HaoqiLu,



Greetings for the day!



This is the first follow-up regarding the issue (I want to use DSA, but can't set CONFIG_INTEL_IOMMU_DEFAULT_ON) you reported to us.


We wanted to inquire whether you had the opportunity to review the plan of action (POA) we provided




Best Regards,


Azeem_Intel


0 Kudos
Azeem_Intel
Employee
414 Views

Hi HaoqiLu,



Greetings for the day!



This is the second follow-up regarding the reported issue. We're eager to ensure a swift resolution and would appreciate any updates or additional information you can provide.




Best Regards,


Azeem_Intel


0 Kudos
Azeem_Intel
Employee
358 Views

Hi HaoqiLu,



Greetings for the day!



This is the third follow-up regarding the reported issue. We're committed to ensuring a swift resolution and would greatly appreciate any updates or additional information you can provide.


If we don't hear back from you soon, we'll assume the issue has been resolved and will proceed to close the case



Best Regards,


Azeem_Intel


0 Kudos
Simon-Intel
Employee
303 Views

Hi HaoqiLu,


Thank you for your prompt response.


Please allow us some time to check this with the internal team, and we will get back to you as soon as we have an update.


If you have any additional information or questions, please feel free to share them with us. We're here to support you every step of the way.


Best regards,

Simon



0 Kudos
Azeem_Intel
Employee
260 Views

Hi HaoqiLu,



Greetings for the day!



Thank you for sharing the logs, screenshots, and information. After analyzing the details, we suspect the issue is related to the BIOS configuration and/or OS driver compatibility.


We believe this might be related to Kernel 6.5, which was not tested and might be missing a driver or have an unidentified issue.


The tested compatibility for the Intel® Xeon® 6438N is with Ubuntu 22.04, which comes with Kernel 5.15.0-73-generic. 


As indicated in the output: “[825.369938] idxd 0000:75:01.0: Intel(R) Accelerator Device (v100),” the DSA Device is present in the system, so there are no hardware issues.


According to the document “Doc#:759709 Intel® Data Streaming Accelerator User Guide to enable DSA on your project” (Page 13), Table 2-1 lists the kernel distribution versions with complete IDXD driver support, specifically “SUSE Linux Enterprise Server SLES 15 SP4.”


We do not provide support for operating systems or BIOS from third-party vendors, so you will need to check with the vendors for further assistance. Please refer to vendor documentation for the latest information.


What we can offer is our best effort to help you:


1. In the logs, we see: “[3.886362] dmar6 SVM disabled, incompatible paging mode.” As per the guide mentioned above, on certain platforms, the VT-d 5-level paging capability is disabled by the BIOS, resulting in “SVM disabled, incompatible paging mode” in the dmesg output. In this case, pass `no5lvl` on the kernel command line. This boot-time parameter disables the 5-level paging mode and forces the kernel to use 4-level paging mode.


2. Also in the logs, we see: “PRI is not enabled and thus SVA cannot be enabled: Capabilities: [240 v1] Page Request Interface (PRI) PRICtl: Enable- Reset-PRISta: RF- UPRGI- Stopped+.” Please check with the OS vendor on how to enable PRI.


3. In the BIOS settings, ensure that "Opt-Out Illegal MSI Mitigation" is enabled, "PRS Capability for PCIe" is enabled, and "Limit CPU PA to 46 bits" is disabled. Consult the system or motherboard documentation for specific BIOS settings related to VT-d and VT-x to ensure no additional configuration is needed at the BIOS level.


4. Lastly, you have the option to use DSA with DPDK, using the “vfio-pci” driver. More details can be found at: [DPDK Guide](https://doc.dpdk.org/guides/dmadevs/idxd.html). Note: We do not provide support for DPDK. Please refer to the DPDK Org documentation.


We hope these steps guide you in the right direction.




Best regards,


Azeem_Intel


0 Kudos
HaoqiLu
Beginner
258 Views

Hi Azeem_Intel,

Thanks for your reply, I will follow up with relevant information after trying your suggestion.

 

 

Best regards!

0 Kudos
Azeem_Intel
Employee
158 Views

Hi HaoqiLu,



Greetings for the day!



This is the first follow-up regarding the issue (I want to use DSA, but can't set CONFIG_INTEL_IOMMU_DEFAULT_ON) you reported to us.


We wanted to inquire whether you had the opportunity to review the plan of action (POA) we provided



Best Regards,


Azeem_Intel


0 Kudos
Azeem_Intel
Employee
91 Views

Hi HaoqiLu,



Greetings for the day!


This is the second follow-up regarding the reported issue. We're eager to ensure a swift resolution and would appreciate any updates or additional information you can provide.



Best regards,


Azeem_Intel


0 Kudos
Azeem_Intel
Employee
33 Views

Hi HaoqiLu,



Greetings for the day!


This is the third follow-up regarding the reported issue. We're committed to ensuring a swift resolution and would greatly appreciate any updates or additional information you can provide.


If we don't hear back from you soon, we'll assume the issue has been resolved and will proceed to close the case



Best regards,


Azeem_Intel


0 Kudos
Reply