Application is stuck in qat_dh_generate_key().

[Alfago12]

Hi Team,

Version information:
driver: QAT1.7.L.4.18.1-00001
engine: QAT_Engine-0.6.14
Openssl: 1.1.1q

Hardware: Intel C627 QAT

My application is stuck in qat_dh_generate_key() after upgrading QAT driver and engine.
Note that the memory driver of QAT engine is qat_contig_mem which can be seen from the configure information at the bottom of the issue.
Stack trace:
Thread 1 (Thread 0x7f8451f77f80 (LWP 6078)):
#0 0x00007f84532c0c77 in sched_yield () from /lib64/libc.so.6
#1 0x00007f844f8a5a2b in qat_dh_generate_key (dh=0x7f8451a3bee0) at qat_hw_dh.c:479
And this is triggered by DH_generate_key(dh) in my application.

And after enabling debug for both QAT driver and engine, I got qat_engine_debug.log and attached it.
qat_engine_debug.log

dmesg:
[ 71.603452] usdm_drv: Loading USDM Module Version 0.7.1 ...
[ 71.603454] usdm_drv: IOCTLs: c0507100, c0507101, 7102, c0047104
[ 71.604862] Loading QAT CONTIG MEM Module ...
[ 71.645856] c6xx 0000:04:00.0: Enabling default configuration
[ 71.956848] c6xx 0000:04:00.0: qat_dev0 started 10 acceleration engines
[ 72.000848] c6xx 0000:05:00.0: Enabling default configuration
[ 72.311839] c6xx 0000:05:00.0: qat_dev1 started 10 acceleration engines
[ 72.354846] c6xx 0000:06:00.0: Enabling default configuration
[ 72.409206] igb 0000:85:00.0 port9: igb: port9 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
[ 72.409325] IPv6: ADDRCONF(NETDEV_CHANGE): port9: link becomes ready
[ 72.665826] c6xx 0000:06:00.0: qat_dev2 started 10 acceleration engines
[ 72.688975] QAT: Stopping all acceleration devices.
[ 72.688980] c6xx 0000:04:00.0: qat_dev0 stopped 10 acceleration engines
[ 72.689464] c6xx 0000:04:00.0: Resetting device qat_dev0
[ 72.689465] c6xx 0000:04:00.0: Function level reset
[ 72.797966] c6xx 0000:05:00.0: qat_dev1 stopped 10 acceleration engines
[ 72.798529] c6xx 0000:05:00.0: Resetting device qat_dev1
[ 72.798530] c6xx 0000:05:00.0: Function level reset
[ 72.901964] c6xx 0000:06:00.0: qat_dev2 stopped 10 acceleration engines
[ 72.902519] c6xx 0000:06:00.0: Resetting device qat_dev2
[ 72.902520] c6xx 0000:06:00.0: Function level reset
[ 74.009175] c6xx 0000:04:00.0: Starting acceleration device qat_dev0.
[ 74.359864] c6xx 0000:04:00.0: qat_dev0 started 10 acceleration engines
[ 74.385812] QDM: iommu_pass_through is set to 0
[ 74.387457] c6xx 0000:05:00.0: Starting acceleration device qat_dev1.
[ 74.735857] c6xx 0000:05:00.0: qat_dev1 started 10 acceleration engines
[ 74.758956] c6xx 0000:06:00.0: Starting acceleration device qat_dev2.
[ 75.092824] c6xx 0000:06:00.0: qat_dev2 started 10 acceleration engines
...
[ 108.954019] WARNING: CPU: 31 PID: 6078 at mm/mmap.c:1807 mmap_region+0x564/0x650
[ 108.954020] Modules linked in: igb_uio(O) qat_c62x(O) qat_contig_mem(O) usdm_drv(O) intel_qat(O) fwbrnd(O) ha(O) network_bypass(O) caswell_bpgen3(O) caswell_pmbus(O) ixgbe(O) cp_uio(O) log_msgid(O) mcast(O) wproxy(O) rp_tproxy(O) tproxy(O) tp_v6_defrag(O) antidos(O) vserver_filter(O) intf_filter(O) bridge_mac(O) ti_bridge(O) share_ip(O) miglog(O)
[ 108.954035] CPU: 31 PID: 6078 Comm: proxyd Kdump: loaded Tainted: G O 5.4.202 #1
[ 108.954036] Hardware name: To be filled by O.E.M. To be filled by O.E.M./To be filled by O.E.M., BIOS 5.6.5 07/27/2015
[ 108.954039] RIP: 0010:mmap_region+0x564/0x650
[ 108.954040] Code: 48 8b 04 24 48 85 c0 0f 84 ea fc ff ff 8b 15 ab 40 10 01 48 f7 d8 48 c7 c7 c0 26 9f 81 48 89 c6 e8 91 49 1f 00 e9 cd fc ff ff <0f> 0b e9 9d fe ff ff 48 8b 7c 24 08 4c 89 4c 24 10 4d 63 e6 49 c7
[ 108.954041] RSP: 0018:ffffc9000ae03d50 EFLAGS: 00010206
[ 108.954043] RAX: 0000000000000000 RBX: ffff88903a399e20 RCX: ffffea0000000000
[ 108.954044] RDX: 8000000000000027 RSI: 000ffffffffff000 RDI: 00007f8441aa0000
[ 108.954044] RBP: 00007f8441a7a000 R08: 0000000000000000 R09: 0000000000000008
[ 108.954045] R10: ffff88903a399ec0 R11: ffff8890597f0780 R12: ffff88903a399e30
[ 108.954046] R13: ffff88905926ec00 R14: 00007f8441a80000 R15: ffff8890597f0780
[ 108.954047] FS: 00007f8451f77f80(0000) GS:ffff88905f8c0000(0000) knlGS:0000000000000000
[ 108.954048] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 108.954049] CR2: 00007f8450330000 CR3: 00000010385a2005 CR4: 00000000001606e0
[ 108.954050] Call Trace:
[ 108.954057] do_mmap+0x397/0x590
[ 108.954060] vm_mmap_pgoff+0x92/0xf0
[ 108.954063] ksys_mmap_pgoff+0x18c/0x230
[ 108.954066] ? ksys_ioctl+0x58/0x70
[ 108.954069] do_syscall_64+0x68/0x3c0
[ 108.954074] ? __do_page_fault+0x23d/0x480
[ 108.954080] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 108.954083] RIP: 0033:0x7f84532d3402
[ 108.954084] Code: 00 00 00 00 0f 1f 40 00 41 f7 c1 ff 0f 00 00 75 27 55 48 89 fd 53 89 cb 48 85 ff 74 3b 41 89 da 48 89 ef b8 09 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 66 5b 5d c3 0f 1f 00 48 8b 05 59 5a 0d 00 64
[ 108.954085] RSP: 002b:00007ffecdc793f8 EFLAGS: 00000246 ORIG_RAX: 0000000000000009
[ 108.954086] RAX: ffffffffffffffda RBX: 0000000000002001 RCX: 00007f84532d3402
[ 108.954087] RDX: 0000000000000003 RSI: 0000000000040000 RDI: 0000000000000000
[ 108.954087] RBP: 0000000000000000 R08: 000000000000003c R09: ffff88903a3e0000
[ 108.954088] R10: 0000000000002001 R11: 0000000000000246 R12: 0000000000000157
[ 108.954089] R13: 00007f8451a23100 R14: 0000000000000000 R15: 0000000000000100
[ 108.954091] ---[ end trace cc675b012022be0a ]---
/#

/# bin/adf_ctl status
Checking status of all devices.
There is 3 QAT acceleration device(s) in the system:
qat_dev0 - type: c6xx, inst_id: 0, node_id: 0, bsf: 0000:04:00.0, #accel: 5 #engines: 10 state: up
qat_dev1 - type: c6xx, inst_id: 1, node_id: 0, bsf: 0000:05:00.0, #accel: 5 #engines: 10 state: up
qat_dev2 - type: c6xx, inst_id: 2, node_id: 0, bsf: 0000:06:00.0, #accel: 5 #engines: 10 state: up

#18
./qat_contig_mem_test
seg mapped to 0x7f3f99c6d000, virtualAddress in seg 0xffff8890531f7000, length 64
Hello world!

PASS Verify for QAT Contig Mem Test.

configure for QAT engine:
./configure
--with-qat_hw_dir=xxxxxx
--with-qat_hw_install_dir=xxxx
--with-openssl_dir=xxxxxx
--with-openssl_install_dir=xxxxx
--enable-qat_hw_multi_thread
--enable-qat_hw_contig_mem
--disable-qat_hw_lenstra_protection ;

Regards,

Allen

[CarlosAM_INTEL]

Hello, @Alfago12:

Thank you for contacting Intel Embedded Community.

Could you please clarify if this thread is related to the one created by you at the following address?

https://community.intel.com/t5/Embedded-Intel-Core-Processors/Intel-C627-Application-is-stuck-in-qat-dh-generate-key/m-p/1413380#M4285

We are waiting for your answer.

Best regards,

@CarlosAM_INTEL.

[Alfago12 · Accepted solution]

Hi @CarlosAM_INTEL ,

Yes, this thread is a duplicate of https://community.intel.com/t5/Embedded-Intel-Core-Processors/Intel-C627-Application-is-stuck-in-qat-dh-generate-key/m-p/1413380#M4285. Pls help to close this thread.

After I post https://community.intel.com/t5/Embedded-Intel-Core-Processors/Intel-C627-Application-is-stuck-in-qat-dh-generate-key/m-p/1413380#M4285, I don't see it appears in the forum. So I post this thread. Sorry for any inconvenience.

Regards,

Alfago