- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello!
I'm having problem with vtune crashing in version 2022.2.0. I did not have these problems with version 2022.1.0.
vtune command:
vtune -collect hotspots -knob sampling-mode=hw -result-dir /tmp/logs/vtune <executable>
cat /etc/*-release output:
NAME="SLES"
VERSION="15-SP3"
VERSION_ID="15.3"
PRETTY_NAME="SUSE Linux Enterprise Server 15 SP3"
ID="sles"
ID_LIKE="suse"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:suse:sles:15:sp3"
DOCUMENTATION_URL="https://documentation.suse.com/"
cat /proc/version output:
Linux version 4.18.0-305.19.1.el8_4.x86_64 (mockbuild@x86-vm-09.build.eng.bos.redhat.com) (gcc version 8.4.1 20200928 (Red Hat 8.4.1-1) (GCC)) #1 SMP Tue Sep 7 07:07:31 EDT 2021
Error log:
vtune: Warning: Access to /proc/kallsyms file is limited. Consider changing /proc/sys/kernel/kptr_restrict to 0 to enable resolution of OS kernel and kernel module symbols.
vtune: Warning: To analyze modules at the kernel level in this configuration, load the Intel sampling driver and set an unlimited (0) value for the Stack size option (if you require stack collection). Alternatively, enable access to kernel-mode monitoring by setting the /proc/sys/kernel/perf_event_paranoid value to 1 or lower.
vtune: Warning: To profile kernel modules during the session, make sure they are available in the /lib/modules/kernel_version/ location.
vtune: Warning: Consider reducing possible collection overhead by setting the /proc/sys/kernel/perf_event_paranoid value to 0 (or less).
vtune: Error: amplxe-perf:
threads_spec: cpu
Using CPUID GenuineIntel-6-55-7
path: /tmp/logs/vtune/data.0
dir-data-file: 1667.perf
dir-reuse: true
Compression enabled, disabling build id collection at the end of the session.
nr_threads: 112
thread_masks[0]: 0x558fd7f36310: maps mask[112]: 0
thread_masks[0]: 0x558fd7f36320: affinity mask[112]: 0
thread_masks[1]: 0x558fd7f36330: maps mask[112]: 1
thread_masks[1]: 0x558fd7f36340: affinity mask[112]: 1
thread_masks[2]: 0x558fd7f36350: maps mask[112]: 2
thread_masks[2]: 0x558fd7f36360: affinity mask[112]: 2
thread_masks[3]: 0x558fd7f36370: maps mask[112]: 3
thread_masks[3]: 0x558fd7f36380: affinity mask[112]: 3
thread_masks[4]: 0x558fd7f36390: maps mask[112]: 4
thread_masks[4]: 0x558fd7f363a0: affinity mask[112]: 4
thread_masks[5]: 0x558fd7f363b0: maps mask[112]: 5
thread_masks[5]: 0x558fd7f363c0: affinity mask[112]: 5
thread_masks[6]: 0x558fd7f363d0: maps mask[112]: 6
thread_masks[6]: 0x558fd7f363e0: affinity mask[112]: 6
thread_masks[7]: 0x558fd7f363f0: maps mask[112]: 7
thread_masks[7]: 0x558fd7f36400: affinity mask[112]: 7
thread_masks[8]: 0x558fd7f36410: maps mask[112]: 8
thread_masks[8]: 0x558fd7f36420: affinity mask[112]: 8
thread_masks[9]: 0x558fd7f36430: maps mask[112]: 9
thread_masks[9]: 0x558fd7f36440: affinity mask[112]: 9
thread_masks[10]: 0x558fd7f36450: maps mask[112]: 10
thread_masks[10]: 0x558fd7f36460: affinity mask[112]: 10
thread_masks[11]: 0x558fd7f36470: maps mask[112]: 11
thread_masks[11]: 0x558fd7f36480: affinity mask[112]: 11
thread_masks[12]: 0x558fd7f36490: maps mask[112]: 12
thread_masks[12]: 0x558fd7f364a0: affinity mask[112]: 12
thread_masks[13]: 0x558fd7f364b0: maps mask[112]: 13
thread_masks[13]: 0x558fd7f364c0: affinity mask[112]: 13
thread_masks[14]: 0x558fd7f364d0: maps mask[112]: 14
thread_masks[14]: 0x558fd7f364e0: affinity mask[112]: 14
thread_masks[15]: 0x558fd7f364f0: maps mask[112]: 15
thread_masks[15]: 0x558fd7f36500: affinity mask[112]: 15
thread_masks[16]: 0x558fd7f36510: maps mask[112]: 16
thread_masks[16]: 0x558fd7f36520: affinity mask[112]: 16
thread_masks[17]: 0x558fd7f36530: maps mask[112]: 17
thread_masks[17]: 0x558fd7f36540: affinity mask[112]: 17
thread_masks[18]: 0x558fd7f36550: maps mask[112]: 18
thread_masks[18]: 0x558fd7f36560: affinity mask[112]: 18
thread_masks[19]: 0x558fd7f36570: maps mask[112]: 19
thread_masks[19]: 0x558fd7f36580: affinity mask[112]: 19
thread_masks[20]: 0x558fd7f36590: maps mask[112]: 20
thread_masks[20]: 0x558fd7f365a0: affinity mask[112]: 20
thread_masks[21]: 0x558fd7f365b0: maps mask[112]: 21
thread_masks[21]: 0x558fd7f365c0: affinity mask[112]: 21
thread_masks[22]: 0x558fd7f365d0: maps mask[112]: 22
thread_masks[22]: 0x558fd7f365e0: affinity mask[112]: 22
thread_masks[23]: 0x558fd7f365f0: maps mask[112]: 23
thread_masks[23]: 0x558fd7f36600: affinity mask[112]: 23
thread_masks[24]: 0x558fd7f36610: maps mask[112]: 24
thread_masks[24]: 0x558fd7f36620: affinity mask[112]: 24
thread_masks[25]: 0x558fd7f36630: maps mask[112]: 25
thread_masks[25]: 0x558fd7f36640: affinity mask[112]: 25
thread_masks[26]: 0x558fd7f36650: maps mask[112]: 26
thread_masks[26]: 0x558fd7f36660: affinity mask[112]: 26
thread_masks[27]: 0x558fd7f36670: maps mask[112]: 27
thread_masks[27]: 0x558fd7f36680: affinity mask[112]: 27
thread_masks[28]: 0x558fd7f36690: maps mask[112]: 28
thread_masks[28]: 0x558fd7f366a0: affinity mask[112]: 28
thread_masks[29]: 0x558fd7f366b0: maps mask[112]: 29
thread_masks[29]: 0x558fd7f366c0: affinity mask[112]: 29
thread_masks[30]: 0x558fd7f366d0: maps mask[112]: 30
thread_masks[30]: 0x558fd7f366e0: affinity mask[112]: 30
thread_masks[31]: 0x558fd7f366f0: maps mask[112]: 31
thread_masks[31]: 0x558fd7f36700: affinity mask[112]: 31
thread_masks[32]: 0x558fd7f36710: maps mask[112]: 32
thread_masks[32]: 0x558fd7f36720: affinity mask[112]: 32
thread_masks[33]: 0x558fd7f36730: maps mask[112]: 33
thread_masks[33]: 0x558fd7f36740: affinity mask[112]: 33
thread_masks[34]: 0x558fd7f36750: maps mask[112]: 34
thread_masks[34]: 0x558fd7f36760: affinity mask[112]: 34
thread_masks[35]: 0x558fd7f36770: maps mask[112]: 35
thread_masks[35]: 0x558fd7f36780: affinity mask[112]: 35
thread_masks[36]: 0x558fd7f36790: maps mask[112]: 36
thread_masks[36]: 0x558fd7f367a0: affinity mask[112]: 36
thread_masks[37]: 0x558fd7f367b0: maps mask[112]: 37
thread_masks[37]: 0x558fd7f367c0: affinity mask[112]: 37
thread_masks[38]: 0x558fd7f367d0: maps mask[112]: 38
thread_masks[38]: 0x558fd7f367e0: affinity mask[112]: 38
thread_masks[39]: 0x558fd7f367f0: maps mask[112]: 39
thread_masks[39]: 0x558fd7f36800: affinity mask[112]: 39
thread_masks[40]: 0x558fd7f36810: maps mask[112]: 40
thread_masks[40]: 0x558fd7f36820: affinity mask[112]: 40
thread_masks[41]: 0x558fd7f36830: maps mask[112]: 41
thread_masks[41]: 0x558fd7f36840: affinity mask[112]: 41
thread_masks[42]: 0x558fd7f36850: maps mask[112]: 42
thread_masks[42]: 0x558fd7f36860: affinity mask[112]: 42
thread_masks[43]: 0x558fd7f36870: maps mask[112]: 43
thread_masks[43]: 0x558fd7f36880: affinity mask[112]: 43
thread_masks[44]: 0x558fd7f36890: maps mask[112]: 44
thread_masks[44]: 0x558fd7f368a0: affinity mask[112]: 44
thread_masks[45]: 0x558fd7f368b0: maps mask[112]: 45
thread_masks[45]: 0x558fd7f368c0: affinity mask[112]: 45
thread_masks[46]: 0x558fd7f368d0: maps mask[112]: 46
thread_masks[46]: 0x558fd7f368e0: affinity mask[112]: 46
thread_masks[47]: 0x558fd7f368f0: maps mask[112]: 47
thread_masks[47]: 0x558fd7f36900: affinity mask[112]: 47
thread_masks[48]: 0x558fd7f36910: maps mask[112]: 48
thread_masks[48]: 0x558fd7f36920: affinity mask[112]: 48
thread_masks[49]: 0x558fd7f36930: maps mask[112]: 49
thread_masks[49]: 0x558fd7f36940: affinity mask[112]: 49
thread_masks[50]: 0x558fd7f36950: maps mask[112]: 50
thread_masks[50]: 0x558fd7f36960: affinity mask[112]: 50
thread_masks[51]: 0x558fd7f36970: maps mask[112]: 51
thread_masks[51]: 0x558fd7f36980: affinity mask[112]: 51
thread_masks[52]: 0x558fd7f36990: maps mask[112]: 52
thread_masks[52]: 0x558fd7f369a0: affinity mask[112]: 52
thread_masks[53]: 0x558fd7f369b0: maps mask[112]: 53
thread_masks[53]: 0x558fd7f369c0: affinity mask[112]: 53
thread_masks[54]: 0x558fd7f369d0: maps mask[112]: 54
thread_masks[54]: 0x558fd7f369e0: affinity mask[112]: 54
thread_masks[55]: 0x558fd7f369f0: maps mask[112]: 55
thread_masks[55]: 0x558fd7f36a00: affinity mask[112]: 55
thread_masks[56]: 0x558fd7f36a10: maps mask[112]: 56
thread_masks[56]: 0x558fd7f36a20: affinity mask[112]: 56
thread_masks[57]: 0x558fd7f36a30: maps mask[112]: 57
thread_masks[57]: 0x558fd7f36a40: affinity mask[112]: 57
thread_masks[58]: 0x558fd7f36a50: maps mask[112]: 58
thread_masks[58]: 0x558fd7f36a60: affinity mask[112]: 58
thread_masks[59]: 0x558fd7f36a70: maps mask[112]: 59
thread_masks[59]: 0x558fd7f36a80: affinity mask[112]: 59
thread_masks[60]: 0x558fd7f36a90: maps mask[112]: 60
thread_masks[60]: 0x558fd7f36aa0: affinity mask[112]: 60
thread_masks[61]: 0x558fd7f36ab0: maps mask[112]: 61
thread_masks[61]: 0x558fd7f36ac0: affinity mask[112]: 61
thread_masks[62]: 0x558fd7f36ad0: maps mask[112]: 62
thread_masks[62]: 0x558fd7f36ae0: affinity mask[112]: 62
thread_masks[63]: 0x558fd7f36af0: maps mask[112]: 63
thread_masks[63]: 0x558fd7f36b00: affinity mask[112]: 63
thread_masks[64]: 0x558fd7f36b10: maps mask[112]: 64
thread_masks[64]: 0x558fd7f36b20: affinity mask[112]: 64
thread_masks[65]: 0x558fd7f36b30: maps mask[112]: 65
thread_masks[65]: 0x558fd7f36b40: affinity mask[112]: 65
thread_masks[66]: 0x558fd7f36b50: maps mask[112]: 66
thread_masks[66]: 0x558fd7f36b60: affinity mask[112]: 66
thread_masks[67]: 0x558fd7f36b70: maps mask[112]: 67
thread_masks[67]: 0x558fd7f36b80: affinity mask[112]: 67
thread_masks[68]: 0x558fd7f36b90: maps mask[112]: 68
thread_masks[68]: 0x558fd7f36ba0: affinity mask[112]: 68
thread_masks[69]: 0x558fd7f36bb0: maps mask[112]: 69
thread_masks[69]: 0x558fd7f36bc0: affinity mask[112]: 69
thread_masks[70]: 0x558fd7f36bd0: maps mask[112]: 70
thread_masks[70]: 0x558fd7f36be0: affinity mask[112]: 70
thread_masks[71]: 0x558fd7f36bf0: maps mask[112]: 71
thread_masks[71]: 0x558fd7f36c00: affinity mask[112]: 71
thread_masks[72]: 0x558fd7f36c10: maps mask[112]: 72
thread_masks[72]: 0x558fd7f36c20: affinity mask[112]: 72
thread_masks[73]: 0x558fd7f36c30: maps mask[112]: 73
thread_masks[73]: 0x558fd7f36c40: affinity mask[112]: 73
thread_masks[74]: 0x558fd7f36c50: maps mask[112]: 74
thread_masks[74]: 0x558fd7f36c60: affinity mask[112]: 74
thread_masks[75]: 0x558fd7f36c70: maps mask[112]: 75
thread_masks[75]: 0x558fd7f36c80: affinity mask[112]: 75
thread_masks[76]: 0x558fd7f36c90: maps mask[112]: 76
thread_masks[76]: 0x558fd7f36ca0: affinity mask[112]: 76
thread_masks[77]: 0x558fd7f36cb0: maps mask[112]: 77
thread_masks[77]: 0x558fd7f36cc0: affinity mask[112]: 77
thread_masks[78]: 0x558fd7f36cd0: maps mask[112]: 78
thread_masks[78]: 0x558fd7f36ce0: affinity mask[112]: 78
thread_masks[79]: 0x558fd7f36cf0: maps mask[112]: 79
thread_masks[79]: 0x558fd7f36d00: affinity mask[112]: 79
thread_masks[80]: 0x558fd7f36d10: maps mask[112]: 80
thread_masks[80]: 0x558fd7f36d20: affinity mask[112]: 80
thread_masks[81]: 0x558fd7f36d30: maps mask[112]: 81
thread_masks[81]: 0x558fd7f36d40: affinity mask[112]: 81
thread_masks[82]: 0x558fd7f36d50: maps mask[112]: 82
thread_masks[82]: 0x558fd7f36d60: affinity mask[112]: 82
thread_masks[83]: 0x558fd7f36d70: maps mask[112]: 83
thread_masks[83]: 0x558fd7f36d80: affinity mask[112]: 83
thread_masks[84]: 0x558fd7f36d90: maps mask[112]: 84
thread_masks[84]: 0x558fd7f36da0: affinity mask[112]: 84
thread_masks[85]: 0x558fd7f36db0: maps mask[112]: 85
thread_masks[85]: 0x558fd7f36dc0: affinity mask[112]: 85
thread_masks[86]: 0x558fd7f36dd0: maps mask[112]: 86
thread_masks[86]: 0x558fd7f36de0: affinity mask[112]: 86
thread_masks[87]: 0x558fd7f36df0: maps mask[112]: 87
thread_masks[87]: 0x558fd7f36e00: affinity mask[112]: 87
thread_masks[88]: 0x558fd7f36e10: maps mask[112]: 88
thread_masks[88]: 0x558fd7f36e20: affinity mask[112]: 88
thread_masks[89]: 0x558fd7f36e30: maps mask[112]: 89
thread_masks[89]: 0x558fd7f36e40: affinity mask[112]: 89
thread_masks[90]: 0x558fd7f36e50: maps mask[112]: 90
thread_masks[90]: 0x558fd7f36e60: affinity mask[112]: 90
thread_masks[91]: 0x558fd7f36e70: maps mask[112]: 91
thread_masks[91]: 0x558fd7f36e80: affinity mask[112]: 91
thread_masks[92]: 0x558fd7f36e90: maps mask[112]: 92
thread_masks[92]: 0x558fd7f36ea0: affinity mask[112]: 92
thread_masks[93]: 0x558fd7f36eb0: maps mask[112]: 93
thread_masks[93]: 0x558fd7f36ec0: affinity mask[112]: 93
thread_masks[94]: 0x558fd7f36ed0: maps mask[112]: 94
thread_masks[94]: 0x558fd7f36ee0: affinity mask[112]: 94
thread_masks[95]: 0x558fd7f36ef0: maps mask[112]: 95
thread_masks[95]: 0x558fd7f36f00: affinity mask[112]: 95
thread_masks[96]: 0x558fd7f36f10: maps mask[112]: 96
thread_masks[96]: 0x558fd7f36f20: affinity mask[112]: 96
thread_masks[97]: 0x558fd7f36f30: maps mask[112]: 97
thread_masks[97]: 0x558fd7f36f40: affinity mask[112]: 97
thread_masks[98]: 0x558fd7f36f50: maps mask[112]: 98
thread_masks[98]: 0x558fd7f36f60: affinity mask[112]: 98
thread_masks[99]: 0x558fd7f36f70: maps mask[112]: 99
thread_masks[99]: 0x558fd7f36f80: affinity mask[112]: 99
thread_masks[100]: 0x558fd7f36f90: maps mask[112]: 100
thread_masks[100]: 0x558fd7f36fa0: affinity mask[112]: 100
thread_masks[101]: 0x558fd7f36fb0: maps mask[112]: 101
thread_masks[101]: 0x558fd7f36fc0: affinity mask[112]: 101
thread_masks[102]: 0x558fd7f36fd0: maps mask[112]: 102
thread_masks[102]: 0x558fd7f36fe0: affinity mask[112]: 102
thread_masks[103]: 0x558fd7f36ff0: maps mask[112]: 103
thread_masks[103]: 0x558fd7f37000: affinity mask[112]: 103
thread_masks[104]: 0x558fd7f37010: maps mask[112]: 104
thread_masks[104]: 0x558fd7f37020: affinity mask[112]: 104
thread_masks[105]: 0x558fd7f37030: maps mask[112]: 105
thread_masks[105]: 0x558fd7f37040: affinity mask[112]: 105
thread_masks[106]: 0x558fd7f37050: maps mask[112]: 106
thread_masks[106]: 0x558fd7f37060: affinity mask[112]: 106
thread_masks[107]: 0x558fd7f37070: maps mask[112]: 107
thread_masks[107]: 0x558fd7f37080: affinity mask[112]: 107
thread_masks[108]: 0x558fd7f37090: maps mask[112]: 108
thread_masks[108]: 0x558fd7f370a0: affinity mask[112]: 108
thread_masks[109]: 0x558fd7f370b0: maps mask[112]: 109
thread_masks[109]: 0x558fd7f370c0: affinity mask[112]: 109
thread_masks[110]: 0x558fd7f370d0: maps mask[112]: 110
thread_masks[110]: 0x558fd7f370e0: affinity mask[112]: 110
thread_masks[111]: 0x558fd7f370f0: maps mask[112]: 111
thread_masks[111]: 0x558fd7f37100: affinity mask[112]: 111
nr_cblocks: 0
affinity: SYS
mmap flush: 1
comp level: 1
Problems creating module maps, continuing anyway...
mmap size 528384B
Failed to start threads: File exists
failed to write feature HYBRID_TOPOLOGY
failed to write feature HYBRID_CPU_PMU_CAPS
[ perf record: Captured and wrote 0.023 MB /tmp/logs/vtune/data.0 ]
vtune: Collection failed.
vtune: Internal Error
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
Thank you for posting in Intel Communities.
We ran hardware event-based sampling from our side on RHEL 8 and also in SLES 15.3 with the latest version of Vtune (2022.3) and it was working fine. Please go through the below screenshots.
Could you please try the latest version of vtune and let us know?
download the latest version of Vtune: Download the Intel® VTune™ Profiler
RHEL8:
SLES15.3:
If the issue still persists, please provide the below details.
1. Processor Information
2. Vtune2022.3 self-checker log (latest)
<Vtune_installation_directory\latest\bin64\vtune-self-checker.bat> for windows.
<Vtune_installation_directory\latest\bin64\vtune-self-checker.sh> for linux.
3. Sample reproducer
If this resolves your issue, make sure to accept this as a solution. This would help others with similar issues.
Thanks,
Jaideep
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I tried 2022.3.0 and I had the same problem. Note that we are running the application in a container.
CPU: Intel(R) Xeon(R) Gold 6238R CPU @ 2.20GHz
Output from self checker is attached.
Sample reproducer (I guess any program will suffice):
#include <unistd.h>
int main() {
while (1) {
sleep(1);
}
}
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
I assume you are running the Vtune profiler and your application inside a Docker container. Please follow the below commands before starting a Docker container.
Run docker image with the below command.
docker run --rm -dti --cap-add CAP_SYS_ADMIN <docker_imagename>
To get the container id.
docker ps
To access the container run the below command.
docker exec -it <container id> bash
We ran hardware event-based sampling from our side inside a Docker container with the latest version of Vtune.
If you want to try container-to-container profiling, check the below link.
If this resolves your issue, make sure to accept this as a solution. This would help others with similar issues.
Thanks,
Jaideep
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I validated that we had the CAP_SYS_ADMIN capability already with
kubectl <pod> -- capsh --print
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
We are working on this internally and will get back to you with an update.
Thanks,
Jaideep
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Please note that the
Failed to start threads: File exists
message may be a bug in perf.
In perf, look at the changes to tools/perf/builtin-record.c made Feb 10 2022 with:
perf record: Start threads in the beginning of trace streaming
SHA: 3217e9fecf118d5dcabdd68d91e0c6afcb4c3e1b
At line 2014 pthread_create() is run and on line 2017 strerror(errno) is printed. Between line 2014 and 2017 record__terminate_thread() is run.
Most likely record__terminate_thread() run in-between the creation and the error message looks like it may overwrite errno, thus messing up the error message.
if (pthread_create(&handle, &attrs, record__thread, &thread_data[t])) {
for (tt = 1; tt < t; tt++)
record__terminate_thread(&thread_data[t]);
pr_err("Failed to start threads: %s\n", strerror(errno));
Just so you know the error message may be misleading.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Any updates here? This problem is still apparent in 2022.4.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi ,
You are using perf. Can you please install SEP drivers and then try running hardware event based analysis on the same application?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
I hope you were able to resolve the issue with the recommendations provided. As I have not received the response, we shall stop monitoring this thread.

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page