Analyzers
Talk to fellow users of Intel Analyzer tools (Intel VTune™ Profiler, Intel Advisor)
4975 Discussions

VTune hotspot collection gives "Internal Error"

Alanna
Beginner
2,760 Views

Hello, I'm having trouble collecting hotspot information with VTune. I've tried both sampling-mode=sw and sampling-mode=hw. The error looks like this in both cases:

 

$ vtune -collect hotspots -knob sampling-mode=sw -target-pid 12345
vtune: Collection failed.
vtune: Internal Error

 

`vtune -collect performance-snapshot -target-pid X` and `vtune -collect uarch-exploration -target-pid X` both work without errors.

The `vtune-self-checker.sh` only reports failures for software sampling, everything else is "Ok":

 

$ vtune-self-checker.sh
Intel(R) VTune(TM) Profiler Self Check Utility
Copyright (C) 2009-2020 Intel Corporation. All rights reserved.
Build Number: 613804

HW event-based analysis (counting mode) (Perf)
Example of analysis types: Performance Snapshot
    Collection: Ok
    Finalization: Ok...
    Report: Ok

Instrumentation based analysis check
Example of analysis types: Hotspots with default knob sampling-mode=sw, Threading with default knob sampling-and-waits=sw
    Collection: Fail

HW event-based analysis check (Perf)
Example of analysis types: Hotspots with knob sampling-mode=hw, HPC Performance Characterization, etc.
    Collection: Ok
vtune: Warning: To profile kernel modules during the session, make sure they are available in the /lib/modules/kernel_version/ location.
    Finalization: Ok...
    Report: Ok

HW event-based analysis check (Perf)
Example of analysis types: Microarchitecture Exploration
    Collection: Ok
vtune: Warning: To profile kernel modules during the session, make sure they are available in the /lib/modules/kernel_version/ location.
    Finalization: Ok...
    Report: Ok

HW event-based analysis with uncore events (Perf)
Example of analysis types: Memory Access
    Collection: Ok
vtune: Warning: To profile kernel modules during the session, make sure they are available in the /lib/modules/kernel_version/ location.
    Finalization: Ok...
    Report: Ok

HW event-based analysis with stacks (Perf)
Example of analysis types: Hotspots with knob sampling-mode=hw and knob enable-stack-collection=true, etc.
    Collection: Ok
vtune: Warning: To profile kernel modules during the session, make sure they are available in the /lib/modules/kernel_version/ location.
    Finalization: Ok...
    Report: Ok

HW event-based analysis with context switches (Perf)
Example of analysis types: Threading with knob sampling-and-waits=hw
    Collection: Ok
vtune: Warning: To profile kernel modules during the session, make sure they are available in the /lib/modules/kernel_version/ location.
    Finalization: Ok...
    Report: Ok

The check observed a product failure on your system.
Review errors in the output above to fix a problem or contact Intel technical support.

Log location: /tmp/vtune-tmp-user/self-checker-2021.04.09_15.10.52/log.txt

 

I've put the full log here, in case it's helpful. https://gist.github.com/alannnna/f82c9fb94510b2424effb1df3486c8e5

I have added my user to the `vtune` group and set /proc/sys/kernel/perf_event_paranoid to -1.

I am trying to collect information on a Java application.

OS: CentOS 8
VTune version: Intel(R) oneAPI VTune(TM) Profiler 2021.1.1 Gold (build 613804) Command Line Tool

0 Kudos
1 Solution
Vladimir_R_Intel
Employee
2,580 Views

Hi Alanna,

I think this is a known issue with permissions of pinbin executable file locating in <vtune>/bin64 directory. You can check it manually, it should be 755. 

You have two options to resolve this: to change pinbin permissions to 755 manually or to update your VTune to 2021 Update 2 or later.

BR,

Vladimir

View solution in original post

0 Kudos
12 Replies
Kirill_U_Intel
Employee
2,736 Views

Hi, Alanna.

Could you provide logs for bad collections?

clean /tmp/amplxe-log-USER directory

run the hotspots collection

share logs from /tmp/amplxe-log-USER 

BTW, are the vtune and java process started from the same user? This is strange that hardware analysis had issue but ue is ok. They are based on the same collector.

Thanks, Kirill

0 Kudos
Alanna
Beginner
2,724 Views

I cleaned out the folder and ran hotspot collection (vtune -collect hotspots -knob sampling-mode=sw -target-pid 12345). The resulting /tmp/amplxe-log-USER is attached. Let me know if it looks OK or if I should try fetching it again.

Yes, the vtune and java processes appear to be started from the same user. I'm starting them both, and the PID of the java process that I get from top has my username on it.

0 Kudos
Kirill_U_Intel
Employee
2,714 Views

Hi.

Could you share one more log? The following command dumps perfun-xxxxx.logs to current directory. 

PERFRUN_DEBUG=TRACE /opt/intel/oneapi/vtune/2021.1.1/bin64/amplxe-runss --stack-stitching --data-limit-mb=1000 --disk-space-limit=0 --mrte-type=java,python --itt-config=frame --itt-config=task,event,counter --stackwalk=offline --mrte-mode=auto --type=cpu:counters:nostack --type=cpu:stack --interval=10 --event-config=CPU_CLK_UNHALTED.THREAD,CPU_CLK_UNHALTED.REF_TSC:sample:sa=2000000,INST_RETIRED.ANY:sample:sa=2000000,CPU_CLK_UNHALTED.REF_XCLK,CPU_CLK_UNHALTED.ONE_THREAD_ACTIVE,FP_ARITH_INST_RETIRED.SCALAR_SINGLE,FP_ARITH_INST_RETIRED.128B_PACKED_SINGLE,FP_ARITH_INST_RETIRED.256B_PACKED_SINGLE,FP_ARITH_INST_RETIRED.SCALAR_DOUBLE,FP_ARITH_INST_RETIRED.128B_PACKED_DOUBLE,FP_ARITH_INST_RETIRED.256B_PACKED_DOUBLE,UOPS_EXECUTED.X87,UOPS_RETIRED.RETIRE_SLOTS,UOPS_EXECUTED.THREAD,FP_ARITH_INST_RETIRED.512B_PACKED_DOUBLE,FP_ARITH_INST_RETIRED.512B_PACKED_SINGLE,UOPS_RETIRED.RETIRE_SLOTS:sample:sa=2000003 --collector=perf --count --target-pid 2965556

 Thanks, Kirill

0 Kudos
Alanna
Beginner
2,636 Views

I am sorry for the delay. Here are the log files. It seems that running your command produces two log files instead of one (or perhaps I did something wrong), so I've attached them both. Let me know if you need more info. Thanks.

0 Kudos
Gopika_Intel
Moderator
2,675 Views

Hi,

We have not heard back from you in a while. Please let us know if the issue persists.

Regards

Gopika


0 Kudos
AthiraM_Intel
Moderator
2,642 Views

Hi,


We have not heard back from you, so we will close this inquiry now. If you need further assistance, please post a new question.


Thanks.


0 Kudos
Alanna
Beginner
2,635 Views

My apologies for the delay. May we please leave the question open?

0 Kudos
AthiraM_Intel
Moderator
2,611 Views

Hi,


We have reopened the question. Thanks for sharing the log files. We are checking on the log files, will get back to you soon with an update.


Thanks


0 Kudos
Alanna
Beginner
2,602 Views
0 Kudos
Vladimir_R_Intel
Employee
2,581 Views

Hi Alanna,

I think this is a known issue with permissions of pinbin executable file locating in <vtune>/bin64 directory. You can check it manually, it should be 755. 

You have two options to resolve this: to change pinbin permissions to 755 manually or to update your VTune to 2021 Update 2 or later.

BR,

Vladimir

0 Kudos
Alanna
Beginner
2,543 Views

Thanks so much, Vladimir!

Indeed my pinbin permissions were not set to 755; changing them fixes it! Thanks again!

0 Kudos
AthiraM_Intel
Moderator
2,527 Views

Hi,


Thanks for the confirmation. If you need any additional information, please submit a new question as this thread will no longer be monitored.


Thanks.


0 Kudos
Reply