Failed to execute sep on vtune amplifier 2019

[chelini__lorenzo]

Hi All,

I am trying out the new vtune amplifier 2019 on Ubuntu 16.04. Ubuntu is a guest system virtualized with Parallel Desktop (with PMU enabled). When running the following command:

amplxe-cl -collect memory-access -knob analyze-mem-objects=true -knob dram-bandwidth-limits=true myApp

I got the following error: Error: Failed to execute sep process. Data collection is interrupted.

I have built the drivers correctly and I can run sep. Using sep -version I got:

Sampling Enabling Product version: 5.1 Beta built on Aug  8 2018 02:28:49
SEP User Mode Version: 5.1.0 Beta
SEP Driver Version: 5.1.0 Beta
PAX Driver Version: 1.0.2
Platform type: 117
CPU name: Intel(R) Processor code named Kabylake
PMU: skylake
Driver configs: Non-Maskable Interrupt
Copyright(C) 2007-2018 Intel Corporation. All rights reserved.

Could you please suggest me how to fix this issue?

Best regards,

Lorenzo

[Kirill_U_Intel]

Hi, Lorenzo.

Looks like amplxe-cl could not validate sep driver on some reason.

Are amplxe-cl and sep from the same vtune build? could you try to use full path 'VTUNE_DIR/bin64/amplxe-cl ....'?

Could you share bad result directory? (something like r000macc-bad)

Also could you run 'VTUNE_DIR/bin64/amplxe-runss --context-value-list' and check 'isSEPDriverAvailable' value?

Thanks, Kirill

[chelini__lorenzo]

Hi Kirill,

Thanks for the reply.

Amplxe-cl and sep are from the same build. I tried to use the full path but got the same problem.

Attached the output of amplxe-runss --context-value-list and the bad result directory. 

Looking forward to hearing from you.

Best,

Lorenzo 

[Kirill_U_Intel]

Hm,

Looks like we don't support precise events under guest system

I investigated perfrun-2018.10.11-08h22m08s.2815.log fro bad result dir and caught

Precise Event Based Sampling (PEBS) is not supported. Please choose non-PEBS events

142 [140474316388096] INFO perfrun.perfcollector <> - SepDryRun RC = false

142 [140474316388096] ERROR perfrun.launcher <> - Messenger::say: Failed to execute sep process. Data collection is interrupted.

Could you try a couple experiments?

run

'VTUNE_DIR/bin64/amplxe-runss --option-file=runsa_no_precise.options

and

'VTUNE_DIR/bin64/amplxe-runss --option-file=runsa_no_precise_sep.options

new options files are in attached zip

Are collections fail or success?

[chelini__lorenzo]

Hi Kirill,

Running: amplxe-runss --option-file=runsa_no_precise.options in the proper directory VTUNE_DIR/bin64/ give me the following error:

amplxe: Error: Failed to execute sep process. Data collection is interrupted.
amplxe: Collection failed.

Running: amplxe-runss --option-file=runsa_no_precise_sep.options in the same directory I get 

amplxe: Error: Cannot configure sampling event groups. The collection is terminated.
Use -knob event-config=? to see the list of available events for the target.
amplxe: Collection failed.

Looking forward to hearing from you.

Best,

Lorenzo 

[Kirill_U_Intel]

Ok,

Let's try to run sep directly to check event config configuration for memory access analysis

what is output and exit code for the following command

/opt/intel/system_studio_2019/vtune_amplifier_2019.0.2.570779/bin64/sep -dry-run -sam 1 -ec "CPU_CLK_UNHALTED.THREAD:sa=2800000,CPU_CLK_UNHALTED.REF_TSC:sa=2800000,INST_RETIRED.ANY:sa=2800000,CPU_CLK_UNHALTED.REF_XCLK:sa=100003,CPU_CLK_UNHALTED.ONE_THREAD_ACTIVE:sa=100003,MEM_TRANS_RETIRED.LOAD_LATENCY_GT_4:sa=10003,CYCLE_ACTIVITY.STALLS_L1D_MISS:sa=2000003,CYCLE_ACTIVITY.STALLS_L2_MISS:sa=2000003,CYCLE_ACTIVITY.STALLS_L3_MISS:sa=2000003,CYCLE_ACTIVITY.STALLS_MEM_ANY:sa=2000003,EXE_ACTIVITY.1_PORTS_UTIL:sa=2000003,EXE_ACTIVITY.2_PORTS_UTIL:sa=2000003,EXE_ACTIVITY.BOUND_ON_STORES:sa=2000003,EXE_ACTIVITY.EXE_BOUND_0_PORTS:sa=2000003,IDQ_UOPS_NOT_DELIVERED.CORE:sa=2000003,INT_MISC.RECOVERY_CYCLES:sa=2000003,UOPS_ISSUED.ANY:sa=2000003,UOPS_RETIRED.RETIRE_SLOTS:sa=2000003,MEM_LOAD_RETIRED.L3_MISS:sa=50003,L1D_PEND_MISS.FB_FULL:cmask=1:sa=2000003,L1D_PEND_MISS.PENDING:sa=2000003,MEM_LOAD_L3_HIT_RETIRED.XSNP_HITM:sa=100003,MEM_LOAD_L3_HIT_RETIRED.XSNP_HIT:sa=100003,MEM_LOAD_L3_HIT_RETIRED.XSNP_MISS:sa=100003,MEM_LOAD_RETIRED.FB_HIT:sa=2000003,MEM_LOAD_RETIRED.L1_MISS:sa=2000003,MEM_LOAD_RETIRED.L2_HIT:sa=2000003,MEM_LOAD_RETIRED.L3_HIT:sa=200003,OFFCORE_REQUESTS_OUTSTANDING.ALL_DATA_RD:cmask=4:sa=2000003,OFFCORE_REQUESTS_OUTSTANDING.CYCLES_WITH_DATA_RD:sa=2000003,MEM_INST_RETIRED.ALL_LOADS:sa=100003,MEM_INST_RETIRED.ALL_STORES:sa=100003,CPU_CLK_UNHALTED.ONE_THREAD_ACTIVE:sa=10003,CPU_CLK_UNHALTED.REF_XCLK:sa=10003,UNC_IMC_DRAM_DATA_READS,UNC_IMC_DRAM_DATA_WRITES" -experimental

[chelini__lorenzo]

Hi Kirill,

Here the results. It says again Precise event-based sampling is not supported. 

Do you know what could be the cause of the problem?

Looking forward to hearing from you.

regards,

Lorenzo 

[Kirill_U_Intel]

Let me check that with somebody else but looks like we could not provide memory-access without precise events.

Kirill

[Andrey_I_Intel]

Hi Lorenzo,

Unfortunately Memory Access analysis requires precise events feature and that is generally not available in virtualized systems. VTune do support Hotspots and Microarchitecture analyses if PMU virtualization is enabled, but for other types it would most probably fail. Please refer the documentation for more details: https://software.intel.com/en-us/vtune-amplifier-help-on-virtual-machine

BR,

Andrey

[chelini__lorenzo]

Hi Andrey,

Thanks for the answer. I already did the procedures highlighted in the documentation you provided me. The PMU are enabled in the virtual machine I am using so not sure why it is failing. I am using Parallel Desktop Pro. Anyway, I will try without virtualization and see if I manage to make it running. Thanks for the support.

Best,

Lorenzo