The Mpi sample program heart_demo keeps reporting errors when using aps to generate the data directo

[liu_wei67]

Hi:

I encountered some problems when using the mpi sample program heart_demo given on the official website, Run the heart_demo application with the Application Performance Snapshot analysis enabled. 2 machines，c1 and c2 The previous operations are all running normally, 8 processes ran on c1, and 8 processes ran on c2

>>> mpirun -n 16 -ppn 2 -f hosts.txt aps ./heart_demo -m ../mesh_mid -s ../setup_mid.txt -t 50

This aps option parameter is added and always report an error. If it is not added, the data result directory cannot be generated. After running for many times, just add aps and there will be a problem. Please help me to see how to solve this problem.

[root@c1 build]# mpirun -aps -n 16 -ppn 2 -f hosts.txt ./heart_demo -m ../Cardiac_demo/mesh_mid -s ../Cardiac_demo/setup_mid.txt -t 50

vtune: Warning: EMON Collector Message: Warning: PMU counter(s) [GP0 GP1 GP2 GP3 GP4 GP5 GP6 GP7 FP3 ] is/are already in use by other software and the data collected on these counter(s) may be incorrect

vtune: Error: EMON Collector Message: ERROR - A collection is already in progress - Abort current run. aps Error: Cannot run the collection. aps Error: Cannot process configs directory.

aps Error: Cannot process configs directory.

aps Error: Cannot process configs directory.

aps Error: Cannot process configs directory.

aps Error: Cannot process configs directory.

aps Error: Cannot process configs directory.

aps Error: Cannot process configs directory.

vtune: Warning: EMON Collector Message: Warning: PMU counter(s) [GP0 GP1 GP2 GP3 GP4 GP5 GP6 GP7 FP0 ] is/are already in use by other software and the data collected on these counter(s) may be incorrect

vtune: Error: EMON Collector Message: ERROR - A collection is already in progress - Abort current run. aps Error: Cannot run the collection.

aps Error: Cannot process configs directory.

aps Error: Cannot process configs directory.

aps Error: Cannot process configs directory.

aps Error: Cannot process configs directory.

aps Error: Cannot process configs directory.

aps Error: Cannot process configs directory.

aps Error: Cannot process configs directory.

Thanks

[Dmitry_P_Intel1]

Hello,

Can this be the case that you have already some tool launched on your machines that uses PMU for performance monitoring/analysis? 

Could you please try to add "--collection-mode=mpi" to see that tracing part of the collection works just fine?

Thanks & Regards, Dmitry

[liu_wei67]

Hi:

@Dmitry_P_Intel1,

Refer to the solution you gave and the report is ready to be printed!

Thanks

[liu_wei67]

Hi:

@Dmitry_P_Intel1 

I followed the example on the official website and continued to execute. When executing the hpc-performance report generated by vtune, a similar problem occurred. I think it is necessary to add some unknown option parameters to execute.. .., like the above question, adding -collection-mode=mpi is normal.

>>> mpirun -n 16 -ppn 2 -f hosts.txt -gtool "vtune -collect hpc-performance -data-limit=0 -r result_second:7" ./heart_demo -m ../Cardiac_demo/mesh_mid -s ../Cardiac_demo/setup_mid.txt -i -t 50

vtune: Analyzing data in the node-wide mode. The hostname (c2) will be added to the result path/name.
vtune: Peak bandwidth measurement started.
vtune: Peak bandwidth measurement finished.
vtune: Warning: To enable hardware event-based sampling, VTune Profiler has disabled the NMI watchdog timer. The watchdog timer will be re-enabled after collection completes.
Warning: PMU counter(s) [GP0 GP1 GP2 GP3 GP4 GP5 GP6 GP7 FP0 ] is/are already in use by other software and the data collected on these counter(s) may be incorrect
PMU resource(s) currently being used by another profiling tool or process: another PMU collection session may be running
vtune: Error: Warning: PMU counter(s) [GP0 GP1 GP2 GP3 GP4 GP5 GP6 GP7 FP0 ] is/are already in use by other software and the data collected on these counter(s) may be incorrect
PMU resource(s) currently being used by another profiling tool or process: another PMU collection session may be running

vtune: Collection failed.
vtune: Internal Error

Thanks

[Dmitry_P_Intel1]

Hello,

The option I recommended for APS switched off HW-counter based collection so you have only MPI tracing based data in your report. For VTune HPC Performance there is no such mode - it requires HW-counter based collection to be available. In your case most likely you have some tool launched on the system that occupies  HW counters and VTune complains that they are busy. Could you please double check if such tool is loaded on your system? If so - please disable it for the time of VTune collection.

Thanks & Regards, Dmitry

[RaeesaM_Intel]

Hi,

Is your issue resolved ? Were you able to run the command successfully ? Let us know the updates.

Regards,

Raeesa

[RaeesaM_Intel]

Hi,

We assume that your issue is resolved. If you need any additional information, please submit a new question as this thread will no longer be monitored.

Thanks and Regards,

Raeesa