topic Re: Unexpected Error with VTune Command Line Interface in Analyzers

Unexpected Error with VTune Command Line Interface

chintan221b — Wed, 20 Nov 2024 13:09:42 GMT

Context: I'm currently working on creating a Datadog integration that utilizes the VTune command line interface to generate CSV summary reports. The integration is running on a Windows environment, using the Datadog Agent as a service account to interact with VTune. The Datadog Agent is running under a service account, which is used to execute the VTune command line interface.

Issue: When running the VTune command line interface to generate a CSV summary report, the process stops unexpectedly and returns an error. The error message and logs are attached below.

Error Message:

vtune: Using result path `C:\Users\admin\Documents\VTune\Projects\enhanced_intensive\r000hs' vtune: Executing actions 0 % vtune: Executing actions 0 % Finalizing results vtune: Executing actions 50 % Finalizing results vtune: Executing actions 50 % Generating a report vtune: Executing actions 50 % Setting data model parameters vtune: Executing actions 75 % Setting data model parameters vtune: Executing actions 75 % Generating a report vtune: Executing actions 100 % Generating a report vtune: Executing actions 100 % done Exception: 0xe06d7363, EXCEPTION_UNKNOWN Module: KERNELBASE.dll Collecting modules information... Collecting system information... Copyright (C) 2009 Intel Corporation. All rights reserved. Intel(R) VTune(TM) Profiler 2024.3.0; 628755 experienced an unexpected error. Please send a problem report: amplxe-feedback --send-crash-report .....

Command: The command being executed is:

vtune -report summary -result-dir <result_dir> -report-output <output_file> -format csv -csv-delimiter comma

Environment:

Windows: Windows 10, 64-bit
VTune: VTune 2024.3.0

Expected Behaviour:
The VTune command line interface should generate a CSV summary report without errors.

Actual Behaviour:
The process stops unexpectedly and returns an error.

I have:

Verified that the Datadog Agent service account has the necessary permissions to access the required files and directories.
Reviewed the logs for any error messages related to the VTune command line interface.

Request:
I would greatly appreciate any help in resolving this issue. If anyone has experienced similar issues or has any suggestions for troubleshooting, please let me know.

Additional Information:
I am including the crash report information below, which may be helpful in diagnosing the issue.

Re: Unexpected Error with VTune Command Line Interface

yuzhang3_intel — Thu, 21 Nov 2024 01:38:11 GMT

Are the VTune profiling and report command lines all called by the Datadog service? I mean the two command lines below.

vtune -collect hotspots xxx

vtune -report summary xxx

Did you try executing 'vtune -report summary -result-dir <result_dir> -report-output <output_file> -format csv -csv-delimiter comma' separately instead of executed by the Datadog service? Can it work or not?

Re: Unexpected Error with VTune Command Line Interface

chintan221b — Thu, 21 Nov 2024 05:43:54 GMT

Hi,

Thank You for the response.

Yes, the VTune report extraction command lines are called by the Datadog service. However, the report generation (i.e. collection) is done through an admin user, and the report extraction (i.e. creating a CSV file) is done through the Datadog service account.

To answer your second question, I did try executing the vtune -report summary -result-dir <result_dir> -report-output <output_file> -format csv -csv-delimiter comma command separately, but it did not work when run as the Datadog service user. The command only works when run as an admin user.

It seems that the issue is related to the permissions or access control of the Datadog service account, which is preventing it from generating the report. I'm still investigating the root cause of the issue and would appreciate any suggestions or guidance on how to resolve it.

Re: Unexpected Error with VTune Command Line Interface

chintan221b — Thu, 21 Nov 2024 05:51:11 GMT

Also, to add to the original query, Giving Admin rights to Datadog service account is not a viable solution as it can be security hazard and is not recommended.

Re: Unexpected Error with VTune Command Line Interface

yuzhang3_intel — Thu, 21 Nov 2024 06:18:54 GMT

Can you check if the service can read/write the working directory and vtune data?

The permissions for data collection and report generation should be the same. If the service has permission to collect data, it can generate the report. Maybe you can try to collect in the service. If it is the reason, we can consider granting minimum permission for vtune support in the service.

Re: Unexpected Error with VTune Command Line Interface

chintan221b — Thu, 21 Nov 2024 06:41:50 GMT

Yes, It has read, write and execute permissions on folder where Vtune reports are stored, the issue persist. I can not give collect permissions to datadog service account, It requires admin permissions to collect through vtune.

Re: Unexpected Error with VTune Command Line Interface

yuzhang3_intel — Thu, 21 Nov 2024 07:05:06 GMT

Add the service account to Group 'vtune' and try again.

Re: Unexpected Error with VTune Command Line Interface

chintan221b — Thu, 21 Nov 2024 12:10:53 GMT

There is no group named “vtune” in windows,
I even tried making service user owner of that folder still the issue persist.

Re: Unexpected Error with VTune Command Line Interface

yuzhang3_intel — Thu, 21 Nov 2024 12:32:19 GMT

The service has permission to access all the files in the vtune data folder, right?

Re: Unexpected Error with VTune Command Line Interface

chintan221b — Thu, 21 Nov 2024 12:45:25 GMT

Yes, the service user has full access to the folder where reports are stored.
I also tried running the command 'vtune -report summary -result-dir <result_dir> -format csv -csv-delimiter comma' i.e. without specifying the report output location, and it still failed with the same error:
Code
CopyInsert
Exception: 0xe06d7363, EXCEPTION_UNKNOWN
Module: KERNELBASE.dll
Collecting modules information...
Collecting system information...
However, what's interesting is that the entire CSV report was printed to the command prompt before the error occurred. This suggests that the report generation itself is working, but there's some issue with the way the report is being output or handled by the VTune tool.
I'm still unsure what the root cause of the issue is

Re: Unexpected Error with VTune Command Line Interface

yuzhang3_intel — Thu, 21 Nov 2024 12:57:17 GMT

Could you share the vtune data?

Re: Unexpected Error with VTune Command Line Interface

chintan221b — Thu, 21 Nov 2024 15:17:31 GMT

I have attached the files here. Let me know if any more files are needed.

Re: Unexpected Error with VTune Command Line Interface

yuzhang3_intel — Fri, 22 Nov 2024 01:35:57 GMT

I can generate the csv based report from r000hs locally. We need further investigation.

Re: Unexpected Error with VTune Command Line Interface

yuzhang3_intel — Thu, 28 Nov 2024 01:03:07 GMT

I don’t see any issues, and I am able to finalize the VTune result you shared and get report without crash.

Can it be the environment when collection and reporting are performed by different versions of VTune? E.g. collected with newer one, reported with older ?

Re: Unexpected Error with VTune Command Line Interface

chintan221b — Thu, 28 Nov 2024 12:32:39 GMT

Thanks for trying. Not sure what is causing issue. Will try my luck with different machines and environments.