Unexpected Error with VTune Command Line Interface

[chintan221b]

Context: I'm currently working on creating a Datadog integration that utilizes the VTune command line interface to generate CSV summary reports. The integration is running on a Windows environment, using the Datadog Agent as a service account to interact with VTune. The Datadog Agent is running under a service account, which is used to execute the VTune command line interface.

Issue: When running the VTune command line interface to generate a CSV summary report, the process stops unexpectedly and returns an error. The error message and logs are attached below.

Error Message:

vtune: Using result path `C:\Users\admin\Documents\VTune\Projects\enhanced_intensive\r000hs'

vtune: Executing actions  0 %                                                  
vtune: Executing actions  0 % Finalizing results                               
vtune: Executing actions 50 % Finalizing results                               
vtune: Executing actions 50 % Generating a report                              
vtune: Executing actions 50 % Setting data model parameters                    
vtune: Executing actions 75 % Setting data model parameters                    
vtune: Executing actions 75 % Generating a report                              
vtune: Executing actions 100 % Generating a report                             
vtune: Executing actions 100 % done                                            

Exception: 0xe06d7363, EXCEPTION_UNKNOWN
Module: KERNELBASE.dll
Collecting modules information...
Collecting system information...

Copyright (C) 2009 Intel Corporation. All rights reserved.

Intel(R) VTune(TM) Profiler 2024.3.0; 628755 experienced an unexpected error.

Please send a problem report: 
amplxe-feedback --send-crash-report .....

Command: The command being executed is:

vtune -report summary -result-dir <result_dir> -report-output <output_file> -format csv -csv-delimiter comma

Environment:

• Windows: Windows 10, 64-bit
• VTune: VTune 2024.3.0

Expected Behaviour: 
The VTune command line interface should generate a CSV summary report without errors.

Actual Behaviour: 
The process stops unexpectedly and returns an error.

I have:

• Verified that the Datadog Agent service account has the necessary permissions to access the required files and directories.
• Reviewed the logs for any error messages related to the VTune command line interface.

Request: 
I would greatly appreciate any help in resolving this issue. If anyone has experienced similar issues or has any suggestions for troubleshooting, please let me know.

Additional Information: 
I am including the crash report information below, which may be helpful in diagnosing the issue.

[yuzhang3_intel]

Are the VTune profiling and report command lines all called by the Datadog service? I mean the two command lines below.

vtune -collect hotspots xxx

vtune -report summary xxx

Did you try executing 'vtune -report summary -result-dir <result_dir> -report-output <output_file> -format csv -csv-delimiter comma' separately instead of executed by the Datadog service? Can it work or not?

[chintan221b]

Hi,

Thank You for the response.

Yes, the VTune report extraction command lines are called by the Datadog service. However, the report generation (i.e. collection) is done through an admin user, and the report extraction (i.e. creating a CSV file) is done through the Datadog service account.

To answer your second question, I did try executing the vtune -report summary -result-dir <result_dir> -report-output <output_file> -format csv -csv-delimiter comma command separately, but it did not work when run as the Datadog service user. The command only works when run as an admin user.

It seems that the issue is related to the permissions or access control of the Datadog service account, which is preventing it from generating the report. I'm still investigating the root cause of the issue and would appreciate any suggestions or guidance on how to resolve it.

[chintan221b]

Also, to add to the original query, Giving Admin rights to Datadog service account is not a viable solution as it can be security hazard and is not recommended.   

[yuzhang3_intel]

Can you check if the service can read/write the working directory and vtune data?

The permissions for data collection and report generation should be the same. If the service has permission to collect data, it can generate the report.  Maybe you can try to collect in the service. If it is the reason, we can consider granting minimum permission for vtune support in the service.

[chintan221b]

Yes, It has read, write and execute permissions on folder where Vtune reports are stored, the issue persist. I can not give collect permissions to datadog service account, It requires admin permissions to collect through vtune.  

[yuzhang3_intel]

Add the service account to Group 'vtune' and try again.

[chintan221b]

There is no group named “vtune” in windows,
I even tried making service user owner of that folder still the issue persist.

[yuzhang3_intel]

The service has permission to access all the files in the vtune data folder, right?

[chintan221b]

Yes, the service user has full access to the folder where reports are stored.
I also tried running the command 'vtune -report summary -result-dir <result_dir> -format csv -csv-delimiter comma' i.e. without specifying the report output location, and it still failed with the same error:
Code
CopyInsert
Exception: 0xe06d7363, EXCEPTION_UNKNOWN
Module: KERNELBASE.dll
Collecting modules information...
Collecting system information...
However, what's interesting is that the entire CSV report was printed to the command prompt before the error occurred. This suggests that the report generation itself is working, but there's some issue with the way the report is being output or handled by the VTune tool.
I'm still unsure what the root cause of the issue is

[yuzhang3_intel]

Could you share the vtune data?

[chintan221b]

I have attached the files here. Let me know if any more files are needed.

[yuzhang3_intel]

I can generate the csv based report from r000hs locally. We need further investigation.

[yuzhang3_intel · Accepted solution]

I don’t see any issues, and I am able to finalize the VTune result you shared and get report without crash.

Can it be the environment when collection and reporting are performed by different versions of VTune? E.g. collected with newer one, reported with older ?

[chintan221b]

Thanks for trying. Not sure what is causing issue. Will try my luck with different machines and environments.