Need help getting information from Vtune using time ranges

[Justin_H_]

Hello,

So I am using Vtune to profile my system while I use a generated workload. What I am trying to do is get the data for 15ms intervals for the entire run. Here are some of the things I am running:

System profiling

General exploration -> Hardware event sample count viewpoint

Bottom up view

Core, HW context, Function, Callstack -> (I am filtering in by 1 cpu. So my data is only for 1 cpu)

Then I am selecting 15ms time frams and filtering so I get all the relevant statistics for this small time frame. My problem is that doing this would take a lifetime. I need to be able to just export all the 15ms intervals to a CSV or excel or something. I found:

C:\Users\User\Documents\Amplifier XE\Projects\Test2\r001ge

There is an sqlite-db folder and I used a db browser to look into it. It LOOKS like this would do what I need it to do. The problem is that I dont know what the command would be to look at the database in a meaningful or really the same way I was looking at the table in the GUI. Can someone help me with that or maybe tell me if there is a better way of doing this?

[Vitaly_S_Intel]

Hi Justin,

Did I get you right that you want to divide all the collection time by 15ms intervals and get Core/HW Context/Function/Callstack bottom-up data for every interval?

[Justin_H_]

Yes. I can use the GUI to get a single 15ms interval. I know how to do that. Manually going through after data collection and manually selecting individual 15ms ranges, then waiting for the table to update is too long. I am running for around 10 minutes an experiment so there will be thousands of these results for core_0, cpu_0.

Does this clear up what I am asking?

Thanks

[Vitaly_S_Intel]

Thanks Justin, I captured this request.

For now you should be able to automate these actions using CLI, look on time-filter option.

Just to understand your case better - what are you trying to achieve? Why do you need aggregated data for exactly 15ms intervals?

[Justin_H_]

There isnt any thing really magical about 15ms. I am choosing 15ms because thats what windows performance recorder outputs its samples in. 

Essentially this is for academic research. I am generating a workload and recording events using WPR and  Vtune to get some information to use in the research. I need the sample times to be the same so WPR since vtune has much more flexibility. The issue here is that I need ALL the samples for the experiment, which may be several minutes long. So trying to capture  individual results, by hand, takes too long. I spent several hours and collected 3 seconds worth of data.

Thanks,

[Bernard]

Why do you need WPR if VTune can give you more accurate results? I suppose that you are performing system wide sampling where WPR can be really helpful.

[Carlos_P_1]

1

[Justin_H_]

Great question. The answer is in the data. I need a few things

1. Archetectural events such as cache miss, etc

2. Individual CPU usage

3.  The enter and exit time for CPU idle states

It looks like Vtune logs 1 and 2 but not 3. WPR records 2 and 3 but not 1

[David_A_Intel1]

One solution to consider would be going in the other direction, that is, VTune Amplifier includes support for importing CSV data.  You could try adding the WPR data to the VTune Amplifier timeline.

[Justin_H_]

I think the issue there might be that unless I was able to start them at exactly the same time (to the smallest possible time denomination), which I cannot, then the results of one will be behind or ahead of the other. Time is really the only constant since there is no guarantee that the CPU usage will be measured as the same value.

Unless you know of a way of doing that better.

[David_A_Intel1]

Yes, and the CSV import requires time-stamps so that the data can be correlated.  Do you get time stamps with WPR?

[Justin_H_]

You do get timestamps but the problem is that youre assuming that both WPR and vtune start at the EXACT moment in time. I dont think thats possible since even when you hit start for both there is a slight delay to when they actually start recording.

[David_A_Intel1]

No, I'm not assuming that.  I'm assuming that they are both collecting data on the same system and using the same system clock. ;)

When I say "time-stamp", I don't mean seconds since collection started, I mean the current time is recorded.  Thus, VTune Amplifier can match it with the time it recorded it's sample.  You are correct, they won't start at exactly the same time, but the samples that were collected at the same time can be displayed on the timeline together.  Check out the documentation I linked to earlier for details.

[Justin_H_]

oh ok, Thanks!

[Bernard]

I am not sure if there is a way to fully synchronize even startup of WPR and VTune and we are talking on full synchronization  of  events recording. One hypothetical solution (in theory) can be insertion of the code which will invoke specific functionality like logging of architectural events in case of WPR.