topic Seems to be some issue with in Analyzers

How do I open a case?

soderstrom__carl — Thu, 21 May 2020 19:22:42 GMT

I have vTune 2020 update 1 and I'm having problems trying to do a hotspot trace. I can't open a case. When I login and click the dashboard access the "My Support" it wants you to select the product and I searched through all the links and can't find vTune. Also searched for vtune on the support page and there's nothing.

Hi,

JananiC_Intel — Fri, 22 May 2020 07:21:00 GMT

Hi,

Thanks for reaching out.

You are at the right place,this forum deals with issues related to Vtune. So you can reach out to this forum for any vtune clarifications.

You have mentioned that you had a problem while collecting hotspots. Could you give us more information about the issue like error you faced and environment details to try from our side?

Seems to be some issue with

soderstrom__carl — Fri, 22 May 2020 13:43:00 GMT

Seems to be some issue with the vtsspp kernal module. Attached a file with OS, System info, the vtune commands I use and the problem I'm facing.

Thanks a lot for such a

Alexey_B_Intel2 — Tue, 26 May 2020 13:08:19 GMT

Thanks a lot for such a detailed report.

First of all vtss driver is unable to catch all hw events, to reduce it please increase "-knob sampling-interval"

As I understand Windriver is RT OS, so could you make sure

#define VTSS_CONFIG_REALTIME 1
#define VTSS_CONFIG_INTERNAL_MEMORY_POOL 1

are defined in "<vtune_dir>/sepdk/src/vtsspp/config.h" or try to define them unconditionally.

If it doesn't help, could you try to replace:

msleep_interruptible(1);
to
msleep_interruptible(10);

in "<vtune_dir>/sepdk/src/vtsspp/transport.c"
Also msleep_interruptible(1); at the end of the loop:

list_for_each_safe(it, tmp, &vtss_transport_list) {
  ...
  msleep_interruptible(1);
}

may help (these sleeps were deleted some time ago as redundant).

Also there are at least 2 workarounds:

1. Switch to driverless collection, to enable it use "-knob stack-size=1024", more details: https://software.intel.com/content/www/us/en/develop/documentation/vtune-cookbook/top/configuration-recipes/profiling-hardware-without-sampling-drivers.html

2. You can use previous vtss driver, it must be binary compatible with new VTune, just put it in <vtune_dir>/sepdk/src/vtsspp rebuild and reload it.

Hi thanks for the help.

soderstrom__carl — Tue, 26 May 2020 15:10:27 GMT

Hi thanks for the help.

WindRiver used to provide a realtime only kernel for embedded systems but that's not what we use. We are using WRL (WindRiver Linux) which is just a packaged linux kernel with support from Wind River. We use WRL like any server would use linux, our programs run as regular user space programs.

I confirmed that these flags are OFF.

/* Realtime patch support */
#if defined(CONFIG_PREEMPT_RT) || defined(CONFIG_PREEMPT_RT_FULL)
#define VTSS_CONFIG_REALTIME 1
#define VTSS_CONFIG_INTERNAL_MEMORY_POOL 1

There's one existing msleep_interruptible() call in vtss_transport_fini(), I changed the 1 to a 10. Was that what you wanted me to do? Maybe you could do a diff of your if not.

I also changed -knob sampling-interval=30

Still getting the same failures.

Thanks,

Alexey_B_Intel2 — Wed, 27 May 2020 17:26:40 GMT

Thanks,

Yes, this single msleep_interruptible() waits tracefile flushing (which according to dmesg Iog somehow hurts linux scheduler). I've attached diff file for it (transport.diff)

What version of vtss or Vtune (w/o craches) did you use perviously (vtss version can be found in <vtune_dir>/sepdk/src/vtsspp/version.h). I will try to analize the difference beetween them.

Hi Alexy -

soderstrom__carl — Wed, 27 May 2020 19:00:17 GMT

Hi Alexy -

Here are the 2 versions:

VTune 2020 (was ok)
#ifndef _VTSS_VERSION_H_
#define _VTSS_VERSION_H_
#define VTSS_VERSION_MAJOR 1
#define VTSS_VERSION_MINOR 8
#define VTSS_VERSION_REVISION 198
#define VTSS_VERSION_STRING "1.8.198-605129"
#endif

2020-Update 1 (problems)
cisco@eseto-cel2 vtsspp]$ cat version.h
#ifndef _VTSS_VERSION_H_
#define _VTSS_VERSION_H_
#define VTSS_VERSION_MAJOR 1
#define VTSS_VERSION_MINOR 8
#define VTSS_VERSION_REVISION 208
#define VTSS_VERSION_STRING "1.8.208-607630"
#endif

Keep in mind that we changes the WindRiverLinux version as well. The last time I profiled and vtune was working the WRL version was: 82.9.1.172. I'm now using WRL version: 82.9.1.258

Thanks

Hi,

Alexey_B_Intel2 — Fri, 29 May 2020 09:45:27 GMT

Hi,

I found only one change that might cause such crash (other changes are related to new CPUs or just code cleanup).
Could you comment out in "<vtune2020u1>/sepdk/src/vtsspp/target.c" the following line:

vtss_task_data_free_buffers(tskd);

In vtss_target_dtr() function (diff is attached)

If this doesn’t help could you please try the old driver (v1.8.198 from VTune 2020) on new WRL 82.9.1.258 to make sure this is new issue
To do it:

cd <vtune2020>/sepdk/src/vtsspp/
./build-driver -ni && ./insmod-vtsspp -r -g $(id -ng)

Unloading of current driver is not necessary.

Thanks,
Alexey