- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
Since last week I have been experiencing a mixture of Signal 7 (SIGBUS) error and failure to load a data file.
In our environment VTune (ver. 2020.0.0.605129) connects over SSH to remote target which is a Skylake server. The Skylake server runs a SEP collector which samples our uplink layer 5G simulation module. The process of collection always completes without any error and successfully creates a samples data file *.tb7.
VTune GUI which is installed on my development machines during parsing phase of .tb7 file displays a following error:
vtune: Error: Cannot load data file `/home/workspace/VRAN/vl1/build_rcp_avx/artifacts/bts_artifacts/PRACH_profiling/profiling/data/data.0/sep7ffff1183700.20200714T151424.068574.tb7' (tbrw call "TBRW_INMEM_initialize_data(tbrwFile->getHandle(), streamIndex)" failed: invalid string (97)).
At random times VTune receives a SIGBUS error which might be sent due to accessing or touching probably undefined portion of memory mapped file(s).
I'm attaching the Ubuntu .crash file and vtune log file.
Thank you for your help.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Yes, at least for now.
The issue happens in a combination of running Vtune collector in non-root privilege and some JIT activities inside container in my understanding. So, it may not happen all the times as you experience.
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hey iliyapolak,
Thanks for reaching out.
Could you provide the following information to help us debug on this.
1)Are you using the same version of vtune on your development machines and the remote server.
2)Are you trying to open the full vtune result using vtune or are you just importing the Tb7 file. If you are importing the full report could you try with importing the tb7 alone.To import Tb7 alone you can select import option from vtune gui and select your tb7. Let us know if this works.
Thanks
Arun Jose
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Arun Jose,
Thank you very much for quick reply!
1) The remote server uses only SEP driver collector, it does not have any VTune installation.
2) From the vtune-gui I open data.vtune file (size of 138 bytes) and in this moment the error materializes. I can not import the .tb7 file because the "import" option is inactive and is greyed out, hence I'm forced to open directly the *vtune.data file.
*We are not creating project, just only communicate with SEP driver and receive samples from remote server.
P.s.
FYI -- this is the version of VTune installed on remoter server
/opt/intel/vtune_profiler_2020.0.0.605129/bin64/vtune
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hey iliyapolak,
What I meant was to download the tb7 file to your development machine and use the Import results option in the GUI. I am also attaching a screenshot on how to select and import the tb7 file.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Now I understood what you have ment.
I will do that.
Thank you
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hey iliyapolak,
Have you tried out the solution provided ?. Does that help.
Thanks
Arun Jose
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Arun,
Yes I tried to apply your workaround unfortunately the "import" option is still inactive. The tb7 file is stored on my local machine.
P.s.
I will try the command line switch --import.
I will let you know.
I'm using Linux version of VTune
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi.
Create New Project and make the import inside it
Kirill
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks Kirill for pointing it out.
I will try it and let you know the outcome.
Thanks
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I opened the raw samples file (tb7) and unfortunately the error persists.
Here is the screenshot.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
As a an additional information I would like to provide a top callstack functions active during the SIGBUS error, which sometimes VTune receives.
I strongly suspect, that somehow referencing the undefined section of memory mapped file (tb7?) might result in Signal 7 error.
The relevant captured register context
The faulting IP and surrounding it assembly.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
Yes of course I will share, that file tomorrow with you.
Thank you.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Min Yeol,
As per your request I attached two zipped files a .tb7 and smaller .sc file. These files do not belong to profiling session plagued by SIGBUS signal.
We observed two different situation which caused directly or indirectly to .tb7 file parsing failure.
1) An aforementioned rare condition of SIGBUS signal
2) Most frequently present failure of "TBRW_INMEM_initialize_data" function
(tbrw call "TBRW_INMEM_initialize_data(tbrwFile->getHandle(), streamIndex)" failed: invalid string (97)).
In case of #1 there is always created a .crash file which I partly analyzed unfortunately I was not able to inspect the arguments (memory addresses) because their content obviously was not collected during the crash data collection phase.
I can hypothesize than somehow a memory reference into the undefined portion of memory mapped file could have triggered the SIGBUS error, I do not suspect a hardware error.
In case of #2 I suspect a possibility of some kind of data/metadata corruption. I suppose that SEP collector might be sending a packets over the network to its receiving end and somehow part of those packets might have been corrupted
Thank you for your help
- Tags:
- tt
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks for sharing the result files. It seems the issue is similar to what we experienced before. To confirm it, I have two questions for you.
1. Can you confirm your target machine runs a docker container?
2. Is the vtune collection through ssh is under the same user as docker container is launched? Or are they under different users?
Regards,
Min
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Min,
Thank you for quick response.
I will contact the person responsible for managing our profiling environment and provide the answers in regards to your questions.
Thank you
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
I have the answers:
1) Our target machine runs Docker container
2) The same user account is used for docker container and for VTune collection session.
Thank you
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks.
Can you try two things?
1. Run VTune collection without docker container.
2. Run Vtune collection AND docker container with root privilege if you need to run a docker container.
Thanks
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Min,
I will test those 2 options and let you know the results.
Thank you for your help
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Min
1) Our simulation environment relies on Docker and we can not drop the usage of it
2) VTune process has a root privilege.
P.s.
Could you explain (if allowed of course) the meaning of invalid string (error 97) failure?
Thank you for your help
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I didn't understand the second answer. Was it working with root privilege? If not, did you see the same error or different?

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page