Community
cancel
Showing results for 
Search instead for 
Did you mean: 
DH123
Beginner
371 Views

advixe crashes immediately

Hi,

I've been trying to run a sample intel example that does matrix multiplication using advixe-cl or advixe. I issue the following command:

advixe-cl -collect survey -project-dir my_result -- ./1_mmult_serial

 

and get the following output (on multiple platforms running fedora 33):

Intel(R) Advisor Command Line Tool
Copyright (C) 2009-2020 Intel Corporation. All rights reserved.
advisor: Collection started. To stop the collection, either press CTRL-C or enter from another console window: advisor -r /home/dhalpern/Downloads/mmult/my_result/e000/hs002 -command stop.
advisor: Warning: [2020.12.27 18:00:23] /lib64/libc.so.6 _init() instrumentation failed. Profiling data may be missing.
vcs/tpss2/tpss/src/tpss/runtime/linux/exe/tpss_deepbind.c:235 tpss_deepbind_notify_on_pthread_loaded: Assertion '((tpss_pthread_key_create_call_t)(((((tpss_probe_t*)g_tpss_probes_table) + g_tpss_pt_id[(tpss_pi_pthread_key_create)]))->trampoline)) != ((void *)0) && ((tpss_pthread_setspecific_call_t)(((((tpss_probe_t*)g_tpss_probes_table) + g_tpss_pt_id[(tpss_pi_pthread_setspecific)]))->trampoline)) != ((void *)0) && ((tpss_pthread_getspecific_call_t)(((((tpss_probe_t*)g_tpss_probes_table) + g_tpss_pt_id[(tpss_pi_pthread_getspecific)]))->trampoline)) != ((void *)0) && ((tpss_pthread_getattr_np_call_t)(((((tpss_probe_t*)g_tpss_probes_table) + g_tpss_pt_id[(tpss_pi_pthread_getattr_np)]))->trampoline)) != ((void *)0) && ((tpss_pthread_attr_getstack_call_t)(((((tpss_probe_t*)g_tpss_probes_table) + g_tpss_pt_id[(tpss_pi_pthread_attr_getstack)]))->trampoline)) != ((void *)0) && ((tpss_pthread_attr_getstacksize_call_t)(((((tpss_probe_t*)g_tpss_probes_table) + g_tpss_pt_id[(tpss_pi_pthread_attr_getstacksize)]))->trampoline)) != ((void *)0) && ((tpss_pthread_attr_setstack_call_t)(((((tpss_probe_t*)g_tpss_probes_table) + g_tpss_pt_id[(tpss_pi_pthread_attr_setstack)]))->trampoline)) != ((void *)0) && ((tpss_pthread_attr_setstacksize_call_t)(((((tpss_probe_t*)g_tpss_probes_table) + g_tpss_pt_id[(tpss_pi_pthread_attr_setstacksize)]))->trampoline)) != ((void *)0) && ((tpss__pthread_cleanup_push_call_t)(((((tpss_probe_t*)g_tpss_probes_table) + g_tpss_pt_id[(tpss_pi__pthread_cleanup_push)]))->trampoline)) != ((void *)0) && ((tpss__pthread_cleanup_pop_call_t)(((((tpss_probe_t*)g_tpss_probes_table) + g_tpss_pt_id[(tpss_pi__pthread_cleanup_pop)]))->trampoline)) != ((void *)0)' failed.

 

I've tried different executable programs (fortran and c) and get the same error message about /lib64/libc.so.6.

Any suggestions would be appreciated.

DH

 

 

0 Kudos
16 Replies
GouthamK_Intel
Moderator
353 Views

Hi David,

Thanks for reaching out to us!

As your issue is related to Advisor tool, we are redirecting this thread to Intel Advisor Forum for faster response.

Have a Good day!

 

Thanks & Regards

Goutham

 

ChithraJ_Intel
Moderator
331 Views

Hi David,


We have tried to run Survey analysis for a sample matrix application in Ubuntu machine(Ubuntu 18.04.4 LTS). But, we couldn't observe any issues. So, could you please try the below work arounds as of now.

1. As checking your error log we found that you are using an older version of advisor. Could you please upgrade it to the latest version and let us know the updates.

2. Please check other analysis types as well.

3. If it's a GUI enabled machine, try to launch advisor through GUI.


Please try any of the above work arounds and let us know the updates.


Regards,

Chithra


ChithraJ_Intel
Moderator
306 Views

Hi David,


Did you try out any of the work arounds provided? Please let us know the updates.


Regards,

Chithra


DH123
Beginner
285 Views

Hi,

Thanks for your reply.

1. I am using the latest version 2021.1 that comes with oneapi. So, I am not sure what you mean by "OLD".

2. I've tried and get the same problems.

3. I used the gui versions of advixe and advisor and get exactly the same errors. Also the gui and non-gui versions of advixe and advisor crash ungracefully. I have to end up doing a kill -9 PID on a separate window,

 

DRW
Beginner
275 Views

I think I am seeing the same problem, /lib64/libc.6.0 _init() instrumentation failed, also with Fedora 33 which I just updated to about a week ago, and Advisor GUI at the 2021.1 version that's part of the Oneapi install archive.

I am using the Advisor GUI to analyze a simple program using AVX512 that runs about 30 seconds.

I created a new project and just specified the name of the executable in the dialog asking for run parameters, and running the  Vectorization and Code insights analysis

Once I get the error, the GUI is stalled, but the UI is still active. I end up manually killing my program and the advisor processes manually to recover.

Maybe there's something unique about the system software, libc and others, in Fedora 33 that is causing this.

DH123
Beginner
265 Views

Hi,

 

Thanks for the response. In the end, do you get any meaningful output from advisor that you can use?

 

DH

DRW
Beginner
252 Views

If you're asking me, no I do not. The GUI is stalled and I have to kill it and my application process. I did try starting the GUI again and saw there is a project folder. I tried opening that and got a popup assert and hung GUI that I cannot copy test from. The assert is at line 722 db_interface_smpl_sqlite.cpp: Attribute table already exists. There's what' apparently a preceding error message 'No data is collected'.

 

ChithraJ_Intel
Moderator
240 Views

Hi David,


We are forwarding this case to Subject Matter Experts(SME) who can help you out and they will get back to you soon.


Regards,

Chithra


Johny_P_Intel
Employee
231 Views

Is this issue occurring only on a particular fedora system? Can you also reproduce this issue on a similar but clean installation? We suspect an issue with your libc and that's the reason behind these additional questions.


Regards,

Johny Paul.



DRW
Beginner
225 Views

I created a virtual machine, VirtualBox 6.1.16, installed a new copy of Fedora 33 and then installed the OneApi code. I get the same error when I run advisor-gui

DH123
Beginner
215 Views

Hi,

 

I have a xeon workstation and an nuc both with fedora 33. However, neither has a clean install. But as pointed out by another user, DRW, who did a clean install of fedora 33, he encountered exactly the same problem.

 

Thanks.

 

DH

DRW
Beginner
207 Views

I decided to try one more thing to narrow this down.

I downloaded a small OpenMP example program https://computing.llnl.gov/tutorials/openMP/samples/C/omp_mm.c and compiled it on my base Fedora 33 system where advisor does not work

icc -qopenmp omp_mm.c omp_mm.c

I have the base docker containers for Fedora 31 and Fedora 33 (presumably from dockerhub)

I have OneApi final December 2020 version, including Advisor installed on my base Fedora system

I started the Fedora 33 container 'docker container run -t -i -v /opt/intel:/opt/intel <imagename> sh' to get a container with a shell prompt and have OneApi available to the container

I copied the omp_mm binary to the container /tmp directory

I installed a few RPMs apparently needed to run, 'dnf install procps-ng stdlibc++ findutils'

I ran the setvars.sh top level script to set the environment

Then I ran 'advisor --collect survey ./omp_mm'

That crashed as already noted

I started another container using the Fedora31 base image and repeated all of the above. This time there were no error messages

Then I ran 'advisor --report survey .omp_mm' in the Fedora 31  and it ran with no errors. There was likely nothing to report since omp_mm is so simple, but advisor seems function.

The difference seems to be that on Fedora 31 /lib64/libc.so.6 is linked to libc-2.30.so and on Fedora 33 linked to libc-2.32.so.

I don't know if there are more differences, but this seems to be one that matters

Ruslan_M_Intel
Employee
190 Views

According to oneAPI System Requirements Fedora 33 is not supported yet. Probably incompatibility between different versions of 'libc' is the reason.

Johny_P_Intel
Employee
175 Views

I managed to reproduce this issue using Fedora 33 (version: Workstation Edition with Linux Kernel 5.8.15-301.fc33.x86_64) and libc (version 2.32). The error message does not exactly resemble the errors posted above. But I can see that the behavior is very similar and leading to a crash related to libc. The errors I observed before the crash/hang are listed below.


advisor: Warning: [2021.01.11 20:33:20] /lib64/libc.so.6 _init() instrumentation failed. Profiling data may be missing.

AMPLXE_TPSSCOLLECTOR: tool:939: g_instr_global.is_libc_so_initialized == 1 : g_instr_global.is_libc_so_initialized == 0


Assertion failed: tool:939: g_instr_global.is_libc_so_initialized == 1 : g_instr_global.is_libc_so_initialized == 0

. Please contact the technical support. advisor: Error: Assertion failed: tool:939: g_instr_global.is_libc_so_initialized == 1 : g_instr_global.is_libc_so_initialized == 0

. Please contact the technical support. 



The issue has been reported to our engineering team and they are taking a look to identify the root-cause of this issue. We will get back to you as soon as possible.



Johny_P_Intel
Employee
105 Views

Hi,


I can now confirm that the issue is related to the glibc (version v2.32). We hope that a fix can be incorporated in the next release. I will provide further updates when a fix is available.


Regards,

Johny


FelixBrakel
Beginner
10 Views

I get the exact same error with glibc version 2.33. One thing to note is that it only happens when sampling in user mode. I also noticed that when running threading analysis my program using pthreads only executes on a single thread and this one thread is pinned at 100% usage. The program never seems to finish.