Intel® oneAPI HPC Toolkit
Get help with building, analyzing, optimizing, and scaling high-performance computing (HPC) applications.
Announcements
The Intel sign-in experience is changing in February to support enhanced security controls. If you sign in, click here for more information.
1990 Discussions

mpigdb: attaching to Cannot access memory when using -gtool option for debugging

xsl
Beginner
489 Views

This is a follow up on my previous question: https://community.intel.com/t5/Intel-oneAPI-HPC-Toolkit/Debugging-MPI-codes-with-gtool-or-gdb-hangs-...

 

I was told the cause of the issue is that I did not use a supported OS at the time but now I am actually using a supported OS (Ubuntu 20.04.4 LTS) that is freshly installed just a couple hours ago.

 

I still get the same error here when I try to use -gtool option to attach GDB to my code.  Here is what I used to run my code 

mpirun -n 2 -gtool "gdb:0,1=attach" ./fogo

and this is what I get 

mpigdb: attaching to Cannot access memory
mpigdb: hangup detected: while read from [0]
mpigdb: attaching to Cannot access memory
mpigdb: hangup detected: while read from [1]

I have also tried to replace gdb with gdb-oneapi but got the same thing. I could use this option on our Compute Canada clusters but most of the nodes are slow for debugging / compilation. 

 

I get the same error when running

mpirun -n 3 -gtool "gdb:0,1=attach" $I_MPI_ROOT/bin/IMB-MPI1

Any idea what is causing this issue?

 

0 Kudos
8 Replies
SantoshY_Intel
Moderator
388 Views

Hi,

 

Thanks for reaching out to us.

 

We have reported this issue to the concerned development team. They are looking into your issue.

 

Thanks & Regards,

Santosh

 

SantoshY_Intel
Moderator
193 Views

Hi,


Could you please try Intel MPI 2021.7 and let us know if the issue persists?


Thanks & Regards,

Santosh


SantoshY_Intel
Moderator
174 Views

Hi,


We haven't heard back from you. Could you please try Intel MPI 2021.7 and let us know if the issue persists?


Thanks & Regards,

Santosh



SantoshY_Intel
Moderator
149 Views

Hi,

 

Please follow the below steps to resolve your issue:

Steps

  1. sudo sysctl -w kernel.yama.ptrace_scope=0
  2. mpirun -gdb -n 4 ./sample

We have tried at our end & we are able to get the desired results as shown below:

image-2023-01-09-16-00-07-836.png

Please try from your end & let us know if you face any issues.

 

Thanks & Regards,

Santosh

 

xsl
Beginner
134 Views

Hi Santosh,

 

Thanks very much for this. I have tried your solution and it does seem that it's also working on my computer. However, it won't work without first changing the settings as you suggested, which is not entirely unacceptable. 

 

SantoshY_Intel
Moderator
119 Views

Hi,

 

The issue is specific to the systems having PTRACE scope to "1"(kernel.yama.ptrace_scope = 1).

 

  • The PTRACE system is used for debugging. With it, a single user process can attach to any other dumpable process owned by the same user. In the case of malicious software, it is possible to use PTRACE to access credentials that exist in memory (re-using existing SSH connections, extracting GPG agent information, etc).
  • A PTRACE scope of "0" is the more permissive mode. A scope of "1" limits PTRACE only to direct child processes (e.g. "gdb name-of-program" and "strace -f name-of-program" work, but gdb's "attach" and "strace -fp $PID" do not). The PTRACE scope is ignored when a user has CAP_SYS_PTRACE, so "sudo strace -fp $PID" will work as before.

For more details see: https://wiki.ubuntu.com/SecurityTeam/Roadmap/KernelHardening#ptrace

 

In general, PTRACE is not needed for the average running Ubuntu system. To that end, the default is to set the PTRACE scope to "1". This value(kernel.yama.ptrace_scope = 1) may not be appropriate for developers or servers with only admin accounts. 

 

Thus, the issue can be simply resolved by setting kernel.yama.ptrace_scope = 0 & it it not an issue related to Intel MPI.

 

Thanks & Regards,

Santosh

 

 

xsl
Beginner
102 Views

Hi Santosh,

 

Thanks very much for your detailed explanation. I now understand!

 

SantoshY_Intel
Moderator
92 Views

Hi,


Thanks for the quick response.


As your issue is resolved, we are closing this thread. If you need any additional information, please post a new question as this thread will no longer be monitored

by Intel.


Thanks & Regards,

Santosh


Reply