- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
I'm encountering an issue with a large, in-house Fortran MPI code where the program consistently hangs at the MPI_RECV call.
Unfortunately, due to the code's sensitivity and the limitations of this public forum, I'm unable to share the entire code or create a simplified version for demonstration purposes.
Despite these constraints, I would be grateful if you could offer any suggestions or troubleshooting steps that might help me identify and resolve the hanging issue.
Thank you for your time and assistance.
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello @Niyas
you might want to consider priority support where we offer a direct channel to Intel engineers and also a clear way how to share confidential data.
https://www.intel.com/content/www/us/en/developer/tools/oneapi/support.html
Without a reproducer or anything to work with, the only advice I can give you is to run with -check_mpi enabled. This should help you to identify some coding errors.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi @TobiasK
Thanks for your reply.
Currently, I using the following script file for MPI execution.
#!/bin/bash
#PBS -N RDE3D04MPI
#PBS -o /home/..../Result_out.out
#PBS -e /home/..../Result_err.out
#PBS -l nodes=lc601:ppn=36+lc602:ppn=36+lc603:ppn=36+lc604:ppn=33
cd /home/..../06RPL3DRDEoldMPI
mpiexec -np 141 ./RPL3DRDEMPI.exe
In this file, where should I add -check_mpi?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@Niyas yes, you can insert -check_mpi directly after mpiexec, for your reference:
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@TobiasK I tried exactly the following lines
mpiexec -check_mpi -np 141 ./RPL3DRDEMPI.exe
But, I get the error as the following file,
ERROR: ld.so: object 'libVTmc.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libVTmc.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libVTmc.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libVTmc.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libVTmc.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libVTmc.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libVTmc.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libVTmc.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libVTmc.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libVTmc.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libVTmc.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libVTmc.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libVTmc.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libVTmc.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libVTmc.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libVTmc.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libVTmc.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libVTmc.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libVTmc.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libVTmc.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libVTmc.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: ld.so: object 'libVTmc.so' from LD_PRELOAD cannot be preloaded: ignored.
Please find the attached.
I turned on tracking by adding "export VT_CHECK_TRACING=on" in the bashrc as follows but still the error persist,
##intel compiler
source /opt/intel/oneapi/compiler/latest/env/vars.sh
source /opt/intel/oneapi/mpi/2021.5.1/env/vars.sh
export I_MPI_HYDRA_BOOTSTRAP=ssh
export VT_CHECK_TRACING=on
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@TobiasK Thanks for the link.
Can I directly install a trace analyzer in the HPC cluster using the link you provided?
We are using Linux-CentOS.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@Niyas CentOS is not supported by our current release.
https://www.intel.com/content/www/us/en/developer/articles/release-notes/intel-trace-analyzer-and-collector-release-notes-linux.html
- Operating systems:
- Amazon Linux 2, 2022
- Debian* 11.x
- Fedora* 37, 38
- Rocky 9
- Red Hat Enterprise Linux* 8.x, 9.x
- SUSE Linux Enterprise Server* 15SP3, 15SP4
- Ubuntu* 20.04, 22.04
You may still try install the package either as part of the HPC kit or as standalone available here:
https://www.intel.com/content/www/us/en/developer/articles/tool/oneapi-standalone-components.html#trace
However, since you are running on an unsupported OS, I cannot help you if something is not working.
![](/skins/images/B7BA37F9CE4B52FF7CAE1FB0388839C8/responsive_peak/images/icon_anonymous_message.png)
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page