topic How can I specify IB Adapter when using mpirun? in Intel® MPI Library

How can I specify IB Adapter when using mpirun?

youn__kihang — Tue, 05 Jan 2021 05:35:47 GMT

Hello everyone,

I have one question to ask, so I'm writing a post.
I would like to choose and use one of the Infiniband adapters.

In previous versions, it could be specified as follows,
I_MPI_OFA_ADAPTER_NAME mlx5_0
That option seems to have disappeared as it has been incorporated into OFI in recent versions.
What environment variable should I set in order to use mlx5_1?

If you have any comments, please tell us.

Thank you.

Re: How can I specify IB Adapter when using mpirun?

youn__kihang — Tue, 05 Jan 2021 07:25:24 GMT

Here is ibstat result.

> ibstat
CA 'mlx5_0'
CA type: MT4117
Number of ports: 1
Firmware version: 14.28.1002
Port 1:
State: Active
Physical state: LinkUp
Link layer: Ethernet
CA 'mlx5_1'
CA type: MT4123
Number of ports: 1
Firmware version: 20.28.1002
Port 1:
State: Active
Physical state: LinkUp
Link layer: InfiniBand
CA 'mlx5_2'
CA type: MT4117
Number of ports: 1
Firmware version: 14.28.1002
Port 1:
State: Active
Physical state: LinkUp
Link layer: Ethernet

Re:How can I specify IB Adapter when using mpirun?

PrasanthD_intel — Wed, 06 Jan 2021 12:21:50 GMT

Hi Kihang,

You can use the variable UCX_NET_DEVICES to select from available adapters.

For example: mpirun -np 2 -env UCX_NET_DEVICES=mlx5_1:1

Let us know if you face any issues.

Regards

Prasanth

Re: Re:How can I specify IB Adapter when using mpirun?

youn__kihang — Mon, 11 Jan 2021 08:08:37 GMT

Using the environment variable you mentioned, I was able to use the adapter I wanted !!

One more thing, are there any Intel MPI options to check the name of the adapter in use?
I tried I_MPI_DEBUG=6 and I_MPI_HYBDA_DEBUG=ON, but the adapter name did not appear.

Thank you!

Re: How can I specify IB Adapter when using mpirun?

PrasanthD_intel — Mon, 11 Jan 2021 10:48:24 GMT

Hi Kihang,

You can set the variable UCX_LOG_LEVEL=info to check which adapter is being used.

eg: UCX_LOG_LEVEL=info UCX_NET_DEVICES=mlx5_1:1 mpiexec.hydra -n 2 ./a.out

Let us know if you any other queries else we can close the thread.

Regards

Prasanth

Re: How can I specify IB Adapter when using mpirun?

youn__kihang — Wed, 13 Jan 2021 08:55:08 GMT

All questions have been resolved.

Thank you!

Re:How can I specify IB Adapter when using mpirun?

PrasanthD_intel — Tue, 19 Jan 2021 07:28:10 GMT

Hi Kihang,

Thanks for the confirmation.

As the issue has been resolved, we will no longer respond to this thread. If you require additional assistance from Intel, please start a new thread. Any further interaction in this thread will be considered community only.

Regards

Prasanth