RDMA DEFAULT MAX SEND/RECV WQE

[karl_lehnberger]

how do MVAPICH2's RDMA_DEFAULT_MAX_SEND_WQE/RDMA_DEFAULT_MAX_RECV_WQE envs translate into
I_MPI_ speak (for both DAPL and DAPL_UD on Linux) ?

cheers.

[Dmitry_K_Intel2]

Hi Karl,

Analog for RDMA_DEFAULT_MAX_RECV_WQE is I_MPI_DAPL_RECV_QUEUE_SIZE ( from version 4.0) and I_MPI_RDMA_RECV_QUEUE_SIZE ( for versions 3.x.x).
Please pay attention that for RDMA mode DAPL recv operations are not used (by default) and this variable has no influence. Only setting I_MPI_DAPL_RECV_QUEUE_SIZE=0 you can save memory for each connection. But some old providers may fail with I_MPI_DAPL_RECV_QUEUE_SIZE=0.

For RDMA_DEFAULT_MAX_SEND_WQE Intel MPI Library has 2variables: I_MPI_DAPL_REQUEST_QUEUE_SIZE and I_MPI_DAPL_REQUEST_QUEUE_RESERVE. Their sum is MVAPICH2's RDMA_DEFAULT_MAX_SEND_WQE.
I_MPI_DAPL_REQUEST_QUEUE_RESERVE is usually quite small.

DAPL UD is connectioless transport and there are no exact analogs. DAPL UD is more like SRQ.

Regards!
Dmitry

[karl_lehnberger]

thanks, Dmitry ! what's the default value I_MPI_DAPL_REQUEST_QUEUE_SIZE ? It seems not to be documented in the reference manual.

cheers.

[Dmitry_K_Intel2]

I_MPI_DAPL_REQUEST_QUEUE_SIZE is 400 right now by default.

Regards!
Dmitry