topic Does Pardiso OOC mode support multi-process environment? in Intel® oneAPI Math Kernel Library

Does Pardiso OOC mode support multi-process environment?

yuchengshuai — Fri, 18 Oct 2024 06:39:32 GMT

We know that we can set the MKL_PARDISO_OOC_FILE_NAME environment variables to store the intermediate matrix factors, there is no question, but if I have multi process to parallel execute matrix calculation, every process will use this environment variable to store theirs intermediate date, does there has conflicts?

In my test, if I run sequential in single process , there has no error, but if I run in parallel with multi-process, the Pardiso will have numerical factor fail error.

Does anyone have any experience?

Re: Does Pardiso OOC mode support multi-process environment?

morskaya_svinka_1 — Fri, 08 Nov 2024 10:58:33 GMT

I ain't got any experience with distributed MKL PARDISO solver, but I guess you can check your process's id given by MPI and set different file names for each process. Does multithreaded single-process version work fine for you? Which way do you check the solution to be correct? May I ask why do you think the probem is in conflict to write into files? It is also helpful if you provide the ouput given by the program together with a post. If you are afraid of the conflict between the threads within a single process, I think each thread calculates its tasks but the data is writeen only with one thread at a time, so there should not be conflicts.

Re:Does Pardiso OOC mode support multi-process environment?

Ruqiu_C_Intel — Fri, 08 Nov 2024 11:48:18 GMT

Sharing the same MKL_PARDISO_OOC_FILE_NAME across multiple processes in a multi-process environment is not recommended. Each process writing to the same Out-of-Core (OOC) file can cause various issues including data corruption, file overwrites, and potential race conditions.