Intel® oneAPI Math Kernel Library
Ask questions and share information with other developers who use Intel® Math Kernel Library.
6956 Discussions

Releasing Sparse Matrix Handle and Arrays

Ahmadi__Afshin
3,196 Views

Hello,

 

I have a question about mkl_sparse_destroy(). In the provided sample code by Intel (sparse_spmmd.c) you first release the matrix handle and then deallocate associated arrays:

 

if( mkl_sparse_destroy( csrB ) != SPARSE_STATUS_SUCCESS)
{ printf(" Error after MKL_SPARSE_DESTROY, csrB \n");fflush(0); status = 1; }
mkl_free(values_B); mkl_free(columns_B); mkl_free(rowIndex_B);

 

However, in cases when I use the mkl_sparse_z_export_csr function to export the CSR handle to individual arrays, I get access violation error when I use mkl_free to deallocate the arrays after calling mkl_sparse_destroy(A_CSR_Handle). And it seems memory for these individual arrays is not released, which is causing memory leak. Any idea what could be the issue and how to address it?

 

mkl_sparse_z_export_csr(A_CSR_Handle, &indexing, &rows, &cols, &A_JA, &pointerE_A, &A_IA, &A_Val);

mkl_sparse_destroy(A_CSR_Handle);

mkl_free(A_JA);

mkl_free(A_IA);

mkl_free(A_Val);

mkl_free(pointerE_A);

 

My second question is that after calling mkl_sparse_destroy(), do I need to NULL the pointers again for reuse in the same subroutine (e.g. A_CSR_Handle = NULL, A_JA = NULL, etc.)?

 

Thank you,

Afshin

0 Kudos
1 Solution
Spencer_P_Intel
Employee
3,117 Views

Hi Afshin,

This is a fantastic question and one which I realize the documentation can probably be improved on.  The question boils down to understanding who "owns" the arrays.  Let us consider two scenarios: 

  1. where the user allocates the arrays and puts them into the sparse matrix handle, and
  2. where an api is called from the library that generates a sparse matrix as output and then mkl_sparse_?_export_csr is used to get access to those arrays. 

Hopefully by laying out what happens in each case and who is responsible for what, it will become clear why you are seeing the behavior you are describing.

Scenario 1:  When the user allocates the arrays themselves it is clear that they own that data and have responsibility to maintain their lifespan.  When the user inputs the arrays into a sparse matrix handle, they are temporarily on loan to the library and there is an implicit agreement that they will not be modified by either party while in the matrix handle unless the user calls an API (like mkl_sparse_order or mkl_sparse_?_set_value or mkl_sparse_?_update_values) that explicitly says it is modifying the original data.  If something needs to be changed outside of these scenarios, the matrix handle should be destroyed, the arrays get adjusted then if desired, a new sparse matrix handle should be created...  

Scenario 2:  When the library creates the arrays in a matrix handle (for instance as part of a call like mkl_sparse_sp2m()  or mkl_sparse_add() ...) then the library itself assumes ownership of those arrays and is responsible for managing their lifetime (done via the lifetime of the matrix_handle that contains them).  When an mkl_sparse_?_export_csr() api is called, the library gives access to those internal arrays to the user as a temporary loan (this is the part that documentation can probably do better at explaining) but still maintains "ownership" for them.  When the matrix handle that contains them is destroyed (by calling mkl_sparse_destroy() ) then all internally owned data in the handle (including the arrays) is actually destroyed as well.  When the user is done with the arrays, they can either let them go out of scope or explicitly set them back to nullptr, it won't affect the internal data at all. If a user wants to keep access to the data beyond the call to mkl_sparse_destroy(), they will have to make a copy of it themselves.  Also note that the same implicit agreement holds on not making modifications to the data while it is in the sparse matrix handle...

I hope this explains what you are seeing and how to work with the data.   Please let me know if you have any questions about this.

Best,

Spencer

View solution in original post

18 Replies
MRajesh_intel
Moderator
3,148 Views

Hi,

 

Thanks for posting your query.

 

Can you please share your MKL version, OS version?

 

As per the documentation, the mkl_sparse_?_export_csr routine exports an m-by-k matrix in CSR format matrix from the internal representation. The routine returns pointers to the internal representation and does not allocate additional memory. So, an error occurs if you use mkl_free() with these pointers.

 

Link: https://software.intel.com/content/www/us/en/develop/documentation/onemkl-developer-reference-c/top/blas-and-sparse-blas-routines/inspector-executor-sparse-blas-routines/matrix-manipulation-routines/mkl-sparse-export-csr.html

 

>>My second question is that after calling mkl_sparse_destroy(), do I need to NULL the pointers again for reuse in the same subroutine 

 

You cannot re-initialize the same handle in the same scope.

 

>>And it seems memory for these individual arrays is not released, which is causing memory leak. Any idea what could be the issue and how to address it?

 

We would need a minimal reproducer to see what is causing a memory leak as no additional memory is being allocated for the pointers used in mkl_sparse_?_export_csr and these arrays will be deallocated together with  A_CSR_Handle when mkl_sparse_destroy(A_CSR_Handle) is called.

 

Please let us know if you have any issues.

 

Regards

Rajesh.

 

0 Kudos
Spencer_P_Intel
Employee
3,118 Views

Hi Afshin,

This is a fantastic question and one which I realize the documentation can probably be improved on.  The question boils down to understanding who "owns" the arrays.  Let us consider two scenarios: 

  1. where the user allocates the arrays and puts them into the sparse matrix handle, and
  2. where an api is called from the library that generates a sparse matrix as output and then mkl_sparse_?_export_csr is used to get access to those arrays. 

Hopefully by laying out what happens in each case and who is responsible for what, it will become clear why you are seeing the behavior you are describing.

Scenario 1:  When the user allocates the arrays themselves it is clear that they own that data and have responsibility to maintain their lifespan.  When the user inputs the arrays into a sparse matrix handle, they are temporarily on loan to the library and there is an implicit agreement that they will not be modified by either party while in the matrix handle unless the user calls an API (like mkl_sparse_order or mkl_sparse_?_set_value or mkl_sparse_?_update_values) that explicitly says it is modifying the original data.  If something needs to be changed outside of these scenarios, the matrix handle should be destroyed, the arrays get adjusted then if desired, a new sparse matrix handle should be created...  

Scenario 2:  When the library creates the arrays in a matrix handle (for instance as part of a call like mkl_sparse_sp2m()  or mkl_sparse_add() ...) then the library itself assumes ownership of those arrays and is responsible for managing their lifetime (done via the lifetime of the matrix_handle that contains them).  When an mkl_sparse_?_export_csr() api is called, the library gives access to those internal arrays to the user as a temporary loan (this is the part that documentation can probably do better at explaining) but still maintains "ownership" for them.  When the matrix handle that contains them is destroyed (by calling mkl_sparse_destroy() ) then all internally owned data in the handle (including the arrays) is actually destroyed as well.  When the user is done with the arrays, they can either let them go out of scope or explicitly set them back to nullptr, it won't affect the internal data at all. If a user wants to keep access to the data beyond the call to mkl_sparse_destroy(), they will have to make a copy of it themselves.  Also note that the same implicit agreement holds on not making modifications to the data while it is in the sparse matrix handle...

I hope this explains what you are seeing and how to work with the data.   Please let me know if you have any questions about this.

Best,

Spencer

jduan
Beginner
1,457 Views

Hi Spencer,

Thank you for the detailed explanation, this explains a lot better than the documentation. 

May I ask if it's safe to move the pointer of exported result to some other variable, and set the exported pointer to nullptr to disable mkl_sparse_destroy from free the memory? As I don't want to keep two copies of data at the same time. 

Example:

 

int* csrRowPtrStart, * csrRowPtrEnd, * csrColInd;
int* csrRowPtrMyself, * csrColIndMyself;
float* csrVal;

status = mkl_sparse_s_export_csr(csrMat, &base, &m, &n, &csrRowPtrStart, &csrRowPtrEnd, &csrColInd, (float**)&csrVal);
csrRowPtrMyself = csrRowPtrStart;
csrColIndMyself = csrColInd;
csrRowPtrStart = nullptr;
csrRowPtrEnd = nullptr;
csrColInd = nullptr;
status = mkl_sparse_destroy(csrMat);

 

0 Kudos
Spencer_P_Intel
Employee
1,437 Views

Unfortunately the way the Inspector-Executor Sparse BLAS C/Fortran APIs are currently designed, you changing the address of your locally exported pointer will not transfer the arrays "ownership" to you.  The matrix handle internally keeps track of the arrays it "owns" with flags and the address of the array is still kept internally -- we are just giving the user access to the same address locations.  The tricky part in a "transfer of ownership" like you are suggesting comes down to making sure that the right malloc/free pairs are used -- if we internally allocate with mkl_malloc,  then the user should use mkl_free, and could run into trouble if they use some other deallocation scheme...  It is an interesting idea to consider a separate transfer CSR API that could give full control over to users (along with a documented manner to destroy them), but at this point we don't have plans to do such a thing.

 

An alternative that we have heard from some users might be of interest is to change the paradigm for sparse * sparse multiplication (and others like add, sypr, syrk, etc) that take one or more sparse matrices and create a new sparse matrix from them so that it is a multistage operation where size of arrays for the new matrix are provided to users and then users allocate the arrays, put them in a matrix handle and then pass that new matrix handle back in to be filled -- thereby ensuring that the ownership of all matrix arrays is completely managed by users.  This has been done for instance in the SYCL C++ api  oneapi::mkl::sparse::matmat() which is a newer API that does sparse * sparse matrix multiplication in the Intel oneMKL product.  At some future time, we hope to introduce a similar possibility for the Inspector-Executor Sparse BLAS C/Fortran APIs like mkl_sparse_sp2m or mkl_sparse_spmm.

 

Best,

Spencer

 

 

0 Kudos
jduan
Beginner
1,405 Views

Thank you for the suggestions Spencer, it helps a lot! 

0 Kudos
Spencer_P_Intel
Employee
3,103 Views

To directly answer the questions instead of giving general understanding of responsibilities, the arrays filled by a call to mkl_sparse_?_export_csr() are on loan to you and do not need to be freed explicitly.  The call to mkl_sparse_destroy() will destroy them internally and your arrays will no longer be valid as they are pointing to internally destroyed data.  You can set them to NULL or let them go out of scope.  

As far as the second question goes, which subroutines do you plan to call afterwards?  It is generally good practice to set pointers to null after you have destroyed them, especially if you plan to use them again for other purposes, but to answer whether it is absolutely necessary,  we will need to have more information.

Ahmadi__Afshin
3,070 Views

Thank you so much @Spencer_P_Intel for your detailed reply. It is very clear now.

One more question, when we destroy handles or free the arrays, is the memory freed immediately? Or it will just become available to the OS for future use? I don't see any decrease in memory usage after destroying the handles or freeing the arrays.

0 Kudos
MRajesh_intel
Moderator
3,041 Views

Hi,

 

Can you please share a minimal reproducer code along with the MKL version, OS details so that we can reproduce the issue from our end? Also can you share how are you viewing the memory usage?

 

Regards

Rajesh.

 

0 Kudos
Ahmadi__Afshin
2,911 Views

Hi Rajesh, 

 

Sorry for the late reply. I am using the MKL library that comes with OneAPI 2021.3 on a Windows 10 machine. To check the memory usage, I use break points together with the heap memory snapshot feature of Visual Studio 2019.  I will provide more information in my below reply to Khang.

 

Thanks,

Afshin

0 Kudos
MRajesh_intel
Moderator
2,962 Views

Hi,


Can you please provide the above mentioned details?


Regards

Rajesh


0 Kudos
Khang_N_Intel
Employee
2,931 Views

Hi Afshin,


How are you doing? Hope that everything is well on your side.


You brought up a very good question.!


Like Rajesh mentioned, it would be very helpful if you can provide a small reproducer so that we can confirm your finding.


Best,

Khang


0 Kudos
Ahmadi__Afshin
2,910 Views

Hi Khang,

 

I am good, how are you? Long time no chat!

 

I will get back to you with a reproducer code by this Friday. Let me know if you need anything else.

 

Thank you very much.

 

Best,

Afshin

0 Kudos
Khang_N_Intel
Employee
2,806 Views

Hi Afshin,

How are you doing?

I don't mean to push you. Do you have the reproducer or do you need any more explanation?

Thanks,

Khang


0 Kudos
Ahmadi__Afshin
2,789 Views

Hi Khan,

 

I am very sorry for the delay. Please find below the information that you need to reproduce the issue. 

 

1. Download the Visual Studio project files from the following link:

https://drive.google.com/file/d/18qDK-SoH5NQbItKElSxVW5j52lrfKWPV/view?usp=sharing

 

2. I have already configured the project file with necessary command line arguments. All you need to do is to run the solution from within Visual Studio. There are three breakpoints in the main routine (lines 15, 95, 97). Use the "Take Snapshot" button to take memory snapshots at each breakpoint. 

 

3. As you can see in the picture, there is about 87MB of the heap memory that was not cleared after deleting the classes. I only use  C++ vectors and MKL arrays in those classes (i.e., MKL is only used in PFCase.h). When a class is deleted, vectors are automatically released and freed from memory (I tested this separately). For MKL arrays, I placed the MKL memory free functions in the class destructor. So it is very likely that MKL is just making memory available to the OS and not releasing it. 

 

Ahmadi__Afshin_0-1630014921965.png

 

I used VS 2019 and OneAPI 2021.3 version to perform this test.

 

Thank you,

Afshin

0 Kudos
shb
Employee
1,573 Views

Hello Afshin,

 

After I debugged in your provided project, I found out that you call "mkl_sparse_z_create_csr()" function unnecessarily for "YbusdiagVnorm_CSR" and "YbusdiagV_CSR" sparse matrix handles after you got them as outputs of the previous "mkl_sparse_spmm()" calls, which have internally allocated memory as part of the handle, in the dS_dV_Sparse() function. 

By calling "mkl_sparse_z_create_csr()" function on the matrix handle you already created, it will overwrite the previous handle's information as newly created handle so that we lost the information of the handle's data is owned by library as well as having memory leak for the old sparse handle itself, even though you are using the same pointers as input for the sparse matrix handle.  Due to the overwriting of the information of "owned by library", when "mkl_sparse_destroy()" is called on those sparse matrix handles, the function didn't release the data assigned on the handle, which supposed to be released and that's why we got those memory leak behavior.

In fact, you don't need to call "mkl_sparse_z_create_csr()" for those handles, since they are remaining even after calling "mkl_sparse_z_export_csr()" on them.  After I just commented out those two function calls, there is no memory leak occurs anymore, as shown below:

output_with_mkl_nightly_ats_20230116_win_after_fixing_the_bug.png

If you have any questions for this, please let us know.

Best,

Seung-hee

0 Kudos
Khang_N_Intel
Employee
2,766 Views

Hi Afshin,


Thank you very much for providing the reproducer. I am aware that you have many projects to take care. So, no worries!


I will find the time to run your code and will let you know what the next step is.


Best regards,

Khang


0 Kudos
Khang_N_Intel
Employee
2,624 Views

Hi Afshin,

I apologize for the delay in response. I encountered an issue when loading the VS2019 solution project that you provided.

As soon as I have it resolved and confirm the issue, I will escalate it.

Best,

Khang


0 Kudos
Khang_N_Intel
Employee
2,501 Views

Hi Afshin,


I just want to let you know that I was finally to reproduce the issue on another system with the latest version of oneMKL, 2021.4. I also used VS 2019,

I will escalate the issue and will let you know the root cause as well as when the fix will be available.


Best regards,

Khang


0 Kudos
Reply