Issue with MKL Data Fitting splines memory usage

[Towie__Ewan]

Hello everyone,

We recently transitioned our code to utilise the MKL data fitting spline and have now noticed a rather large issue with memory usage.

In our largest examples we can allocate in excess of 1e6 splines using the MKL spline library and we are seeing an additional overhead of ~100GB of RAM from just the MKL splines.

Previously we used our own code for the natural cubic spline and have compared directly between implementations to understand and isolate the memory increase. It seems that the additional memory is allocated within dfdNewTask1D.

The splines themselves are nothing to complicated or large, typically consist of ~27 knots and use the natural cubic spline method.

I have tried to disable the fast memory management, and to free unused memory for MKL as discussed at avoiding-memory-leaks-in-intel-mkl.html. These options has no impact on the overall memory usage.

Is there anything that can be done to reduce the memory overhead of the MKL data fitting splines?

Is this memory usage expected?

Thanks,

Ewan

[Gennady_F_Intel]

yes, this issue has been resolved in MKL version 2020 

[Gennady_F_Intel]

Are there any updates here, Ewan?

[Towie__Ewan]

Hi Gennady,

I did come to a conclusion on my side regarding the memory usage of the MKL splines.

After much testing (unwillingly!) it was apparent that there wasn't a memory leak or other issue leading to the excessive memory usage. I tried newer versions of MKL and TBB libraries with no change in the result.

In the end the conclusive answer came from extending the reproducer code to compare spline implementations. I switched from MKL splines to a simple natural cubic spline model and the difference in memory usage was concerning.

For reference, I was comparing with the construction of 429,300 splines with each spline containing 26 knots (26 [x,y]data points). In this example the x-sample locations are identical in all cases. For the MKL cases I also compared use of splines with shared x sample points (e.g. single data fitting task for all splines), with independent splines (e.g. separate data fitting task per spline).

Minimum memory usage for a natural cubic spline (with zero storage overhead) of doubles for this example can be estimated assuming only one extra value per knot to store the second-derivative as: 429300*26*3*8 bytes = 267,883,200 bytes = ~256MB.

With independent MKL Data Fitting splines peak mem usage was ~3163MB, with the grouped MKL splines this dropped to ~2787 MB, and finally with a simple natural cubic spline model the peak mem was ~386MB.

So it is very clear the MKL Data Fitting allocates a *lot* of extra memory and I have no idea what for! I understand that the MKL library offers more functionality, but it was neither quicker or more memory efficient in my example here. For a natural cubic spline it does seem bloated.

In the end I dropped the use of MKL Data Fitting splines as I have a premium on memory usage.

Can you tell me where the extra memory is spent? 

[Pavel_D_Intel1]

Hi Ewan,

thank you for the detailed answer.

Regarding your question “Can you tell me where the extra memory is spent? ” – the additional memory is used for the optimizations of cell-search, interpolation and integration routines.

I would like to inform you that we have optimized memory consumption of oneMKL DataFitting routines based on your earlier feedback.

Now we can see the next picture: memory consumption for 429,300 splines (with 26 knots) is

• 6.88 GB before the fix (I am not sure what is the difference between our parameters as you observed ~3 GB)
• 0.43 GB after the fix (about ~16 times reduction)

The fix is available starting coming oneMKL 2021.2 release.
I hope that this fix will unblock you in using of oneMKL DataFitting routines.

Best regards,
Pavel

[Towie__Ewan]

The difference in reported memory is most likely due to the method I'm using to determine memory usage.

For comparison between the splines w/wo MKL I reverted to getrusage() available in 'sys/resource.h' to determine the peak memory usage. It's most likely an estimate but I can confirm that using the MKL memory stats output I see ~ 6GB for the example I discussed above. No idea why the difference in measurements.

The improvements made in the newer version of MKL look very promising, when I get some free time I will check out the new version.

Is there any update on the release of oneMKL on GitHub? 

[Gennady_F_Intel]

Are there any updates here, Ewan?

[Gennady_F_Intel]

The new update ( 2021 u2) is not available on GitHub and only planning to be released ~the next month. We will keep you informed when this release will happen.

[Gennady_F_Intel · Accepted solution]

Ewan,

The memory consumption issue has been resolved in oneMKL 2021.2 which available for download.

[Gennady_F_Intel]

The issue is closing and we will no longer respond to this thread. If you require additional assistance from Intel, please start a new thread. Any further interaction in this thread will be considered community only.