Garbage free Java GBT prediction

[mbazero]

I'm running GBT prediction using DAAL java library version 2020.3.013 from maven.

I'm running in a latency-sensitive system with strict limits on garbage generation. The official java example for GBT prediction allocates a new array internally in the DAAL java library whenever a new prediction result is retrieved.

Is there any way to run GBT prediction without generating garbage like this?

Thanks!

[AthiraM_Intel]

Hi,

Thanks for reaching out to us.

We are working on it internally, will get back to you soon.

Thanks.

[mbazero]

Thanks!

And for more details, this is the line in the DAAL java library that appears to allocate the new java array from the prediction result c object:

https://github.com/oneapi-src/oneDAL/blob/master/java/com/intel/daal/algorithms/gbt/regression/prediction/PredictionResult.java#L63

I'm wondering if it's possible to copy the prediction result c object to existing java array or otherwise read it without creating garbage.

[Rachel_O_Intel]

Hi,

I am an engineer working on investigating your case. We are working on it internally, please feel free to provide any other information that you think may be helpful as well as we look into the details for this (OS system, hardware, etc.).

Best,

Rachel

[mbazero]

Any suggestions here? I think the only relevant information is that I'm using the DAAL java library version 2020.3.013 from maven. I'm wondering if there's any way with the Java API to perform GBT prediction without creating garbage.

[Rachel_O_Intel]

Hi,

Thank you for the information. We are working on this internally and will update this thread when we can provide further steps.

Best,

Rachel

[mbazero]

Any updates by chance?

[kpetrov]

Hi, @mbazero

We create an object every time it is accessed, since we are compute algorithms in C ++ and converting from the C++ table of result to Java object. I think this cannot be avoided in the current implementation.  

How real is it for you critical? We can think about how to fix this for the future release.

[mbazero]

Thanks for the reply @kpetrov .

The big concern here is the allocation of a new Java array inside heap memory for every prediction result. This is a critical blocker for using DAAL in our production systems.

Our production systems can tolerate some garbage--e.g. some wrapper java objects per prediction result--but not a full array allocation.

Is there any way with the current API to write prediction results to a direct DoubleBuffer, for example?

[Rachel_O_Intel]

Hi,

I have spoken with @kpetrov. This functionality is not currently available in oneDAL Java library API. Potentially, the problem would be solved by adding setters of result tables for algorithms. We will add this as a feature request into our roadmap for a future release.

Best,

Rachel

[Rachel_O_Intel]

This issue has been closed and we will no longer respond to this thread. If you require additional assistance from Intel, please start a new thread. Any further interaction in this thread will be considered community only.