Can't extract NumericTable after serializing and deserializing

[Mayank_Jindal]

I have reported this issue here also.

Summary:

If I serialize and deserialize DistributePartialResult received at here, then I can't get Numeric Table from DistributePartialResult.

Steps to reproduce: 

Go to line no. 158 here and Insert this code after line 158 - 

byte[] resultbyte = serializeDistributedPartialResult(result);
            try{
                DistributedPartialResult deserult = deserializeDistributedPartialResult(resultbyte);
                System.out.println("testdeser end");
                TrainingResult tr = deserult.get(DistributedPartialResultId.resultFromMaster);
                System.out.println("test tr got");
                TrainingModel tm = tr.get(TrainingResultId.model);
                System.out.println("test tm got");
                NumericTable yont = tm.getWeightsAndBiases();
                System.out.println("test yont got");
              } catch (Exception e) 
              {
                System.out.println("Fail to serilization trainingModel" + e.toString());
                e.printStackTrace();
              } 

Define two function serializeDistributedPartialResult and deserializeDistributedPartialResult as following -

private static byte[] serializeDistributedPartialResult(DistributedPartialResult dataTable) throws IOException {
        /* Create an output stream to serialize the DistributedPartialResult */
        ByteArrayOutputStream outputByteStream = new ByteArrayOutputStream();
        ObjectOutputStream outputStream = new ObjectOutputStream(outputByteStream);

        /* Serialize the DistributedPartialResult into the output stream */
        dataTable.pack();
        outputStream.writeObject(dataTable);

        /* Store the serialized data in an array */
        byte[] buffer = outputByteStream.toByteArray();
        return buffer;
    }

    private static DistributedPartialResult deserializeDistributedPartialResult(byte[] buffer) throws IOException, ClassNotFoundException {
        /* Create an input stream to deserialize the DistributedPartialResult from the array */
        ByteArrayInputStream inputByteStream = new ByteArrayInputStream(buffer);
        ObjectInputStream inputStream = new ObjectInputStream(inputByteStream);

        /* Create a DistributedPartialResult object */
        DistributedPartialResult restoredDataTable = (DistributedPartialResult) inputStream.readObject();
        restoredDataTable.unpack(context);

        return restoredDataTable;
    }

These functions are written from here.

Expected behavior:

All the newly added print statements should work.

Observed behavior:

NumericTable yont = tm.getWeightsAndBiases();

This line is giving error - 

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007fc4de4fc32d, pid=106185, tid=0x00007fc8c8e8d700
#
# JRE version: Java(TM) SE Runtime Environment (8.0_131-b11) (build 1.8.0_131-b11)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.131-b11 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [libJavaAPI.so+0x1dc132d]  Java_com_intel_daal_algorithms_neural_1networks_training_TrainingModel_cGetWeightsAndBiases+0x5d
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /N/u/mayank/daal/__release_lnx/daal/examples/java/hs_err_pid106185.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

[VictoriyaS_F_Intel · Accepted solution]

Hello Mayank,

The present version of Intel DAAL library does not support serialization of the neural network model.

We are investigating the options to enable the serialization in one of future releases.

One of the possible workarounds is to serialize the numeric table with the trained weights and biases. Use getWeightsAndBiases() method to get the numeric table with trained weights and biases. And, after deserialization use setWeightsAndBiases(NumericTable weightsAndBiases) method to set the table with weights and biases into the neural network model.

Best regards,

Victoriya

[Mayank_Jindal]

Thanks.