Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.
6503 Discussions

Wrong weights conversion during graph generation

EscVM
Beginner
1,341 Views

Hi to everyone,

I'm trying to make it work a very simple neural network with the following structure:

- Input of 3 84 3 (row, col, channels)

- Conv2D with kernel 3x3, filter 1 and stride 3

- Flatten layer where the output is only 28

- A series of Dense layers of 28 ---> 256 ---> 128 ---> 64 ---> 32 ---> 5 with all Relu activations except for the last one that is linear.

initial_model = load_model('stage_4_1510.h5') #initial_model.load_weights('stage_4_1510_weigths.h5') initial_model.summary() initial_model.trainable = False   input = Input(shape=(3, 84, 3), name='input') x = Conv2D(filters=1, kernel_size=(3,3), use_bias=False, strides=(1,3), padding='valid', trainable=False, name='conv2d')(input) x = Flatten()(x) x = initial_model(x)   model = Model(inputs=input, outputs=x) #model.compile(loss='mse', optimizer='rmsprop')   shape = (1,3,3,3,1)   model.load_weights('stage_4_1510.h5', by_name=True) weights = np.empty(shape) weights.fill(1.0) model.get_layer('conv2d').set_weights(weights)   model.summary()

The network has been trained in Keras and so from .h5 I had to transform the model in .pb file.

At that point, in order to verify that everything was correct , I printed out the weights and everything was perfect.

So, I used the mvNCCompiler to get the .graph file and I generated it without problems with the following message:

 

No Bias No Bias No Bias No Bias No Bias Fusing DeptwiseConv + Pointwise Convolution into plain Convolution Fusing Add and Batch after Convolution Eliminate layers that have been parsed as NoOp Fusing Pad and Convolution2D Fusing Scale after Convolution or FullyConnect Fusing standalone postOps Fusing Permute and Flatten Fusing Eltwise and Relu Fusing Concat of Concats   Evaluating input and weigths for each hw layer Network Input tensors ['input:0#17'] Network Output tensors ['sequential_1/dense_5/BiasAdd:0#33'] Blob generated

It seems everything fine so far, but unfortunately the network, during the inference time, gives random outputs ( not coherent with the original network).

I'm sure that the problem is in the conversion and in particular in the fully connected part because with some tests I verified that the output of the convolutional part is correct.

So, considering that I can touch and get my hand the compiler, what I have to do? Do you have any idea? Thx

 

0 Kudos
6 Replies
JesusE_Intel
Moderator
975 Views

Hi EscVM,

 

I responded to your discussion on the other forum. We can continue troubleshooting here since we are migrating the https://ncsforum.movidius.com to this forum next week.

 

Could answer the following questions:

  • What are you using to test your model? If you are using your own code, do you mind sharing that with me to give it a try.
  • Have you tried to test with mvNCCheck? Make sure you are specifying an input image with -i followed with the path of the file
  • What version of the NCSDK are you using?

https://movidius.github.io/ncsdk/tools/check.html

 

By the way, the OpenVINO toolkit supports both the Intel Movidius Neural Compute Stick and the Intel Neural Compute Stick 2. There are no further updates planned for the Intel Neural Compute SDK.

 

Regards,

Jesus

 

EscVM
Beginner
975 Views

Thank you @Intel_Jesus​  for your interest. I was starting loosing all hopes in getting a reply :)

 

In the meantime, I also tried to use the Movidius Neural Compute Stick 2 (I bought also that one) with OpenVIINO, but unfortunately I obtained the same result. Before converting the network everything works fine, but then the converted network gives completely different results :(

 

  • I share a .zip with all weights. There is the original Keras file "test.h5", its translation in .pb "test.pb" and finally, weights for the Movidius 1 and 2 (.graph, .bin, .xml). There is also a final file, "stage_4_1510.h5" , that is the one described in the original question. That file is a simple Multilayer Perceptron (only dense layers) that unfortunately is not directly compatible with the two neural sticks (input is not a 4D tensor). For this reason I wrote the code reported in the first message to a create a first convolutional layer that simply extracts the vector padded with zeros. (kernel are fixed to one and stride is (one three).
  • No because I'm not using the Movidius with Images
  • The second version

 

I'm sure that the problem is the dense layer because I've done an experiment creating a graph for the neural stick only with the convolutional and flatten layer and the results were the desired vector. So, the input vector of the fully connected part is correct, but then the results are different with the Keras/Tensorflow ones. So, the problem is for sure in the conversion of the weights of this last part.

 

Anyway, it pretty paradoxical how a device like this can easily handle convolutional neural networks like Inception, Yolo ecc... and it easily fail at handling a simple feed forward neural network.

 

Thank you very much your help, if you require more information let me know.

0 Kudos
JesusE_Intel
Moderator
975 Views

Hi EscVM,

 

Could you try to use SAME padding instead of VALID? We have seen issues in the past when using valid padding.

 

I am trying to understand what you are trying to accomplish to better assist you. Could you answer a couple more questions for me?

  • What kind of network is it?
  • What kind of input does it take?
  • Can you provide a sample input?
  • Could you share your application to run some tests?

 

Please note that the OpenVINO toolkit and Intel Movidius NCSDK are primarily used with CNNs and an input image.

 

Regards,

Jesus

 

EscVM
Beginner
975 Views
0 Kudos
EscVM
Beginner
975 Views

I'll try to be more clear as possible about what I'm trying to accomplish. In this way, I hope I'll answer all your questions (also why it doesn't make any sense to use padding SAME).

 

I have a very simple fully connected network (dense layers, "stage_4_1510.h5" weights) that takes as input a 2D tensor of 28 elements (-1, 28) and it has four hidden layers and an output of 5 neurons. So, the complete architecture is 28 ---> 256 ---> 128 ---> 64 ---> 32 ---> 5 with all Relu activations except for the last one that is linear.

 

As you perfectly know, both Neural Sticks cannot handle a structure like this, because they need a 4D tensor as input. So, I've found a workaround solution writing this code:

initial_model = load_model('stage_4_1510.h5') #initial_model.load_weights('stage_4_1510_weigths.h5') initial_model.summary() initial_model.trainable = False input = Input(shape=(3, 84, 3), name='input') x = Conv2D(filters=1, kernel_size=(3,3), use_bias=False, strides=(1,3), padding='valid', trainable=False, name='conv2d')(input) x = Flatten()(x) x = initial_model(x) model = Model(inputs=input, outputs=x) #model.compile(loss='mse', optimizer='rmsprop') shape = (1,3,3,3,1) model.load_weights('stage_4_1510.h5', by_name=True) weights = np.empty(shape) weights.fill(1.0) model.get_layer('conv2d').set_weights(weights) model.summary()

I loaded the previous network and then I simply stack on top of it a 2D convolution with kernel (3,3) with all weights set at one, only one filter and a stride of (1, 3). In this way I can have an input with shape (-1, 3, 84, 3) where the second and third channels have all zeros and the first channel has the second and third row zeros and the first is made like this: [v1, 0, 0, v2, 0, 0, v3...] where [v1, v2, v3...] is the original (-1, 28) 2D tensor. So, after the convolutional and a flatten layer I have my original vector [v1, v2, v3...] that will feed the fully connected network.

 

Practically my solution is a kind of simple trick to use Movidius also with simple multilayer perceptron networks with an overhead of an additional convolutional layer :) It should perfectly work, because I use only basic layers compatible with the Sticks!

 

As I said, on both Neural Sticks, everything works fine at the flatten output: the 2D tensor is the right one. Then, after the fully connected part, outputs are without sense. So, the problem should be in the weights of the fully connected layers.

 

To test the network you simply have to load, in the same script, with Keras the test.h5 and with the Movidius API the test.graph file. Feed both with a tensor of shape (-1, 3, 84, 3) made in the following way:

def generateInput(): input_state = np.zeros((3,84,3)) for i in range(0, 84, 3): input_state[0, i, 0] = 1   return input_state

and you will see that the output of the networks are different.

 

I hope that I have made this clear. Thanks again for your help.

0 Kudos
JesusE_Intel
Moderator
975 Views

Hi EscVM,

 

I checked with a couple of my peers and we do not see any issues why it's not working for you, what you are doing is correct. I will file a bug to get additional input from the development team. Could you please share the command used to convert your model to IR format on OpenVINO? It would be of great help if you can share a simple python application to replicate your issue.

 

Regards,

Jesus

 

 

0 Kudos
Reply