Community
cancel
Showing results for 
Search instead for 
Did you mean: 
idata
Community Manager
418 Views

Does the SDK support Tensorflow slice operations?

Hi all,

 

I am trying to implement an autoencoder architecture on Tensorflow.

 

Specifically, the tensor Z is a compressed representation of the original input, with size (1, 1, 1, 192). I am trying to apply a deconvolution layer for multiple times to scale back up to the input size (1, 1024, 32, 1).

 

The first deconvolution layer would be this:

 

def Decoder_test_same(self, Z): with tf.variable_scope("Decoder") as scope: conv1 = tf.layers.conv2d_transpose(Z, 192, 3, strides=2, padding='same', activation=tf.nn.relu) return conv1

 

which uses the 'SAME' padding so the dimension of conv1 will be (1, 2, 2, 192). This is followed by another 8 deconvolution layers with different kernel sizes / no. of channels.

 

Unfortunately it seems that NCSDK does not support 'SAME' padding:

 

[Error 5] Toolkit Error: Stage Details Not Supported: Wrong deconvolution output shape.

 

After looking up for solutions on the internet I decided to use the 'VALID' padding instead. This should give me a tensor with size (1, 3, 3, 192). By removing the last row and column, I should get the exact same (1, 2, 2 ,192) tensor as previously mentioned.

 

The new code now looks like this:

 

def Decoder_test_valid(self, Z): with tf.variable_scope("Decoder") as scope: conv1 = tf.layers.conv2d_transpose(Z, 192, 3, strides=2, padding='valid', activation=tf.nn.relu) conv2 = tf.slice(conv1, [0, 0, 0, 0], [1, 2, 2, 192]) return conv2

 

mvNCProfile can now handle the deconvolution layer, but it now complains that the Slice operation is not supported:

 

[Error 5] Toolkit Error: Stage Details Not Supported: Slice type not supported

 

Is the tf.slice op completely unsupported, or are there some restrictions on the input that I failed to observe? If slicing is indeed not supported, is there some alternative where I can obtain the subset of a tensor? I have also tried tf.split but that resulted in Error 5 as well.

 

Thanks in advance

Tags (1)
0 Kudos
5 Replies
idata
Community Manager
147 Views

@ferrophile The current NCSDK (v 2.08.1) doesn't have support for slicing a tensor dimension like how you showed. The NCSDK currently requires index 1 and index 2 from the input tensor and sliced tensor to be equal.

idata
Community Manager
147 Views

@Tome_at_Intel Thanks for the response. I realized that even if I got past the slicing problem, my network still wouldn't compile since I use batch norm layers after every deconvolution layer, which results in Error 5 again:

 

[Error 5] Toolkit Error: Stage Details Not Supported: FusedBatchNorm must be preceded by convolution or fully connected layer

 

Can I know if there is some official documentation on what rules I have to follow when using a particular layer in Tensorflow? I think I saw a list of supported layers somewhere in the forum, but apparently there are certain limitations on the layer arguments, dimensions etc.

idata
Community Manager
147 Views

By the way, I also tried to compute the batch normalization process manually using stored parameters. NCSDK reported that it does not support the tf.sqrt operation:

 

var = tf.get_variable('batch_normalization/moving_variance', initializer=tf.zeros(192, tf.float32)) std = tf.sqrt(var + 0.001)

 

[Error 5] Toolkit Error: Stage Details Not Supported: Sqrt

 

Is tf.sqrt completely unsupported, or are there specific requirements regarding the input tensor?

idata
Community Manager
147 Views

@ferrophile There are still some TensorFlow ops that the NCSDK doesn't have support for yet and it seems like we don't have support for the sqrt operation at the moment.

idata
Community Manager
147 Views

@Tome_at_Intel

 

Thanks. I am trying to find a workaround solution using the ops supported right now, so I had a look at the release notes of the current version:

 

https://github.com/movidius/ncsdk/releases

 

For V1.12.01 of the SDK, "Crop" is a firmware feature according to the release notes. Could you tell me which Tensorflow op could I use to make use of this feature? (assuming this feature is not for Caffe only)

Reply