Does the SDK support Tensorflow slice operations?

[idata]

Hi all,

I am trying to implement an autoencoder architecture on Tensorflow.Specifically, the tensor Z is a compressed representation of the original input, with size (1, 1, 1, 192). I am trying to apply a deconvolution layer for multiple times to scale back up to the input size (1, 1024, 32, 1).The first deconvolution layer would be this:def Decoder_test_same(self, Z): with tf.variable_scope("Decoder") as scope: conv1 = tf.layers.conv2d_transpose(Z, 192, 3, strides=2, padding='same', activation=tf.nn.relu) return conv1

which uses the 'SAME' padding so the dimension of conv1 will be (1, 2, 2, 192). This is followed by another 8 deconvolution layers with different kernel sizes / no. of channels.

Unfortunately it seems that NCSDK does not support 'SAME' padding:

[Error 5] Toolkit Error: Stage Details Not Supported: Wrong deconvolution output shape.

After looking up for solutions on the internet I decided to use the 'VALID' padding instead. This should give me a tensor with size (1, 3, 3, 192). By removing the last row and column, I should get the exact same (1, 2, 2 ,192) tensor as previously mentioned.

The new code now looks like this:def Decoder_test_valid(self, Z): with tf.variable_scope("Decoder") as scope: conv1 = tf.layers.conv2d_transpose(Z, 192, 3, strides=2, padding='valid', activation=tf.nn.relu) conv2 = tf.slice(conv1, [0, 0, 0, 0], [1, 2, 2, 192]) return conv2

mvNCProfile can now handle the deconvolution layer, but it now complains that the Slice operation is not supported:

[Error 5] Toolkit Error: Stage Details Not Supported: Slice type not supported

Is the tf.slice op completely unsupported, or are there some restrictions on the input that I failed to observe? If slicing is indeed not supported, is there some alternative where I can obtain the subset of a tensor? I have also tried tf.split but that resulted in Error 5 as well.

Thanks in advance

[idata]

@ferrophile The current NCSDK (v 2.08.1) doesn't have support for slicing a tensor dimension like how you showed. The NCSDK currently requires index 1 and index 2 from the input tensor and sliced tensor to be equal.

[idata]

@Tome_at_Intel Thanks for the response. I realized that even if I got past the slicing problem, my network still wouldn't compile since I use batch norm layers after every deconvolution layer, which results in Error 5 again:

[Error 5] Toolkit Error: Stage Details Not Supported: FusedBatchNorm must be preceded by convolution or fully connected layer

Can I know if there is some official documentation on what rules I have to follow when using a particular layer in Tensorflow? I think I saw a list of supported layers somewhere in the forum, but apparently there are certain limitations on the layer arguments, dimensions etc.

[idata]

By the way, I also tried to compute the batch normalization process manually using stored parameters. NCSDK reported that it does not support the tf.sqrt operation:

var = tf.get_variable('batch_normalization/moving_variance', initializer=tf.zeros(192, tf.float32)) std = tf.sqrt(var + 0.001)

[Error 5] Toolkit Error: Stage Details Not Supported: Sqrt

Is tf.sqrt completely unsupported, or are there specific requirements regarding the input tensor?

[idata]

@ferrophile There are still some TensorFlow ops that the NCSDK doesn't have support for yet and it seems like we don't have support for the sqrt operation at the moment.

[idata]

@Tome_at_Intel

Thanks. I am trying to find a workaround solution using the ops supported right now, so I had a look at the release notes of the current version:

https://github.com/movidius/ncsdk/releases

For V1.12.01 of the SDK, "Crop" is a firmware feature according to the release notes. Could you tell me which Tensorflow op could I use to make use of this feature? (assuming this feature is not for Caffe only)