- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

Hi all,

I am trying to implement an autoencoder architecture on Tensorflow.

Specifically, the tensor

`Z`

is a compressed representation of the original input, with size (1, 1, 1, 192). I am trying to apply a deconvolution layer for multiple times to scale back up to the input size (1, 1024, 32, 1).The first deconvolution layer would be this:

```
def Decoder_test_same(self, Z):
with tf.variable_scope("Decoder") as scope:
conv1 = tf.layers.conv2d_transpose(Z, 192, 3, strides=2, padding='same', activation=tf.nn.relu)
return conv1
```

which uses the 'SAME' padding so the dimension of `conv1`

will be (1, 2, 2, 192). This is followed by another 8 deconvolution layers with different kernel sizes / no. of channels.

Unfortunately it seems that NCSDK does not support 'SAME' padding:

`[Error 5] Toolkit Error: Stage Details Not Supported: Wrong deconvolution output shape.`

After looking up for solutions on the internet I decided to use the 'VALID' padding instead. This should give me a tensor with size (1, 3, 3, 192). By removing the last row and column, I should get the exact same (1, 2, 2 ,192) tensor as previously mentioned.

The new code now looks like this:

```
def Decoder_test_valid(self, Z):
with tf.variable_scope("Decoder") as scope:
conv1 = tf.layers.conv2d_transpose(Z, 192, 3, strides=2, padding='valid', activation=tf.nn.relu)
conv2 = tf.slice(conv1, [0, 0, 0, 0], [1, 2, 2, 192])
return conv2
```

mvNCProfile can now handle the deconvolution layer, but it now complains that the Slice operation is not supported:

`[Error 5] Toolkit Error: Stage Details Not Supported: Slice type not supported`

Is the `tf.slice`

op completely unsupported, or are there some restrictions on the input that I failed to observe? If slicing is indeed not supported, is there some alternative where I can obtain the subset of a tensor? I have also tried `tf.split`

but that resulted in Error 5 as well.

Thanks in advance

- Tags:
- Tensorflow

Link Copied

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

@ferrophile The current NCSDK (v 2.08.1) doesn't have support for slicing a tensor dimension like how you showed. The NCSDK currently requires index 1 and index 2 from the input tensor and sliced tensor to be equal.

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

@Tome_at_Intel Thanks for the response. I realized that even if I got past the slicing problem, my network still wouldn't compile since I use batch norm layers after every deconvolution layer, which results in Error 5 again:

`[Error 5] Toolkit Error: Stage Details Not Supported: FusedBatchNorm must be preceded by convolution or fully connected layer`

Can I know if there is some official documentation on what rules I have to follow when using a particular layer in Tensorflow? I think I saw a list of supported layers somewhere in the forum, but apparently there are certain limitations on the layer arguments, dimensions etc.

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

By the way, I also tried to compute the batch normalization process manually using stored parameters. NCSDK reported that it does not support the `tf.sqrt`

operation:

```
var = tf.get_variable('batch_normalization/moving_variance', initializer=tf.zeros(192, tf.float32))
std = tf.sqrt(var + 0.001)
```

`[Error 5] Toolkit Error: Stage Details Not Supported: Sqrt`

Is `tf.sqrt`

completely unsupported, or are there specific requirements regarding the input tensor?

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

@ferrophile There are still some TensorFlow ops that the NCSDK doesn't have support for yet and it seems like we don't have support for the sqrt operation at the moment.

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

@Tome_at_Intel

Thanks. I am trying to find a workaround solution using the ops supported right now, so I had a look at the release notes of the current version:

https://github.com/movidius/ncsdk/releases

For V1.12.01 of the SDK, "Crop" is a firmware feature according to the release notes. Could you tell me which Tensorflow op could I use to make use of this feature? (assuming this feature is not for Caffe only)

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page