float16 vs float32

[idata]

Hi there,

I just started experimenting with the movidius stick and its ncsdk. I read somewhere that the stick supports both 32-bit and 16-bit floating point calculation. It is however not clear to me how this can be configured using the ncsdk. I suspect it is currently only using 16-bit floats and that this is not configurable at the time. Can someone confirm this or am I wrong? And also, if 32-bit is not supported, will it be in the future, or is this deliberately ignored?

I'm using the current master branch of the ncsdk.

[idata]

@maartenvds Thanks for your interest in the NCS. To answer your question, the NCS was designed to use 16 bit floats for power, efficiency and precision reasons. Currently we have no plans to support 32 bit floating point data at the moment, however this doesn't mean that you can't use 32 bit floating point data. The typical workflow is to use 32 bit floats and convert the tensor to 16 bit values before sending the data to the NCS for inference processing.

If you are interested, you can check out the ncappzoo @ https://github.com/movidius/ncappzoo and you can see Python and C++ examples where this conversion is performed. Image-classifier is a good one to start off with (https://github.com/movidius/ncappzoo/blob/master/apps/image-classifier/image-classifier.py). Check line 61 of the python script.

[idata]

Thanks for the detailed answer :smile: