provide integer type BLAS routines?

Adam_S_5 — Tue, 14 Mar 2017 00:44:10 GMT

Hi,

Is there any plan to provide BLAS routines for integer types in MKL? GEMM is the prime example I'm thinking of.

There is a small but growing literature in deep learning on the use of reduced precision (relative to float32) to speed up these computations. A few manual implementations have been published using Intel vectorized integer math instructions and have demonstrated good computation speedup (which to me is the important part, rather than the memory savings). This seems like an excellent opportunity for Intel-MKL to lead. :) Would gladly use float16 if that was an option, but 8 bit values are likely to be useful as well, at least for fast inference.

Perhaps the main difficulty relative to floating point GEMM is handling overflow? Prior deep learning work has sorted out at least one reasonable solution which is to accumulate the answer in extended precision (e.g. int8 -> int16) and then let the user decide how to round back to the input precision, if desired. The risk of saturation in intermediate computation might just have to be accepted by the user?

Seems like otherwise the blocking and cache management would be the same as existing GEMM. But I'm no expert...is there something more fundamental which makes this a bad idea or rather difficult to implement relative to floating point?

Thoughts?

(I could re-post in MKL-DNN git? But since GEMM is proprietary in MKL I would imagine that so would integer GEMM.)

Thank you,

Adam

Great question ... please

Shane_S_Intel — Tue, 14 Mar 2017 01:00:17 GMT

Great question ... please take a look at this recent presentation (www.netlib.org/utk/people/JackDongara/WEB-PAGES/Batched-BLAS-2017/talk12-gurney.pdf) from the February 2017 Workshop on Batched, Reproducible, and Reduced Precision BLAS - main site located here:(www.netlib.org/utk/people/JackDongarra/WEB-PAGES/Batched-BLAS-2017/).

Wow! Great to see a lot of

Adam_S_5 — Tue, 14 Mar 2017 03:23:20 GMT

Wow! Great to see a lot of work is already going into this. Looking forward to MKL 2018!!

Adam, we will post

Gennady_F_Intel — Tue, 14 Mar 2017 12:21:34 GMT

Adam, we will post announsment when MKL 2018 beta will be available. Then you may take it and evaluate how it will work.

topic provide integer type BLAS routines? in Intel® oneAPI Math Kernel Library

provide integer type BLAS routines?

Great question ... please

Wow! Great to see a lot of

Adam, we will post