https://community.intel.com/t5/Intel-oneAPI-Math-Kernel-Library/why-MKL-deep-learning-much-slow-for-backpropagation/m-p/1097433#M23640 <P>according to baidu's benchmark (<A href="https://github.com/baidu-research/DeepBench">https://github.com/baidu-research/DeepBench</A>), mkl's deep learning with convolution (not gemm) has a much slower backward speed than the forward pass.</P> <P>for example (<A href="https://github.com/baidu-research/DeepBench/tree/master/results">https://github.com/baidu-research/DeepBench/tree/master/results</A>), for W=341, H=79,C=32,N=4, K=32, R=5, S=10, in KNL7250 platform, forward 0.91ms, backward with input is 68.79 ms, with weight is 74.98 ms! so backward is 68 times slower than forward.</P> <P>as a comparison, in titanx, forward is 0.74ms, backward with input is 3.09 ms, with weight is 0.76 ms. For forward, titanx is only a little faster than KNL7250, but for backward, KNL7250 is much slower. This is similar with other W,H,C configuration.</P> <P>can any one give me the reason? is it because mkl has not made much optimization for backward yet? it seems mkl-dnn (<A href="https://github.com/01org/mkl-dnn">https://github.com/01org/mkl-dnn</A>) only supports forward operations now.</P> Fri, 30 Sep 2016 17:57:14 GMT frank_z_1 2016-09-30T17:57:14Z https://community.intel.com/t5/Intel-oneAPI-Math-Kernel-Library/why-MKL-deep-learning-much-slow-for-backpropagation/m-p/1097433#M23640 <P>according to baidu's benchmark (<A href="https://github.com/baidu-research/DeepBench">https://github.com/baidu-research/DeepBench</A>), mkl's deep learning with convolution (not gemm) has a much slower backward speed than the forward pass.</P> <P>for example (<A href="https://github.com/baidu-research/DeepBench/tree/master/results">https://github.com/baidu-research/DeepBench/tree/master/results</A>), for W=341, H=79,C=32,N=4, K=32, R=5, S=10, in KNL7250 platform, forward 0.91ms, backward with input is 68.79 ms, with weight is 74.98 ms! so backward is 68 times slower than forward.</P> <P>as a comparison, in titanx, forward is 0.74ms, backward with input is 3.09 ms, with weight is 0.76 ms. For forward, titanx is only a little faster than KNL7250, but for backward, KNL7250 is much slower. This is similar with other W,H,C configuration.</P> <P>can any one give me the reason? is it because mkl has not made much optimization for backward yet? it seems mkl-dnn (<A href="https://github.com/01org/mkl-dnn">https://github.com/01org/mkl-dnn</A>) only supports forward operations now.</P> Fri, 30 Sep 2016 17:57:14 GMT https://community.intel.com/t5/Intel-oneAPI-Math-Kernel-Library/why-MKL-deep-learning-much-slow-for-backpropagation/m-p/1097433#M23640 frank_z_1 2016-09-30T17:57:14Z