- Marcar como novo
- Marcador
- Subscrever
- Silenciar
- Subscrever fonte RSS
- Destacar
- Imprimir
- Denunciar conteúdo inapropriado
Hi all, hope you could help me on this.
I have tried to run mobilenetv1 with dynamic batch size but got the "RuntimeError: MKLDNNGraph::CreateGraph: such topology cannot be compiled for dynamic batch!" error. Properly due to the squeeze layer inside mobilenetv1 that changes the shape of the tensor.
So in the end, I've decided to create multiple models for each batch size and did some benchmarks. But I got weird latency/throughput behavior.
My program is pretty simple
images = np.random.uniform(-1,1, size=[64, 3, 224, 224]).astype(np.float32) for batch_size in range(1,16): #create model net = IENetwork(model=model_xml, weights=model_bin) net.batch_size = batch_size exec_net = plugin.load(network=net) input_blob = next(iter(net.inputs)) #run inference batch = images[np.arange(batch_size)] res = exec_net.infer(inputs={input_blob: batch})
However, after 4 iterations, openvino started using only 1 single CPU core instead of all my CPU cores (I'm using Intel(R) Xeon(R) Gold 6140).
Batch_size: 1, Throughput: 643.86 imgs/s Batch_size: 2, Throughput: 924.83 imgs/s Batch_size: 3, Throughput: 1064.74 imgs/s Batch_size: 4, Throughput: 1245.72 imgs/s Batch_size: 5, Throughput: 168.25 imgs/s Batch_size: 6, Throughput: 168.66 imgs/s
Do you have any suggestions to fix this problem?
Thank you
Link copiado
- Marcar como novo
- Marcador
- Subscrever
- Silenciar
- Subscrever fonte RSS
- Destacar
- Imprimir
- Denunciar conteúdo inapropriado
Hi Loc,
Based on the error message that you provided, it seems like your application has something to do with DNN and you are not using Intel(R) MKL.
This is the Intel(R) MKL forum. Let me see if I can find the correct forum for you to post your question.
Best regards,
Khang
- Marcar como novo
- Marcador
- Subscrever
- Silenciar
- Subscrever fonte RSS
- Destacar
- Imprimir
- Denunciar conteúdo inapropriado
Oops, sorry. I posted in a wrong forum. Should have posted into OpenVino forum.
- Marcar como novo
- Marcador
- Subscrever
- Silenciar
- Subscrever fonte RSS
- Destacar
- Imprimir
- Denunciar conteúdo inapropriado
Hi Loc,
No problem. Yes, all the questions relating to OpenVino should post over there.
Best regards,
Khang
- Marcar como novo
- Marcador
- Subscrever
- Silenciar
- Subscrever fonte RSS
- Destacar
- Imprimir
- Denunciar conteúdo inapropriado
Nguyen, Khang T (Intel) wrote:Hi Loc,
No problem. Yes, all the questions relating to OpenVino should post over there.
Best regards,
Khang
Can you help me delete this post? I'll repost to OpenVino forum.
Thanks a lot :)
- Marcar como novo
- Marcador
- Subscrever
- Silenciar
- Subscrever fonte RSS
- Destacar
- Imprimir
- Denunciar conteúdo inapropriado
the thread has been moved to the OpenVINO forum

- Subscrever fonte RSS
- Marcar tópico como novo
- Marcar tópico como lido
- Flutuar este Tópico para o utilizador atual
- Marcador
- Subscrever
- Página amigável para impressora