How to convert YOLO v2 model by the Model Optimizer for TensorFlow?

Taro_Y_ · ‎09-27-2017

Hello.

I am trying to convert YOLOv2 network from TensorFlow model to IR using Model Optimizer for TendsorFlow.

What I'm trying to convert is https: //github.com/thtrieu/darkflow .
This is a Tensorflow implementation of YOLOv2 (originally is https://pjreddie.com/darknet/yolo/ ).

I'm trying to convert with ModelOptimizer using the .pb file exported by Tendorflow, but I get the following error.
Is there any way to solve it?

command:

python3.5 modeloptimizer/scripts/model_optimizer.py --input_model=yolo.pb --input=input --output=output --transforms="calc_shapes(input_types=float; input_shapes=1,608,608,3) create_ir(model_name=yolo; output_dir=./)"

log:

2017-09-28 12:52:46.401808: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1050 Ti, pci bus id: 0000:01:00.0)

List is: [('calc_shapes', {'input_types': 'float', 'input_shapes': '1,608,608,3'}), ('create_ir', {'model_name': 'yolo', 'output_dir': './'})]

Xml file is /opt/intel/deeplearning_deploymenttoolkit_2017.1.0.4463/deployment_tools/model_optimizer/ModelOptimizerForTensorFlow/config.xml

Run command is: /opt/intel/deeplearning_deploymenttoolkit_2017.1.0.4463/deployment_tools/model_optimizer/ModelOptimizerForTensorFlow/bin/new_transform_graph --in_graph=/tmp/graph_transphorm_calc_shapes_input.pb --out_graph=/tmp/graph_transphorm_calc_shapes_output.pb --inputs=input --outputs=output --transforms="calc_shapes(input_types=\"float\", input_shapes=\"1,608,608,3\")"

Convert_to_IR transform uses FP32 precision as default

Convert_to_IR transform uses scale to be 1 as default

Convert_to_IR transform uses 1.0 norm_factor as default

Convert_to_IR transform uses 1 batch_size as default

Convert_to_IR transform uses RGB input_format as default

Loading yolo model from file... Done (0.000 sec)

Deploying yolo model into IR format

Target precision : FP32

Batch size : 1

Artifacts path : ./

Restoring shapes and sizes... Done (0.006 sec)

Translating tensorflowutils graph... Done (0.306 sec)

Reducing unused nodes

Assign (matches: 0)

Noop (matches: 3)

Restore Slices (matches: 0)

Truncated Normal (matches: 0)

Const (matches: 159)

Variable (matches: 0)

Save Slices (matches: 0)

Identity (matches: 2)

Done (0.032 sec)

Aggregating to IR nodes

input (matches: 1)

MeanValues (matches: 22)

Convolution (matches: 0)

Convolution (matches: 1)

Convolution (matches: 22)

ReLU (matches: 0)

Pooling (matches: 5)

Pooling (matches: 0)

FullyConnected (matches: 0)

BatchNorm (matches: 0)

Concat (matches: 0)

Concat (matches: 1)

Reshape (matches: 0)

StridedSlice (matches: 0)

SoftMax (matches: 0)

Done (1.219 sec)

Validating IR graph...

IR graph validation FAILED for the following nodes

id: 4 node_name: Pad node_type: op type: Pad

id: 5 node_name: Pad:0 node_type: tensor type:

id: 16 node_name: truediv node_type: op type: RealDiv

id: 17 node_name: truediv:0 node_type: tensor type:

id: 20 node_name: mul node_type: op type: Mul

id: 21 node_name: mul:0 node_type: tensor type:

id: 24 node_name: BiasAdd node_type: op type: BiasAdd

id: 25 node_name: BiasAdd:0 node_type: tensor type:

id: 28 node_name: mul_1 node_type: op type: Mul

id: 29 node_name: mul_1:0 node_type: tensor type:

id: 30 node_name: 1-leaky node_type: op type: Maximum

id: 31 node_name: 1-leaky:0 node_type: tensor type:

id: 36 node_name: Pad_1 node_type: op type: Pad

id: 37 node_name: Pad_1:0 node_type: tensor type:

id: 48 node_name: truediv_1 node_type: op type: RealDiv

id: 49 node_name: truediv_1:0 node_type: tensor type:

id: 52 node_name: mul_2 node_type: op type: Mul

id: 53 node_name: mul_2:0 node_type: tensor type:

id: 56 node_name: BiasAdd_1 node_type: op type: BiasAdd

id: 57 node_name: BiasAdd_1:0 node_type: tensor type:

id: 60 node_name: mul_3 node_type: op type: Mul

id: 61 node_name: mul_3:0 node_type: tensor type:

id: 62 node_name: 4-leaky node_type: op type: Maximum

id: 63 node_name: 4-leaky:0 node_type: tensor type:

id: 68 node_name: Pad_2 node_type: op type: Pad

id: 69 node_name: Pad_2:0 node_type: tensor type:

id: 80 node_name: truediv_2 node_type: op type: RealDiv

id: 81 node_name: truediv_2:0 node_type: tensor type:

id: 84 node_name: mul_4 node_type: op type: Mul

id: 85 node_name: mul_4:0 node_type: tensor type:

id: 88 node_name: BiasAdd_2 node_type: op type: BiasAdd

id: 89 node_name: BiasAdd_2:0 node_type: tensor type:

id: 92 node_name: mul_5 node_type: op type: Mul

id: 93 node_name: mul_5:0 node_type: tensor type:

id: 94 node_name: 7-leaky node_type: op type: Maximum

id: 95 node_name: 7-leaky:0 node_type: tensor type:

id: 98 node_name: Pad_3 node_type: op type: Pad

id: 99 node_name: Pad_3:0 node_type: tensor type:

id: 110 node_name: truediv_3 node_type: op type: RealDiv

id: 111 node_name: truediv_3:0 node_type: tensor type:

id: 114 node_name: mul_6 node_type: op type: Mul

id: 115 node_name: mul_6:0 node_type: tensor type:

id: 118 node_name: BiasAdd_3 node_type: op type: BiasAdd

id: 119 node_name: BiasAdd_3:0 node_type: tensor type:

id: 122 node_name: mul_7 node_type: op type: Mul

id: 123 node_name: mul_7:0 node_type: tensor type:

id: 124 node_name: 9-leaky node_type: op type: Maximum

id: 125 node_name: 9-leaky:0 node_type: tensor type:

id: 128 node_name: Pad_4 node_type: op type: Pad

id: 129 node_name: Pad_4:0 node_type: tensor type:

id: 140 node_name: truediv_4 node_type: op type: RealDiv

id: 141 node_name: truediv_4:0 node_type: tensor type:

id: 144 node_name: mul_8 node_type: op type: Mul

id: 145 node_name: mul_8:0 node_type: tensor type:

id: 148 node_name: BiasAdd_4 node_type: op type: BiasAdd

id: 149 node_name: BiasAdd_4:0 node_type: tensor type:

id: 152 node_name: mul_9 node_type: op type: Mul

id: 153 node_name: mul_9:0 node_type: tensor type:

id: 154 node_name: 11-leaky node_type: op type: Maximum

id: 155 node_name: 11-leaky:0 node_type: tensor type:

id: 160 node_name: Pad_5 node_type: op type: Pad

id: 161 node_name: Pad_5:0 node_type: tensor type:

id: 172 node_name: truediv_5 node_type: op type: RealDiv

id: 173 node_name: truediv_5:0 node_type: tensor type:

id: 176 node_name: mul_10 node_type: op type: Mul

id: 177 node_name: mul_10:0 node_type: tensor type:

id: 180 node_name: BiasAdd_5 node_type: op type: BiasAdd

id: 181 node_name: BiasAdd_5:0 node_type: tensor type:

id: 184 node_name: mul_11 node_type: op type: Mul

id: 185 node_name: mul_11:0 node_type: tensor type:

id: 186 node_name: 14-leaky node_type: op type: Maximum

id: 187 node_name: 14-leaky:0 node_type: tensor type:

id: 190 node_name: Pad_6 node_type: op type: Pad

id: 191 node_name: Pad_6:0 node_type: tensor type:

id: 202 node_name: truediv_6 node_type: op type: RealDiv

id: 203 node_name: truediv_6:0 node_type: tensor type:

id: 206 node_name: mul_12 node_type: op type: Mul

id: 207 node_name: mul_12:0 node_type: tensor type:

id: 210 node_name: BiasAdd_6 node_type: op type: BiasAdd

id: 211 node_name: BiasAdd_6:0 node_type: tensor type:

id: 214 node_name: mul_13 node_type: op type: Mul

id: 215 node_name: mul_13:0 node_type: tensor type:

id: 216 node_name: 16-leaky node_type: op type: Maximum

id: 217 node_name: 16-leaky:0 node_type: tensor type:

id: 220 node_name: Pad_7 node_type: op type: Pad

id: 221 node_name: Pad_7:0 node_type: tensor type:

id: 232 node_name: truediv_7 node_type: op type: RealDiv

id: 233 node_name: truediv_7:0 node_type: tensor type:

id: 236 node_name: mul_14 node_type: op type: Mul

id: 237 node_name: mul_14:0 node_type: tensor type:

id: 240 node_name: BiasAdd_7 node_type: op type: BiasAdd

id: 241 node_name: BiasAdd_7:0 node_type: tensor type:

id: 244 node_name: mul_15 node_type: op type: Mul

id: 245 node_name: mul_15:0 node_type: tensor type:

id: 246 node_name: 18-leaky node_type: op type: Maximum

id: 247 node_name: 18-leaky:0 node_type: tensor type:

id: 252 node_name: Pad_8 node_type: op type: Pad

id: 253 node_name: Pad_8:0 node_type: tensor type:

id: 264 node_name: truediv_8 node_type: op type: RealDiv

id: 265 node_name: truediv_8:0 node_type: tensor type:

id: 268 node_name: mul_16 node_type: op type: Mul

id: 269 node_name: mul_16:0 node_type: tensor type:

id: 272 node_name: BiasAdd_8 node_type: op type: BiasAdd

id: 273 node_name: BiasAdd_8:0 node_type: tensor type:

id: 276 node_name: mul_17 node_type: op type: Mul

id: 277 node_name: mul_17:0 node_type: tensor type:

id: 278 node_name: 21-leaky node_type: op type: Maximum

id: 279 node_name: 21-leaky:0 node_type: tensor type:

id: 282 node_name: Pad_9 node_type: op type: Pad

id: 283 node_name: Pad_9:0 node_type: tensor type:

id: 294 node_name: truediv_9 node_type: op type: RealDiv

id: 295 node_name: truediv_9:0 node_type: tensor type:

id: 298 node_name: mul_18 node_type: op type: Mul

id: 299 node_name: mul_18:0 node_type: tensor type:

id: 302 node_name: BiasAdd_9 node_type: op type: BiasAdd

id: 303 node_name: BiasAdd_9:0 node_type: tensor type:

id: 306 node_name: mul_19 node_type: op type: Mul

id: 307 node_name: mul_19:0 node_type: tensor type:

id: 308 node_name: 23-leaky node_type: op type: Maximum

id: 309 node_name: 23-leaky:0 node_type: tensor type:

id: 312 node_name: Pad_10 node_type: op type: Pad

id: 313 node_name: Pad_10:0 node_type: tensor type:

id: 324 node_name: truediv_10 node_type: op type: RealDiv

id: 325 node_name: truediv_10:0 node_type: tensor type:

id: 328 node_name: mul_20 node_type: op type: Mul

id: 329 node_name: mul_20:0 node_type: tensor type:

id: 332 node_name: BiasAdd_10 node_type: op type: BiasAdd

id: 333 node_name: BiasAdd_10:0 node_type: tensor type:

id: 336 node_name: mul_21 node_type: op type: Mul

id: 337 node_name: mul_21:0 node_type: tensor type:

id: 338 node_name: 25-leaky node_type: op type: Maximum

id: 339 node_name: 25-leaky:0 node_type: tensor type:

id: 342 node_name: Pad_11 node_type: op type: Pad

id: 343 node_name: Pad_11:0 node_type: tensor type:

id: 354 node_name: truediv_11 node_type: op type: RealDiv

id: 355 node_name: truediv_11:0 node_type: tensor type:

id: 358 node_name: mul_22 node_type: op type: Mul

id: 359 node_name: mul_22:0 node_type: tensor type:

id: 362 node_name: BiasAdd_11 node_type: op type: BiasAdd

id: 363 node_name: BiasAdd_11:0 node_type: tensor type:

id: 366 node_name: mul_23 node_type: op type: Mul

id: 367 node_name: mul_23:0 node_type: tensor type:

id: 368 node_name: 27-leaky node_type: op type: Maximum

id: 369 node_name: 27-leaky:0 node_type: tensor type:

id: 372 node_name: Pad_12 node_type: op type: Pad

id: 373 node_name: Pad_12:0 node_type: tensor type:

id: 384 node_name: truediv_12 node_type: op type: RealDiv

id: 385 node_name: truediv_12:0 node_type: tensor type:

id: 388 node_name: mul_24 node_type: op type: Mul

id: 389 node_name: mul_24:0 node_type: tensor type:

id: 392 node_name: BiasAdd_12 node_type: op type: BiasAdd

id: 393 node_name: BiasAdd_12:0 node_type: tensor type:

id: 396 node_name: mul_25 node_type: op type: Mul

id: 397 node_name: mul_25:0 node_type: tensor type:

id: 398 node_name: 29-leaky node_type: op type: Maximum

id: 399 node_name: 29-leaky:0 node_type: tensor type:

id: 404 node_name: Pad_13 node_type: op type: Pad

id: 405 node_name: Pad_13:0 node_type: tensor type:

id: 416 node_name: truediv_13 node_type: op type: RealDiv

id: 417 node_name: truediv_13:0 node_type: tensor type:

id: 420 node_name: mul_26 node_type: op type: Mul

id: 421 node_name: mul_26:0 node_type: tensor type:

id: 424 node_name: BiasAdd_13 node_type: op type: BiasAdd

id: 425 node_name: BiasAdd_13:0 node_type: tensor type:

id: 428 node_name: mul_27 node_type: op type: Mul

id: 429 node_name: mul_27:0 node_type: tensor type:

id: 430 node_name: 32-leaky node_type: op type: Maximum

id: 431 node_name: 32-leaky:0 node_type: tensor type:

id: 434 node_name: Pad_14 node_type: op type: Pad

id: 435 node_name: Pad_14:0 node_type: tensor type:

id: 446 node_name: truediv_14 node_type: op type: RealDiv

id: 447 node_name: truediv_14:0 node_type: tensor type:

id: 450 node_name: mul_28 node_type: op type: Mul

id: 451 node_name: mul_28:0 node_type: tensor type:

id: 454 node_name: BiasAdd_14 node_type: op type: BiasAdd

id: 455 node_name: BiasAdd_14:0 node_type: tensor type:

id: 458 node_name: mul_29 node_type: op type: Mul

id: 459 node_name: mul_29:0 node_type: tensor type:

id: 460 node_name: 34-leaky node_type: op type: Maximum

id: 461 node_name: 34-leaky:0 node_type: tensor type:

id: 464 node_name: Pad_15 node_type: op type: Pad

id: 465 node_name: Pad_15:0 node_type: tensor type:

id: 476 node_name: truediv_15 node_type: op type: RealDiv

id: 477 node_name: truediv_15:0 node_type: tensor type:

id: 480 node_name: mul_30 node_type: op type: Mul

id: 481 node_name: mul_30:0 node_type: tensor type:

id: 484 node_name: BiasAdd_15 node_type: op type: BiasAdd

id: 485 node_name: BiasAdd_15:0 node_type: tensor type:

id: 488 node_name: mul_31 node_type: op type: Mul

id: 489 node_name: mul_31:0 node_type: tensor type:

id: 490 node_name: 36-leaky node_type: op type: Maximum

id: 491 node_name: 36-leaky:0 node_type: tensor type:

id: 494 node_name: Pad_16 node_type: op type: Pad

id: 495 node_name: Pad_16:0 node_type: tensor type:

id: 506 node_name: truediv_16 node_type: op type: RealDiv

id: 507 node_name: truediv_16:0 node_type: tensor type:

id: 510 node_name: mul_32 node_type: op type: Mul

id: 511 node_name: mul_32:0 node_type: tensor type:

id: 514 node_name: BiasAdd_16 node_type: op type: BiasAdd

id: 515 node_name: BiasAdd_16:0 node_type: tensor type:

id: 518 node_name: mul_33 node_type: op type: Mul

id: 519 node_name: mul_33:0 node_type: tensor type:

id: 520 node_name: 38-leaky node_type: op type: Maximum

id: 521 node_name: 38-leaky:0 node_type: tensor type:

id: 524 node_name: Pad_17 node_type: op type: Pad

id: 525 node_name: Pad_17:0 node_type: tensor type:

id: 536 node_name: truediv_17 node_type: op type: RealDiv

id: 537 node_name: truediv_17:0 node_type: tensor type:

id: 540 node_name: mul_34 node_type: op type: Mul

id: 541 node_name: mul_34:0 node_type: tensor type:

id: 544 node_name: BiasAdd_17 node_type: op type: BiasAdd

id: 545 node_name: BiasAdd_17:0 node_type: tensor type:

id: 548 node_name: mul_35 node_type: op type: Mul

id: 549 node_name: mul_35:0 node_type: tensor type:

id: 550 node_name: 40-leaky node_type: op type: Maximum

id: 551 node_name: 40-leaky:0 node_type: tensor type:

id: 554 node_name: Pad_18 node_type: op type: Pad

id: 555 node_name: Pad_18:0 node_type: tensor type:

id: 566 node_name: truediv_18 node_type: op type: RealDiv

id: 567 node_name: truediv_18:0 node_type: tensor type:

id: 570 node_name: mul_36 node_type: op type: Mul

id: 571 node_name: mul_36:0 node_type: tensor type:

id: 574 node_name: BiasAdd_18 node_type: op type: BiasAdd

id: 575 node_name: BiasAdd_18:0 node_type: tensor type:

id: 578 node_name: mul_37 node_type: op type: Mul

id: 579 node_name: mul_37:0 node_type: tensor type:

id: 580 node_name: 42-leaky node_type: op type: Maximum

id: 581 node_name: 42-leaky:0 node_type: tensor type:

id: 584 node_name: Pad_19 node_type: op type: Pad

id: 585 node_name: Pad_19:0 node_type: tensor type:

id: 596 node_name: truediv_19 node_type: op type: RealDiv

id: 597 node_name: truediv_19:0 node_type: tensor type:

id: 600 node_name: mul_38 node_type: op type: Mul

id: 601 node_name: mul_38:0 node_type: tensor type:

id: 604 node_name: BiasAdd_19 node_type: op type: BiasAdd

id: 605 node_name: BiasAdd_19:0 node_type: tensor type:

id: 608 node_name: mul_39 node_type: op type: Mul

id: 609 node_name: mul_39:0 node_type: tensor type:

id: 610 node_name: 44-leaky node_type: op type: Maximum

id: 611 node_name: 44-leaky:0 node_type: tensor type:

id: 618 node_name: Pad_20 node_type: op type: Pad

id: 619 node_name: Pad_20:0 node_type: tensor type:

id: 630 node_name: truediv_20 node_type: op type: RealDiv

id: 631 node_name: truediv_20:0 node_type: tensor type:

id: 634 node_name: mul_40 node_type: op type: Mul

id: 635 node_name: mul_40:0 node_type: tensor type:

id: 638 node_name: BiasAdd_20 node_type: op type: BiasAdd

id: 639 node_name: BiasAdd_20:0 node_type: tensor type:

id: 642 node_name: mul_41 node_type: op type: Mul

id: 643 node_name: mul_41:0 node_type: tensor type:

id: 644 node_name: 47-leaky node_type: op type: Maximum

id: 645 node_name: 47-leaky:0 node_type: tensor type:

id: 646 node_name: ExtractImagePatches node_type: op type: ExtractImagePatches

id: 647 node_name: ExtractImagePatches:0 node_type: tensor type:

id: 654 node_name: Pad_21 node_type: op type: Pad

id: 655 node_name: Pad_21:0 node_type: tensor type:

id: 666 node_name: truediv_21 node_type: op type: RealDiv

id: 667 node_name: truediv_21:0 node_type: tensor type:

id: 670 node_name: mul_42 node_type: op type: Mul

id: 671 node_name: mul_42:0 node_type: tensor type:

id: 674 node_name: BiasAdd_21 node_type: op type: BiasAdd

id: 675 node_name: BiasAdd_21:0 node_type: tensor type:

id: 678 node_name: mul_43 node_type: op type: Mul

id: 679 node_name: mul_43:0 node_type: tensor type:

id: 680 node_name: 51-leaky node_type: op type: Maximum

id: 681 node_name: 51-leaky:0 node_type: tensor type:

id: 684 node_name: Pad_22 node_type: op type: Pad

id: 685 node_name: Pad_22:0 node_type: tensor type:

Something went wrong. Please check the error and try again:

Error: Graph validation failed

Failed to run transform create_ir

Failed to run transforms!!!

Best regards,
Taro

Anna_B_Intel · ‎10-03-2017

Hi Taro,

Currently Yolo2 is unsupported by Model Optimizer. Why is it critical for you to use this topology?

Best wishes,

Anna

Taro_Y_ · ‎10-03-2017

Hi Anna,

Thank you for your reply.

It is the reason I want to use it, I would like to improve recognition accuracy of small objects.
I think that it is effective to increase the input size of model in order to improve the recognition accuracy of small objects (objects far from the camera).
The input size of the SSD is 300x300 or 500x500, but the input size of Yolo v2 is 608 x608, so i think it will be hopeful.

Also, Yolo v2 is claimed to be executable at a speed(FPS) close to SSD 300x300, faster than SSD 500x500, according to its official page ( https://pjreddie.com/darknet/yolo/ ).

I am currentry running the SSD 300x300 on the Intel HD Graphics GPU, the speed was about 4 fps and it was close to the lower limit speed of my application.
So, I believe that I can not use the SSD 500x500 in terms of speed(FPS).

Therefore, I hope to use Yolo v2 which is faster than SSD and has a larger input size.

Can I use Yolo v2 by other means, such as using OpenVX without using Model Optimiser?

Best regards,
Taro

Nineteen__Seleven · ‎10-29-2017

Hi

I met similar problem converting resnetv1_50 model:

Validating IR graph...
IR graph validation FAILED for the following nodes
   id: 4 node_name: resnet_v1_50/Pad node_type: op type: Pad
   id: 5 node_name: resnet_v1_50/Pad:0 node_type: tensor type:
   id: 30 node_name: resnet_v1_50/conv1/BatchNorm/batchnorm/add node_type: op type: Add
   id: 31 node_name: resnet_v1_50/conv1/BatchNorm/batchnorm/add:0 node_type: tensor type:
   id: 32 node_name: resnet_v1_50/conv1/BatchNorm/batchnorm/Rsqrt node_type: op type: Rsqrt
   id: 33 node_name: resnet_v1_50/conv1/BatchNorm/batchnorm/Rsqrt:0 node_type: tensor type:
   id: 34 node_name: resnet_v1_50/conv1/BatchNorm/batchnorm/mul node_type: op type: Mul
   id: 35 node_name: resnet_v1_50/conv1/BatchNorm/batchnorm/mul:0 node_type: tensor type:
   id: 36 node_name: resnet_v1_50/conv1/BatchNorm/batchnorm/mul_1 node_type: op type: Mul
   id: 37 node_name: resnet_v1_50/conv1/BatchNorm/batchnorm/mul_1:0 node_type: tensor type:
   id: 38 node_name: resnet_v1_50/conv1/BatchNorm/batchnorm/mul_2 node_type: op type: Mul
   id: 39 node_name: resnet_v1_50/conv1/BatchNorm/batchnorm/mul_2:0 node_type: tensor type:
   id: 42 node_name: resnet_v1_50/conv1/BatchNorm/batchnorm/add_1 node_type: op type: Add
   id: 43 node_name: resnet_v1_50/conv1/BatchNorm/batchnorm/add_1:0 node_type: tensor type:
   id: 72 node_name: resnet_v1_50/block1/unit_1/bottleneck_v1/shortcut/BatchNorm/batchnorm/add node_type: op type: Add
   id: 73 node_name: resnet_v1_50/block1/unit_1/bottleneck_v1/shortcut/BatchNorm/batchnorm/add:0 node_type: tensor type:
   id: 74 node_name: resnet_v1_50/block1/unit_1/bottleneck_v1/shortcut/BatchNorm/batchnorm/Rsqrt node_type: op type: Rsqrt
   id: 75 node_name: resnet_v1_50/block1/unit_1/bottleneck_v1/shortcut/BatchNorm/batchnorm/Rsqrt:0 node_type: tensor type:

...

I do something wrong? While resnets are supported by Model Optimiser