topic Re: I would like to know how to re-build caffe in devcloud, in order to train detectnet model with custom layer in Intel® Distribution for Python*

I would like to know how to re-build caffe in devcloud, in order to train detectnet model with custom layer

GMath7 — Tue, 11 Dec 2018 00:09:35 GMT

I tried to clone Intel dist. of caffe in devcloud and build after uncommenting support for python. But I am getting an error as in below. Could you please support us in building the same. We tried install version of protobuf

libprotoc: v3.6.0

protocol buffer: v3.6.1

.build_release/src/caffe/proto/caffe.pb.h:12:2: error: #error This file was generated by a newer version of protoc which is

#error This file was generated by a newer version of protoc which is

.build_release/src/caffe/proto/caffe.pb.h:13:2: error: #error incompatible with your Protocol Buffer headers. Please update

#error incompatible with your Protocol Buffer headers. Please update

.build_release/src/caffe/proto/caffe.pb.h:14:2: error: #error your headers.

#error your headers.

.build_release/src/caffe/proto/caffe.pb.h:22:35: fatal error: google/protobuf/arena.h: 没有那个文件或目录

#include <google/protobuf/arena.h>

Re: I would like to know how to re-build caffe in devcloud, in order to train detectnet model with custom layer

Rajeswari_P_Intel — Thu, 13 Dec 2018 17:32:14 GMT

Hi Gina, You need to update Makefile to use conda environment paths, below is the link for reference. https://gist.github.com/arundasan91/b432cb011d1c45b65222d0fac5f9232c Comparison between sample and updated with Anacona config. Please change as per your user account paths and try to rebuild. [u14677@c002 caffe]$ diff Makefile.config.example Makefile.config 143c143 < --- > # 146,147c146,147 < PYTHON_INCLUDE := /usr/include/python2.7 \ < /usr/lib/python2.7/dist-packages/numpy/core/include --- > #PYTHON_INCLUDE := /usr/include/python2.7 \ > #/usr/lib/python2.7/dist-packages/numpy/core/include 150,153c150,153 < # ANACONDA_HOME := $(HOME)/anaconda < # PYTHON_INCLUDE := $(ANACONDA_HOME)/include \ < # $(ANACONDA_HOME)/include/python2.7 \ < # $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include \ --- > ANACONDA_HOME := /home/u14677/.conda/envs/caffe_build/ > PYTHON_INCLUDE := $(ANACONDA_HOME)/include \ > $(ANACONDA_HOME)/include/python3.6 \ > $(ANACONDA_HOME)/lib/python3.6/site-packages/numpy/core/include \ 158c158 < # /usr/lib/python3.5/dist-packages/numpy/core/include --- > #/usr/lib/python3.5/dist-packages/numpy/core/include 161,162c161,162 < PYTHON_LIB := /usr/lib < # PYTHON_LIB := $(ANACONDA_HOME)/lib --- > #PYTHON_LIB := /usr/lib > PYTHON_LIB := $(ANACONDA_HOME)/lib [u14677@c002 caffe]$ Thanks, Rajeswari Ponnuru.

Re: I would like to know how to re-build caffe in devcloud, in order to train detectnet model with custom layer

Rajeswari_P_Intel — Thu, 13 Dec 2018 18:17:49 GMT

Hi Gina, Please install protobuf 3.2.0 using conda. conda install -c https://conda.anaconda.org/anaconda protobuf==3.2.0 Thanks, Rajeswari Ponnuru.

Re: I would like to know how to re-build caffe in devcloud, in order to train detectnet model with custom layer

Rajeswari_P_Intel — Mon, 17 Dec 2018 12:06:24 GMT

Hi Gina, Can you please update the status of the installation after following the mentioned steps. Assuming it is successful after following those steps. Thanks, Rajeswari Ponnuru.

Re: I would like to know how to re-build caffe in devcloud, in order to train detectnet model with custom layer

Rajeswari_P_Intel — Tue, 18 Dec 2018 15:06:51 GMT

Hi Gina, As discussed in mail thread, please make sure you have installed cmake 2.8.11, If you still face issues with protoc, try installing protoc 3.3.0 version. Thanks, Rajeswari Ponnuru

Re: I would like to know how to re-build caffe in devcloud, in order to train detectnet model with custom layer

Rajeswari_P_Intel — Wed, 19 Dec 2018 20:36:02 GMT

Hi Gina, Once you create new conda environment, please follow the below steps, I have executed second time and it was successful for me. I have placed Makefile.config in /tmp. You please copy asap otherwise it will be deleted. Commands: git clone https://github.com/intel/caffe conda install -c https://conda.anaconda.org/anaconda protobuf==3.2.0 conda install -c xavier cmake export PATH=/usr/bin/:$PATH Output: CXX/LD -o .build_release/examples/mnist/convert_mnist_data.bin CXX examples/siamese/convert_mnist_siamese_data.cpp CXX/LD -o .build_release/examples/siamese/convert_mnist_siamese_data.bin CXX examples/ssd/ssd_detect.cpp CXX/LD -o .build_release/examples/ssd/ssd_detect.bin (caffe_build) [u5653@c009-n023 caffe]$ cmake --version cmake version 2.8.11.1 (caffe_build) [u5653@c009-n023 caffe]$ which cmake ~/.conda/envs/caffe_build/bin/cmake (caffe_build) [u5653@c009-n023 caffe]$ gcc --version gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-16) Copyright (C) 2015 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. (caffe_build) [u5653@c009-n023 caffe]$ which gcc /usr/bin/gcc (caffe_build) [u5653@c009-n023 caffe]$ (caffe_build) [u5653@c009-n023 caffe]$ protoc --version libprotoc 2.5.0 (caffe_build) [u5653@c009-n023 caffe]$ Thanks, Rajeswari Ponnuru

Re: I would like to know how to re-build caffe in devcloud, in order to train detectnet model with custom layer

Rajeswari_P_Intel — Thu, 20 Dec 2018 18:23:43 GMT

Hi Gina, Sure, please try tomorrow. We will try to close this issue by end of tomorrow as it is opened from considerable number of days. If it fails then we will have a call tomorrow, let me know your convenient time tomorrow afternoon. Thanks, Rajeswari Ponnuru. From: Gina Mathew <ginammathew@gmail.com> Sent: 20 December 2018 13:26 To: Ponnuru, RajeswariX <rajeswarix.ponnuru@intel.com> Subject: Re: I would like to know how to re-build caffe in devcloud, in order to train detectnet model with custom layer Hi Rajeswari, I was able to copy Makefile.config. Thank you for your support. I will give you an update tomorrow after re-building. Today I am a little busy with other work. So I may not be able to check re-build today. regards, Gina On Thu, Dec 20, 2018 at 1:14 PM Ponnuru, RajeswariX <rajeswarix.ponnuru@intel.com> wrote: Hi Gina, If re-build is not successful after taking new Makefile.config, let us have a skype call around 3:30 PM today. Let me know if it is convenient for you, I will schedule the meeting notice. We can resolve the issue online. Thanks, Rajeswari Ponnuru.

Re: I would like to know how to re-build caffe in devcloud, in order to train detectnet model with custom layer

Rajeswari_P_Intel — Fri, 21 Dec 2018 20:46:15 GMT

Hi Gina, Closing this thread as the issue is resolved. Attached screen shot of "make all". Thanks, Rajeswari Ponnuru.