AI Tools from Intel
Find answers to your toolkit installation, configuration, and get-started questions.

Tensorflow 2 optimization documentation (Python)

AdamMiltonBarker
5,007 Views

Hi guys, I am hoping to optimize an existing, or train a new Acute Lymphoblastic Leukemia classifier, could some direct me to documentation about training/optimizing models with oneAPI please.

0 Kudos
22 Replies
JananiC_Intel
Moderator
4,499 Views

Hi,

 

Thanks for posting in Intel forums.

 

Tensorflow framework has been optimized using  Intel oneAPI Deep Neural Network Library(Intel oneDNN) primitives and this Intel-optimized Tensorflow is available as a part of Intel AI Analytics Toolkit.We suggest you to refer the below links for tensorflow optimization.

https://github.com/IntelAI/models(step-by-step tutorial for open-source machine learning models optimized by Intel)and https://software.intel.com/content/www/us/en/develop/articles/intel-optimization-for-tensorflow-installation-guide.html

 

I have attached a webinar link below which has covered the latest optimizations for Intel® Optimization for TensorFlow

https://techdecoded.intel.io/essentials/ai-analytics-part-2-enhance-deep-learning-workloads-on-3rd-gen-intel-xeon-scalable-processors/?cid=em&source=elo&campid=iags_WW_iagstd2_EN_2020_8.19%20AI%20Pt2%20Webinar_C-MKA-16350_T-MKA-19410&content=iags_WW_iagstd2_EMPW_EN_2020_8.19%20AI%20Pt2%20WebinarTY_C-MKA-16350_T-MKA-19410&elq_cid=3195182&em_id=60738...

 

Hope this helps!

0 Kudos
AdamMiltonBarker
4,487 Views

Hi thanks for the reply, this was the document I was looking for:

https://github.com/IntelAI/models/blob/master/docs/image_recognition/tensorflow/Tutorial.md

Is there any features of oneAPI that can be used for improving training results or is it solely for improving inference times, was wondering if there was anything available to improve training results? This is the model I wanted to improve, I will also make an Inception and Resnet version. 

https://github.com/AMLResearchProject/ALL-Tensorflow-2020/tree/master/CNN 

The end goal is to use these as part of our network on Raspberry Pi & UP2 are they supported ?

0 Kudos
JananiC_Intel
Moderator
4,466 Views

Hi,

Thanks for the update.

 

Below are our suggestions for better results

 


1) Intel optimized tensorflow leverages the use of oneAPI libraries like oneDNN,oneMKL,oneCCL underthehood to use the full power of Intel architecture and yield high performance for training and inference. All the optimizations are integrated into the framework and the developer doesn't need to worry about these optimizations.

 

2)You can further enhance the performance of intel optimzed tensorflow by configuring certain parameters(inter_op_parallelism_threads,intra_op_parallelism_threads) and environment variables(OMP_NUM_THREADS,KMP_BLOCKTIME,KMP_AFFINITY) while running tensorflow. 

Refer the link for more information(https://github.com/IntelAI/models/blob/master/docs/general/tensorflow/GeneralBestPractices.md).

 

3)Transfer learning and Fine tuning can help you in saving training time and can provide better performance.You can refer this link for more details(https://www.tensorflow.org/guide/keras/transfer_learning)

 

4)Another suggestion is to increase epoch to increase accuracy.

 

And for your information,UP2 is supported but Raspberry pi is not supported since it is based on ARM.


0 Kudos
AdamMiltonBarker
4,458 Views

Thanks

- Regarding 1, what versions of Tensorflow are supported, UP2 only supports up to 1.4 is the highest I have got working properly. Re 2 thanks I had read that. 

- 2 Thanks I had read that

- 3 I know about transfer learning that isn't the goal for the project I listed but will be doing on based on Resnet and Inception V3 also.

-4 Ok brings me back to point 1 which versions of TF are supported as if TF1.4 min is not supported we can't use in our projects. 

Thanks for the info.

 

 

0 Kudos
AdamMiltonBarker
4,454 Views

Thinking about my last comment, I believe TF2 is supported by OpenVINO now, what I could do is train the model with optimized TF and convert it to IR for OpenVINO so it can run on RPI and UP2.

0 Kudos
AdamMiltonBarker
4,436 Views

To update, I have now trained the model using Optimized Tensorflow and Intel Distribution for Python, I receive an error I have never received before. 

--- Logging error ---
Traceback (most recent call last):
File "C:\Users\intel-vr-1\Anaconda3\lib\logging\handlers.py", line 70, in emit
self.doRollover()
File "C:\Users\intel-vr-1\Anaconda3\lib\logging\handlers.py", line 394, in doRollover
self.rotate(self.baseFilename, dfn)
File "C:\Users\intel-vr-1\Anaconda3\lib\logging\handlers.py", line 111, in rotate
os.rename(source, dest)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process:


Which refers to the logging script, I have never got this before on Windows or Linux, including using Anaconda.  Solution should be here: https://www.programmersought.com/article/54173998537/ however this does not work in Anaconda.

Regarding the project I am now working on the code to freeze the model and convert to IR, will keep you updated.

0 Kudos
JananiC_Intel
Moderator
4,414 Views

Hi,


Kindly provide the below mentioned details.


1) The version of tensorflow and python that you used for training

2) Platform where you trained the model

3) Are you using CPU/IGPU in UP2?

4) Do you want to know about the intel optimized tensorflow version supported in UP2?


We suggest you to train your model on Intel scalable Xeon processors and use the trained model for inference in UP2 for better performance. Using openvino for inference is a good option. For more details regarding openvino you can post in openvino forum(https://community.intel.com/t5/Intel-Distribution-of-OpenVINO/bd-p/distribution-openvino-toolkit)


Thanks.


0 Kudos
AdamMiltonBarker
4,412 Views

Hi in this case I Installed Intel Distribution for Python and then Intel Optimized TF in Conda on Windows using the following: 

conda update conda
conda create -n idp intelpython3_full python=3
conda install tensorflow -c intel

 That was the set up where I was getting the issue, the model trained fine, and I have converted it to IR, about to test it with OpenVINO just finding out the best way to use the IR. 

I am also now training with a different setup, with a standard Python 3 conda env and using

conda install tensorflow-mkl

 That is training right now I will tell you if I get the same. I am replicate both of the above on a Linux machine also. 

On the UP2 I am only using for Inference but sure let me know which version works on it, my plan right now (If this converted TF2 model works ok, last time I did this it was classifying 0 for everything even though using the model without OpenVINO was perfectly fine and never got an answer from anyone at Intel in months). I was planning on using CPU as I was never able to get GPU working with OpenVINO it always just crashes out.

 

 

0 Kudos
AdamMiltonBarker
4,404 Views

By the way in Linux after installing Intel Python the following error is given:

32/1180 [..............................] - ETA: 59sTraceback (most recent call last):
File "ALLoneAPI.py", line 98, in <module>
main()
File "ALLoneAPI.py", line 83, in main
ALLoneAPI.do_train()
File "ALLoneAPI.py", line 42, in do_train
self.Core.do_train()
File "/home/genisys/Desktop/Dev/oneAPI-ALL-Classifier/CNN/Classes/Model.py", line 123, in do_train
validation_steps=self.val_steps, epochs=self.epochs)
File "/home/genisys/anaconda3/envs/IDP/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training.py", line 728, in fit
use_multiprocessing=use_multiprocessing)
File "/home/genisys/anaconda3/envs/IDP/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_v2.py", line 324, in fit
total_epochs=epochs)
File "/home/genisys/anaconda3/envs/IDP/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_v2.py", line 123, in run_one_epoch
batch_outs = execution_function(iterator)
File "/home/genisys/anaconda3/envs/IDP/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_v2_utils.py", line 86, in execution_function
distributed_function(input_fn))
File "/home/genisys/anaconda3/envs/IDP/lib/python3.7/site-packages/tensorflow_core/python/eager/def_function.py", line 457, in __call__
result = self._call(*args, **kwds)
File "/home/genisys/anaconda3/envs/IDP/lib/python3.7/site-packages/tensorflow_core/python/eager/def_function.py", line 520, in _call
return self._stateless_fn(*args, **kwds)
File "/home/genisys/anaconda3/envs/IDP/lib/python3.7/site-packages/tensorflow_core/python/eager/function.py", line 1823, in __call__
return graph_function._filtered_call(args, kwargs) # pylint: disable=protected-access
File "/home/genisys/anaconda3/envs/IDP/lib/python3.7/site-packages/tensorflow_core/python/eager/function.py", line 1141, in _filtered_call
self.captured_inputs)
File "/home/genisys/anaconda3/envs/IDP/lib/python3.7/site-packages/tensorflow_core/python/eager/function.py", line 1224, in _call_flat
ctx, args, cancellation_manager=cancellation_manager)
File "/home/genisys/anaconda3/envs/IDP/lib/python3.7/site-packages/tensorflow_core/python/eager/function.py", line 511, in call
ctx=ctx)
File "/home/genisys/anaconda3/envs/IDP/lib/python3.7/site-packages/tensorflow_core/python/eager/execute.py", line 67, in quick_execute
six.raise_from(core._status_to_exception(e.code, message), None)
File "<string>", line 3, in raise_from
tensorflow.python.framework.errors_impl.AbortedError: Operation received an exception:Status: 5, message: could not create a view primitive descriptor, in file tensorflow/core/kernels/mkl_slice_op.cc:433
[[node Slice_1 (defined at /home/genisys/anaconda3/envs/IDP/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py:1751) ]] [Op:__inference_distributed_function_2037]

Function call stack:
distributed_function

I also found a number of issues with the documentation for Windows OpenVINO and bugs.

0 Kudos
AdamMiltonBarker
4,400 Views

You have loads of issues on Linux. Installing with 

conda create -n IDP intelpython3_full -c intel

gives the error in my last post when trying to run tensorflow. Someone suggested install MKL and upgrade to TF 2.1.0, after doing that, any time I activate the environment Python is no longer Intel distribution of Python and code fails with: 


/home/genisys/anaconda3/envs/IDP/lib/python3.7/site-packages/numpy/__init__.py:156: UserWarning: mkl-service package failed to import, therefore Intel(R) MKL initialization ensuring its correct out-of-the box operation under condition when Gnu OpenMP had already been loaded by Python process is not assured. Please install mkl-service package, see http://github.com/IntelPython/mkl-service
from . import _distributor_init
ImportError: numpy.core.multiarray failed to import
Traceback (most recent call last):
File "ALLoneAPI.py", line 19, in <module>
from Classes.Model import Model
File "/home/genisys/Desktop/Dev/oneAPI-ALL-Classifier/CNN/Classes/Model.py", line 16, in <module>
import cv2
ImportError: numpy.core.multiarray failed to import

Those are the exact steps to reproduce.

0 Kudos
AdamMiltonBarker
4,397 Views

I guess this is the issue, I missed it last time, I need to install OpenCV, when doing so it removes Intel Python


The following packages will be downloaded:

package | build
---------------------------|-----------------
hdf5-1.10.2 | hba1933b_1 5.2 MB
libtiff-4.1.0 | h2733197_1 607 KB
lz4-c-1.9.2 | he6710b0_1 253 KB
zstd-1.4.5 | h9ceee32_0 716 KB
------------------------------------------------------------
Total: 6.7 MB

The following NEW packages will be INSTALLED:

_tflow_select pkgs/main/linux-64::_tflow_select-2.3.0-mkl
astor pkgs/main/linux-64::astor-0.8.1-py37_0
cairo pkgs/main/linux-64::cairo-1.14.12-h8948797_3
ffmpeg pkgs/main/linux-64::ffmpeg-4.0-hcdf2ecd_0
fontconfig pkgs/main/linux-64::fontconfig-2.13.0-h9420a91_0
freeglut pkgs/main/linux-64::freeglut-3.0.0-hf484d3e_5
glib pkgs/main/linux-64::glib-2.65.0-h3eb4bd4_0
graphite2 pkgs/main/linux-64::graphite2-1.3.14-h23475e2_0
harfbuzz pkgs/main/linux-64::harfbuzz-1.8.8-hffaf4a1_0
icu pkgs/main/linux-64::icu-58.2-he6710b0_3
jasper pkgs/main/linux-64::jasper-2.0.14-h07fcdf6_1
jpeg pkgs/main/linux-64::jpeg-9b-h024ee3a_2
keras-applications pkgs/main/noarch::keras-applications-1.0.8-py_1
libglu pkgs/main/linux-64::libglu-9.0.0-hf484d3e_1
libopencv pkgs/main/linux-64::libopencv-3.4.2-hb342d67_1
libopus pkgs/main/linux-64::libopus-1.3.1-h7b6447c_0
libtiff pkgs/main/linux-64::libtiff-4.1.0-h2733197_1
libuuid pkgs/main/linux-64::libuuid-1.0.3-h1bed415_2
libvpx pkgs/main/linux-64::libvpx-1.7.0-h439df22_0
libxcb pkgs/main/linux-64::libxcb-1.14-h7b6447c_0
lz4-c pkgs/main/linux-64::lz4-c-1.9.2-he6710b0_1
opencv pkgs/main/linux-64::opencv-3.4.2-py37h6fd60c2_1
pcre pkgs/main/linux-64::pcre-8.44-he6710b0_0
pixman pkgs/main/linux-64::pixman-0.40.0-h7b6447c_0
py-opencv pkgs/main/linux-64::py-opencv-3.4.2-py37hb342d67_1
zstd pkgs/main/linux-64::zstd-1.4.5-h9ceee32_0

The following packages will be REMOVED:

intelpython3_full-2020.2-0

The following packages will be UPDATED:

certifi intel::certifi-2020.4.5.2-py37_0 --> pkgs/main::certifi-2020.6.20-py37_0

The following packages will be SUPERSEDED by a higher-priority channel:

gast pkgs/main/noarch::gast-0.3.3-py_0 --> pkgs/main/linux-64::gast-0.2.2-py37_0
tensorflow intel::tensorflow-2.2.0-py37_0 --> pkgs/main::tensorflow-2.1.0-mkl_py37h80a91df_0
tensorflow-base intel::tensorflow-base-2.2.0-0 --> pkgs/main::tensorflow-base-2.1.0-mkl_py37h6d63fb7_0

The following packages will be DOWNGRADED:

h5py 2.10.0-py37hd6299e0_1 --> 2.8.0-py37h989c5e5_3
hdf5 1.10.6-hb1b8bf9_0 --> 1.10.2-hba1933b_1
tensorflow-estima~ 2.2.0-pyh208ff02_0 --> 2.1.0-pyhd54b08b_0

 

0 Kudos
AdamMiltonBarker
4,393 Views

The training finished on the installation using 

conda install tensorflow-mkl

and the issue is exactly the same, same errors with the logs.  

0 Kudos
AdamMiltonBarker
4,388 Views

I upgraded to Anaconda 2020, reinstalled the Intel Distribution of Python, checked Python and it is Intel Python, but training is using the GPU, it seems to be entirely ignoring Intel TF.  So far only been successful using Intel Tensorflow on Windows. GPU is much better, trained the whole model in a few minutes. Don't see any benefit of oneAPI. Also zero problems with the logs.

0 Kudos
AdamMiltonBarker
4,380 Views

More problems with OpenVINO, environment variables are set permanently and it cannot find the DLLs, even if I run the .bat file and then navigate to the Python demos folder and run one of the demos. This has been fun lol

0 Kudos
AdamMiltonBarker
4,371 Views

Last update for the day, I everything is working with OpenVINO on another device. Cannot get Windows to work, but the IR conversion and classification works Project complete Hope the bugs I found help. 


1
2020-09-04 10:52:07,605 - OpenVINO - INFO - Acute Lymphoblastic Leukemia correctly detected (True Positive)
2020-09-04 10:52:07,605 - OpenVINO - INFO - Images Classifier: 20
2020-09-04 10:52:07,606 - OpenVINO - INFO - True Positives: 9
2020-09-04 10:52:07,606 - OpenVINO - INFO - False Positives: 1
2020-09-04 10:52:07,606 - OpenVINO - INFO - True Negatives: 9
2020-09-04 10:52:07,606 - OpenVINO - INFO - False Negatives: 1

0 Kudos
AdamMiltonBarker
4,357 Views

If you would like to see the project so far here is it on Github https://github.com/AMLResearchProject/oneAPI-ALL-Classifier/tree/master/CNN

I have tried again with no luck with the above mentioned issues. To list them:

- Cannot get Intel TF working on Linux, GPU version of Tensorflow was never installed yet when training it uses the NVIDIA GPU

- Errors on Windows with logging.

- OpenVINO will not work on Windows, despite environment variables being set, this is true in my project and the demos. Specific error is:

C:\Users\intel-vr-1\Desktop\Dev\AI\oneAPI-ALL-Classifier\CNN>python OpenVINO.py
Traceback (most recent call last):
File "OpenVINO.py", line 18, in <module>
from openvino import inference_engine as ie
File "C:\Program Files (x86)\IntelSWTools\openvino\python\python3.6\openvino\inference_engine\__init__.py", line 1, in <module>
from .ie_api import *
ImportError: DLL load failed: The specified module could not be found.

C:\Users\intel-vr-1\Desktop\Dev\AI\oneAPI-ALL-Classifier\CNN>

Cannot resolve.

 

 

0 Kudos
AdamMiltonBarker
4,327 Views
Raspberry Pi 4 /OpenVINO:
 
 
AttributeError: 'openvino.inference_engine.ie_api.IECore' object has no attribute 'read_network'
 
After downgrading to the last version:
 
AttributeError: 'openvino.inference_engine.ie_api.IECore' object has no attribute 'read_network'
 
Working on using cv2.dnn now and failing with:
terminate called after throwing an instance of 'InferenceEngine::details::InferenceEngineException'
what(): Check 'axis < static_cast<size_t>(input_rank)' failed at /teamcity/work/scoring_engine_build/releases_2020_1/ngraph/src/ngraph/op/gather.cpp:140:
While validating node 'Gather[Gather_99](patternLabel_95: float{10,20,30}, patternLabel_96: int64_t{5}, patternLabel_98: int64_t{1}) -> (??)':
The axis must => 0 and <= input_rank (axis: 4294967295).
 
The IR works fine with UP2 and classifies perfectly, with  --generate_deprecated_IR_V7 it loads on RPI 4. 
 
Next error cv2.error: OpenCV(4.2.0-openvino) ../opencv/modules/dnn/src/op_inf_engine.cpp:831: error: (-2:Unspecified error) Failed to initialize Inference Engine backend (device = CPU): Device with "CPU" name is not registered in the InferenceEngine in function 'initPlugin'
0 Kudos
JananiC_Intel
Moderator
4,319 Views

Hi Adam,


Sorry to inform you that GPU is currently not supported by Intel Optimized Tensorflow.Also your issues are been noted and will be addressed soon.For queries regarding OpenVino, you can post in Openvino toolkit forum(https://community.intel.com/t5/Intel-Distribution-of-OpenVINO/bd-p/distribution-openvino-toolkit) for a quicker response.


Thanks.


0 Kudos
AdamMiltonBarker
4,315 Views

Thanks I am well aware of that, it wasn't what I was saying. I installed Tensorflow MKL on my linux Laptop, it was completely ignored and my training script ran on the GPU.

0 Kudos
AdamMiltonBarker
3,886 Views

I have finished the project now, the information in this thread is for Intel's benefit. Thanks for the help.

0 Kudos
Reply