topic Re: Error in testing ImageNet Dataset in IntelAI models in AI Tools from Intel

Error in testing ImageNet Dataset in IntelAI models

TaejinKim — Tue, 31 May 2022 02:19:11 GMT

Hello,

I'm testing https://github.com/IntelAI/models/tree/master/datasets/imagenet this, and I've faced problem.

My error outputs are here.

2022-05-31 06:06:36.620505: F tensorflow/core/platform/default/env.cc:73] Check failed: ret == 0 (11 vs. 0)Thread tf_ForEach creation via pthread_create() failed.
Fatal Python error: Aborted

Thread 0x00007fddd0bf6b80 (most recent call first):
File "/home/mscho/tf_env/lib64/python3.6/site-packages/tensorflow/python/lib/io/file_io.py", line 443 in get_matching_files_v2
File "/home/mscho/tf_env/lib64/python3.6/site-packages/tensorflow/python/lib/io/file_io.py", line 383 in get_2022-05-31 06:06:36.620610: F tensorflow/core/platform/default/env.cc:73] Check failed: ret == 0 (11 vs. 0)Thread tf_ForEach creation via pthread_create() failed.
matching_files
File "/home/mscho/imagenet_dir/imagenet_to_gcs.py", line 366 in convert_to_tf_records
File "/home/mscho/imagenet_dir/imagenet_to_gcs.py", line 419 in main
File "/home/mscho/tf_env/lib64/python3.6/site-packages/absl/app.py", line 258 in _run_main
File "/home/mscho/tf_env/lib64/python3.6/site-packages/absl/app.p./imagenet_to_tfrecords.sh: line 76: 2518293 Aborted (core dumped) python3 ${WORKDIR}/imagenet_to_gcs.py --raw_data_dir=$IMAGENET_HOME --local_scratch_dir=$IMAGENET_HOME/tf_records
mv: cannot stat 'train/*': No such file or directory
mv: cannot stat 'validation/*': No such file or directory

I searched for pthread_create() failed error, but I couldn't find something to solve this.

Please let me know, if there's something to be added here.

Re:Error in testing ImageNet Dataset in IntelAI models

VaradJ_Intel — Thu, 02 Jun 2022 12:30:37 GMT

Hi,

Thank you for posting in Intel communities.

Thanks for reporting this issue. We were able to reproduce it and we have informed the development team about it.

Thank You.

Re: Error in testing ImageNet Dataset in IntelAI models

TaejinKim — Thu, 02 Jun 2022 12:53:55 GMT

Hi Varad,

Thank you for checking it. How will I know it is modified? Would you notice me by reply this post?

Regards,

Jason.

Re:Error in testing ImageNet Dataset in IntelAI models

Preethi_V_Intel — Tue, 30 Aug 2022 20:30:54 GMT

Can you please provide the command you ran and understand where exactly you are facing the issue

Re:Error in testing ImageNet Dataset in IntelAI models

Preethi_V_Intel — Thu, 06 Oct 2022 22:27:43 GMT

Can you please share a reproducer, without one I'll not be able to assist this ticket

Re: Error in testing ImageNet Dataset in IntelAI models

TaejinKim — Sun, 09 Oct 2022 07:01:02 GMT

Hi,

Sorry for the late reply.

I'd just followed git and run the code ./imagenet_to_tfrecords.sh .

Regards,

Jason.

Re: Error in testing ImageNet Dataset in IntelAI models

Preethi_V_Intel — Wed, 19 Oct 2022 06:10:20 GMT

this is core dump error. Can you try to fecth core dump files on your working dir if possible or may be run your code on debug mode

bash -x imagenet_to_tfrecords.sh

Re:Error in testing ImageNet Dataset in IntelAI models

Preethi_V_Intel — Thu, 20 Oct 2022 22:11:00 GMT

Core dump messages can help us understand it better. Just letting you know that this is a public imagenet script, are you getting this using from Intel packages?

Re: Error in testing ImageNet Dataset in IntelAI models

TaejinKim — Tue, 25 Oct 2022 00:06:55 GMT

Hi,

I'll reply to this feed after downloading the dataset again.

Sorry for the delay.

Regards,

Jason.

Re: Re:Error in testing ImageNet Dataset in IntelAI models

TaejinKim — Thu, 03 Nov 2022 01:51:54 GMT

+ tar xf n15075141.tar -C n15075141
+ rm n15075141.tar
+ cd /root/mason
+ [[ ! -f /root/mason/imagenet_to_gcs.py ]]
+ wget https://raw.githubusercontent.com/IntelAI/models/master/datasets/imagenet/imagenet_to_gcs.py
--2022-11-03 19:34:59-- https://raw.githubusercontent.com/IntelAI/models/master/datasets/imagenet/imagenet_to_gcs.py
Resolving proxy-dmz.intel.com (proxy-dmz.intel.com)... 172.30.7.176
Connecting to proxy-dmz.intel.com (proxy-dmz.intel.com)|172.30.7.176|:911... connected.
Proxy request sent, awaiting response... 200 OK
Length: 15260 (15K) [text/plain]
Saving to: ‘imagenet_to_gcs.py’

imagenet_to_gcs.py 100%[==============================================================================================================>] 14.90K --.-KB/s in 0.1s

2022-11-03 19:35:00 (126 KB/s) - ‘imagenet_to_gcs.py’ saved [15260/15260]

+ python3 /root/mason/imagenet_to_gcs.py --raw_data_dir=/root/imagenet_raw_data --local_scratch_dir=/root/imagenet_raw_data/tf_records
2022-11-03 19:35:03.011210: F tensorflow/core/platform/default/env.cc:73] Check failed: ret == 0 (11 vs. 0)Thread tf_ForEach creation via pthread_create() failed.
Fatal Python error: Aborted

Thread 0x00007f20f1da2dc0 (most recent call first):
File "/usr/local/lib64/python3.6/site-packages/tensorflow/py2022-11-03 19:35:03.011261: F tensorflow/core/platform/default/env.cc:73] Check failed: ret == 0 (11 vs. 0)Thread tf_ForEach creation via pthread_create() failed.
thon/lib/io/file_io.py", line 443 in get_ma2022-11-03 19:35:03.011320: F tensorflow/core/platform/default/env.cc:73] Check failed: ret == 0 (11 vs. 0)Thread tf_ForEach creation via pthread_create() failed.
tching_files_v2
File "/usr/local/lib64/python3.6/site-packages/tensorflow/python/lib/io/file_io.py", line 383 in get_matching_files
File "/root/mason/imagenet_to_gcs.py", line 370 in convert_to./imagenet_to_tfrecords.sh: line 74: 173558 Aborted (core dumped) python3 ${WORKDIR}/imagenet_to_gcs.py --raw_data_dir=$IMAGENET_HOME --local_scratch_dir=$IMAGENET_HOME/tf_records
+ rm /root/mason/imagenet_to_gcs.py
+ cd /root/imagenet_raw_data/tf_records
+ mv 'train/*' /root/imagenet_raw_data/tf_records
mv: cannot stat 'train/*': No such file or directory
+ mv 'validation/*' /root/imagenet_raw_data/tf_records
mv: cannot stat 'validation/*': No such file or directory
+ rm -rf train
+ rm -rf validation
+ cd /root/mason

Re: Re:Error in testing ImageNet Dataset in IntelAI models

TaejinKim — Thu, 03 Nov 2022 01:52:19 GMT

this is where the error occurs.

Re: Re:Error in testing ImageNet Dataset in IntelAI models

TaejinKim — Mon, 07 Nov 2022 02:18:03 GMT

Hi Preethi,

I solved this problem by editing imagenet_to_gcs.py.

There was a problem that occurred when processing images by tf.glob.

I replaced tf.glob to glob.glob function.

Regards,

Jason.