Have Run asl_recognition_demo but the Demo Screen shows only black screen no video images.

[Chiu__Hsien_Hsiang]

Hi,

Have Run asl_recognition_demo but the Demo Screen shows only black screen no video images.

Have followed the ASL Open Model demo page

https://github.com/openvinotoolkit/open_model_zoo/tree/master/demos/python_demos/asl_recognition_dem...

ASL Recognition Python* Demo

and the commands:

python asl_recognition_demo.py \
-m_a /home/user/asl-recognition-0003.xml \
-m_d /home/user/person-detection-asl-0001.xml \
-i 0 \
-c ./classes.json

PC is using OS: Windows 10, 

Command Prompt Run as Administrator

setupvars.bat set correct

asl-recognition-0004 download 2021.1.110 updated.

person-detection-asl-0001 download 2021.1.110 updated. 

Video camera ok

C:\Program Files (x86)\IntelSWTools\openvino_2021.1.110\deployment_tools\open_model_zoo\demos\python_demos\asl_recognition_demo>python asl_recognition_demo.py -m_a asl-recognition-0004.xml -m_d person-detection-asl-0001.xml -i 1 -c classes.json

and

have Error when use Video Camera as "-i 1" source.  (my PC use video camera 1 as source)

the Demo windows will display "Black" screen, no video images ...etc.

on CMD screen shows message 

"

Process Process-1:
Traceback (most recent call last):
File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\multiprocessing\process.py", line 258, in _bootstrap
self.run()
File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\multiprocessing\process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "C:\Program Files (x86)\IntelSWTools\openvino_2021.1.110\deployment_tools\open_model_zoo\demos\python_demos\asl_recognition_demo\asl_recognition_demo\video_stream.py", line 123, in _frame_generator
trg_time_step = 1.0 / float(source_fps)
ZeroDivisionError: float division by zero

"

and the pop up Demo screen is Black screen. Program keep running , but Black screen, not show anything. When Click "ESC" key the Demo window program will terminated back to CMD window.

Attached Demo Windows screen show Black screen error picture for reference.

Does someone know How to solve this error?

Demo windows shows black screen , no images display.

in CMD display "

video_stream.py", line 123, in _frame_generator
trg_time_step = 1.0 / float(source_fps)
ZeroDivisionError: float division by zero

"

Program do not crash, still running, but demo windows has no video image.

When click ESC key, will terminated windows and back to CMD windows prompt.

[Chiu__Hsien_Hsiang]

Hi,

Here are the detail information of Environment Details

1. Which version of the OpenVINO™ toolkit are you using?

Using openvino_2021.1.110 version

2. Operating System

Windows 10 Pro 64 bits,
Version: 1909

3. Hardware specifications

System:
Processor: Intel(R) Core(TM) i7-10510U CPU @ 1.8GHz 2.30 GHz
Installed memory (RAM): 16.0 GB
System type: 64-bit Operating System, x64-based processor

4. Additional environment information

Video Camera:
Intel RealSense(TM) Depth Camera 415 RGB
Intel RealSense(TM) Depth Camera 415 Depth

CMAKE version 3.18.0-rc2

Python 3.6.5

C:\Users\User>pip -V
pip 20.3 from c:\users\user\appdata\local\programs\python\python36\lib\site-packages\pip (python 3.6)

5. Steps to reproduce

Here are the details steps:

CMD run as Admin

Step 1.

C:\Program Files (x86)\IntelSWTools\openvino_2021.1.110\bin
Run setupvars.bat

Step 2.

Have use Open_Model_Zoo downloader to Download newest asl .xml and .bin fies, using FP32 files to Run

C:\Program Files (x86)\IntelSWTools\openvino_2021.1.110\deployment_tools\open_model_zoo\tools\downloader\intel\asl-recognition-0004\FP32

asl-recognition-0004.XML
asl-recognition-0004.bin

Step 3.

The application is orginal from OpenViNO_2021.1.110 installation
C:\Program Files (x86)\IntelSWTools\openvino_2021.1.110\deployment_tools\open_model_zoo\demos\python_demos\asl_recognition_demo

Step 4.

Have moved the asl .xml and .bin files to same directory

PC connect USB with RealSense Camera is 1

Step 5.

In CMD Run command:
C:\Program Files
(x86)\IntelSWTools\openvino_2021.1.110\deployment_tools\open_model_zoo\demos\python_demos\asl_recognition_demo>python
asl_recognition_demo.py -m_a asl-recognition-0004.xml -m_d
person-detection-asl-0001.xml -i 1 -c classes.json

Step 6.

Program Error:

Program Run but the pop up Demo window only display black screen, no image.

During the program run
The CMD windows shows message:
"
Process Process-1:Traceback (most recent call last):File
"C:\Users\User\AppData\Local\Programs\Python\Python36\lib\multiprocessing\process.py",
line 258, in _bootstrapself.run()File
"C:\Users\User\AppData\Local\Programs\Python\Python36\lib\multiprocessing\process.py",
line 93, in runself._target(*self._args, **self._kwargs)File "C:\Program
Files
(x86)\IntelSWTools\openvino_2021.1.110\deployment_tools\open_model_zoo\demos\python_demos\asl_recognition_demo\asl_recognition_demo\video_stream.py",
line 123, in _frame_generatortrg_time_step = 1.0 /
float(source_fps)ZeroDivisionError: float division by zero

"

Step 7.

Program No crash, when click ESC key, then program will terminated, and back to CMD windows prompt.

[Rizal_Intel · Accepted solution]

Hi Chiu__Hsien_Hsiang,

It seems the fps property is not extractable for cameras. It seems the property can be set but not extracted as discussed in this Stack Overflow thread.

I added

if source_fps == 0 : source_fps = 30 #replace this number with your camera fps.

just above

trg_time_step = 1.0 / float(source_fps)

in

..\asl_recognition_demo\video_stream.py

Hope this helps.

Regards,

Rizal

[Chiu__Hsien_Hsiang]

Hi Rizal,

Thanks for reply message solution.

Yes, I have follow instruction and edited video_stream.py, and the solution solved the Demo Window Black screen error.

C:\Program Files (x86)\IntelSWTools\openvino_2021.1.110\deployment_tools\open_model_zoo\demos\python_demos\asl_recognition_demo\asl_recognition_demo\video_stream.py

For Intel RealSense D415 camera,

The video frame is 60 PFS.

Solution:

if source_fps == 0 : source_fps = 60 #replace this number with your camera fps.

just above

trg_time_step = 1.0 / float(source_fps)

in Video_stream.py file

The program can Run and display video images now.

But have another new ASL demo error need to solved.

The Demo Windows only able shows user tracking, but not do any ASL hand signs detection or readings.

Program can only do tracking user when walking around with square lines border and number '0'.

But program does not do any ASL hand sign recognition , nor reading ASL hand signs at all.

When user do any ASL sign , the program did not shows anything detections, and not shows any reading user's hands  ASL hand signs.

For reference: The command that I Run 

"python
asl_recognition_demo.py -m_a asl-recognition-0004.xml -m_d
person-detection-asl-0001.xml -i 1 -c classes.json"

at 

C:\Program Files (x86)\IntelSWTools\openvino_2021.1.110\deployment_tools\open_model_zoo\demos\python_demos\asl_recognition_demo\

Have moved all the .xml and bin files in same directory

both asl-recognition-0004 and person-detection-asl-0001

ASL Demo program error:

Demo Windows program Run, and show video images, program able to tracking user,  but program has not do any ASL hand signs detection , program not do any reading user's hand signs.

Anyone know how to enable the ASL detection and Reading?

or where might be the problem need to be corrected?

[Rizal_Intel]

Hi Chiu__Hsien_Hsiang,

The demo starts in person tracking mode and to switch it in the action recognition mode you should press 0-9 button with appropriate detection ID (the number in top-left of each bounding box). After that you can switch back to tracking mode by pressing space button.

You need to press key number 0-9 to read ASL for that particular person. For example, if you are the only person in the camera feed and it highlights you with number 0, press 0. You can read more about it under running section at the repo.

I tested it a bit but as I am not familiar with ASL I flailed like a madman and got some response like "fish" or "man". I am not sure the accuracy of these as the sample videos is not available in the links.

Regards,

Rizal

[Chiu__Hsien_Hsiang]

Hi Rizal,

I have Run the program and click the user ID.

The Demo Windows will highlight the user.

I use Video camera as input source, program is not able to detect and reading simple ABC...etc hand sings.

and  then

Have use  ASL Videos with hand signs abc, number 1 - 10 to test ASL program,

The demo program can not reading ASL hand signs.

I have reviewed ASL Recognition Python* Demo

the https://github.com/openvinotoolkit/open_model_zoo/tree/master/demos/python_demos/asl_recognition_demo

Have try do the Hello sign, need repeat do several times,

then the program can detect the "Hello" ASL hand sign.

Summary:

ASL demo program not able to correctly reading hand signs.

The ASL demo program has shows Latency to display image on Demo window.

and fromthe basic ASL hand signs test, the ASL reading is poorly, not works.

Question:

Where might be the poorly ASL reading cause? Where need to fixed in order to have accuracy ASL reading?

Could be the .xml and .bin files from the download (newest version) not working correctly?

or

the Pre-Train Models Map cause the reading badly?

or

asl_recognition_demo.py?

Anyone has refine the ASL Demo program and works? 

[Vladimir_Dudnik]

Note, in model description it stated that MS-ASL-100 dataset used in model training. From dataset specification and from supplied with model classes.json file you may clearly see what gestures/signs model should be able to recognize. I do not see "simple ABC...etc hand sings" listed there.

[Rizal_Intel]

Hi Chiu__Hsien_Hsiang,

You can check the list of the 100 trained signs in classes.json.

Intel will no longer monitor this thread since this issue has been resolved. If you need any additional information from Intel, please submit a new question.

Regards,

Rizal