Using Multi Camera Multi Person Python* Demo 2020.3 to activate 2 cameras and re-identify person

[Lenard-Lim]

Hi

May I know that how to solve the error when I run the command line of Multi Camera Multi Person Python* Demo 2020.3 to activate my camera.

This is the error show out:
python3 multi_camera_multi_person_tracking.py \

-i 0 1 \

--m_detector /home/lenard/Desktop/person-detection-retail-0013.xml \

--m_reid /home/lenard/Desktop/person-reidentification-retail-0103.xml \

--config config.py 

INFO: 2020-10-13 14:07:48: Reading configuration file config.py
INFO: 2020-10-13 14:07:48: Connection cam 0
INFO: 2020-10-13 14:07:48: Connection cam 1
[ WARN:0] global ../opencv/modules/videoio/src/cap_v4l.cpp (893) open VIDEOIO(V4L2:/dev/video1): can't open camera by index
Traceback (most recent call last):
File "multi_camera_multi_person_tracking.py", line 259, in <module>
main()
File "multi_camera_multi_person_tracking.py", line 233, in main
capture = MulticamCapture(args.i)
File "/home/lenard/Downloads/openvino/open_model_zoo/demos/python_demos/multi_camera_multi_person_tracking/utils/video.py", line 38, in __init__
assert cap.isOpened()
AssertionError

[Rizal_Intel]

Hi Lenard,

It is likely that the camera at index 1 is not a standard RGB camera or the camera is not detected.

I tested using RealSense D435 as my secondary camera.

For my case, I needed to use index 0 and 2 as index 1 is the depth camera.

Could you test using other indexes to find your second camera?

Regards,

Rizal

[Lenard-Lim]

Hi Rizal,

Are you mean that I need to set the index 1 to index 2 to find my camera? Then the need any camera connect to index 1 or not?

For my case, I am using two EP-DCOV2735-F36 USB 1080P HD cameras . Can I use these 2 cameras connect to index 0 and index 2 and let the index 1 blank?

Regards,
Lenard

[Rizal_Intel]

Hi Lenard,

I used the input argument like so

-i 0 2 \

The index is based on the order the camera is found.

There is a possibility your camera is not on index 1.

There is also a possibility that your camera is not detected.

Regards,

Rizal

[Lenard-Lim]

Hi Rizal,

Thank you for your solution. I try it then it work but when it show out the video. The side of two frame are large and place together in one frame. It cause the second camera video is not able to see. 
So, may I know how can I change the size of the frame to see 2 video.

And, the next question is when i use to re-identify the same person. I place two camera in same place but the ID are not the same. Suppose the same person is same ID right but why they are not the same when I run the person-reidentification-retail-0103.xml.

Regards,
Lenard

[Rizal_Intel]

Hi Lenard,

You can change output size by changing the utils/visualization.py on line 82

target_width, target_height = get_target_size(frames, vis, max_window_size, stack_frames)

  target_width = 400 #add these width and height

  target_height = 400

  vis = cv.resize(vis, (target_width, target_height))

It is not an elegant solution but it will resize your output as needed.

Try using a more recent version of the reidentification model, I used person-reidentification-retail-0288 and seems to be working fine. You could get this model by downloading OpenVINO 2021.1 and using the downloader.py (but I'll attach it as a zip file for you).

Regards,

Rizal

[Lenard-Lim]

Hi Rizal,

Thank a lot of your solution to solve my question, because I used the OPENVINO 2020.3 to conduct my project. So, It is a bit difficult to replace the current version to latest version in the project. I will try the person-reidentification-retail-0288 later. I still have one question want to ask.

If I use two Intel movidius stick 2 in person re-identification. May I know that how the first and second movidius stick 2 function well in the neural network analysis. I mean we can use first movidius stick 2 to analyze the pixel data and track the object. Then, how the second movidius stick 2 improve or assist to get a good result.

Regards,
Lenard

[Rizal_Intel]

Hi Lenard,

If you are using 2 NCS2 (Neural Compute Stick 2) for this demo, one stick will run the person-detection model and the other will run the person-reidentification model. This behavior is explained in the MYRIAD Plugin document.

"Each IExecutableNetwork instance tries to allocate new device on InferenceEngine::Core::LoadNetwork, but if all available devices are already allocated it will use the one with the minimal number of uploaded networks. The maximum number of networks single device can handle depends on device memory capacity and the size of the networks."

There are other methods of using multiple devices such as the Heterogeneous Plugin (splitting the model neural layers across devices as some layer perform faster or not supported on some devices).

The Multi-Device Plugin will assign the inferencing on multiple device based on availability (this does not split the inference, it just sends the next inference to other devices if the first inference is still ongoing).

Regards,

Rizal

[Lenard-Lim]

Hi Rizal,

Thank you for your solution. It is very useful for me learning the demo and computer vision.

I did it and it can re-identify person. May I know that if the cameras are difference view point (non-overlapping cameras) then will it be able remember the ID. Take a example, camera 1 is place in the meeting room, camera 2 is place in the outside of the company entrance. Target is coming from the outside then go to the meeting room. Will it able to remember the ID of the person?

Do you have face recognition model? If yes, can you send it to me?

Thank you.

Regards,
Lenard

[Rizal_Intel]

Hi Lenard,

I have not tested the reidentification model as per your scenario.

You could try experimenting with it to see if it works, it would be quite interesting.

You don’t have to put the camera so far away, maybe just pointing to different directions.

Test if it remembers by delaying the time when you reappear on the other camera.

I picked the model- 201 from the Model Zoo for you.

Regards,

Rizal

[Lenard-Lim]

Hi Rizal,

Thank your solution. May I know how can I test it? Is it change the python file inside or need to add some python coding inside the file?

May I know this model 201 can detect face or not? May I know how can I write the command line to access the file?

Regards,
Lenard

[Rizal_Intel]

Hi Lenard,

I am sorry the model I shared person-detection-201 is not compatible with the demo.

multi_camera_multi_person_tracking.py has the following models.lst

>person-detection-retail-0013

>person-reidentification-retail-????

which means those are the only model compatible.

As for the testing scenario, yes it should be able to recognize the person (as long as you keep the program running because as far as I know it does not save the reidentification data).

Regards,

Rizal

[Lenard-Lim]

Hi Rizal,

For testing scenario, I tested the non-overlapping cameras which can re-identify the same person with same ID when the view point is difference place. So, can you help me to confirm this system can solve the re-identification data or not? If it cannot store it, how any method can store it like create a database?

That day you give me the person-reidentification-retail-0288. I replace the person-reidentification-retail-0103 to 0288 model then I running the model. It show the runtime error, so may I know how can I solve it?

$ python3 multi_camera_multi_person_tracking.py \
> -i 0 2 \
> --m_detector /home/lenard/Downloads/openvino/open_model_zoo/demos/python_demos/multi_camera_multi_person_tracking/person-detection-retail-0013/FP16/person-detection-retail-0013.xml \
> --m_reid /home/lenard/Downloads/openvino/open_model_zoo/demos/python_demos/multi_camera_multi_person_tracking/person-reidentification-retail-0288/FP16/person-reidentification-retail-0288.xml \
> --config config.py
INFO: 2020-10-21 19:03:23: Reading configuration file config.py
INFO: 2020-10-21 19:03:23: Connection cam 0
INFO: 2020-10-21 19:03:24: Connection cam 2
INFO: 2020-10-21 19:03:24: Creating Inference Engine
INFO: 2020-10-21 19:03:24: Initializing Inference Engine plugin for CPU
INFO: 2020-10-21 19:03:24: Loading network
INFO: 2020-10-21 19:03:25: Preparing input blobs
INFO: 2020-10-21 19:03:25: Loading model to the plugin
INFO: 2020-10-21 19:03:26: Initializing Inference Engine plugin for CPU
INFO: 2020-10-21 19:03:26: Loading network
Traceback (most recent call last):
File "multi_camera_multi_person_tracking.py", line 259, in <module>
main()
File "multi_camera_multi_person_tracking.py", line 250, in main
person_recognizer = VectorCNN(ie, args.m_reid, args.device, args.cpu_extension)
File "/home/lenard/Downloads/openvino/open_model_zoo/demos/python_demos/multi_camera_multi_person_tracking/utils/network_wrappers.py", line 99, in __init__
self.net = load_ie_model(ie, model_path, device, None, ext_path, num_reqs=self.max_reqs)
File "/home/lenard/Downloads/openvino/open_model_zoo/demos/python_demos/multi_camera_multi_person_tracking/utils/ie_tools.py", line 70, in load_ie_model
net = ie.read_network(model_xml, os.path.splitext(model_xml)[0] + ".bin")
File "ie_api.pyx", line 136, in openvino.inference_engine.ie_api.IECore.read_network
File "ie_api.pyx", line 157, in openvino.inference_engine.ie_api.IECore.read_network
RuntimeError: Cannot create ShapeOf layer 611/input_rank/shape_of id:35

Regards,
Lenard

[Lenard-Lim]

Hi Rizal

Sorry I forget to tell you that I am require face detection model because I am try to put in this demo.
Can you give me a suitable/ latest face detection model and what command line should I write when I put this additional model to this demo?

Thank you

Regard,
Lenard

[Vladimir_Dudnik]

every Open Model Zoo demo has models.lst file which list models validated to work with this demo. Also, this models.lst file can be provided as a parameter for model downloader script, which will download only models which are needed for particular demo application

Regards,
  Vladimir

[Rizal_Intel]

Hi Lenard,

If you want to utilize person-reidentification-retail-0288.xml, I would recommend you download OpenVINO 2021.1.

The current iteration of the demo is now called

Multi Camera Multi Target Python* Demo.

You can then download the models yourself using model downloader.

Regards,

Rizal

[Rizal_Intel]

Hi Lenard,

Intel will no longer monitor this thread since we have provided a solution. If you need any additional information from Intel, please submit a new question.

Regards,

Rizal