Multi Tracking Multi Camera with Re-Identification crawls.

[Duggy]

Hi,

I am running the demo: https://docs.openvinotoolkit.org/latest/omz_demos_multi_camera_multi_target_tracking_demo_python.html

I have run this on a core i5 with NVidia T600 GPU and 8 Gigs Ram as well as on a core i9 with 16 Gigs Ram with the time_window set to 1 in the person.py configs file to ensure REID. 

The system works great but slows to an absolute crawl within 5 minutes, like 1 frame per 10 seconds at best. The GPU/CPU and memory are not exploited, they are not effected or at a maximum (under 60% utilization) but still the system crawls=.

Initially it starts off with 20 or 30 (or more) frames a second but then just bottoms out.

Utilizing the :
FP16-INT8\person-reidentification-retail-0277 (for reidentification)

and 

FP16-INT8\person-detection-retail-0013.xml (for detection)

Have tried both FP16 and FP32. 

When the config is set to time_window=10 then it flies but reidentification is at 10 or 20% accuracy at best, most people are not reidentified, when set to 1 second, all are identified but system crawls to a grinding halt. 

Any help would be appreciated.

Thanks

[Zulkifli_Intel]

Hello Elad Segev,

Thank you for your question.

We are checking this issue and will get back to you soon.

Sincerely,

Zul

[Zulkifli_Intel]

Hi Elad Segev,

I test the Multi Camera Multi Tracker demo using pre-trained models, person-reidentification-retail-0277, and person-detection-retail-0013 with different precisions. To improve the inference speed, we suggest you use the FP16-INT8 model and window_time-20. Below are the results from my test on a 1080p video.

window_time=1

FP32

Processing frame: 340, fps = 1.6 (avg_fps = 1.72)

FP16

Processing frame: 340, fps = 1.8 (avg_fps = 2.28)

FP16-INT8

Processing frame: 340, fps = 1.9 (avg_fps = 3.41)

window_time=20

FP32

Processing frame: 340, fps = 2.7 (avg_fps = 2.34)

FP16

Processing frame: 340, fps = 2.7 (avg_fps = 2.66)

FP16-INT8

Processing frame: 340, fps = 5.3 (avg_fps = 4.26))

For your information, OpenVINO might not work with NVidia GPU.

Sincerely,

Zulkifli

[Duggy]

Hi,

What GPU do you recommend for Intel?

Also the window_time has to be 1 otherwise the people cross the view without any REID. That was our initial issue, most of the people were just bounding black boxes without REID as they crossed the view. Changing to window_time=1 ensured that 95% of the people in view had reid and therefore could be counted/tracked.

Is there any documentation on the meaning of all the config variables?

Thanks

[Zulkifli_Intel]

Hello Elad Segev,

The recommended GPU to use is Intel® Processor Graphics, including Intel® HD Graphics and Intel® Iris® Graphics. You also can use the Intel® Neural Compute Stick 2 powered by the Intel® Movidius™ Myriad™ X, Intel® Vision Accelerator Design with Intel® Movidius™ VPUs to run the inference.

We do not have the documentation describing all the config variables. We suggest you refer to the Device-Specific Optimizations documentation to increase the performance.

Sincerely,

Zulkifli 

[Duggy]

Thanks.

At the moment we are finding the system starts off great, runs great in real time. Then after about an hour (and depending on the number of people seen) it will slow down to a crawl (under 1 fps). Do you believe the NC2 stick will correct this phenomenon?

Struggling to find the Intel GPU recommended, can you share a link?

[Zulkifli_Intel]

Hello Elad Segev,

You can find the information on Intel® Graphics Processing Units (GPUs) here:

https://www.intel.com/content/www/us/en/products/discrete-gpus.html

https://docs.openvinotoolkit.org/latest/openvino_docs_IE_DG_supported_plugins_GPU.html

NCS2 is an alternative that you can choose. But we do not guarantee it will resolve the current issue as it was not validated.

Sincerely,

Zulkifli

[Vladimir_Dudnik]

@Duggy  could you please share exact command line options and link to the input video (or describe somehow what input data you are using) ? We'll investigate that issue and would like to reproduce it.

[Duggy]

Hi,

Am using the commands in the python multi tracking multi camera demo (as recommended) with people identification and reidentification :

(FP16-INT8\person-detection-retail-0013.xml and FP16-INT8\person-reidentification-retail-0277.xml - have tried FP16 and FP32 as well).

(https://docs.openvinotoolkit.org/latest/omz_demos_multi_camera_multi_target_tracking_demo_python.html)

The only change that was made was to the person.py file to change the time_window=1 instead of 10 as the reidentification was missing most people with time_window=10.

sct_config = namespace(
time_window=1, #10,
continue_time_thresh=2,
track_clear_thresh=3000,
match_threshold=0.25,
merge_thresh=0.15,
n_clusters=4,
max_bbox_velocity=0.2,
detection_occlusion_thresh=0.7,
track_detection_iou_thresh=0.5, #0.5,
process_curr_features_number=0,
interpolate_time_thresh=10,
detection_filter_speed=0.6,
rectify_thresh=0.1
)

The machine specs are:

Intel Core i9-7900X CPU @ 3.30Ghz with 16Gigs RAM
Running off RTSP (or Video) - RTSP cameras set at 20 frames per second - same occurrences when running avi videos.

The CPU and Memory are not effected, do not climb or peak but the fps drops from over 10 to :

Processing frame: 7605, fps = 0.1 (avg_fps = 2.96)

in about 10 minutes. There were about 100 or 200 people who went through the view before the fps drops and the video view slows down to stuttering - shows 1 fame every 5 seconds on view also the reidentification seems to degrade tremendously and most people identified are just black boxes (not reid)

I'm lead to believe that the more the system has data for reid then the more it slows down the more inaccurate it becomes, but this is a guess on my part.

Even with time_window=10 the same issues occur (just might take longer than the 10 minutes). The views in the live windows slow down to show image every 5 seconds (person appears in view and then the next view is of him leaving the view). The reidentification degrades incredibly where 1 in 10 would get a reid colored box the rest are all black boxes). See image below example:

 

[Vladimir_Dudnik]

@Duggy  thanks, we'll look into this

[Duggy]

Thanks, have run a  series of tests. It seems to be an impact of quantity of people in the view, the more the people the faster the slowdown in fps and speed in live view. 

The same RTSP (as above) run when the location is closed runs for hours with high fps. The moment there is high activity it slows down. 

Have tried not feeding in the m_reid argument (leaving blank), have tried all FP16, FP32 and FP16-INT8 (both person-detection and person-reidentification) and same events happen.

After further test seems the person.py parameter of track_clear_thresh plays a role in the slowdown. If this number is set to low (1 or 2 seconds) then the slowdown doesn't seem to occur. So its a factor of number of people and the duration of this parameter in retaining the tracks or features? that causes the system to slow down to a halt. A low number and the system continues, a high number and lots of traffic or lots of traffic over time and the system grinds to a halt at under 1 fps.

[Vladimir_Dudnik]

@Duggy  We have investigated tha issue amd it seems that the tracker itself work fine. The problem occurs due to accumulation of tracks, the more objects we have to track, the more it slows down. You should try to reduce track_clear_thresh in configs/person.py. Right now it is set to 3000, so old tracks are not cleared and get accumulated. Try to set it to 300 or less, then demo shouldn't slow down so much.

[Munesh_Intel]

Hi Duggy,

This thread will no longer be monitored since we have provided a solution. If you need any additional information from Intel, please submit a new question.

Regards,

Munesh