Moving object detection using D5M/DE1-SoC

[Altera_Forum]

Hi, 

 I have implemented a video streaming design using DE1-SoC interfaced to D5M camera and VGA display. It works fine for still objects or very slowly moving objects. But if I move the object a little faster I could not see the object on VGA display.  I configured the D5M camera to export 640x480 frame and from the signal tap results it seems i am receiving more than 20 frames per second. With such a frame rate I should be able to detect moving object too. What could be the reason for that?  Is their any setting I can tweak to improve moving object detection??  Please suggest

[Altera_Forum]

This will be a problem with the algorithm. Depending on the speed it could have a large vector and maybe appear as two separate objects? I know fast objects are hard to detect because they are not on screen very long.

[Altera_Forum]

My design just use Avalon Streaming to transmit video from camera to VGA. It just transfer the pixels to VGA as it get them from camera. 

  question: How does camera work?? Assuming camera is giving a 30fps video. I can think of two cases:  1. Is it some thing like it captures one whole frame at once and transmits it pixel by pixel. And while it is transmitting, it doesn't capture the other frame (so the captured pixels are unaffected while camera module is transmitting them). And as soon it completes transmitting the previous frame it captures another frame and starts transmitting it. In this case, every frame is buffered in camera module first and then transmitted.  OR  2. It captures the frame pixel by pixel and transmits it simultaneously. In the case, the whole frame is not buffered in the camera module.    Issue: If it is the first case, the moving object should be visible (movement may not be that smooth) because it is just discretely capturing what is visible. Also, In this particular case, I tried keeping the whole frame total white and the object as black. And in between the streaming protocol, their is a small logic which check for the presence of black pixel and raises a flag if it found one. So when object is still or moving really slowly I can see that flag raised but if i increase the speed slightly the flag is de-asserted. That means their is no black pixel in the whole frame. How is that possible.  I doubt it has to do something with the camera specs like exposure time/shutter speed. Please suggest.

[Altera_Forum]

Any help will really be appreciated.

[Altera_Forum]

What help do you need. Like I said, this sounds like an algorith problem - what is the algorithm? are you using the simplest of just taking a reference background image and subtracting it from the incoming video, or do you have some filtering going on to detect images? 

 The camera will transmit you real time video - ie. pixel by pixel - it is up to you to do any buffering if you really need it (plenty of algorithms can be done on streaming video).

[Altera_Forum]

To debug the design, I removed whole algorithm from the design. For now, its just transferring pixel data as given by camera, to VGA. And it doesn't buffer the frame. It transfer pixel by pixel, as it gets from camera using Avalon Streaming. The data bus in Avalon Streaming is 24bit [8:8:8][RGB]. The only algo i have is a small comparator which checks for the presence of black pixels[0x000000]. If i move my black object slowly the algo asserts a flag but as soon I increase its speed, it de-asserts flag. It means it couldn't even detect the object in frame because it was moving. What can be done to achieve the moving object detection to a more better extent? 

 I hope I described it better this time.

[Altera_Forum]

The D5M camera has an image sensor with rolling shutter. This may explain why your algorithm doesn't work for fast moving objects. At 640 x 480 you should be able to get more frames per second, by shortening the exposure time and at the same time increasing the illumination of the scene. 

Also detecting black pixels comparing them to (only) 0 is a bit optimistic. When moving (fast) a pixel sees a mixture of black and white, especially when you have a relatively long exposure time. Setting a threshold of 1/10th (or so) would probably work better.