Problem with grainy raw video stream

[Jack_K_1]

I'm having a problem with the raw video stream: The image is kind of grainy - the pixels dance around.  Perhaps this is due to a YUV conversion issue.  Also, the FPS is only 15.

I'm running the RealSense camera on an i5 NUC using Windows 10.  I installed the Gold SDK.  All of the samples work. 

The firmware on my camera is apparently not up-to-date: I'm at 2.36.9.0, not 2.38.5.0.  I didn't update it because I read somewhere that it was unnecessary if your camera was shipped after November 17th.

Not sure where the conversion is happening, but suspect it is being done in firmware because I get the same grainy image when I use the Windows camera app.

Is anyone else experiencing this problem?  Anyone know whether a firmware upgrade will fix the problem?

Regards,

Jack

[Christopher_C_]

I'm experiencing the same problems on both my laptop and my desktop (i5 4670k running on windows 8.1). I was assuming it was just the quality of the cameras. I'd like some clarification on the matter as well (whether it's the camera's fault of firmware's).

[Jack_K_1]

Christopher,

What firmware version is running on your camera?

Regards, Jack

[Christopher_C_]

I'm not sure how to check the exact firmware version, but I just recently (5 minutes ago to be exact) downloaded the firmware update from this link: (just a heads up, the link will download the driver for you)

files.creative.com/ManualDn/Applications/AVP/13316/0x3AF9CAD5/C3DG_0800_FW_EN_2_38_5_0.exe

If it's not too much trouble, if I have the wrong firmware, can you direct me to where I can find the most up to date firmware which will give me a clearer stream (unless the camera originally gives grainy streams).

[Christopher_C_]

I'm not sure how to check the exact firmware version, but I just recently (5 minutes ago to be exact) downloaded the firmware update from this link: (just a heads up, the link will download the driver for you)

http://files.creative.com/ManualDn/Applications/AVP/13316/0x3AF9CAD5/C3DG_0800_FW_EN_2_38_5_0.exe

If it's not too much trouble, if I have the wrong firmware, can you direct me to where I can find the most up to date firmware which will give me a clearer stream (unless the camera originally gives grainy streams).

[Jack_K_1]

Christopher,

You can check the firmware version by invoking utility "Intel(R) RealSense(TM) SDK Information Viewer."  The path to this utility is C:\Program Files (x86)\Intel\RSSDK\bin\x64\sdk_info.exe.

To check your firmware version:

1. Invoke the SDK Information Viewer (i.e. sdk_info.exe)

2. Click the Camera tab.

3. You'll find the version number under "FW version."

My understanding is that the most recent firmware version is  2.38.5.0, which is in the file you download (C3DG_0800_FW_EN_2_38_5_0.exe).

I think the driver on the Creative site is the audio driver for the camera - the 0800 in the filename matches the number used for the name of the audio driver "Creative VF0800" in the device manager under "Sound, video and game controllers".

Regards,

Jack

[Christopher_C_]

This is what I have. Under the Camera tab, it says the Depth Camera Manager is 1.2.14.24922 (same as what it says in the version tab).

If I need to update, can you direct me to a link where I can update the firmware of the camera?

[Jack_K_1]

Hi Christopher,

You need to click the "Camera" tab and then expand "Device: 1" under "Depth Camera Manager Service".  See the attached image.

To update the firmware you need to invoke the file you downloaded from the Creative website.  That is, invoke C3DG_0800_FW_EN_2_38_5_0.exe.

EDIT:  I just noticed that the firmware update also has 0800 in its name.  I guess my earlier guess about the meaning of 0800 in the driver name is not correct.   Perhaps the driver update applies to more than just the audio portion.  It would be nice if Intel could provide a change log or release notes for these updates so we would not have to guess what they are for.

Regards,

Jack

[Christopher_C_]

Yeah, I already updated the firmware a while ago by opening that file I downloaded from Creative. The firmware version is now 2.38.5.0, but I'm still seeing a somewhat grainy color stream. I'm assuming the problem lies with the hardware of the camera. Below is a picture to show the quality I'm getting, it isn't as HD as I expected it to be. Are you experiencing the same quality or even worse? (Open the image in a new tab for a bigger and clearer image).

[Jack_K_1]

Hi Christopher,

Yep.  You're seeing what I'm seeing.  The image looks less grainy in natural light (below) than when the room is illuminated by florescent light:

And, it might look better in incandescent light than in florescent light.  With florescent lights in white and pink (don't ask) cones, the pink twinkles or dances in place through out the image.  I hope the production version of the camera looks better.

It would be nice if someone from Intel could join this discussion after the holidays.

Regards,

Jack

[Colleen_C_Intel]

You can try playing with some settings
device.SetDepthConfidenceThreshold(1);

device.SetIVCAMFilterOption(6);

[Jack_K_1]

Hi Colleen,

I think I'm seeing high frequency low magnitude random Chroma noise that is more noticeable as I enlarge the video image.

I tried using SetIVCAMFilterOption but the smoothing had little if any visible effect.

The problem is much less noticeable with good natural ambient light.  In fact, if I position the camera so that the scene has less natural light to one side, the portion of the video with good natural light looks pretty good while the less lit portion has more noticeable noise.

In the evening, both florescent and tungsten light look pretty bad when you go to full screen mode - It looks like I'm picking up more low frequency noise, but maybe it's just the effect of enlarging the video image.

I tried adjusting the white balance and the color exposure but the APIs returned PXCM_STATUS_DEVICE_FAILED.  Are these APIs supported by the Real Sense camera?

Regards,

Jack

P.S. Happy New Year!

[Colleen_C_Intel]

To the API support, according to documentation - there's a long list of what's supported here  - :https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/manuals_member_functions23.html

obviously you can query a lot more settings than you can alter but SetColorWhiteBalance (and SetColorAutoWhiteBalance)  and SetColorExposure (and SetColorAutoExposure) are there. - and failure of those should return PXC_STATUS_ITEM_UNAVAILABLE.

[Jack_K_1]

Hi Colleen,

I used the query versions of these APIs to get default and range information and had no problem doing so.  On failure, I expected the status code you listed per the documentation.  But, instead received PXCM_STATUS_DEVICE_FAILED.  Perhaps the documentation is wrong.

EDIT: Could this be a firmware version issue?

Regards,

Jack

[Colleen_C_Intel]

2.38.5 is the latest available firmware. And I assume you're on DCM software 1.2.x (which should install with Gold R1 SDK)
Since only a few people are seeing the grainy-ness, it would seem more likely it's a problem with just a few cameras and not the firmware.

On the device failure, something to be checked which message it gives....But on your attempted changes, did you try reading them right after to see if by any chance it did change?

[Jack_K_1]

Hi Colleen,

Yes, I did try querying the values I attempted to change.  Nothing changed.

Regards,

Jack

[Colleen_C_Intel]

On SETting the camera.  Turns out (documentation bug) that you need to add the parameter auto=false. This changes the setting to manual to allow what you enter to become the setting. 
And the error messages expected should include the PXCM_STATUS_DEVICE_FAILED. (will also be documented next release.)

[Jack_K_1]

Colleen,

Thanks.  Using the paired API to set auto false does work.

So forum readers understand the context:

For example, if you want to change the white balance of the camera, you need to call SetColorAutoWhiteBalance(false) before calling SetColorWhiteBalance to set the balance.

Regards,

Jack

[Jonathan_M_Intel]

Hi Guys - just to jump in, I'm a software guy and not a camera hardware guy, but I can say from my experience that when the frame rate drops and the image gets grainy, it's usually just as easy as turning on a light nearby so that the whole scene gets a little brighter.  

The camera can be a little... overeager at times, trying to keep the brightness of the image up, and so it resorts to lowering its frame rate to give the sensor more time to gather light, and also boosting the "gain" of the image (which is just an artificial way of doing the same thing), but this causes the natural sensor noise to be amplified and look like multi-colored graininess.  

You can try disabling the auto-exposure function, and manually setting your level, but remember that the longer your exposure, the lower your frame rate, and you might find that your scene will be way darker than your expected.  

So, easy answer, turn on a couple more lights.

[Jack_K_1]

Hi Jonathan,

Is 10 lights enough?   With 10 lights on - 5 per pole - and no sunlight, I still get a grainy image.  However, my smartphone can take crisp video with only a couple lights on.  And, the RGB on my Kinect looks just fine in the same light that causes Real Sense to look grainy.

My camera only produces passable imagest during the day when there is sufficient ambient light from the sun. 

I've tried changing some of the camera settings but to no available - this problem is not solvable by software.  There is something wrong with my camera.  The image I provided above is when there is still some sun light left in the late afternoon, so it doesn't look nearly as bad as when the sun sets.

As Colleen suggested, it is likely that Christopher and I got bad cameras since no one else has complained.  I certainly hope so because I'd hate to have to tell customers that they can only use my app during the day.

Regards,

Jack

[samontab]

My camera also looks similar to that. I reckon it is just the quality of the sensor.