- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello
I am working with the RealSense R200 camera using Visual Studio & C++.
I need to use OpenCV to do image processing so I converted the PXCImage * into cv::Mat.
But, as you can see, the two images are different (like if I am loosing information).
Does anybody can help me?
Render display screenshot Imshow display screenshot
Thanks,
Lucie
- Tags:
- Difference
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Lucie,
Thank you for contacting us.
We've investigated about your case and found two useful links that relate to this case. The first one is a community thread where the user has a similar question and the second one is a guide to use LibRealSense and OpenCV to stream RGB and Depth Data. The latter has a complete example that explains how to use LibRealSense and OpenCV to display both an RGB stream as well as Depth stream into two separate OpenCV created windows.
https://software.intel.com/en-us/forums/realsense/topic/599415
https://software.intel.com/en-us/articles/using-librealsense-and-opencv-to-stream-rgb-and-depth-data
Let us know if you find this useful.
Regards,
-Sergio A
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Sergio A,
Thank you for you answer ,
Unfortunately, I already looked those two links and I am still blocked. It might be a format problem or I don't know..
This is my code (a mix of what I found on internet), if someone can help me :
# include "opencv2/core.hpp"
# include "opencv2/highgui.hpp"
# include
# include "pxcsensemanager.h"
# include "pxcmetadata.h"
# include "util_cmdline.h"
# include "util_render.h"
# include
# include
using namespace std;
using namespace cv;
void RecordORPlayback(pxcCHAR *file, bool record) {
// Create a SenseManager instance
PXCSenseManager *sm = PXCSenseManager::CreateInstance();
PXCSession *session;
// Set file recording or playback
sm->QueryCaptureManager()->SetFileName(file, record);
// Select the color stream
sm->EnableStream(PXCCapture::STREAM_TYPE_DEPTH, 320, 240, 30);
// Initialize and Record 300 frames
sm->Init();
for (int i = 0; i<100; i++) {
// This function blocks until a color sample is ready
if (sm->AcquireFrame(true)
// Retrieve the sample
PXCCapture::Sample *sample = sm->QuerySample();
// Go fetching the next sample
sm->ReleaseFrame();
}
// close down
sm->Release();
}
cv::Mat PXCImage2CVMat(PXCImage *pxcImage, PXCImage::PixelFormat format)
{
PXCImage::ImageData data;
pxcImage->AcquireAccess(PXCImage::ACCESS_READ, format, &data);
int width = pxcImage->QueryInfo().width;
int height = pxcImage->QueryInfo().height;
if (!format)
format = pxcImage->QueryInfo().format;
int type;
if (format == PXCImage::PIXEL_FORMAT_Y8)
type = CV_8UC1;
else if (format == PXCImage::PIXEL_FORMAT_RGB24)
type = CV_8UC3;
else if (format == PXCImage::PIXEL_FORMAT_DEPTH_F32)
type = CV_32F;
cv::Mat ocvImage = cv::Mat(cv::Size(width, height), type, data.planes[0], data.pitches[0]);
pxcImage->ReleaseAccess(&data);
return ocvImage;
}
int main(int argc, WCHAR* argv[]) {
//Define some parameters for the camera
cv::Size frameSize = cv::Size(320, 240);
float frameRate = 30;
cv::Mat frameColor = cv::Mat::zeros(frameSize, CV_8UC3);
cv::Mat frameDepth = cv::Mat::zeros(frameSize, CV_32F);
/* Creates an instance of the PXCSenseManager */
PXCSenseManager *pp = PXCSenseManager::CreateInstance();
if (!pp) {
wprintf_s(L"Unable to create the SenseManager\n");
return 3;
}
/* Sets file recording or playback */
PXCCaptureManager *cm = pp->QueryCaptureManager();
pxcCHAR *filename = (L"DEPTHRenderTEST");
RecordORPlayback(filename, true);
// Create stream renders
UtilRender renderc(L"Color"), renderd(L"Depth"), renderi(L"IR"), renderr(L"Right"), renderl(L"Left");
pxcStatus sts;
do {
/* Apply command line arguments */
pxcBool revert = false;
PXCVideoModule::DataDesc desc = {};
desc.deviceInfo.streams = PXCCapture::STREAM_TYPE_COLOR | PXCCapture::STREAM_TYPE_DEPTH;
revert = true;
pp->EnableStreams(&desc);
/* Initializes the pipeline */
sts = pp->Init();
if (sts
if (revert) {
/* Enable a single stream */
pp->Close();
pp->EnableStream(PXCCapture::STREAM_TYPE_DEPTH);
sts = pp->Init();
if (sts
pp->Close();
pp->EnableStream(PXCCapture::STREAM_TYPE_COLOR);
sts = pp->Init();
}
}
if (sts
wprintf_s(L"Failed to locate any video stream(s)\n");
pp->Release();
return sts;
}
}
/* Reset all properties */
PXCCapture::Device *device = pp->QueryCaptureManager()->QueryDevice();
device->ResetProperties(PXCCapture::STREAM_TYPE_ANY);
for (;;) {
// for (int i=0;i<51;i++){
/* Waits until new frame is available and locks it for application processing */
sts = pp->AcquireFrame(true);
if (sts
if (sts == PXC_STATUS_STREAM_CONFIG_CHANGED) {
wprintf_s(L"Stream configuration was changed, re-initilizing\n");
pp->Close();
}
break;
}
const PXCCapture::Sample *sample = pp->QuerySample();
if (sample) {
if (sample->depth && !renderd.RenderFrame(sample->depth)) break;
if (sample->depth) {
frameDepth = PXCImage2CVMat(sample->depth, PXCImage::PIXEL_FORMAT_DEPTH_F32);
/* string result_depthCV = "DepthOpenCV.png";
imwrite(result_depthCV, frameDepth);*/
namedWindow("Depth", CV_WINDOW_NORMAL);
imshow("Depth", frameDepth);
waitKey(1);
}
if (sample->color) {
frameColor = PXCImage2CVMat(sample->color, PXCImage::PIXEL_FORMAT_RGB24);
namedWindow("Color", CV_WINDOW_NORMAL); //works without this line too
imshow("Color", frameColor);
waitKey(1);
}
/* Releases lock so pipeline can process next frame */
pp->ReleaseFrame();
}
if (_kbhit()) { // Break loop
int c = _getch() & 255;
if (c == 27 || c == 'q' || c == 'Q') break; // ESC|q|Q for Exit
}
}
} while (sts == PXC_STATUS_STREAM_CONFIG_CHANGED);
wprintf_s(L"Exiting\n");
// Clean Up
pp->Release();
system("Pause");
return 0;
}
Thanks,
Lucie
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Lucie,
We looked online and even though we didn't find any official guides that might help you with the code, we did found a couple of links that may be helpful
https://software.intel.com/en-us/forums/realsense/topic/624897
http://stackoverflow.com/questions/37266390/how-to-convert-realsense-rgb-frame-to-cvmat-in-ubuntu-or-other-linux-env
From the looks of both pictures, they're obviously different, but, the depth patterns don't seem to be so different from each other.
It looks as if one of the images needs further calibration in order to look the same as the first image.
I looked at some examples and found that frameColor and frameDepth used this way solved the issue for some customers:
cv::Mat frameColor = cv::Mat::zeros(resolutionColor.height, resolutionColor.width, CV_8UC3);
cv::Mat frameDepth = cv::Mat::zeros(resolutionDepth.height, resolutionDepth.width, CV_32FC1);
Keep us updated.
-Sergio A
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Lucie,
Do you still need assistance with this case? We'll be waiting for your response.
-Sergio A
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Sergio,
Thanks for your interest in my issue.
I just solved it
Finally, I used :
cv::Mat frameDepth = cv::Mat::zeros(frameSize, CV_16U);
cv::Mat frameDepth8b = cv::Mat::zeros(frameSize, CV_8UC1);
and then,
if (sample->depth) {
frameDepth = PXCImage2CVMat(sample->depth, PXCImage::PIXEL_FORMAT_DEPTH);
frameDepth.convertTo(frameDepth8b, CV_8UC1, 255.0 / 4096.0);
// imwrite
string result_depthCV = "DepthOpenCV.png";
imwrite(result_depthCV, frameDepth8b, { CV_IMWRITE_PNG_STRATEGY_DEFAULT });
// imshow
namedWindow("Depth", CV_WINDOW_AUTOSIZE);
imshow("Depth", frameDepth8b);
waitKey(1);
}
I don't really understand with I have to convert frameDepth in CV_8UC1 because normally imwrite can save an 16-bit image in PNG format. But it now at least, it works
Thanks again ,
Lucie
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Lucie,
Thank you for confirming you've fixed the issue and sharing your results with the community. Don't hesitate to come back if help is needed, we'd be happy to assist you.
Regards,
-Sergio A
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page