- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
hi,
I want to do some image processing using depth and RGB stram of realsense camera. but i am not able to integrate opencv with realsense sdk. how can i receive depth frame in Mat variable of opencv so that i can do some processing in it.
thanks !!
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
RealSense SDK doesn't provide frames in mat format, but you can easily convert to mat
first you access frame data as follows
// image is a PXCImage instance
PXCImage::ImageData data;
image->AcquireAccess(PXCImage::ACCESS_READ,&data);
Note: image planes are in data.planes[0-3] with pitch data.pitches[0-3]
Then you create an empty Mat using a constructor like this
Mat mat(int rows, int cols, int type)
after that you can copy data in to your mat from data above.
finally release access to data
image->ReleaseAccess(&data);
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
how can i take video stream such as rgb, depth, ir in mat variable for processing.
thanks !!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Well the type of stream you get by the AquireAccess() is based on the type of PXCImage you called that method on :)
So if you want get RGB image frame, for example then you do the following
psm = PXCSenseManager::CreateInstance();
then for each frame you do
PXCCapture::Sample *sample = psm->QuerySample();
PXCImage *rgbImage = sample->color;
PXCImage::ImageData frmData;
rgbImage->AcquireAccess(PXCImage::ACCESS_READ, PXCIMAGE::PIXEL_FORMAT_NV12, &frmData);
then you create a mat and copy the data over from frmData. Once you no longer need frmData, you just release the access
rgbImage->ReleaseAccess(&frmData);
Hope this helps
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
This is what I am currently using:
cv::Mat frameColor = cv::Mat::zeros(resolutionColor.height, resolutionColor.width, CV_8UC3); cv::Mat frameDepth = cv::Mat::zeros(resolutionDepth.height, resolutionDepth.width, CV_32FC1); cv::Mat frameIR = cv::Mat::zeros(resolutionIR.height, resolutionIR.width, CV_8UC1);
color stream:
PXCImage::PIXEL_FORMAT_RGB32, and then read the image as BGRA unsigned ints.
depth stream:
PXCImage::PIXEL_FORMAT_DEPTH_F32, and then read the image as floats.
IR stream:
PXCImage::PIXEL_FORMAT_Y8, and then read the image as unsigned chars.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
this is what i am doing to get depth stream.
/* Creates an instance of the PXCSenseManager */
PXCSenseManager *pp = PXCSenseManager::CreateInstance();
if (!pp)
{
wprintf_s(L"Unable to create the SenseManager\n");
return 3;
}
/* Collects command line arguments */
UtilCmdLine cmdl(pp->QuerySession());
if (!cmdl.Parse(L"-listio-nframes-sdname-csize-dsize-isize-file-record-noRender-mirror",argc,argv)) return 3;
// Create stream renders
UtilRender renderd(L"Depth");
pxcStatus sts;
do {
/* Apply command line arguments */
if (cmdl.m_dsize.size()>0)
{
pp->EnableStream(PXCCapture::STREAM_TYPE_DEPTH, cmdl.m_dsize.front().first.width, cmdl.m_dsize.front().first.height, (pxcF32)cmdl.m_dsize.front().second);
}
/* Sets file recording or playback */
PXCCaptureManager *cm=pp->QueryCaptureManager();
cm->SetFileName(cmdl.m_recordedFile, cmdl.m_bRecord);
if (cmdl.m_sdname) cm->FilterByDeviceInfo(cmdl.m_sdname,0,0);
if (cmdl.m_csize.size()==0 && cmdl.m_dsize.size()==0 && cmdl.m_isize.size()==0)
{
PXCVideoModule::DataDesc desc={};
if (cm->QueryCapture())
{
cm->QueryCapture()->QueryDeviceInfo(0, &desc.deviceInfo);
} else
{
desc.deviceInfo.streams = PXCCapture::STREAM_TYPE_DEPTH;
}
pp->EnableStreams(&desc);
}
/* Initializes the pipeline */
sts = pp->Init();
if (sts<PXC_STATUS_NO_ERROR) {
wprintf_s(L"Failed to locate any video stream(s)\n");
break;
}
/* Stream Data */
for (int nframes=0;nframes<cmdl.m_nframes;nframes++) {
/* Waits until new frame is available and locks it for application processing */
sts=pp->AcquireFrame(false);
if (sts<PXC_STATUS_NO_ERROR) {
if (sts==PXC_STATUS_STREAM_CONFIG_CHANGED) {
cerr << "Stream configuration was changed, re-initilizing\n" << endl;
pp->Close();
}
break;
}
/* Render streams, unless -noRender is selected */
if (cmdl.m_bNoRender == false)
{
const PXCCapture::Sample *sample = pp->QuerySample();
if (sample)
{
if (sample->depth && !renderd.RenderFrame(sample->depth)) break;
}
}
/* Releases lock so pipeline can process next frame */
pp->ReleaseFrame();
if( _kbhit() ) { // Break loop
int c = _getch() & 255;
if( c == 27 || c == 'q' || c == 'Q') break; // ESC|q|Q for Exit
}
}
} while (sts == PXC_STATUS_STREAM_CONFIG_CHANGED);
cerr <<"Exiting..."<<endl;
now where should i add
cv::Mat frameDepth = cv::Mat::zeros(resolutionDepth.height, resolutionDepth.width, CV_32FC1);
in the code so that i can get depth stream in Mat frameDepth.
i am not able to get depth stream in Mat variable.
can you please provide me full source code where mat access any stream . I am stucked from last 10 days into same place.
please help me
thanks !!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You will find a solution here: http://stackoverflow.com/questions/32609341/comvert-a-pxcimage-into-an-opencv-mat/32609342
Regards,
Rémy
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
hey!
how do u get a good depth result with librealsense?
I am currently using this:
const uint16_t * depth_frame = reinterpret_cast<const uint16_t *>(dev->get_frame_data(rs::stream::depth));
cv::Mat depth16(360, 480, CV_16U, (void*)depth_frame);
cv::Mat depthM(depth16.size().height, depth16.size().width, CV_8UC1);
depth16.convertTo(depthM, CV_8UC1);
// min/max distance from the camera
unsigned short min = 0.5, max = 2.5;
cv::Mat img0 = cv::Mat::zeros(depthM.size().height, depthM.size().width, CV_8UC1);
cv::Mat depth_show;
double scale_ = 255.0 / (max-min);
depthM.convertTo(img0, CV_8UC1, scale_);
cv::applyColorMap(img0, depth_show, cv::COLORMAP_COOL);
cv::imshow("DEPTH", depth_show);
it returns a "depth image" from the video, but it is really noise with grains, not possible to see details of depth.
thanks
![](/skins/images/872293744008A34B36F8ABF94A46CC66/responsive_peak/images/icon_anonymous_message.png)
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page