[Realsense SDK] Merging Scan & Align Object

[AHoan1]

Dear,

I'm working on Realsense SDK and SR300,

1 - How to merge two PXCM3DScan or create multi PXCM3DScan in a PXCMSenseManager;

2 - Having two parts of an object, how to auto align?

3 - In PXCM3DScan, can you show me how to Reconstruct 2 PXCM3DScan to a file?

4 - Having *rssdk files of a model, how to merge?

Thanks!

[MartyG]

The approach taken by Intel with the 400 Series cameras is an open-source, community focused one that encourages using the camera with software from non-Intel sources. This does mean though that features that were included with previous SDKs may be harder to integrate into RealSense applications for the 400 Series. I can appreciate how this may cause frustration for users.

Support by third-party software for the 400 Series is likely to increase over time in the same way that support for past RealSense camera models were added to packages like ReconstructMe and ItSeez3D over time. I know this is not helpful if you need those functions in your application right now, and I apologize.

[AHoan1]

Dear MartyG,

I want to scan the large model 3D, what is the best Intel Realsense's camera to do that?

[MartyG]

If you wanted to scan something large, the D435 would make sense, as it has a wider field of view than the D415, meaning that you can fit more into the camera's field of view. You might have to go through the trouble of taking a point cloud scan and converting it to a mesh though.

An easier option for you may be the R200 camera. That was the camera that used to be the best choice for scanning large objects or the human body. The R200 will also work with the commercial 3D modeling software that I suggested earlier. The R200 is still available in the Intel Click online store inside the Robotic Development Kit.

https://click.intel.com/intelr-realsensetm-robotic-development-kit-2487.html Intel® RealSense™ Robotic Development Kit - Intel® RealSense™ Depth Cameras

[AHoan1]

The R200 Camera was discontinued and I think SR300 is better.

But SR300 working not stability, I need a camera has a wider field of view and low-power than SR300

[MartyG]

Then the D435 is likely your best choice. You will unfortunately have to go to the trouble of making a point cloud and converting it though until someone makes a 3D model scanning program for the 400 Series cameras.

Before there were 400 Series cameras with wider FOVs, it was possible to put more than one camera together to widen the field of view, like user McCool did with three cameras/

[AHoan1]

If I put 3 the camera together and scan, then I will receive 3 model object?

I just want an Object from scanning.

[MartyG]

If you take 3D model scans with multiple cameras then you can stitch the separate models together into a single model in 3D modeling software.

If you want to take a 3D model scan of a large object with a single camera that is not a retired model then the D435 is the best option. But it will not be easy due to the need to convert a point clould.

[AHoan1]

The SDK or there is any intel-source of support to align multi-part of an object 3D?

[MartyG]

You usually have to do 3D model stitching in a separate program.

[AHoan1]

I tried with 2 SR300 cameras but the infrared of them is the conflict.

[MartyG]

IR interference with SR300 cameras whose projections overlap is a problem that people have been trying to deal with for a long time. Intel support team member Jesus G recently said that if interference between cameras occurs, it can be reduced by lowering the exposure setting.

[AHoan1]

Dear MartyG,

I need your support,

I scan full model 3d with RealSense SR300, but I usually receive the status "lost tracking" or "lost frame" while I turnaround the camera.

How can I resolve this issue?

[MartyG]

A product reviewer encountered a similar problem with lost tracking with the Sense scanner (a scanner product with an SR300 camera inside). They were scanning round a teddy bear to try to scan it but would lose tracking when they got to the 90 degree point at the side of the bear. Their solution was to put a printed-out picture underneath the bear to add geometry detail to the scene, and this fixed the lost tracking.

https://3dscanexpert.com/3dsystems-sense-for-realsense-review/ https://3dscanexpert.com/3dsystems-sense-for-realsense-review/

Adding items to a scene is an old trick with 3D model scanning for getting better results in cases where the camera is having difficulty seeing some of the detail of the object, as it provides the camera with more geometry to lock on to when scanning.

[AHoan1]

Dear MartyG,

Thank you for your idea, but I scan a lager model (body) and do not use that idea.

By the way, I have a question:

Why can I not receive picture and frame of the model's hair and jean paint while scanning?

[MartyG]

Hair may be easier to scan if it is brushed down flat so the camera can see the thin strands easier, otherwise it will have trouble seeing the edges of the hairs. If the hair cannot be brushed down, having a complex texture near the hair, such as a hair accessory with a pattern on it, may help the camera to lock onto that area.

The visibility of jeans may depend on their color. The camera would find it easier to see colors such as blue than ones such as black or grey. This is because of physics related to the absorbing of the camera's light by black / grey shades. Spraying on a fine powder such as baby powder or foot powder can make an object easier for the camera to see.

[AHoan1]

Dear MartyG,

Please refer to the source code below:

PXCMPersonTrackingData.PersonJoint ptj=ptp.QuerySkeletonJoints();

https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/index.html?doc_pt_tracking_joints.html at the link: Intel® RealSense™ SDK 2016 R2 Documentation

Why can I get SkeletonPoint[] from PersonJoint?

I using source c# .

[MartyG]

Usually, if you cannot get a reading of the joint then it means that skeleton tracking is not enabled in your application.

Skeleton tracking does not work well in the old RealSense SDKs. It was called a 'Preview' feature, meaning that it had not been developed much and has limited functionality. Development on the old R2 and R3 SDKs has ended now, so it will not be fixed, unfortunately.

Skeleton tracking is now available for the new cameras and SDK 2.0. 3diVi is a company that offers skeleton tracking software called Nuitrack.

https://www.youtube.com/watch?v=gMPtV4NXtUo Intel RealSense D415/D435 and Nuitrack skeletal tracking SDK replace Kinect SDK - YouTube

[AHoan1]

Dear MartyG,

I watched the video https://www.youtube.com/watch?v=VWVwKylq3dk Intel RealSense SR300 3D Scanning + DSLR. Powered by Cappasity Easy 3D Scan - YouTube

and https://software.intel.com/en-us/articles/cappasity-3d-people-full-body-scan-with-intel-realsense-and-intel-edison Archived - 3D People Full-Body Scanning System With Intel® RealSense™ 3D Cameras and Intel® Edison: How We Did It | Inte…

about the https://www.youtube.com/watch?v=VWVwKylq3dk CAPPASITY .

Is it the product of Intel, Why do I not find any the sample source code of Capasity?

Could You please show me?

[AHoan1]

I really want to know the algorithm "Multi-Fusion" of Intel using to merge model 3D

[MartyG]

Easy 3D Scan is a commercial software product made by the company Cappacity. It is not the same as the similarly named '3DScan' sample program by Intel that comes with the old '2016 R2' and '2016 R3' RealSense Windows SDKs.

Source code is available for the Intel '3DScan' sample program, but it would not work with the D415 and D435 camera models. It uses the '2016 R2' or '2016 R3' SDKs.

[AHoan1]

Could you show me the source code to merge 2 raw stream records (rssdk)