What is the best setup to capture dog heads and necks for 3D capture for the purposes of a measurement study?

[APepe2]

I am looking capture live dog head and necks for a study I am doing. The animals can remain (relatively) still during the image capture session. We are planning a multi-cam setup do enable simultaneous capture, since the animals will be alive not able to sit still long enough for a single camera scan.

I believe I can use the following, but would very much appreciate help on what the best setup would be.

Four (4) D435 cameras > RecFusionPro

RecFusion Pro > GeoMagic

GeoMagic > Solidworks

Ultimately the files need to be brought into Solidworks for some analysis work.

What would the best setup be to capture the dog head and neck? Number of cameras? Lighting? Backdrop? Software?

We do not need to do real-time process of the image, but would like someway to know if the data we captured is "accurate" as we will not have the ability to recapture subjects after the initial session.

Any recommendations would be greatly appreciated!

Thank you,

AJ

[MartyG]

It should certainly be possible to do 3D model scanning with the multi-camera setup that you have in mind. I wonder if it would be easier though for you to use a Windows 10 tablet equipped with a built-in RealSense camera so that you could walk around the dog whilst holding the tablet to take the scan.

Solidworks recommend the RealSense-equipped Wacom MobileStudio Pro tablet, which comes with a year's subscription to the Artec 3D scanning software.

Wacom MobileStudio Pro

https://www.solidworks.com/partner-product/wacom-mobilestudio-pro

Artec

Although the Solidworks article about the Wacom does not not provide a link to the specific Artec software that they mention, Wacom's own official page says that the software provided is Artec Studio 12 Ultimate.

https://www.wacom.com/en-cn/products/pen-computers/wacom-mobilestudio-pro-overview#MobileStudio-3D

https://www.artec3d.com/3d-software/solidworks-cad

A 3D model created with Artec Studio can be directly imported into Solidworks, as Artec has a Geomagic plugin. In regard to accuracy checking, the above link states that "Geomagic for SOLIDWORKS also allows you to check the feature accuracy of your model with the available standard deviation analysis tool". This could potentially provide a fast enough workflow to check the data whilst the dog is still with you so you can take another scan if required.

If the Wacom MobileStudio Pro tablet does not meet your needs, there are a few other models of RealSense Windows 10 tablet available that contain a built-in D410 or D415 camera and could potentially run the Artec software.

https://support.intelrealsense.com/hc/en-us/community/posts/360033966114-DT-Research-article-on-using-RealSense-in-rugged-tablets-and-other-tablet-suppliers

If you feel that a tablet-based approach is not the way that you want to go, I will be happy to discuss a multi-camera setup with you.

[APepe2]

Hi,

Thanks for the reply. Unfortunately, I do not feel a tablet based approach will work. The animals we are studying will be live, and therefore will only be able to remain still for so long. So, using a single camera/tablet would have all sorts of distortion and artifacts from the resultant image. This is why I was thinking about a multi-camera setup since it would allow us to capture the animal with one shot and we could stitch the images together to create the model.

[MartyG]

The CONIX Research Center at Carnegie Mellon has taken the approach of capturing point-cloud type scans with multiple RealSense cameras and then using an ethernet network to stitch the point clouds together into a single cloud. A point cloud can then be subsequently converted into a solid mesh object (so this approach requires a bit more labor than just outputting a finished 3D model like the kind produced by RecFusion).

https://forums.intel.com/s/question/0D50P00004Fjfd9SAB/a-conix-research-center-program-for-combining-point-clouds-from-multiple-400-series-cameras

[APepe2]

Hey Marty,

Thanks for the information. I spoke with one of my engineers and he isn't too keen on the idea - seems like the approach might be better used when mapping a room for the purposes of creating a 3D model...?

[MartyG]

No problem at all. I will return to the scope of your original question, based around a RecFusion Pro setup, and your query about number of cameras, lighting, etc.

(a) For a 360 degree capture, 8 cameras would be optimal, though I have seen it done with six. For a 180 degree view, four cameras (like the setup in the Intel demo at the Sundance festival that I linked to earlier).

The more cameras you have, the better the data quality (less blind-spots in the data and more redundancy due to the same areas being covered by the field of view of more than one camera).

(b) In regard to lighting, try to avoid fluorescent lights such as ceiling strip-lights (as unlike bulb-based lights, they can create noise by flickering at frequencies hard to see with the human eye). LED-based lights may be beneficial due to creating less shadow in the scene.

(c) If you can light the scene well, you may find that you get less noise on the image by turning off the camera's IR Projector (which projects a pattern of semi-random dots onto the scene) and let the camera use the room's ambient light instead to process the depth in the scene. Unlike most depth cameras, the 400 Series cameras actually work even better in strong light. The dogs may feel more comfortable about not being covered in lots of dots too!

(d) It looks like you can record multi-cam footage with RecFusion Pro and edit it at your leisure later. There is also the option of creating a custom setup where you start and stop capture with a trigger, though if you are happy to edit the raw RecFusion footage into what you need then this may not be necessary. You can certainly craft a trigger setup later if RecFusion's capture method does not best suit your needs.

(e) In regard to software for editing the captured solid 3D model and cleaning it up, a good free option is MeshLab.

http://www.meshlab.net/

[APepe2]

Thank you for the thoughtful reply! I am going to try testing the cameras on some mundane objects (boxes and spheres) to see what sort of end result I am able to achieve before moving to dogs.

Do you have experience putting these types of rigs together? Wouldn't mind getting connected to someone professionally who has direct experience that I could potentially work with directly to ensure I am putting together the best system for the captures I need to do.

[MartyG]

I have not constructed a multiple camera system myself so my knowledge is theoretical. There are extensive resources available on the subject though, and I hope that my own input can be helpful to you.

Intel's online seminar on multiple cameras would be a very good reference source. It looks at camera arrangements from 3 mins 20 seconds onward in the video. The type of camera arrangement you would need is an 'inward' facing one.

​https://youtube.com/watch?v=drBCxHhbxI0

​The video is 30 minutes of formal presentation and then a lengthy Q&A session with attendees afterward. A text transcript I created of the Q&A is available here:

https://forums.intel.com/s/question/0D50P00004B5zslSAB/transcript-of-text-faq-questions-from-the-intel-multiple-realsense-cameras-webinar?language=en_US

​The video touches upon full body capture with multiple cameras, a subject that is covered far more deeply in a later seminar.

https://youtube.com/watch?v=VSHDyUXSNqY

[jeniliapeter]

Thank you so much all of you for the thoughtful reply! I am going to try testing the cameras for capture dog heads and necks for 3D capture for the purposes of a measurement study...