Some advice or guidance required...

[JRoge4]

Hi this is my first post so please be gentle,

I'm currently working with a Children's Neurological Charity in the UK called 'Cerebra' that manufactures products for children with neurological disorders to aid in daily life. We are currently making bespoke helmets for children with different skull morphology's that can't buy units that are sufficient in the shops, enabling them to participate in sporting activities such as Horse Riding & Cycling.

We need to develop a technique for gathering the children's head geometry (3D scaled mesh) that can be used in modelling the helmet using CAD applications. We have been previously using a Artec EVA 3D (borrowed) scanner and travelling around the country but need to develop a way of gathering this information remotely as we are a very small design team and can't keep up with demand.

I have been looking at a range of different options including both Photogrammetry and 3D Scanning. Having experimented with the Xbox 360 Kinect camera and 'Skanect / ReconstructMe' software, I was able to gain fairly accurate results which are suitable for our needs. However the equipment involved is fairly dated and is on the bulky side, in terms of being incorporated into a design solution that could be customised and simplified to the point a parent or carer would be able to operate.

For this reason I am very interested in understanding the fundamentals of operating both the:

Intel® RealSense™ Robotic Development Kit & Intel® RealSense™ Depth Camera D415. (Or anything else that maybe suitable)

• Would someone be able to shed some light on how complicated the process of connecting the Up board to the R200 in order to start scanning? What additional hardware/software/code is required if we chose this option?

• What are the minimum computer requirements for operating the D415 and are there currently any software applications that can be used in conjunction for gathering 3D geometry?

Our background is product design with a very basic knowledge of electronics and even less with software development or UI design. From what I've read It sounds perfect for incorporating into a set of apparatus that we can send out by courier, however I don't want to purchase a unit for testing unless i'm certain it will work as we have a very tight budget.

I'm sorry for the length of this post but any advice or guidance would be gratefully received!

Thanks for reading,

Josh

[MartyG · Accepted solution]

Connecting the R200 to the Up Board is relatively easy. Additional parts not in the box that you may need to purchase are an HDMI cable for connecting the Up Board's output to a TV or monitor, and a USB storage stick for transferring an operating system such as Linux onto the Up Board's internal storage. The R200 is now a retired product though and will not be available once the Robotic Development Kits have sold out, so it may not be an appropriate option for you if you are seeking future-proofing.

The D415 and D435 work with any Intel or ARM processor chip, so the main requirement is just a USB 3.0 port. These cameras are planned to have a 3-year life cycle until 2020 and then a new next-generation model called the D600 will be introduced. It is planned that transition between the D415 / D435 and D600 will be straightforward for those who want to upgrade their existing projects to the newest hardware in 2020.

If custom design is important to you, the new cameras can be purchased in caseless module form for integration into your custom hardware designs.

Support for 3D model scanning on the new D415 and D435 cameras is still a bit awkward but getting better. At the moment, it is necessary to create a type of 3D scan called a Point Cloud (which the RealSense SDK 2.0 software can do) and then convert it into a solid 3D model by converting the points to a 'mesh'. The article below discusses one way to do the process.

https://www.andreasjakl.com/capturing-3d-point-cloud-intel-realsense-converting-mesh-meshlab/ https://www.andreasjakl.com/capturing-3d-point-cloud-intel-realsense-converting-mesh-meshlab/

[JRoge4]

Thanks for such a quick response.

That's all really useful info!

In terms of future proofing we only need 1 unit that is hopefully going to be returned to us after each scan, is this a problem in terms of the SDK or software being no longer available / supported? For example could I use a similar setup with the 'Skanect' software I mentioned having tested with the Kinect, installed on the Up board? The ability to send out apparatus without an expensive computer setup would be greatly beneficial to us, as the risk of equipment being stolen is more likely.

Having seen some video's of people experimenting with the R200 camera module the results look far superior to my current Kinect / Skanect setup.

I read another forum discussion you were helping out with about and can see there are issues with the fact there is no USB-C connection, is there a suitable alternative to this option that you could think of? I'm not sure how many tablets support USB-C 3.0 but does going down this route involve any additional computing power? The Skanect website sells an Occipital (https://structure.io/support/how-do-i-use-skanect-pro-my-structure-sensor Structure Sensor Support Center | How do I use Skanect Pro with my Structure Sensor? ) but although the scanning process is done using the iPad it still requires a PC/Mac to aid in the capture process.

Being able to buy just the camera unit does sound like an additional benefit if we did go down the D415 route and I have worked with point cloud data previously when exploring the photogrammetry method. Is it possible to try the SDK 2.0 software without a camera module? I'm interested in how 'easy to use' the user interface is as the solution we need to come up with has to be operated by a parent or carer.

Thanks again so much for your help!

Josh

[MartyG]

Whilst the old SDKs will no longer be updated, they can be installed and run without an internet connection. So you could store the installation packages for the SDK and camera drivers on a couple of backup dvds for years (assuming the programs remain compatible with future versions of Windows).

If your budget allows, it might be a good idea to get two of the Robotic Development Kits instead of one in case one R200 camera breaks, in case you cannot get a replacement R200 in future.

I wonder if it would be cheaper and easier for you to keep the camera at your base and send your clients a package with some kind of flexible material in it that keeps its shape once depressed. This is similar to how dentists make dentures - they have the patient bite into a plate of plaster-like material and then use that mould to make the dentures. In your case, the returned material could have its dimensions scanned by the camera back at base to get the correct skull proportions for the helmet.

The type of material used would obviously have to be convenient and sensitive to the end-user and their parents / carers. For example, a current kids craze in the UK is a material called Slime. This can be purchased in £5 tubs or in packets of powder that mix in water to make the slime. If slime can be set, then the client could dip their head in a bowl of slime (I guess you could call it 'moulding powder'!) and then have the appropriate person mail back the mould that is left. A microwave would be a good way to set the slime hard - ovens or microwaves are sometimes used in making home-made slime.

Regarding using SDK 2.0 without a camera: once it is installed, you can load pre-made sample files called 'ros bags' into it and use that file for tests instead of live camera data. the link below provides a set of pre-made sample files you can load in.

https://github.com/IntelRealSense/librealsense/blob/master/doc/sample-data.md librealsense/sample-data.md at master · IntelRealSense/librealsense · GitHub

Regarding the D415: it uses a Type-A to Type-C, also known as A to Micro-B. The full-size USB connector goes in the computer / board and the thin micro-connector end goes in the side of the camera.

Mobile devices tend to provide USB compatibility via a similar USB port to the Up Board called a USB 3.0 OTG (On The Go). This enables a full USB 3.0 device to be connected to the thinner OTG port via an adapter. The Robotic Development Kit includes one of these adapters, but you can also easily get them from retailers such as Amazon.

[JRoge4]

Hi MartyG,

That's fantastic news and will solve all our problems if we can get the R200 up and working, at least as well as the Xbox 360 Kinect & Skanect software I mentioned previously. I will probably take your advice and buy a development kit for myself while they're still available too, as I often need to be able to scan objects for use in 3D printing. We did actually test the Sony 3D creator App on the XZ1 when it initially came out but it required quite a lot of practice to get usable data from and for some reason the App only lets you export the data in 5cm,10cm & 15cm Models. I know these could be scaled up using CAD but at that scale for the size of a human head we lost the level of accuracy we need for making the helmets. We need roughly a 2mm level of accuracy across the whole shape as padding gives us a little design leeway.

I did initially think of using a similar type method to using the dentist material you referred to, but a few of my colleagues said that some of the autistic children we work with wouldn't be able to deal with a process of this nature. Not sure why really as we're going to be putting a rubber cap on their head if they have much hair anyway, but I've got to continue to develop a few alternative methods as that's what my supervisors have asked me to do. I may re-visit this option if I can't get the above option working.

I have previously tried the CAPPASITY EASY 3D SCAN - App which I think is compatible with R200 but I'm not sure whether the data captured is exportable as a Scale 3D mesh / STL file?

Do you have any idea if it would be possible to power the Up Board & R200 unit using a battery to power the setup? I'm thinking then it would be a handheld device I could attach a touchscreen display to and load the software permanently so it loads straight up.

With the SDK 2.0 do you mean if I installed it on any PC? Just to test the UI.

Thanks again so much for your help,

Josh

[MartyG]

I ca understand very well the hesitation that some parents / carers might have, and the fear that younger special needs kids might have, with applying a mould material to the head, hence my mention of taking sensitivities into account. I have done tech work with disability organisations and online events related to the promotion of accessibility. I also have family members with autism / Aspergers. So I very much get the complexities involved and can appreciate your work.

The R200 model was especially suited to scanning the human head and body, so it will likely be a good choice for your needs.

This may be a dumb idea, but I wonder if you could produce a generic helmet size for the solid outer part and put separate pads inside that can be individually air-inflated - like a combination between a pro leather football's toughness and the inflatability of swimming armbands. If each padding piece was individually inflatable then the inside could be moulded around the skull because adjustments to each side would be unique and not affect the shape of the pads on the other sides.

The 3D modeling software ReconstructMe can export STL files and be used with R200.

http://reconstructme.net/reconstructme-sdk/ ReconstructMe for Developers | ReconstructMe

http://reconstructme.net/2017/03/13/reconstructme-sdk-support-for-intel-realsense/ ReconstructMe SDK support for Intel RealSense | ReconstructMe

An alternative to using a single-board computer may be a 'computer on a stick' like Intel's own Compute Stick. The fist two models on the page below that use the Intel Atom Z8300 "Cherry Trail" processor would probably be best suited, since the R200 can be used with Intel Atom Cherry Trail processors (the Up Board has a similar one, the Cherry Trail Z8350).

https://www.intel.com/content/www/us/en/products/boards-kits/compute-stick.html Intel® Compute Stick

Yes, you can install the SDK 2.0 software to Windows, Linux or Mac computers.

[JRoge4]

Hi Marty,

Sorry for the delay in response. Yeah the team do some really good work for all kinds of projects, I'm studying for my masters and this is my 'Major Project'.

I opted for purchasing both the Robot Development Kit and the Intel Compute Stick so I'll have a play about with them as soon as they arrive. I'm sure I may need a little more advice at some point...

The larger helmet idea is a good one as we've already got a mould that we lay up with a mixture of Resin, Fiberglass and Kevlar to creat the hard outer shell. I'll do a little test with some kind of 2 part setting material like we discussed yesterday as I think that will at least show the possibilities available by casting. I've been looking into the dentist material you mentioned previously but the only stuff i've found is incredibly expensive here in the UK. I guess it needs to be something that is used in medical applications so that it's a non-toxic substance, just in case the silicone cap splits or breaks!

I did try using ReconstructMe previously but couldn't get my Xbox 360 Kinect to sync with the software, I achieved a good level of detail with 'Skanect' but am using a demo version that only lets you export at 5000 faces.

Have a good weekend anyway and once I've got some results I'll upload them so people can see.

Thanks

Josh

[MartyG]

If you Google for 'home made non toxic slime' then you can find recipes made from materials such as corn starch.

https://coolmompicks.com/blog/2017/03/02/natural-safe-homemade-slime-recipes-no-borax/ 5 natural, safe homemade slime recipes - no Borax! | Cool Mom Picks

Look forward to your test results!

[JRoge4]

Hi Marty,

Sorry for such a delay in updating!

I had a problem with delivery of my 'Robotic Development Kit' and had to pay lots of customs tax for some reason.

Anyway, I got it up and running with Ubuntu Desktop and followed the installation guide: https://01.org/developerjourney/recipe/intel-realsense-robotic-development-kit Intel RealSense Robotic Development Kit | Developer Journey up until step 5 - 'sudo reboot' command and for some reason the UPboard restarted in Aptio Setup Utility using UEFI. Whatever I seemed to try at this point didn't seem to work as the boot options were limited and my knowledge minimal. At this point, I tried to install the SDK and relevant software packages on my laptop just to get some usable data to try and test the camera. My laptop is a fairly new HP Spectre - i7 2.7GHz so I knew it should be able to run all the software/SDK camera fairly easily.

The greatest success I have had has been using the R200 module with itSeez3D. It works very well with gathering 'Bust' (Sholders up portrait) but I found the 'Full Body Scanning' method a little harder to get right.

Most of the other applications I tried even on my laptop wouldn't work:

ReconstructMe crashes every time I attempt to set up the R200 as the device, Skanect doesn't work with the R200, most of the Intel SDK Sample Browser tools didn't seem to work (3D Scanner does work but I couldn't seem to record anything in terms of exporting as an STL/OBJ or point cloud data.). On the Cappasity Easy 3D Scan for Intel Realsense app I get the following error message:

'13:04:22.280: ERROR: 3D camera is not connected or DCM drivers are not installed.

If you don't have DCM drivers, please download them from Intel website - https://downloadcenter.intel.com/download/25044/Intel-RealSense-Depth-Camera-Manager https://downloadcenter.intel.com/download/25044/Intel-RealSense-Depth-Camera-Manager

For Creative BlasterX, Razer Stargazer cameras download SR300 driver. '

I've then re-installed the https://downloadcenter.intel.com/download/25044/Intel-RealSense-Depth-Camera-Manager Download Intel® RealSense™ Depth Camera Manager R200 Drivers and also the R200 Camera Calibrator to no avail. I also tried versions of RecFusion, KScan3D and most other things I came across as it would be helpful for me to be able to scan objects as well as people (for a little scientific proof/accuracy.).

I did purchase an Intel Compute Stick to try out too however when I realised that this unit also needs to be connected to the mains for power it didn't seem to offer an alternative to the UP board really.

I then installed Windows 10 running on the UP Board and got the itSeez3D App working on there, the next step was looking at integrating a power supply that could power the UP Board, R200, Touchscreen Display and a Voltage/current regulator module (with backup power supply to ensure the OS can always shut down properly) to run the 'itSeez3D' application. From a little research online I found a few items that would possibly suit our application: (I think I came across another forum topic you helped out on when these were discussed previously!)

https://up-shop.org/up-peripherals/113-3000mah-lipo-battery-use-for-ups-hat.html https://up-shop.org/up-peripherals/113-3000mah-lipo-battery-use-for-ups-hat.html

https://up-shop.org/up-peripherals/112-ups-and-wide-range-dc-in-hat.html https://up-shop.org/up-peripherals/112-ups-and-wide-range-dc-in-hat.html

At this point, before looking into pricing up touchscreen displays I wanted to get some detail on whether a suitable tablet with either USB 3.0 Type A,B & C may be a better solution, used in conjunction with the R200 module? I got sent two types of cable with the development kit (USB 3.0 Type: B-B + USB 3.0 Type: B-A) but was wondering:

• Would a USB 3.0 Type: B-C work as there seem's to be a lot more options out there if so?
• Could you see a problem using a tablet like this: https://www.amazon.co.uk/dp/B06ZZ3F8NH/ref=psdc_429892031_t1_B00A9308NM https://www.amazon.co.uk/dp/B06ZZ3F8NH/ref=psdc_429892031_t1_B00A9308NM - it seems to have USB 3.0 Type: A, runs Windows and has similar processor/speed as the Intel Compute stick? This would provide a fairly straightforward solution for us as all CE Testing etc would be completed if we want to send this setup out to parents/carers.
• I know the D415 module has a USB 3.0 Type C connection but we're keen on using an application such as 'itSeez3D' as the software if fairly user-friendly and easy to operate with tip's on how to scan, sensors to highlight poor lighting and can pick up on low-level movement without having to re-start the whole scanning process. Is there any alternative programs emerging that I may not have stumbled across?

Sorry for the lengthy slightly muddled message but it feels like I've covered quite a lot of ground in the last few weeks without a huge amount of success.

Thanks in advance if you can shed any light on any of the above,

Josh

[MartyG]

Regarding your Ubuntu boot problem: have you removed the USB boot stick before you reboot? Once you have installed the OS onto the Up Board's internal storage, you should not need the stick inserted as you can boot off the OS installed on the board now.

Regarding the Compute Stick: yes, I have discussed battery packs before.

You should be able to power it off a USB-based battery pack instead of the mains, though I believe it isn't an officially Intel-recommended method of powering it. Here's some example discussions on the subject

On the topic of using a tablet with the camera: history shows that tablets are more likely to have USB power problems with RealSense cameras than full laptops and desktops.

Regarding 3D modeling support on the 400 Series cameras: it is still a bit awkward and I have not heard of any commercial packages supporting these cameras yet due to their newness. You can do 3D scanning by taking a point cloud scan with the RealSense SDK 2.0 software and then converting it into a sold object with software called MeshLab. but there isn't anything with the ease of use of the commercial packages like ReconstructMe, Easy 3D Scan and ItSeez3D yet.

Regarding using a different cable with the Up Board: I would stick to the original cable, as it is tested by Intel for use with the Up Board's USB OTG port that provides USB 3.0 connectivity. There's no guarantee a different kind of cable would work. There's no harm in trying if you can afford to write off the cost of a cable that doesn't work though.

[JRoge4]

Hi Marty,

Yeah, I realised at the end of the message there, that I was jumping from topic to topic.

The problem I've got is finding a solution that can power not just the hardware/software but also the display and R200 as well. If possible to make a wireless mobile device that is easy to operate. My laptop was able to capture good data using the itSeez3D application but I really need to find something that is a lot cheaper than that.

When you say 'history shows' what types of tablets have had the power issues in the past? Do you know that the tablet I gave a link to in my last message(as this has a full USB 3.0 Type A) or for example a Microsoft Surface (lest expensive one I can find) will definitely not work?

No problem about the different cables, I was just wondering if a USB 3.0 Type B - C could work theoretically between a tablet and the R200 module.

Thanks again for the swift response,

Josh

[MartyG]

Surface Books and Surface Pros are particularly known for troubles with RealSense. The problem is that it only affects some people but not others, so it is a lottery whether it will work well or not.

There is a pen-driven tablet called the Wacom MobileStudio Pro that has a built-in R200. It is known for having troubles with its inbuilt camera calibration app not working, but the camera comes well calibrated when you buy it anyway. Having the R200 built in avoids the problems that tablets have with the R200's USB connection.

Someone did a review of the tablet's built in R200 camera.

https://3dscanexpert.com/wacom-mobilestudio-pro-intel-realsense-r200-3d-scanning-review/ https://3dscanexpert.com/wacom-mobilestudio-pro-intel-realsense-r200-3d-scanning-review/

A similar pen-driven model with built-in R200 is is the Acer Aspire Switch 12. This also has had the calibrator app issue reported.

https://www.windowscentral.com/acer-switch-12-s-new-4k-2-1-laptop-set-dazzle-you https://www.windowscentral.com/acer-switch-12-s-new-4k-2-1-laptop-set-dazzle-you

[MartyG]

It also occurred to me to mention the Structure Sensor, a 3D scanner containing a RealSense SR300 camera that can be attached to an iPad (including cheap models like the iPad Mini, apparently).

https://3dscanexpert.com/structure-sensor-review-part-1/ https://3dscanexpert.com/structure-sensor-review-part-1/

[JRoge4]

Thanks Marty.

Sorry, I'm preparing for a conference tomorrow. I am hoping to give a practical overview of the R200 using my laptop running the itSeez3D app as I think it'll be interesting to show a scan in progress.

The Wacom tablet looks like a fantastic bit of kit but it may be a little out of our budget at the minute, I have looked at the structure sensor that can be used with an iPad previously too but will use that as a backup plan if none of the following work...

Am thinking like you said to take a risk on one of the following products that are slightly cheaper and hope that it works!

https://www.amazon.co.uk/dp/B06ZZ3F8NH/ref=as_li_ss_tl?slotNum=6&ie=UTF8&linkCode=g12&linkId=2654fed73da5aa6204b2d43059436b72&imprToken=LNXVJbznhKVjXEK8XsVgYA&creativeASIN=B00YCX7SGU&tag=brijeshgohil-21&th=1 10.1" Fusion5 Windows Tablet PC - Windows 10, Intel Quad core, Touch screen Tablet, 64-bit Chipset, 1280*800 IPS Screen,…

https://www.amazon.co.uk/ASUS-T102HA-GR035T-Transformer-Processor-Bluetooth/dp/B01LWJPS2Y/ref=sr_1_fkmr1_1?s=computers&ie=UTF8&qid=1524483358&sr=1-1-fkmr1&keywords=ASUS+10.1%E2%80%9D+Transformer+Mini+T102HA-D4-GR%2C+2in1+Touchscreen+Laptop%2C+Intel+Quad-Core+Atom%2C+4GB+RAM%2C+128GB+EMMC%2C+Pen+and+Keyboard+Included ASUS T102HA-GR035T 10.1 inch 2-in-1 Transformer Mini Tablet (Intel Atom X5-Z8350 Processor, 4 GB RAM, 64 GB eMMC, Window…

Both of the above have full-size USB 3.0 and run Windows 10 with the Atom X5-Z8350 (like the Computer Stick we discussed previously) so I'm hoping we'll get something functional.

If either work, I'll be sure to keep the board updated.

Kind regards,

Josh

[MartyG]

Both models seem to be an appropriate choice for R200, as they have Windows 10, at least one full USB 3.0 port and an Intel Atom Cherry Trail processor,