Scan Module Feedback\Ideas

[Bradley_F_]

I've been working with the 3d scan module for some time now and I have a few thoughts:

---

Resolution:

---

We need better documentation for the Resolution parameter.  I assume it's a raw (linear) width (in cells) of an oct-tree or similar acceleration structure.  In my app, I've found it best to connect it to a slider (1-32) where the value is set to 2^n.  Meaning a notch up on the slider, doubles the value.

However, I've noticed that somwhere between 24 and 32, I get some kind of resolution decrease.  Probably this is a maximum value?  Or a memory limit?  Anyhow, to me it feels like the SDK is doing something behind my value in that case.  And since there is no "ParamaterInfo" for that value, I can't query what the limits are.

Anyhow, better documenation of this value would be nice.

---

Control of Meshing algorithm:

---

I'm not exactly sure which meshing algorithm you are using.  But if there are parameters to be had, I'd like access to them.  In-particular, parameters that affect smoothing, interpolation and especially extrapolation.  Even without "solid" set, there seems to be some rather large amount of interpolation or extrapolation going on.  And I'd like to be able to get rid of it in favor of getting more "raw" data.

---

Raw Point Mode:

---

I've noticed the obj files you export have some raw points available in them.  They seem to be the outliers or rejected points.  But the mesh I get seems to be made from smoothed points, decimated points or interpolated points.  Probably a poisson mesh reconstruction?  I occasionally take multiple scans and try and align them in MeshLab afterward.  sometimes, the object you intend to scan just needs to be handled in passes.  And in that case, I would very much like to have the original "good" points that were used to build the mesh.

So I wonder if it might be possible to include some better access to raw point scan data in the form of raw point obj files.  Possibly even the ability to call the export multiple times on the same scan and get different data-sets each time?

---

BackFacing points:

---

I've noticed an oddity when I scan complex objects over some time, where the points seem to become clear or the object appears to "bald."  It's almost like they're back-facing and I've crossed behind them.  The points are not actually being culled (which is what I first thought) because if I reconstruct at that point, the "balding" areas show up in the reconstruction with data. So clearly they're still part of the structure and being processed for the mesh.  Which leads me to think this is a visualization problem.

---

Scan Volume Visualization:

---

It can be difficult to figure out where the scan volume is once you set it in VARIABLE mode.  But really, the preview window should be able to visualize the rectangular solid.  You've got the depth, so depth clipping should be easy.  I'd say it's something that the application should be able to do.  But really, the way you have constructed the preview is too complex for the application developer to take apart and divine in this module.  So the visualization issues fall to Intel in this case.

---

Automatic Align and Rig Removal:

---

Ideally, I'd prefer to remove meshlab and other scan aligning and combing methods from my workflow.  I have thoughts as to how to go about it.  But without specific knowledge of exactly which algorithms you are using, I don't want to suggest too specific an implementation.

But at a high (use) level:

I'd like to be able to scan in my platform/rig such that the SDK can recognize it and use it for localization, but also hold it out of later scans when I put my object in scene in different orientations.

Subsequent scans of the object placed on the platform/rig in multiple orientations should attempt to localize or align with one another.  Feedback is necessary to determine if the localization was successful at runtime (user.)

Each of the subsequent scans should be able to be flagged as "good" or not (reasonably done by the user if the localization or alignment is visualized).  Possibly even deleted (dump useless/mistaken scans from memory).  And finally, a call to construct the mesh should combine the flagged scans in a single meshing run (and obviously not include the original platform/rig scan.)

---

Clarification on Filters and such.

---

As far as I can tell, changing the camera's filter modes (F200) has significant effect on the scan.  I've exposed those paremters to the user in my app.  But I suspect there are certain values that are more or less appropriate for scanning.  Probably this should be in the docs.

---

Anyhow, those are my top thoughts/requests after having taken the scan module apart (to the extent I've been able to) and integrated it into a custom application.  I've been scanning objects straight for a couple of weeks now with both the F and R cameras with many many capture, device and scan parameters exposed.  And also I've been post-processing scans.  And this is what I've come up with.  Thanks for taking a look and considering my requests.

As a one-off sloppy low-res scan, it works quite well.  But really, in that manner, it's not a SDK at all.  It's an application that has no UI.  Flushing out the SDK further means access to more data and more control, such that developers can optionally offer something more complexities/finess/workflow than just a single point and click scan.

-brad

[Xusheng_L_Intel]

Great feedback. I will give those feedback to my colleague and i am pretty sure it will help us to improve 3D scan mw in the future release. Thanks!