FilterBoxBorder, still not perfect?

[Dambrin_D_]

I'm migrating my code from the deprecated FilterBox to the new FilterBoxBorder, and I was expecting it to fully address the problem with the old function: the need to create a temp image with proper borders.

Unfortunately, unless I'm missing something, it's still not able to do this, because it's missing a region of interest for the *source* image. Instead, it guesses it from the destination ROI, and that's only partially addressing the problem.

On this picture, if A was your image (the square itself being the full image), and you were to blur it (5x5 mask here), you would normally get B, with 2 pixels inside & outside of the image.
But all the API allows, because of the missing source ROI, is C. It's already better than the original FilterBox, but not by much, since you -still- need to create a temp image with borders, in this case, a version of A enlarged by 2 pixels & filled with the same color passed to FilterBoxBorder in ippBorderConst mode. The required border is smaller than what FilterBox required, but IppiCopyConstBorder is still needed I'm afraid.

[Dambrin_D_]

On the same subject btw, I've seen "stack blur" used in more than one libraries, and it might be interesting in IPP:

http://www.quasimondo.com/StackBlurForCanvas/StackBlurDemo.html

This said, I understand that it's not just the algo that makes the speed, but its ability to be optimized for the CPU, and I don't know if this algo is well parallelizable (I haven't looked at it), so I don't know if in practice it would be so much faster than a true gaussian.

[Dambrin_D_]

Last thing, as for the out-of-place FilterBoxBorder being a replacement for in-place FilterBox, I'm measuring strange things here, blurring a medium RGBA 500x500 image:

-FilterBox, in-place or out-of-place: around 900 microseconds
-FilterBoxBorder, out-of-place: around 600 microseconds
-FilterBoxBorder, in-place: around 1500 microseconds. This is strange, and way larger than the time it'd take to create another temporary image, so that's not what it's doing. I thought, maybe an in-place blur prevents multithreading, but then the old in-place FilterBox was still performing better.

Considering that, as I wrote above, blurring an image still requires making a copy with a border, then it makes in-place blurring still valuable, but a 2.5x slower function pretty much bans it.

[Igor_A_Intel]

Hi Dambrin,

could you be more specific on what you need from ippiFilterBoxBorder, what is wrong with IPP border concept? Could you provide a formula? Or a reference to some "standard de facto" like Matlab or OpenCV?

As regarding performance - yes, in-place flavor of FilterBoxBorder is not optimized yet, will be improved in the next IPP version. Also you should take into account that in-place variant of FilterBoxBorder is an exclusion from the general rule - IPP doesn't support in-place flavors for all other 2D filtering operations.

regards, Igor.

[Dambrin_D_]

Well, I provided a picture, my only need would be a blur that allows result B straight from image A, without the need for another temporary image.

I don't know how it would work in practice, but I think that it would also work without reformatting FilterBoxBorder to support a source ROI, but by adding a flag to it.

Right now, say you have a source rectangle of 10x10, FilterBoxBorder will produce a result of 10x10. With a new flag, for a mask size of 5x5, it would produce, for the same 10x10 image, a result of 14x14, the outer 2x2 border being what's missing to produce result B.

So yes, imagine a new flag added to the function's border mode, then in order to blur a 10x10 image, you would set the target ROI to 10+Mask.X-1 x 10+Mask.Y-1, it would produce B from the original 10x10 source ROI. This is pretty much what I was expecting FilterBoxBorder to do.

[Igor_A_Intel]

This approach contradicts IPP concept - in IPP (and in Matlab, and in OpenCV, etc.) all filtering operations are reducing operations - this is the mathematics (and physics) of the filtering process. For your purpose you can try the general purpose convolution functionality - the "Full" flavor of this functionality produces exactly what you want: dstSize.w = srcSize.w + maskSize.w - 1. For rather small masks you should expect significantly worse performance as FilterBox is based on the running sum algorithm while ippiConv is based on convolution theorem and doesn't take into account the internals of the mask/kernel. There are 2 types of API for 2D convolution:

New:

/* ////////////////////////////////////////////////////////////////////////////
//   Names: ippiConv_32f_C1R, ippiConv_32f_C3R, ippiConv_32f_C4R
//          ippiConv_16s_C1R, ippiConv_16s_C3R, ippiConv_16s_C4R
//          ippiConv_8u_C1R,  ippiConv_8u_C3R,  ippiConv_8u_C4R
//  Purpose: Performs full or valid 2-D convolution of two images.
//           The result image size depends on operation shape selected in algType mask as follows:
//             (Wa+Wb-1)*(Ha+Hb-1) for ippiROIFull mask
//             (Wa-Wb+1)*(Ha-Hb+1) for ippiROIValid mask,
//           where Wa*Ha and Wb*Hb are the sizes of the image and template, respectively.
//          If the IppAlgMask value in algType is equal to ippAlgAuto, the optimal algorithm is selected
//          automatically. For big data size, the function uses 2D FFT algorithm.
 

Old (deprecated):

/* ////////////////////////////////////////////////////////////////////////////
//   Names:     ippiConvFull_32f_C1R
//              ippiConvFull_32f_C3R
//              ippiConvFull_32f_AC4R
//              ippiConvFull_16s_C1R
//              ippiConvFull_16s_C3R
//              ippiConvFull_16s_AC4R
//              ippiConvFull_8u_C1R
//              ippiConvFull_8u_C3R
//              ippiConvFull_8u_AC4R
//
//  Purpose: Performs full 2-D convolution of matrices (images). If IppiSize's
//           of matrices are Wa*Ha and Wb*Hb correspondingly, then the
//           IppiSize of the resulting matrix (image) will be
//              (Wa+Wb-1)*(Ha+Hb-1).
//           For big data size, the function uses 2D FFT algorithm.
 

regards, Igor.

[Dambrin_D_]

Igor Astakhov (Intel) wrote:
This approach contradicts IPP concept - in IPP (and in Matlab, and in OpenCV, etc.) all filtering operations are reducing operations - this is the mathematics (and physics) of the filtering process.

Mmh I don't understand what this means, but what was the point of changing FilterBox to FilterBoxBorder, if it wasn't to get to the result in less time?

Proper blurring using FilterBox = IppiCopyConstBorder + FilterBox

Proper blurring using FilterBoxBorder = IppiCopyConstBorder + FilterBoxBorder

So I don't see why "Border" was needed, it's only doing half of the work of IppiCopyConstBorder. Well, if I sum it up,
FilterBoxBorder = FilterBox + requirement for memory allocation + 250% slower in in-place version, with zero benefit except it's 30% faster in not in-place version.

But maybe I'm wrong and there is a scenario where case C is really useful.

Igor Astakhov (Intel) wrote:
this is the mathematics (and physics) of the filtering process.

but wasn't FilterBox already "the mathematics of the filtering process"? Why enhance it with a border feature?

(& yes I know about the convolution, but the reason I'm using FilterBox is purely CPU usage, otherwise I would rather go for a proper gaussian blur)

[Igor_A_Intel]

Hi Dambrin,

A lot of new APIs with "border" suffix developed for filtering functionality in the latest IPP versions. The main purpose is to provide the non-reducing capability according to a lot of customers' requests, compatibility with OpenCV, and possibility of external threading (as internal is deprecated).

regards, Igor.

[Dambrin_D_]

Igor Astakhov (Intel) wrote:
compatibility with OpenCV

ok I see

Igor Astakhov (Intel) wrote:
and possibility of external threading (as internal is deprecated).

I didn't know it was deprecated.

This said, I don't see how the borders makes external threading more possible. It's just the in-place version that was making external threading impossible, for a 2D blur. (I've threaded a horizontal-only blur using the old in-place FilterBox with no problem)