Issues with ippiRemap when applied line-wise

[BenjaminS]

Dear all,

We are currently using IPP 2020 and encountered an issue with the application of IppRemap on an image. We are using the ippiRemap variant with 8uC3 images and 32fC1 maps. 

Since we only wanted to remap the x-coords, we applied the ippiRemap function in a line-wise manner, with the yMap containing only zeros and the xMap containing the actual displacements.

We did so using NN, LINEAR and CUBIC interpolations, and the ippiRemap did so without any errors. However, the function did not yield the correct result. Instead, it always (silently, since no error was raised) returned the NN result. In addition the docmentation is quite rare regarding the internals of this function (especially the preliminaries/internals of "Interpolate()" are nowhere to be found).

Further investigations have shown, that there is (probably) an implicit fallback of the interpolation function inside the ippiRemap functions. The following table shows the actually used interpolation mode in relation to the passed interpolation and the number of consecutive lines used for each partial remapping of the image:

	1 line at once	2 lines at once	3 lines at once	4 lines at once
NN	does NN	does NN	does NN	does NN
Linear	does NN	does Linear	does Linear	does Linear
Cubic	does NN	does Linear	does Linear	does Cubic

The comparision reference has been estimated by "pumping up" the x- and y-Maps to the complete image size.

The question is, if this behaviour is wanted. If it is,  it should be documented (IMHO). Otherwise, eg. if a call to the ippiRemap function is done and the interpolation could (for whatever reason) not be used, an error should be the way to go (again, IMHO). Maybe someone at Intel could have a look into this issue and provide a statement regarding this issue. 

Best wishes,
Benjamin

[ShanmukhS_Intel]

Hi,

Thank you for posting in Intel Communities. Could you please share us a sample reproducer, so that we could investigate your issue.

Best Regards,

Shanmukh.SS

[BenjaminS]

Hi,

Thank you for your response. It took me some time to make the code a more-or-less minimal working example for you. We use a lot of stuff around the Ipp for our image processing, but this C++ file (using our own remap-x-data) should do the trick.

Best wishes,
Benjamin

[Andrey_B_Intel]

Hi Benjamin.

The linear(cubic) interpolation uses 2(4) points for interpolation. These points exist inside image but interpolation near image borders is ambiguous. Therefore ippiRemap replicates points on the left/right/top/bottom image edges. So we get effect of changing interpolation mode. I've modified your code to avoid this effect  when processing image by slices. See attachment

Thanks.

Andrey

[BenjaminS]

Hi Andrey,

thank you for your fast response and code example. What I get from your example is that the ROI-based approach you posted works, since IPP knows about the whole image (size). Thank you for this solution!

However, could you please be a bit more precise on the effect of the interpolation mode wrt. the image size and especially the given remapping factors?  If IPP used pixel replication here, I would assume that the linewise approach I committed would have yield the same results as the "complete image" remapping.

The justification for my assumption is that we have no interpolation in y-direction (by design, we always pass zeros here). Thus no replicated pixels in y-direction would ever be accessed and the result should (IMHO) only depend on the remapping values for the x-direction. For the repication of pixels on the x-edge, this should also not result in any differences when compared with the "complete image" remapping.

Thank you,
Benjamin

[Andrey_B_Intel]

Hi.

Ok, in case of interpolation is "cubic" to get 1 dst point we need square 4x4 of src points(anchor is 1,1). So we always need [x-1],[x],[x+1],[x+2] points in both directions. If we are at the edge (x==0) we replace src[-1] with src[0], i.e [-1][0][1][2]->[0][0][1][2]. In situation when we process "complete image" the borders pixels are replicated too. This function checks input size and roi size to understand if we are at the edge. When srcSize equal roiSize and roiSize is small (by height in this case) we always "at the edge" and replicate pixels vertically(1 or 2 lines) without changing of interpolation mode.  The API of this function was developed specially for processing by tiles using approach from "modified reproducer". 

Thanks.

Andrey.  

[BenjaminS]

Hi,

I am aware of Image Processing as well as of those interpolation schemes. However, if the pixels were just repeated in the y-direction, we should get the same result for any interpolation scheme, even when applied (purely) line wise, shouldn't we? The (virtually) repeated pixels in y-direction should never be accessed, since y==0 for the complete yMap (in pure line wise application), should it? IMHO the interpolation would just be performed on the same base of values (namely only the one from one line), no matter what kind of interpolation shall be carried out. So, for me, the observered behaviour still looks like an interpolation fallback based on the (minimum) dimension of the srcSize.
I also added an image to this post, which (hopefully) illustrates, what I had assumed to be the outcome of the pure linewise application.

Thanks
Benjamin

[Andrey_B_Intel]

Hi Benjamin.

Sorry for long delay.

I have checked code of this function. You are absolutely right. ippiRemap changes interpolation mode if source image is too small. By performance reasons interpolation changes in both dimensions although width of input image is enough. We will update documentation with details in one of next IPP releases.

I hope previous workaround is acceptable for you?

Thanks.

Andrey B.

[BenjaminS]

Thank you Andrey,

Yes, the current workaround is okay for us. However, could you please notfy me, once this issue is fixed and the fix is released?  

Thanks in advance
Benjamin

[ShanmukhS_Intel]

Hi,

We are working on your issue. We will get back to you soon.

Best Regards,

Shanmukh.SS

[Ruqiu_C_Intel · Accepted solution]

Hello Benjamin,

We are happy let you know that the fixed will be available in IPP next new version.

Best,

Ruqiu

[Ruqiu_C_Intel]

Hello Benjamin,

Thank you for your patience. We would like to let you know that the fixed is available in oneAPI 2023.0 launched last week.

Regards,

Ruqiu