Intel® Integrated Performance Primitives
Deliberate problems developing high-performance vision, signal, security, and storage applications.

possible bug when resizing images with transparency

David_N_
Beginner
1,435 Views

Hello,

In the post here, you indicated that IPP resizing supported transparency. For the most part it seems to work okay, but there are problems around the border.

In the attached file "in.png" there are circles that are 100% opaque (alpha 255) and the rest of the image is 100% transparent (alpha value is 0). The pixel values around the circles are black with 0 alpha, and the rest of the transparent area is white with zero alpha.

When you resize the image using IppResizeLinear, based on the resize_mt example and the code here.  to 50% of the size you see a grey border (see attached resize_mt_out.png). Somehow, the black pixels are "bleeding" into the resized image, even though the black pixels are 100% transparent and should not contribute to the final resized image.

In some cases, the borders of the resized circles have some transparency, and become semi-transparent in the original color that they were (black arrow in bleedexample.png) and others are semi transparent with black (red arrow in bleedExample.png).

Is this a bug, or is there something we can do about this? It definately seems to me that pixels that were 100% transparent in the original image should not contribute anything to the resized image.

We saw the same effect with cubic filter as well
 

0 Kudos
1 Solution
BMart1
New Contributor II
1,426 Views

I am dealing with RGBA images that either have fully transparent or fully opaque pixels. After premultiplication, the fully transparent pixels should all have values of (0,0,0,0) and the fully opaque pixels will be untouched.

Correct.

Since I am resizing the image, I don't have a "source" and "destination" image as described in your previous link so I presume premultiplication will just do the multiplication on the source image based on its own alpha value.

Correct. Note that before saving the resized image back to .png you will have to undo the premultiplication.

So if I make a 50x50 image that is all (0,0,0,0) except for a 14 pixel red rectangle in the middle that is (255,0,0,255). This is starting off as effectively "premultiplied". Is that correct?

Correct.

Then I shrink that by 3 in each dimension (i.e. ippResize to 1/3 the size). I would naively expect that inside the shrunk rectangle it would be (255,0,0,255).

Correct

However, since 14 does not divide equally by 3, I would expect pixels on the border to be semi-transparent.

Actually, proper downsampling with the antialiasing resize functions will read several input pixels to generate each output pixel. Normally linear resizing uses two pixels to generate each output pixel. Since you are shrinking by a factor of 3, ResizeAntialiasingLinear will read 6 input pixels. You can avoid this by having your non transparent pixels be a multiple of 3 and also start at a multiple of 3 and disabling antialiasing or using supersampling.  It won't look as good.

On the border of the shrunk rectangle it would be (255,0,0,x) where x is the transparency of that pixel as determined by whatever filter I was using. It should not vary in the RGB portion, which should always be (255,0,0) but may have different values in the alpha pixel depending on the filter.

No. Border pixels will be (x, 0, 0, x).

View solution in original post

0 Kudos
22 Replies
Jonghak_K_Intel
Employee
1,279 Views

Hi David,

 could you let me know which version of IPP you are using?

 and please describe your development environment, such as OS, your platform information and compiler options you used if applicable.

 

0 Kudos
BMart1
New Contributor II
1,279 Views

Hi David,

You need to premultiply your alpha before resizing the image.

Bruno

0 Kudos
David_N_
Beginner
1,279 Views

@ Bruno. Thank you for the comments. Are you sure that is the case? Fiona from Intel, in the post here, seems to indicate that premultiplying is necessary for combining (merging) images, but not for resizing.

0 Kudos
BMart1
New Contributor II
1,279 Views

I'm 100% sure.  See https://developer.nvidia.com/content/alpha-blending-pre-or-not-pre​ A web search can give you tons of other articles in the subject. It's rediscovered often.

While you are at it, I also believe you should gamma convert the input image so that the resizing is done with linear light. This however is not as noticeable. I don't bother in my programs.

 

0 Kudos
David_N_
Beginner
1,279 Views

HI Bruno,

Thanks again for the comments. What I meant was whether or not IPP resize methods do the pre- multiplication for you.

Fiona wrote that

If you point to use premultiplied alpha that would be used for alpha composition function that when two 4 channel images cover each other, you probably need to calculate alpha(A)*RGB(A)+alpha(B)*RGB(B). But other functions, like resize, the size of alpha also need to be changed by interpolation method. it is not using premultiplied alpha, it process with alpha channel. 

Based on that, I thought that for iPP composition functions you needed to premultiply, but iPPresize did whatever premultiplication was necessary as part of the filtering/interpolation process and we did not need to do it manually.

In addition, when composing (merging) images, the IPP docs are fairly clear about how/when to premultiply, but this is not mentioned at all for resizing.

0 Kudos
BMart1
New Contributor II
1,279 Views

You can resize a 10x10 image with ipp and by hand to confirm that ipp doesn't treat the alpha channel specially. Remember that ipp contains primitives that you are supposed to assemble into your own pipelines. You may want to operate multiple times with an image. It would be wasteful to pre multiplicate inside each operation, compared with once only at the beginning.

0 Kudos
David_N_
Beginner
1,279 Views

Bruno, thanks so much for the help.

Ok, before I submit any code I will continue testing with premultiplication. But just to ensure my expectations are correct, can you confirm the following:

I am dealing with RGBA images that either have fully transparent or fully opaque pixels. After premultiplication, the fully transparent pixels should all have values of (0,0,0,0) and the fully opaque pixels will be untouched. Since I am resizing the image, I don't have a "source" and "destination" image as described in your previous link so I presume premultiplication will just do the multiplication on the source image based on its own alpha value.

So if I make a 50x50 image that is all (0,0,0,0) except for a 14 pixel red rectangle in the middle that is (255,0,0,255). This is starting off as effectively "premultiplied". Is that correct?

Then I shrink that by 3 in each dimension (i.e. ippResize to 1/3 the size). I would naively expect that inside the shrunk rectangle it would be (255,0,0,255). However, since 14 does not divide equally by 3, I would expect pixels on the border to be semi-transparent. On the border of the shrunk rectangle it would be (255,0,0,x) where x is the transparency of that pixel as determined by whatever filter I was using. It should not vary in the RGB portion, which should always be (255,0,0) but may have different values in the alpha pixel depending on the filter.

Is that the proper expectation for the behavior?

0 Kudos
BMart1
New Contributor II
1,427 Views

I am dealing with RGBA images that either have fully transparent or fully opaque pixels. After premultiplication, the fully transparent pixels should all have values of (0,0,0,0) and the fully opaque pixels will be untouched.

Correct.

Since I am resizing the image, I don't have a "source" and "destination" image as described in your previous link so I presume premultiplication will just do the multiplication on the source image based on its own alpha value.

Correct. Note that before saving the resized image back to .png you will have to undo the premultiplication.

So if I make a 50x50 image that is all (0,0,0,0) except for a 14 pixel red rectangle in the middle that is (255,0,0,255). This is starting off as effectively "premultiplied". Is that correct?

Correct.

Then I shrink that by 3 in each dimension (i.e. ippResize to 1/3 the size). I would naively expect that inside the shrunk rectangle it would be (255,0,0,255).

Correct

However, since 14 does not divide equally by 3, I would expect pixels on the border to be semi-transparent.

Actually, proper downsampling with the antialiasing resize functions will read several input pixels to generate each output pixel. Normally linear resizing uses two pixels to generate each output pixel. Since you are shrinking by a factor of 3, ResizeAntialiasingLinear will read 6 input pixels. You can avoid this by having your non transparent pixels be a multiple of 3 and also start at a multiple of 3 and disabling antialiasing or using supersampling.  It won't look as good.

On the border of the shrunk rectangle it would be (255,0,0,x) where x is the transparency of that pixel as determined by whatever filter I was using. It should not vary in the RGB portion, which should always be (255,0,0) but may have different values in the alpha pixel depending on the filter.

No. Border pixels will be (x, 0, 0, x).

0 Kudos
David_N_
Beginner
1,279 Views

No. Border pixels will be (x, 0, 0, x).

Is that because after resizing, I have a "premultiplied, resized" image. If I want to display that to a user, I have to un-premultiply it. Doing that will divide by R value to be x/x hence back to 255 all the time. Therefore, in my "real" resized image, I only have red pixels with varying degrees of transparency?

Is there an IPP function to un-premultiply an image?

 

0 Kudos
BMart1
New Contributor II
1,279 Views

Yes, resizing a premultiplied image produces a premultipied image. I don't know how to un-premultiply with ipp.

0 Kudos
David_N_
Beginner
1,279 Views

@Bruno,

Thank you for your help. Based on the discussion above we got everything working and there does not seem to be any bug. It is kind of non-intuitive that you need to pre-multiply to resize, and end up with a premultiplied image, and you can also premultiply to compose, but then you end up with a "normal" image.  I guess to unpremultiply you need to compose the image against another pre-multiplied image pixels that are (0,0,0,0), or do it yourself.

 

0 Kudos
BMart1
New Contributor II
1,279 Views

I'm glad you got it working. I think that when you compose you also get a pre multiplied image, since the table at https://software.intel.com/en-us/node/503865 has no division. I hope someone from Intel reads this and answers how to convert to straight alpha/

0 Kudos
David_N_
Beginner
1,279 Views

We did the following tests:

I took an opaque red background (255,0,0,255) and blended unpremul green (0,255,0,128) onto it:

  • GDIPlus gives (127,128,0,255)
  • IPP Alphacomp with ippAlphaOver gives (127,128,0,255)

Next, I took an opaque red background (255,0,0,255) with premul green (0,128,0,128):

  • IPP Alphacomp with ippAlphaOver gives (127,64,0,255)
  • IPP AlphaComp with ippAlphaOverPremul gives (127,128,0,255)

So it seems that when you take premultiplies background composed with a premultiplied foreground and call Alphacomp with ippAlphaOverPremul  you get the correct "non-premultiplied" image. At least that was our interpretation.

0 Kudos
BMart1
New Contributor II
1,279 Views

Since the result has alpha=255, you cannot tell if it's pre multiplied or not. Try a (128, 0, 0, 128) background and a (0, 128, 0, 128) foreground with ippAlphaOverPremul. You should get (64, 128, 0, 192).

0 Kudos
David_N_
Beginner
1,279 Views

Hi Bruno,

If you do (128,0,0,128) background and (0,128,0,128) foreground with ippAlphaOverPremul you get (64, 128, 0, 192)

If you do (256,0,0,128) background and (0,256,0,128) foreground with ippAlphaOver you also get (64, 128, 0, 192)

Strangely if you do the same in GDI+ ( semi red background (255,0,0,128) and blended unpremul green (0,255,0,128)) you get (85,170,0,192). I guess GDI+ is doing some extra processing..

 

0 Kudos
BMart1
New Contributor II
1,279 Views

85 = 64*256/192

170 = 128*256/192

GDI+ is converting the result back to straight alpha.

0 Kudos
David_N_
Beginner
1,279 Views

@Bruno,

I thought that (64, 128, 0, 192) is the non-premultiplied result.

If you take (256,0,0,128) background and (0,256,0,128) foreground and do the math by hand, you get (64, 128, 0, 192)

If you take (256,0,0,128) background and (0,256,0,128) foreground (both not premultiplied) and use AlphaComp with ippAlphaOver you also get (64, 128, 0, 192), again, seeming to confirm that (64, 128, 0, 192) is the not premultiplied result.

If you pass in the premultiplied version ( (128,0,0,128) background and (0,128,0,128) foreground) to AlphaComp with ippAlphaOverPremul you also get (64, 128, 0, 192), so we interpreted that AlphaComp with ippAlphaOverPremul is also returning the non-premultiplied result of (64, 128, 0, 192).

By "straight alpha" do you mean non-premultiplied alpha? If so, does that mean that you think that (64, 128, 0, 192) is a premultiplied value? If so, then I am a little confused why AlphaComp with ippAlphaOver with non-premultiplied inputs returns a premultiplied result.

0 Kudos
BMart1
New Contributor II
1,279 Views

Yes, (64, 128, 0, 192)​ is premultiplied. You can confirm that AlphaComp returns a premultiplied image by using a (0,0,0,0) background. Then ippAlphaOver ​just premultiplies.

0 Kudos
David_N_
Beginner
1,279 Views

@Bruno,

You are correct. Amazing... AlphaComp always produces pre-multiplied results. The documentation on Alpha Composition could be improved in this regard, especially for beginners. While it lists the equations, it would be helpful to mention that these equations result in getting pre-multiplied results. I guess if I was an image processing wizard, I would have recognized that.

I guess the fact that running AlphaComp against a completely opaque background results in an opaque image, so the fact that the opaque image is actually "premultiplied" is irrelevant since for opaque images pre-multiplied and native-alpha are the same.

I guess that is why there is no un-premultiply since you just AlphaComp against some opaque background to get back to native-alpha. Although if trying to save as PNG or just inspect the values of a blending operation in a "native alpha" it would be useful. I will ask this as a separate thread

Thank you so much for your help. We never would have figured all this out on our own.

0 Kudos
BMart1
New Contributor II
1,189 Views

Glad to help.  I only recently learned this.  I'm only giving back.

0 Kudos
Reply