- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I noticed when attempting to match the behavior of ippiScale_32f8u that I was getting unexpected results from the intel function.
Specifically, scaling the following float rounds down to 0 instead of up to 1.
float a = 0.00196078443;
Since the float range is 0.0 to 1.0 and the 8u range is inherently 0 to 255, a * 255 = 0.50000002965. And thus the function should round up I believe, but the scaling function outputs 0 for this. Am I misunderstanding how this should work?
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
Thank you for contacting us!
Could you help provide out the information: IPP version, CPU platform, OS, etc? As well as a simple reproducer.
Have a nice day.
Regards,
Ruqiu
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I literally just gave you the code to try.
I think I figured it out ... this is due to the use of "bankers rounding" aka "round to nearest, ties to even". This makes sense, but should be documented in the function behavior.
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page