Intel® IPP 7.1 update 1 is now available

[Jeffrey_M_Intel1]

Intel® IPP 7.1 update 1 is now available. The library may be obtained as a stand-alone product or as a component in Intel® Composer XE 2013, Intel® Parallel Studio XE 2013, and other Intel® Tool Suites. Please visit the Intel Software Evaluation Center to download evaluation versions of these and other Intel software products.

What's new in 7.1:

• Intel® AVX2 speedup: performance boost for many image and signal processing operations
• Improved image resize
• Improved sample build system
• ippsResamplePolyphase reinstated

For more information, please see the Intel® IPP landing page.

[raghu_getlasterror]

Does IPP support XDCAM related profile(422) and level for encoding? Currently we are using IPP samples : w_ipp-samples_p_7.0.4.054 and IPP Library Version : w_ipp_7.0.4.221

[Jeffrey_M_Intel1]

Unfortunately 422 is a bit of a grey area. Media SDK doesn't support it internally, and UMC validation follows MSDK in many cases. Sorry there isn't a better answer, but as you know UMC is a sample. It is a starting point that can be extended any way you like, but it is not productized and does not fully implement every feature. I'll follow up with a private message just to make sure your use case is understood so we can make the best suggestions.

[SergeyKostrov]

Hi everybody, >>Intel® IPP 7.1 (Initial Release) is now available... Could you provide some technical details on image size limitations? I remember that on 32-bit platforms, and even on 64-bit platforms, it was impossible to create / use an image if its size was greater than ~1.5GB or so. Best regards, Sergey

[Jeffrey_M_Intel1]

There are some limitations around the signed integer maximum (2 GB). IPP malloc uses integer sizes, while C/C++ standard malloc uses size_t. In some cases it is possible to use giant image buffers from standard malloc, but this isn't generally supported by all functions. Hopefully with the new directions started in IPP 7.1 you won't have to. With external threading it should become easier to approach many large image processing tasks as tiles instead of needing to work with monolithic buffers. Regards, Jeff

[SergeyKostrov]

>>There are some limitations around the signed integer maximum (2 GB)... Two kinds of platforms are used by many software developers, that is 32-bit and 64-bit. Could you clarify the statement "...There are some limitations ...", please? What platform(s) do you mean? Best regards, Sergey

[Jeffrey_M_Intel1]

This should only be a 32-bit limitation, but I'll do some more investigation to make sure. In the meantime, any feedback on the acceptability of tiling for very large images could help make sure your original question is answered thoroughly. Regards, Jeff

[SergeyKostrov]

>>This should only be a 32-bit limitation, but I'll do some more investigation to make sure... Thanks in advance, Jeffrey! >>...In the meantime, any feedback on the acceptability of tiling for very large images could help... I'll also try to provide some technical details later. If somebody will try to use IPP to process panoramic images ( B&W, uncompressed, 14-bit depth, for example ) some problems are possible.

[Jeffrey_M_Intel1]

Regarding the 2GB-1 limit for image buffers, this applies to 32 and 64 bit systems. IPP's image mallocs use the int type to specify size. For the Intel compiler, the int type is 4 bytes on 32 and 64 bit systems for all OSes supported by IPP (Windows, Linux, OSX). The 2GB-1 limit is a frequent internal assumption. I'll see what can be done to make this clearer in the future via documentation, return codes, etc.

[SergeyKostrov]

>>Regarding the 2GB-1 limit for image buffers, this applies to 32 and 64 bit systems. IPP's image mallocs use the int type to specify size. >>For the Intel compiler, the int type is 4 bytes on 32 and 64 bit systems for all OSes supported by IPP (Windows, Linux, OSX). >> >>The 2GB-1 limit is a frequent internal assumption. I'll see what can be done to make this clearer in the future via documentation, >>return codes, etc. Thank you, Jeffrey! Is there any chance to increase the 2GB limit for 64-bit platforms? What do you think regarding a solution like: ... #undef ippMalloc #ifdef _PLATFORM_32BIT IPPAPI( void *, ippMalloc32, ( int length ) ) #define ippMalloc ippMalloc32 #end #ifdef _PLATFORM_64BIT IPPAPI( void *, ippMalloc64, ( __int64 length ) ) #define ippMalloc ippMalloc64 #end ... Please consider this as a feature request.

[Jeffrey_M_Intel1]

Thanks for your reply. Your suggestion makes a lot of sense. However, before pursuing as a feature request it will help to understand if the 2GB-1 limitation is blocking any development. The reason for not following up on what must seem like an obvious gap is the move to external threading. If IPP primitives need to internally partition work on full images then of course they need to work with arbitrarily large inputs. However, going forward applications will need to handle their own partitioning. Large partitions for a single thread are probably not ideal. Applications need to consider cache sizes for efficiency, so keeping partitions far smaller than 2GB will usually be best. This will also align with validation, where coverage is deeper for cache-friendly sizes for the same reason.

[SergeyKostrov]

>>...However, going forward applications will need to handle their own partitioning... You shouldn't expect that real life applications in Geomatics for example will be redesinged to work with partitions. Could you imagine an overhead of a such redevelopment? I can't because I know that complexity of Geomatics applications that work with large images is very high. Image sizes are growing ( 'Panorama Mode's are integrated in almost all P&S digital cameras now! ) and what if somebody will need to work with an image that is larger than 2.5GB. Here is an example of its size: 2.5GB = 2^31 + 2^19 = 2147483648 + 524288 = 2148007936 ( bytes ) / sizeof( float ) = 537001984 single-precision elements sqrt( 537001984 ) = 537001984^0.5 ~= 23173 ( for a square image ) So, an image size is ~23173x23173 for a 'float' / 'Ipp32f' Single-Precision data type. Once again, you shouldn't even try to force companies to do partitioning in image processing because many complex legacy applications couldn't not be easily redesigned (!). What about edge or boundary problems in a partitioned image? How should somebody apply some filtering, etc? When a Geomatics application does a classification of some object / target it works with a whole image in most cases. This is how algorithms for classification(s) are designed in the first place and they are very complex. However, what IPP library needs to provide is a relatevely simple thing and this is a new 'ippMalloc'-like function that allows to allocate a block of memory greater than 2GB.

[Jeffrey_M_Intel1]

Very good points. I agree with you. Two related issues have been entered for this. DPD200239164: Feature request for arbitrary buffer size support (not limited to 2GB-1 when not a system limitation) DPD200238286: Add errors/warning messages to indicate requested size is beyond an internal limit Please keep in mind that these are potentially large projects, and not as simple as updating IPP's malloc functions. Some functions will work fine with larger buffers, others have internal limitations. Work on these issues will require project-level commitment, which is still TBD. I cannot make any guarantees at this point about timelines. Performance with multi-GB buffer sizes is not likely to be ideal, and one goal of documentation going forward is to highlight ways to transition to cache-aware external threading/partitioning. Many thanks for this helpful feedback.

[SergeyKostrov]

>>DPD200239164: Feature request for arbitrary buffer size support (not limited to 2GB-1 when not a system limitation) >>DPD200238286: Add errors/warning messages to indicate requested size is beyond an internal limit Thank you, Jeffrey! >>Please keep in mind that these are potentially large projects, and not as simple as updating IPP's malloc functions. Some functions >>will work fine with larger buffers, others have internal limitations. Work on these issues will require project-level commitment, >>which is still TBD. I cannot make any guarantees at this point about timelines. I understand it and I don't expect to see that new functionality by the end of 2012. The most important thing is that 2GB limitation should be removed for 64-bit applications on 64-bit platforms and 32-bit applications on 32-bit Windows platforms that support Microsoft's Address Windowing Extensions ( AWE ) technology. Note: I will try to provide a test-case for AWE technology. Best regards, Sergey

[SergeyKostrov]

Hi Jeffrey, >>...IPP malloc uses integer sizes, while C/C++ standard malloc uses size_t... It looks like I finally understood the origins of the problem and this is Intel IPL ( Image Processing Library / a predecessor of IPP ). In IPL a similar memory allocation function is declared as follows: ... IPLAPI( void *, iplMalloc, ( int length ) ) ...

[SergeyKostrov]

Hi everybody, >>>>...In the meantime, any feedback on the acceptability of tiling for very large images could help... >> >>I'll also try to provide some technical details later. If somebody will try to use IPP to process panoramic images ( B&W, >>uncompressed, 14-bit depth, for example ) some problems are possible. This is a short follow up and it gives an idea on modern trends in imaging and I found that article as very interesting: Article: 320-gigapixel photo of London is the world's largest panoramic photo Web-link: www.dpreview.com/news/2013/02/21/320-gigapixel-photo-of-London-is-the-worlds-largest-panoramic-photo A quote: ... - The raw images were then processed over a multi-week period using Fujitsu Technology Solutions' Celsius R920 workstation with 256GB of RAM and 16 cores at 3.1GHz, and Autopano Giga panorama stitching software from Kolor. ...