ippiMalloc of small sizes fails

[psnvm]

I am using ippiMalloc to allocate memory for an image. When (width, height) are within some (usually small) ranges, ippiMalloc returns a NULL pointer, indicating that it was unable to allocate the memory. The error appears consistently and is reproducible in a certain context but not in others.

An odd thing is that the allocation works for values that do not lie within the range. An example:

1. Allocating (1, 1) is successful.
2. Allocating (50, 50) returns a NULL pointer => is unsuccessful.
3. Allocating (1000, 1000) is successful.

The problem is seen regardless of the order of the allocations and how many times the allocations are performed.

This is using version 5.2 of IPP. ippiMalloc is being called from managed .NET code using P/Invoke with the following import statement:

[DllImport("ippi-5.2.dll", EntryPoint="ippiMalloc_8u_C1")]
private static extern IntPtr ippiMalloc_8u_C1(int widthPixels, int heightPixels, out int stepBytes);

The program is running on Windows Vista 32-bit on a Core 2 Duo T7300.

Are we doing something wrong or is it to be expected that the allocation fails? I could understand if there was not a sufficiently large contiguous block of memory available for the allocation, but we can repeatedly allocate both larger and smaller blocks.

/Peter

[psnvm]

It seems that a nearby ippiPyrDown using a slightly wrong size for the buffer was causing this. After correcting it everything seems fine. Problem solved!

/Peter

[Vladimir_Dudnik]

Ok, thanks for updating

Regards,
Vladimir

[d_scain_farenzena]

Hello,

eventually I forget to ippiFree or call ippi functions with "harazdous" parameters and this cause the ippiMalloc returning me a null pointer.
(1) But sometimes (currently) I can't allocate with ippiMalloc even if everything in the code is ok. Subseqent calls to it keep failing, so it is not the case of trying again.
The image I am trying to allocate is 112 x 27 and the step returns 128 (which makes sense). The pointer is initialized with 0xCCCCCCC before calling ippiMalloc_8u_C1 and ends with 0x00000000, which I assume is an allocation fail.
(2) Anyhow, there wouldn't be a way to find why the ippiMalloc failed? I can allocate lots of images but suddenly it starts to give me null pointers everywhere. What I do to solve is search all code for places where I can deallocate something temporarly. But as the code is getting bigger and time is upping, I can't spare more time searching for ippiMalloc killers.
(3) Anyanyhow, why would some function that was previously called affect the next? Like one function corrupting another.

Thank you in advance.

Regards,
Daniel

[d_scain_farenzena]

More info on it: I haven't realized that I was too calling external DLL from C#.NET. So I paid more attention to the standard output and read a

"0xC0000005: Access violation reading location 0x01b80038."

"Access violation"I read in some posts that this may be heap corruption by previous code. Indeed I did found two "Heap Corruption" messages in the output. Cleaned the bad code and now it do not deliver more "Heap Corruption" messages, but the "Access Violation" still shows up (and the pointer stills returns null). For the sake of curiosity, the two lines that made the Heap Corrupted was allocating ippiMalloc_8u_C1(10, 10, &x) images (for testings purposes, that's why I could remove them).

I'll try again tomorrow.

Regards,
Daniel

[d_scain_farenzena]

Hi,

well I'm still getting null pointers and it is always in the same place. Then, I tried the following:

Ipp8u * Temp;
int i = 0, width = 112, height = 27;
for(i = 0; i < 100000; i++)
{
Temp= ippiMalloc_8u_C1(width+i, height+i, &step);
if (Temp != 0)
break;
}
printf("Area %d and i =%d", (width+i)*(height+i), i);

And always it successfully allocate when i = 651 (corresponding area 517314). More curious is that only the second time I try to allocate with 112 x 27 it returns me null, the first time is fine.
To solve the problem, I'm switching back part of the code to malloc instead of ippiMalloc, since it still works. Not perfectly aligned always, but it works. Hints are still welcome, I'm very curious about this, maybe I have passed hazardous parameteres back in the code and now it is manifestating itself, or maybe a bug in the ippiMalloc.

By the way, there is any intention of release ipp as open source, like Framewave of AMD? There is a better forum to know about this?

Regards,
Daniel

[d_scain_farenzena]

Hi,

after introducing more Heap Corruption tests, finding them and correcting them, I started to change ippiMalloc's for simples malloc's, since malloc succeeds in allocating memory (although not aligned). But internally, some ippi functions (currently "ippiResize_8u_C1R") uses ippiMalloc, which probably returns "0x0000000" and the function returns

"ippStsMemAllocErr" "Not enough memory for the operation"

Now, I am out of bullets. The MSVC++ memory dump

_CrtMemState temp;
_CrtMemCheckpoint( &temp );
_CrtMemDumpStatistics( &temp);

returns:

0 bytes in 0 Free Blocks.
335464 bytes in 922 Normal Blocks.
20531 bytes in 127 CRT Blocks.
0 bytes in 0 Ignore Blocks.
0 bytes in 0 Client Blocks.
Largest number used: 395238 bytes.
Total allocations: 55413182 bytes.

The 300kb is expected as well as the 55MB, because there is lots os text processing before reaching that. I am using the "_CrtCheckMemory()" to find heap corruptions and all found till the faulty code were corrected. I would apreciate any new idea.

Regards,
Daniel

[Vladimir_Dudnik]

Hi Daniel,

what is total memory requirements of your application? And what is the system you are working on? How much memory is available on your system? It would be nice if you can attach here test case to reproduce that issue, as you are only one who reports IPP memory allocation issue for the years of IPP life.

There is no plan to release IPP as open source library.

Regards,
Vladimir

[d_scain_farenzena]

Hello Vladimir,

thank you for your answers.
- Memory requirements of application: the memory dump statistics show the system currently in full production: about 300kB, and this will be the top for a while. I think this dump statistics disconsider ippiMalloc allocations, but they are few anyway. Our images have a maximum of 300x300, 32f, C1, but in the future they'll be high-def camera sized. So, I think its memory requirements are negligible.
- System: the system is a Windows XP SP2, Pentium D CPU 2.8 GHz (Dell Optiplex GX620).
- System memory: 2 GB RAM.
- Test case: well, this is complicated. I can allocate many times, but "sometimes", and here is the mystery, it returns me 0x0000000 for the rest of the applications life, whereas malloc can return me correctly allocated pointer (I allocate with malloc everytime ippiMalloc starts to fail).

I will try to reproduce the problem in other parts of the application, but I don't have much hope, since its kind of random. I would apreciate some help trying to ellaborate a test case. Since I know it is very dificult to analyse without a test case, soon I will be posting here again with the test case (if I can get one).

Best regards,
Daniel

[Vladimir_Dudnik]

What is the structure of your application? Do you use IPP in managed runtime environment? Is it possible that .NET garbage collector somehow infuence that?

Vladimir

[d_scain_farenzena]

Hi Vladimir,

- Structure of the application: there is a C# .NET Windows Forms Application that calls a library, coded in Win32 C/C++ (compiled with MSVC in Visual Studio 2008). This C/C++ library runs outside .NET Framework, its the so called "unmanaged" library, then outside the .NET Framework, thus I think the garbage collector can't influence memory management on the library. The image parameter passed to the library is copied as soon as the function in the library is called. This library calls IPP and other libraries as well.

Last friday I upgraded the library to 5.3 Update 2 (it was Update 1 before that), which is said to be compatible with Visual Studio 2008. Then, a lot more IppiMalloc's worked correctly, and I could remove almost all malloc's, but not all! Some of the IppiMalloc still fails and IppiResize still fails too.
Hint: now, with Update 2, when IppiMalloc fails it outputs on the console "Invalid Cache depth" and breaks the debuging. Before only a "Access violation" and 0x0000000 as return, but not exception was thrown, so the debugger don't stopped.

Some libraries have options to track memory issues or other types of problem trought console or logging, that makes it slow while activated. There is this kind of tracing in Ipp?

Thank you again.

Regards,
Daniel

[d_scain_farenzena]

Hi Vladimir,

the problem was solved, here is the summary:

- Visual Studio 2008 was being used for Debugging with IPP 5.3 Update2, whichisnot compatible, as stated by the release document of IPP 5.3 Update 3 of IPP. With IPP 5.3 Update 3, half the memory allocation errors were corrected.

- There was code unrelated to IPP the was causing memory corruption (1 byte out of bound being write). Using dynamic program analysis tool such as IBM'sPurify identified the problem. All errorsvanished then, including failing IppiResize function (http://software.intel.com/en-us/forums//topic/58449). Isuspect thatprevious errors that were apparently being caused by using not up to date version of IPP compatible with Visual Studio 2008 were caused too by this memory corruption, but now it is impossible to confirm.

I am linking this thread to the failing ippiResize one. Thank you for your help on both threads.

Best regards,

Daniel