Intel® Integrated Performance Primitives
Deliberate problems developing high-performance vision, signal, security, and storage applications.
6746 Discussions

Test for possible IppiConvValid bug

Ryan2
Beginner
‎08-15-2012 07:26 AM
1,351 Views
 
Hi,
 
I'm getting a -9 mem error with ippiConvValid_32f_C1R even though  my system has plenty of memory.
 
I allocated a 25GB with malloc just to make sure my program's memory allocation wasn't being limited by Windows.  No problems there.
 
Attached to this email is a compiled Windows x64 console app.  Everyone please download it and reply with the MAX size you can enter before you get a -9 err along with your system specs.  If you want to build it yourself, please go to my other post to download the VS project.  http://redfort-software.intel.com/en-us/forums/showthread.php?t=106382&o=a&s=lr
 
Hopefully there is a solution to this because it's really hampering me!
 
Best,
 
Ryan
mem_test_exe.rar
0 Kudos

Link Copied

18 Replies
SergeyKostrov
Valued Contributor II
‎08-15-2012 08:12 PM
1,351 Views
Quoting Ryan
...I allocated a 25GB with malloc just to make sure my program's memory allocation wasn't being limited by Windows. No problems there...


Ryan,

Did you try to allocate more memory with a 'malloc' CRT function, for example 64GB and higher? Please take a look at
a test-application I created some time ago and attached to that post:

     http://redfort-software.intel.com/en-us/forums/showpost.php?p=189221

I'll take a look at your test-application.

Best regards,
Sergey

0 Kudos
Copy link
SergeyKostrov
Valued Contributor II
‎08-15-2012 08:19 PM
1,351 Views
I'll take a look at your test-application...


What about source codes of your test-application? At least simplified version that reproduces the problem
with ippiConvValid_xxx IPP function.

0 Kudos
Copy link
Ryan2
Beginner
‎08-15-2012 08:55 PM
1,351 Views
Hi Sergey,
 
Attached is the source code for my test app.  It's very easy to follow.
 
Thanks for looking into this for me!
 
Best,
 
Ryan
IppiConvValid_mem_test.rar
0 Kudos
Copy link
Ryan2
Beginner
‎08-15-2012 09:03 PM
1,351 Views
Hi Sergey,
 
I ran your mem test app with the CM3 line uncommented:
 
 __int64 _MEM_SIZE = ( 1073741824 * 64.0 ); // 64GB Recommended VM =  N/A  / 96GB
 
64-bit Windows platform
Allocating memory with 'malloc' CRT-function - 1D
Succesfully Allocated 64.00GB
64-bit Windows platformAllocating memory with 'malloc' CRT-function - 1DSuccesfully Allocated 64.00GB
 
0 Kudos
Copy link
Ryan2
Beginner
‎08-16-2012 07:26 AM
1,351 Views
Update,
 
I also tried my mem test app with IppiConvValid_16s_C1R, IppiConvValid_16u_C1R, and IppiConvValid_8u_C1R and the resutls are the same.
 
On my machine:
 
(Dual socket Intel Xeon E5645, 48GB DDR3) - 1.06 before I get -9 mem error.
  
On three other computers I've tried:
 
(Dual socket Xeon E5420, 16GB DDR2) -1.89.
(Dual socket Xeon E5420, 16GB DDR2) - 1.89.
(Dual socket i7, 12GB DDR2) - 1.89.
 
 
Ryan
0 Kudos
Copy link
SergeyKostrov
Valued Contributor II
‎08-16-2012 07:02 PM
1,351 Views
Quoting Ryan
Attached is the source code for my test app.  It's very easy to follow...


Hi Ryan,

Could you upload as a zip archive, please? I couldn't extract your sources from the rar archive:

   1   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\mem_test.cpp" header broken
   2   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\mem_test.cpp
   3   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\mem_test.vcproj" header broken
   4   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\mem_test.vcproj
   5   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\stdafx.cpp" header broken
   6   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\stdafx.cpp
   7   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\stdafx.h" header broken
   8   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\stdafx.h
   9   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\targetver.h" header broken
  10   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\targetver.h
  11   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\x64\Release\BuildLog.htm" header broken
  12   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\x64\Release\BuildLog.htm
  13   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\x64\Release\mem_test.exe.intermediate.manifest" header broken
  14   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\x64\Release\mem_test.exe.intermediate.manifest
  15   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\x64\Release\mem_test.obj" header broken
  16   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\x64\Release\mem_test.obj
  17   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\x64\Release\mem_test.pch" header broken
  18   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\x64\Release\mem_test.pch
  19   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\x64\Release\mt.dep" header broken
  20   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\x64\Release\mt.dep
  21   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\x64\Release\stdafx.obj" header broken
  22   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\x64\Release\stdafx.obj
  23   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\x64\Release\vc90.idb" header broken
  24   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\x64\Release\vc90.idb
  25   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\x64\Release\vc90.pdb" header broken
  26   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\x64\Release\vc90.pdb
  27   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test.ncb" header broken
  28   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test.ncb
  29   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test.sln" header broken
  30   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test.sln
  31   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test.suo" header broken
  32   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test.suo
  33   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\x64\Release\libiomp5md.dll
  34   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\x64\Release\mem_test.exe" header broken
  35   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\x64\Release\mem_test.exe
  36   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\x64\Release\mem_test.pdb" header broken
  37   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\x64\Release\mem_test.pdb
  38   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\x64\Release" header broken
  39   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\x64" header broken
  40   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\x64\Release" header broken
  41   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test" header broken
  42   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\x64" header broken
  43   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test" header broken

Best regards,
Sergey

PS: That is really strange but it looks like older versions of WinRAR are not compatible with the newer versions.

0 Kudos
Copy link
SergeyKostrov
Valued Contributor II
‎08-16-2012 07:32 PM
1,351 Views
Hi Ryan, This ia a short follow up regarding the error -9...

Quoting Ryan
...I'm getting a -9 mem error with ippiConvValid_32f_C1R even though  my system has plenty of memory.
 
I allocated a 25GB with malloc just to make sure my program's memory allocation wasn't being limited by Windows.  No problems there.
 
Attached to this email is a compiled Windows x64 console app. Everyone please download it and reply with the MAX size you can enter before you get a -9 err...


This is how it is declared:

     typedef enum
     {
         ...
         ippStsMemAllocErr           = -9,
         ...
     } IppStatus;

and I think the problem could be possibly related to limitation(s) of some IPP functions, including ippiConvValid_xxx. Your 64-bit system
allows to allocate up to 64GB of memory (!) and you've done two tests. These "magic" limitations were
disccused a couple of times on the IPP forum in the past. Unfortunately, there is no article that describes these limitations
in some functions of IPP library.

Best regards,
Sergey

0 Kudos
Copy link
Ryan2
Beginner
‎08-16-2012 07:33 PM
1,351 Views
Hi Sergey,
 
That's weird.  Sorry about that.
 
Please see attached for zip. 
 
Best,
 
Ryan
IppiConvValid_mem_test.zip
0 Kudos
Copy link
Ryan2
Beginner
‎08-16-2012 07:37 PM
1,351 Views
Hi Sergey,
 
Yes I looked up that enum.  That's exactly my thinking.  I want to know why IppConvValid is having that problem.  For it to run out of memory on my system it would have to be doing a HUGE FFT.  What I find even stranger is that an i7 with 12gigs of DD2 memory can handle more than my Xeon with 48gigs of DD3.
  
Ryan
0 Kudos
Copy link
Ryan2
Beginner
‎08-16-2012 07:44 PM
1,351 Views
OK!
 
I tried using IppConvFull instead of Valid.  I was able to enter 3.2 into my mem test program before I got the -9 error code.  So WHY, does ConvValid crap out at 1.06?
[cpp]//Allocate repImg
	Image repImg(srcImg.getSize().width + width_pad * 2, srcImg.getSize().height + height_pad * 2);

	//Repeat borders of imgSrc
	ippiCopyReplicateBorder_32f_C1R(srcImg.getPtr(), srcImg.getStepBytes(), srcImg.getSize(), repImg.getPtr(), repImg.getStepBytes(), repImg.getSize(), height_pad, height_pad);

	//Allocate convImg
	Image convImg(repImg.getSize().width + kerImg.getSize().width - 1, repImg.getSize().height + kerImg.getSize().height - 1);

	IppStatus status = ippiConvFull_32f_C1R(repImg.getPtr(), repImg.getStepBytes(), repImg.getSize(), kerImg.getPtr(), kerImg.getStepBytes(), kerImg.getSize(), convImg.getPtr(), convImg.getStepBytes());[/cpp]
0 Kudos
Copy link
SergeyKostrov
Valued Contributor II
‎08-17-2012 06:58 PM
1,351 Views
Quoting Ryan
Please see attached for zip... 

Received and no errors detected. Thank you.

    Testing ...
    testing: IppiConvValid_mem_test/   OK
    testing: IppiConvValid_mem_test/mem_test/   OK
    testing: IppiConvValid_mem_test/mem_test/mem_test.cpp   OK
    testing: IppiConvValid_mem_test/mem_test/mem_test.vcproj   OK
    testing: IppiConvValid_mem_test/mem_test/mem_test.vcproj.AFX-RYAN.afx-admin.user   OK
    testing: IppiConvValid_mem_test/mem_test/stdafx.cpp   OK
    testing: IppiConvValid_mem_test/mem_test/stdafx.h   OK
    testing: IppiConvValid_mem_test/mem_test/targetver.h   OK
    testing: IppiConvValid_mem_test/mem_test/x64/   OK
    testing: IppiConvValid_mem_test/mem_test/x64/Release/   OK
    testing: IppiConvValid_mem_test/mem_test/x64/Release/BuildLog.htm   OK
    testing: IppiConvValid_mem_test/mem_test/x64/Release/mem_test.exe.intermediate.manifest   OK
    testing: IppiConvValid_mem_test/mem_test/x64/Release/mem_test.obj   OK
    testing: IppiConvValid_mem_test/mem_test/x64/Release/mem_test.pch   OK
    testing: IppiConvValid_mem_test/mem_test/x64/Release/mt.dep   OK
    testing: IppiConvValid_mem_test/mem_test/x64/Release/stdafx.obj   OK
    testing: IppiConvValid_mem_test/mem_test/x64/Release/vc90.idb   OK
    testing: IppiConvValid_mem_test/mem_test/x64/Release/vc90.pdb   OK
    testing: IppiConvValid_mem_test/mem_test.ncb   OK
    testing: IppiConvValid_mem_test/mem_test.sln   OK
    testing: IppiConvValid_mem_test/mem_test.suo   OK
    testing: IppiConvValid_mem_test/x64/   OK
    testing: IppiConvValid_mem_test/x64/Release/   OK
    testing: IppiConvValid_mem_test/x64/Release/libiomp5md.dll   OK
    testing: IppiConvValid_mem_test/x64/Release/mem_test.exe   OK
    testing: IppiConvValid_mem_test/x64/Release/mem_test.pdb   OK
    No errors detected in compressed data of C:/Documents and ../Desktop/IppiConvValid_mem_test.zip.

Best regards,
Sergey
0 Kudos
Copy link
Ryan2
Beginner
‎08-22-2012 09:43 AM
1,351 Views
What are the results of your testing?
0 Kudos
Copy link
SergeyKostrov
Valued Contributor II
‎08-27-2012 04:18 PM
1,351 Views
Quoting Ryan
What are the results of your testing?


Hi Ryan,

I'm sorry for some delay with my tests. I confirm that some internal problem with 'ippiConvValid_32f_C1R' exists. Please take a look at my results:

>> Test-case with Kernel size = 3 <<
     ...
     Enter '0' to quit
     Kernel size relative to input size 1080p: 3
     Input width: 7680, Input height: 4320
     Kernel width: 5761, Kernel height: 3241
     IppStatus code for ippiConvValid_8u_C1R: 0
     ...

>> Here is a summary for different Kernel sizes <<

     Kernel size relative to input size 1080p: 1
     IppStatus code for ippiConvValid_8u_C1R: 0

     Kernel size relative to input size 1080p: 2
     IppStatus code for ippiConvValid_8u_C1R: 0

     Kernel size relative to input size 1080p: 3
     IppStatus code for ippiConvValid_8u_C1R: 0

     Kernel size relative to input size 1080p: 4
     IppStatus code for ippiConvValid_8u_C1R: -9 ( Error: ippStsMemAllocErr )

     Kernel size relative to input size 1080p: 5
     IppStatus code for ippiConvValid_8u_C1R: N/A because the test application crashed

Best regards,
Sergey

0 Kudos
Copy link
SergeyKostrov
Valued Contributor II
‎08-27-2012 04:18 PM
1,351 Views
Quoting Ryan
What are the results of your testing?


Please take a look at a screenshot:

IppiConvolveProblem.jpg

'ippiConvValid_32f_C1R' uses lots of memory, especially when kernel size gets bigger, and it crashes
when it tries to allocate more than 1.65GB of memory.

Best regards,
Sergey

0 Kudos
Copy link
SergeyKostrov
Valued Contributor II
‎08-27-2012 04:19 PM
1,351 Views
Here is your test-case with my small modifications:

[cpp]#include "stdafx.h"

#include 
#include 

#include "..IPPIncludeippi.h"

void doConvolution( float size );

struct Image
{
private:
	Ipp32f *ptr;
	IppiSize size;
	int stepBytes;

public:
	Image( int width, int height )
	{
		ptr = NULL;
		size.width = width;
		size.height = height;
		stepBytes = -1;

		ptr = ::ippiMalloc_32f_C1( width, height, &stepBytes );
	}

	Ipp32f * getPtr()
	{
		return ptr;
	}

	Ipp32f * getPtr( int x, int y )
	{
		Ipp32f *pixelPtr = ( Ipp32f * )( ( char * )ptr + x + y * size.width );
		return pixelPtr;
	}

	IppiSize getSize()
	{
		return size;
	}

	int getStepBytes()
	{
		return stepBytes;
	}

	~Image()
	{
		if( ptr != NULL )
		{
			::ippiFree( ptr );
			ptr = NULL;
		}
	}
};

int _tmain( int argc, _TCHAR *argv[] )
{
	float size = 0.0f;

	std::cout << "Enter '0' to quit" << std::endl;

	while( true )
	{
		std::cout << "Kernel size relative to input size 1080p: ";
		std::cin >> size;

		if( size == 0.0f )
			break;

		doConvolution( size );
	}

	return 0;
}

void doConvolution( float size )
{
	// Allocate srcImg 1920x1080
	Image srcImg( 1920, 1080 );
	_tprintf( _T("Allocate srcImg 1920x1080 - donen") );

	// Initialize srcImg
	{
		Ipp32f *pixelPtr = NULL;
		for( int y = 0; y < srcImg.getSize().height; y++ )
		{
			pixelPtr = srcImg.getPtr( 0, y );
			for( int x = 0; x < srcImg.getSize().width; x++ )
			{
				*pixelPtr = 100;
				pixelPtr++;
			}
		}
	}
	_tprintf( _T("Initialize srcImg - donen") );

	// Allocate kerImg. Size multiplied by srcImg rounded to next odd integer
	Image kerImg( ( int )ceil( srcImg.getSize().width * size ) | 1, ( int )ceil( srcImg.getSize().height * size ) | 1 );
	_tprintf( _T("Allocate kerImg - donen") );

	// Initialize kerImg
	{
		Ipp32f *pixelPtr = NULL;
		for( int y = 0; y < kerImg.getSize().height; y++ )
		{
			pixelPtr = kerImg.getPtr( 0, y );
			for( int x = 0; x < kerImg.getSize().width; x++ )
			{
				*pixelPtr = 100;
				pixelPtr++;
			}
		}
	}
	_tprintf( _T("Initialize kerImg - donen") );

	// Padding is half of kerSize rounded down to nearest even number
	int width_pad = kerImg.getSize().width >> 1;
	int height_pad = kerImg.getSize().height >> 1;

	// Allocate repImg
	Image repImg( srcImg.getSize().width + width_pad * 2, srcImg.getSize().height + height_pad * 2 );
	_tprintf( _T("Allocate repImg - donen") );

	// Repeat borders of imgSrc
	::ippiCopyReplicateBorder_32f_C1R( srcImg.getPtr(), srcImg.getStepBytes(), srcImg.getSize(),
									   repImg.getPtr(), repImg.getStepBytes(), repImg.getSize(),
									   height_pad, height_pad );

	// Allocate convImg
	Image convImg( srcImg.getSize().width, srcImg.getSize().height );
	_tprintf( _T("Allocate convImg - donen") );

	IppStatus status = ::ippiConvValid_32f_C1R( repImg.getPtr(), repImg.getStepBytes(), repImg.getSize(),
												kerImg.getPtr(), kerImg.getStepBytes(), kerImg.getSize(),
												convImg.getPtr(), convImg.getStepBytes() );

	_tprintf( _T("Convolve - donen") );

	std::cout << "Input width: " << repImg.getSize().width << ", Input height: " << repImg.getSize().height << std::endl;
	std::cout << "Kernel width: " << kerImg.getSize().width << ", Kernel height: " << kerImg.getSize().height << std::endl;
	std::cout << "IppStatus code for ippiConvValid_8u_C1R: " << status << std::endl << std::endl;
}
[/cpp]
0 Kudos
Copy link
Thomas_Jensen1
Beginner
‎08-28-2012 12:28 AM
1,351 Views
It seems that getPtr( int x, int y ) is wrong, since it calculates y * size.width instead of y * stepBytes.

0 Kudos
Copy link
SergeyKostrov
Valued Contributor II
‎08-28-2012 04:00 PM
1,351 Views
It seems that getPtr( int x, int y ) is wrong, since it calculates y * size.width instead of y * stepBytes.


You have a working test-case ( Thank you, Ryan ) and you could easily verify it. Please let us know results...

0 Kudos
Copy link
SergeyKostrov
Valued Contributor II
‎08-28-2012 04:05 PM
1,351 Views
Quoting Ryan
What are the results of your testing?


When the test application crashes these two 'ippi' functions are shown in the Call Stack of a Visual Studio 2005:

     ...
     ippiZigzagFwd8x8_16s_C1()               // crash was here
     ...
     ippiCopy_32f_C1R()                               // it worked
     ...

and Output Window shows

     Unhandled exception ... 0xC0000005: Access violation writing location ...

0 Kudos
Copy link
Reply

Community support is provided Monday to Friday. Other contact methods are available here.

Intel does not verify all solutions, including but not limited to any file transfers that may appear in this community. Accordingly, Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade.

For more complete information about compiler optimizations, see our Optimization Notice.