Intel® Integrated Performance Primitives
Deliberate problems developing high-performance vision, signal, security, and storage applications.
6814 Discussions

Test for possible IppiConvValid bug

Ryan2
Débutant
‎08-15-2012 07:26 AM
2 475 Visites
 
Hi,
 
I'm getting a -9 mem error with ippiConvValid_32f_C1R even though  my system has plenty of memory.
 
I allocated a 25GB with malloc just to make sure my program's memory allocation wasn't being limited by Windows.  No problems there.
 
Attached to this email is a compiled Windows x64 console app.  Everyone please download it and reply with the MAX size you can enter before you get a -9 err along with your system specs.  If you want to build it yourself, please go to my other post to download the VS project.  http://redfort-software.intel.com/en-us/forums/showthread.php?t=106382&o=a&s=lr
 
Hopefully there is a solution to this because it's really hampering me!
 
Best,
 
Ryan
mem_test_exe.rar
0 Compliments

Lien copié

18 Réponses
SergeyKostrov
Précieux contributeur II
‎08-15-2012 08:12 PM
2 475 Visites
Quoting Ryan
...I allocated a 25GB with malloc just to make sure my program's memory allocation wasn't being limited by Windows. No problems there...


Ryan,

Did you try to allocate more memory with a 'malloc' CRT function, for example 64GB and higher? Please take a look at
a test-application I created some time ago and attached to that post:

     http://redfort-software.intel.com/en-us/forums/showpost.php?p=189221

I'll take a look at your test-application.

Best regards,
Sergey

0 Compliments
Copier le lien
SergeyKostrov
Précieux contributeur II
‎08-15-2012 08:19 PM
2 475 Visites
I'll take a look at your test-application...


What about source codes of your test-application? At least simplified version that reproduces the problem
with ippiConvValid_xxx IPP function.

0 Compliments
Copier le lien
Ryan2
Débutant
‎08-15-2012 08:55 PM
2 475 Visites
Hi Sergey,
 
Attached is the source code for my test app.  It's very easy to follow.
 
Thanks for looking into this for me!
 
Best,
 
Ryan
IppiConvValid_mem_test.rar
0 Compliments
Copier le lien
Ryan2
Débutant
‎08-15-2012 09:03 PM
2 475 Visites
Hi Sergey,
 
I ran your mem test app with the CM3 line uncommented:
 
 __int64 _MEM_SIZE = ( 1073741824 * 64.0 ); // 64GB Recommended VM =  N/A  / 96GB
 
64-bit Windows platform
Allocating memory with 'malloc' CRT-function - 1D
Succesfully Allocated 64.00GB
64-bit Windows platformAllocating memory with 'malloc' CRT-function - 1DSuccesfully Allocated 64.00GB
 
0 Compliments
Copier le lien
Ryan2
Débutant
‎08-16-2012 07:26 AM
2 475 Visites
Update,
 
I also tried my mem test app with IppiConvValid_16s_C1R, IppiConvValid_16u_C1R, and IppiConvValid_8u_C1R and the resutls are the same.
 
On my machine:
 
(Dual socket Intel Xeon E5645, 48GB DDR3) - 1.06 before I get -9 mem error.
  
On three other computers I've tried:
 
(Dual socket Xeon E5420, 16GB DDR2) -1.89.
(Dual socket Xeon E5420, 16GB DDR2) - 1.89.
(Dual socket i7, 12GB DDR2) - 1.89.
 
 
Ryan
0 Compliments
Copier le lien
SergeyKostrov
Précieux contributeur II
‎08-16-2012 07:02 PM
2 475 Visites
Quoting Ryan
Attached is the source code for my test app.  It's very easy to follow...


Hi Ryan,

Could you upload as a zip archive, please? I couldn't extract your sources from the rar archive:

   1   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\mem_test.cpp" header broken
   2   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\mem_test.cpp
   3   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\mem_test.vcproj" header broken
   4   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\mem_test.vcproj
   5   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\stdafx.cpp" header broken
   6   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\stdafx.cpp
   7   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\stdafx.h" header broken
   8   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\stdafx.h
   9   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\targetver.h" header broken
  10   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\targetver.h
  11   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\x64\Release\BuildLog.htm" header broken
  12   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\x64\Release\BuildLog.htm
  13   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\x64\Release\mem_test.exe.intermediate.manifest" header broken
  14   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\x64\Release\mem_test.exe.intermediate.manifest
  15   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\x64\Release\mem_test.obj" header broken
  16   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\x64\Release\mem_test.obj
  17   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\x64\Release\mem_test.pch" header broken
  18   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\x64\Release\mem_test.pch
  19   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\x64\Release\mt.dep" header broken
  20   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\x64\Release\mt.dep
  21   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\x64\Release\stdafx.obj" header broken
  22   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\x64\Release\stdafx.obj
  23   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\x64\Release\vc90.idb" header broken
  24   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\x64\Release\vc90.idb
  25   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\x64\Release\vc90.pdb" header broken
  26   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test\x64\Release\vc90.pdb
  27   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test.ncb" header broken
  28   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test.ncb
  29   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test.sln" header broken
  30   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test.sln
  31   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test.suo" header broken
  32   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\mem_test.suo
  33   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\x64\Release\libiomp5md.dll
  34   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\x64\Release\mem_test.exe" header broken
  35   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\x64\Release\mem_test.exe
  36   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\x64\Release\mem_test.pdb" header broken
  37   IppiConvValid_mem_test.rar: Unknown method in IppiConvValid_mem_test\x64\Release\mem_test.pdb
  38   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\x64\Release" header broken
  39   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test\x64" header broken
  40   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\x64\Release" header broken
  41   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\mem_test" header broken
  42   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test\x64" header broken
  43   IppiConvValid_mem_test.rar: File "IppiConvValid_mem_test" header broken

Best regards,
Sergey

PS: That is really strange but it looks like older versions of WinRAR are not compatible with the newer versions.

0 Compliments
Copier le lien
SergeyKostrov
Précieux contributeur II
‎08-16-2012 07:32 PM
2 475 Visites
Hi Ryan, This ia a short follow up regarding the error -9...

Quoting Ryan
...I'm getting a -9 mem error with ippiConvValid_32f_C1R even though  my system has plenty of memory.
 
I allocated a 25GB with malloc just to make sure my program's memory allocation wasn't being limited by Windows.  No problems there.
 
Attached to this email is a compiled Windows x64 console app. Everyone please download it and reply with the MAX size you can enter before you get a -9 err...


This is how it is declared:

     typedef enum
     {
         ...
         ippStsMemAllocErr           = -9,
         ...
     } IppStatus;

and I think the problem could be possibly related to limitation(s) of some IPP functions, including ippiConvValid_xxx. Your 64-bit system
allows to allocate up to 64GB of memory (!) and you've done two tests. These "magic" limitations were
disccused a couple of times on the IPP forum in the past. Unfortunately, there is no article that describes these limitations
in some functions of IPP library.

Best regards,
Sergey

0 Compliments
Copier le lien
Ryan2
Débutant
‎08-16-2012 07:33 PM
2 475 Visites
Hi Sergey,
 
That's weird.  Sorry about that.
 
Please see attached for zip. 
 
Best,
 
Ryan
IppiConvValid_mem_test.zip
0 Compliments
Copier le lien
Ryan2
Débutant
‎08-16-2012 07:37 PM
2 475 Visites
Hi Sergey,
 
Yes I looked up that enum.  That's exactly my thinking.  I want to know why IppConvValid is having that problem.  For it to run out of memory on my system it would have to be doing a HUGE FFT.  What I find even stranger is that an i7 with 12gigs of DD2 memory can handle more than my Xeon with 48gigs of DD3.
  
Ryan
0 Compliments
Copier le lien
Ryan2
Débutant
‎08-16-2012 07:44 PM
2 475 Visites
OK!
 
I tried using IppConvFull instead of Valid.  I was able to enter 3.2 into my mem test program before I got the -9 error code.  So WHY, does ConvValid crap out at 1.06?
[cpp]//Allocate repImg
	Image repImg(srcImg.getSize().width + width_pad * 2, srcImg.getSize().height + height_pad * 2);

	//Repeat borders of imgSrc
	ippiCopyReplicateBorder_32f_C1R(srcImg.getPtr(), srcImg.getStepBytes(), srcImg.getSize(), repImg.getPtr(), repImg.getStepBytes(), repImg.getSize(), height_pad, height_pad);

	//Allocate convImg
	Image convImg(repImg.getSize().width + kerImg.getSize().width - 1, repImg.getSize().height + kerImg.getSize().height - 1);

	IppStatus status = ippiConvFull_32f_C1R(repImg.getPtr(), repImg.getStepBytes(), repImg.getSize(), kerImg.getPtr(), kerImg.getStepBytes(), kerImg.getSize(), convImg.getPtr(), convImg.getStepBytes());[/cpp]
0 Compliments
Copier le lien
SergeyKostrov
Précieux contributeur II
‎08-17-2012 06:58 PM
2 475 Visites
Quoting Ryan
Please see attached for zip... 

Received and no errors detected. Thank you.

    Testing ...
    testing: IppiConvValid_mem_test/   OK
    testing: IppiConvValid_mem_test/mem_test/   OK
    testing: IppiConvValid_mem_test/mem_test/mem_test.cpp   OK
    testing: IppiConvValid_mem_test/mem_test/mem_test.vcproj   OK
    testing: IppiConvValid_mem_test/mem_test/mem_test.vcproj.AFX-RYAN.afx-admin.user   OK
    testing: IppiConvValid_mem_test/mem_test/stdafx.cpp   OK
    testing: IppiConvValid_mem_test/mem_test/stdafx.h   OK
    testing: IppiConvValid_mem_test/mem_test/targetver.h   OK
    testing: IppiConvValid_mem_test/mem_test/x64/   OK
    testing: IppiConvValid_mem_test/mem_test/x64/Release/   OK
    testing: IppiConvValid_mem_test/mem_test/x64/Release/BuildLog.htm   OK
    testing: IppiConvValid_mem_test/mem_test/x64/Release/mem_test.exe.intermediate.manifest   OK
    testing: IppiConvValid_mem_test/mem_test/x64/Release/mem_test.obj   OK
    testing: IppiConvValid_mem_test/mem_test/x64/Release/mem_test.pch   OK
    testing: IppiConvValid_mem_test/mem_test/x64/Release/mt.dep   OK
    testing: IppiConvValid_mem_test/mem_test/x64/Release/stdafx.obj   OK
    testing: IppiConvValid_mem_test/mem_test/x64/Release/vc90.idb   OK
    testing: IppiConvValid_mem_test/mem_test/x64/Release/vc90.pdb   OK
    testing: IppiConvValid_mem_test/mem_test.ncb   OK
    testing: IppiConvValid_mem_test/mem_test.sln   OK
    testing: IppiConvValid_mem_test/mem_test.suo   OK
    testing: IppiConvValid_mem_test/x64/   OK
    testing: IppiConvValid_mem_test/x64/Release/   OK
    testing: IppiConvValid_mem_test/x64/Release/libiomp5md.dll   OK
    testing: IppiConvValid_mem_test/x64/Release/mem_test.exe   OK
    testing: IppiConvValid_mem_test/x64/Release/mem_test.pdb   OK
    No errors detected in compressed data of C:/Documents and ../Desktop/IppiConvValid_mem_test.zip.

Best regards,
Sergey
0 Compliments
Copier le lien
Ryan2
Débutant
‎08-22-2012 09:43 AM
2 475 Visites
What are the results of your testing?
0 Compliments
Copier le lien
SergeyKostrov
Précieux contributeur II
‎08-27-2012 04:18 PM
2 475 Visites
Quoting Ryan
What are the results of your testing?


Hi Ryan,

I'm sorry for some delay with my tests. I confirm that some internal problem with 'ippiConvValid_32f_C1R' exists. Please take a look at my results:

>> Test-case with Kernel size = 3 <<
     ...
     Enter '0' to quit
     Kernel size relative to input size 1080p: 3
     Input width: 7680, Input height: 4320
     Kernel width: 5761, Kernel height: 3241
     IppStatus code for ippiConvValid_8u_C1R: 0
     ...

>> Here is a summary for different Kernel sizes <<

     Kernel size relative to input size 1080p: 1
     IppStatus code for ippiConvValid_8u_C1R: 0

     Kernel size relative to input size 1080p: 2
     IppStatus code for ippiConvValid_8u_C1R: 0

     Kernel size relative to input size 1080p: 3
     IppStatus code for ippiConvValid_8u_C1R: 0

     Kernel size relative to input size 1080p: 4
     IppStatus code for ippiConvValid_8u_C1R: -9 ( Error: ippStsMemAllocErr )

     Kernel size relative to input size 1080p: 5
     IppStatus code for ippiConvValid_8u_C1R: N/A because the test application crashed

Best regards,
Sergey

0 Compliments
Copier le lien
SergeyKostrov
Précieux contributeur II
‎08-27-2012 04:18 PM
2 475 Visites
Quoting Ryan
What are the results of your testing?


Please take a look at a screenshot:

IppiConvolveProblem.jpg

'ippiConvValid_32f_C1R' uses lots of memory, especially when kernel size gets bigger, and it crashes
when it tries to allocate more than 1.65GB of memory.

Best regards,
Sergey

0 Compliments
Copier le lien
SergeyKostrov
Précieux contributeur II
‎08-27-2012 04:19 PM
2 475 Visites
Here is your test-case with my small modifications:

[cpp]#include "stdafx.h"

#include 
#include 

#include "..IPPIncludeippi.h"

void doConvolution( float size );

struct Image
{
private:
	Ipp32f *ptr;
	IppiSize size;
	int stepBytes;

public:
	Image( int width, int height )
	{
		ptr = NULL;
		size.width = width;
		size.height = height;
		stepBytes = -1;

		ptr = ::ippiMalloc_32f_C1( width, height, &stepBytes );
	}

	Ipp32f * getPtr()
	{
		return ptr;
	}

	Ipp32f * getPtr( int x, int y )
	{
		Ipp32f *pixelPtr = ( Ipp32f * )( ( char * )ptr + x + y * size.width );
		return pixelPtr;
	}

	IppiSize getSize()
	{
		return size;
	}

	int getStepBytes()
	{
		return stepBytes;
	}

	~Image()
	{
		if( ptr != NULL )
		{
			::ippiFree( ptr );
			ptr = NULL;
		}
	}
};

int _tmain( int argc, _TCHAR *argv[] )
{
	float size = 0.0f;

	std::cout << "Enter '0' to quit" << std::endl;

	while( true )
	{
		std::cout << "Kernel size relative to input size 1080p: ";
		std::cin >> size;

		if( size == 0.0f )
			break;

		doConvolution( size );
	}

	return 0;
}

void doConvolution( float size )
{
	// Allocate srcImg 1920x1080
	Image srcImg( 1920, 1080 );
	_tprintf( _T("Allocate srcImg 1920x1080 - donen") );

	// Initialize srcImg
	{
		Ipp32f *pixelPtr = NULL;
		for( int y = 0; y < srcImg.getSize().height; y++ )
		{
			pixelPtr = srcImg.getPtr( 0, y );
			for( int x = 0; x < srcImg.getSize().width; x++ )
			{
				*pixelPtr = 100;
				pixelPtr++;
			}
		}
	}
	_tprintf( _T("Initialize srcImg - donen") );

	// Allocate kerImg. Size multiplied by srcImg rounded to next odd integer
	Image kerImg( ( int )ceil( srcImg.getSize().width * size ) | 1, ( int )ceil( srcImg.getSize().height * size ) | 1 );
	_tprintf( _T("Allocate kerImg - donen") );

	// Initialize kerImg
	{
		Ipp32f *pixelPtr = NULL;
		for( int y = 0; y < kerImg.getSize().height; y++ )
		{
			pixelPtr = kerImg.getPtr( 0, y );
			for( int x = 0; x < kerImg.getSize().width; x++ )
			{
				*pixelPtr = 100;
				pixelPtr++;
			}
		}
	}
	_tprintf( _T("Initialize kerImg - donen") );

	// Padding is half of kerSize rounded down to nearest even number
	int width_pad = kerImg.getSize().width >> 1;
	int height_pad = kerImg.getSize().height >> 1;

	// Allocate repImg
	Image repImg( srcImg.getSize().width + width_pad * 2, srcImg.getSize().height + height_pad * 2 );
	_tprintf( _T("Allocate repImg - donen") );

	// Repeat borders of imgSrc
	::ippiCopyReplicateBorder_32f_C1R( srcImg.getPtr(), srcImg.getStepBytes(), srcImg.getSize(),
									   repImg.getPtr(), repImg.getStepBytes(), repImg.getSize(),
									   height_pad, height_pad );

	// Allocate convImg
	Image convImg( srcImg.getSize().width, srcImg.getSize().height );
	_tprintf( _T("Allocate convImg - donen") );

	IppStatus status = ::ippiConvValid_32f_C1R( repImg.getPtr(), repImg.getStepBytes(), repImg.getSize(),
												kerImg.getPtr(), kerImg.getStepBytes(), kerImg.getSize(),
												convImg.getPtr(), convImg.getStepBytes() );

	_tprintf( _T("Convolve - donen") );

	std::cout << "Input width: " << repImg.getSize().width << ", Input height: " << repImg.getSize().height << std::endl;
	std::cout << "Kernel width: " << kerImg.getSize().width << ", Kernel height: " << kerImg.getSize().height << std::endl;
	std::cout << "IppStatus code for ippiConvValid_8u_C1R: " << status << std::endl << std::endl;
}
[/cpp]
0 Compliments
Copier le lien
Thomas_Jensen1
Débutant
‎08-28-2012 12:28 AM
2 475 Visites
It seems that getPtr( int x, int y ) is wrong, since it calculates y * size.width instead of y * stepBytes.

0 Compliments
Copier le lien
SergeyKostrov
Précieux contributeur II
‎08-28-2012 04:00 PM
2 475 Visites
It seems that getPtr( int x, int y ) is wrong, since it calculates y * size.width instead of y * stepBytes.


You have a working test-case ( Thank you, Ryan ) and you could easily verify it. Please let us know results...

0 Compliments
Copier le lien
SergeyKostrov
Précieux contributeur II
‎08-28-2012 04:05 PM
2 475 Visites
Quoting Ryan
What are the results of your testing?


When the test application crashes these two 'ippi' functions are shown in the Call Stack of a Visual Studio 2005:

     ...
     ippiZigzagFwd8x8_16s_C1()               // crash was here
     ...
     ippiCopy_32f_C1R()                               // it worked
     ...

and Output Window shows

     Unhandled exception ... 0xC0000005: Access violation writing location ...

0 Compliments
Copier le lien
Répondre

L'assistance à la communauté est fournie du lundi au vendredi. D'autres moyens d'entrer en contact sont disponibles ici.

Intel ne vérifie pas l'intégralité des solutions, y compris, entre autres, les transferts de fichiers qui peuvent avoir lieu au sein de cette communauté. Intel décline donc toutes garanties explicites et implicites, y compris, sans s'y limiter, les garanties implicites de qualité marchande, d'adéquation à un usage particulier et de respect des droits de propriété intellectuelle, ainsi que toute garantie découlant des performances, des transactions ou de l'utilisation.

Pour obtenir des informations plus complètes sur l'optimisation des compilateurs, voir notre Avis d'optimisation.