COM

[Christopher_Pisz]

Why does inspector complain about virtually every single COM related function call?

I've got some 50 or so entries claiming:
CoInitialize - Invalid memory access.
CoUnitialize - Invalid Memory access.

I've got hundreds complaining about Uninitialized partial memory access or invalid memory access:
Calls to ADO generate a good 200 entries for MSWSOCK.dll, MSDART.dll, OLEAUT32.dll, iclit09b.dll, MSVCR80.dll, ole32.dll, RPCRT4.dll

All of the entires that complain about any module that I actually have source for point somewhere down into assembly where something COM related is happening.

My project obviously leaks, because if I iterate over it a number of times great enough, I can sit and watch process explorer climb and climb until it crashes due to running out of memory. Yet, nothing points to anything in my source.

What is the official word? Can we not use this product with ATL/COM code? Are there some special instructions somewhere for finding leaks in projects involving ATL/COM? How can I go about testing my code for leaks when ATL/COM is such a large part of the system?

Going through and checking every single one of 500+ entries, digging through their call stacks, and making test cases to isolate them on the chance that there is some problem with MS code, is just not reasonable. Suppressing them is not reasonable either. There are just too many, and in order to surpress them we'd have to do the former to truly rule them out.

[Peter_W_Intel]

Inspecting memory error or thread error on Huge project (with hundreds of module), you can use "Include only the following module(s)" function and put your interest of modules into list, in Project Properties. Thus, Inspector XE only tracks your interest of modules. In command line - (for example)

inspxe-cl -collect mi3-module-filter-mode=include -module-filter=myprogram.exe -- myprogram.exe

Regards, Peter

[Christopher_Pisz]

I will try that. Thank you.

However, the question to whether or not Intel Inspector XE "supports COM and ATL" is unanswered. I suspect not. Should I be getting all of these entries from the inspector? How do I know if any of them really leak or if they are all false positives? Is it known that every COM call results in an inspector entry? Is there some chance of leakage in mashalling and unmarshalling?

I know my project leaks or grows. Something is amiss. However, inspector is not giving me any entries in source code that I control. I am rather lost as to how to narrow down the problem.

[Peter_W_Intel]

I don't think that Inspector XE supports COM/ATL. Inspectorsupports applications whichare launched by the tool, not for service application, device drivers, etcin system.

Inspector can support Native code implemented by languages (e.g C/C++, Fortran), and Managed code implemented by C# (full support for .NET below 4.0, and limited support for .NET 4.0). Inspector also supportsMixed code (Native & Managed code).

Regards, Peter

[SergeyKostrov]

Quoting Christopher Pisz

Why does inspector complain about virtually every single COM related function call?
...
My project obviously leaks, because if I iterate over it a number of times great enough, I can sit and watch process explorer climb and climb until it crashes due to running out of memory. Yet, nothing points to anything in my source...

Please take into account that ATL classes are full of C++ operators '=', '->', etc. It means, that some number
of objects could be created and not released. It is impossible to guess what is going on in your application, or a Test-Case,
withoutseeing it.

Some time ago it took for me more than 9 months to find a bug inATL codes related to some C++ operator '='when
it wasused withsmart pointers for some MIDL generated class wrappers.

Best regards,
Sergey

[Christopher_Pisz]

Here is one small example and it doesn't even instantiate a COM object yet. It causes inspector to generate 1 missing allocation item, 2 kernal resource leak items, 2 invalid memory acces items, when running m1:

#include

#include

#include

#include

//-----------

int main()

{

CoInitialize(NULL);

CoUninitialize();

return 0;

}


Does CoInitialize really leak? Probably not.
Add in Any _com_ptr and call create instance on it and it gets worse.
Add in a call to any method on that instance and you get more.
Now bring some threads into the mix and you can get spammed with hundreds of entries.

[Peter_W_Intel]

Thank you for test code. I think that most of users focus memory check in their code, not in ole or COM.

C:\temp>inspxe-cl -version

Intel Inspector XE 2011 Update 9 (build 218166) Command Line tool

Copyright (C) 2009-2012 Intel Corporation. All rights reserved.

C:\temp>inspxe-cl -versionIntel Inspector XE 2011 Update 9 (build 218166) Command Line toolCopyright (C) 2009-2012 Intel Corporation. All rights reserved.

C:\temp>icl /Zi comtest.cpp

Intel C++ Compiler XE for applications running on IA-32, Version 12.1.3.300 B

uild 20120130

Copyright (C) 1985-2012 Intel Corporation. All rights reserved.

comtest.cpp

Microsoft Incremental Linker Version 10.00.30319.01

Copyright (C) Microsoft Corporation. All rights reserved.

-out:comtest.exe

-debug

-pdb:comtest.pdb

comtest.obj

C:\temp>inspxe-cl -collect mi3 -module-filter-mode=include -module-filter=comtes

t.exe -- comtest.exe

Used suppression file(s): []

0 new problem(s) found

[Christopher_Pisz]

It's really notas simple as "in or out of COM". user code could be inside a COM object. Sometimes the profiler complains about marshalling/unmarshalling and the user could have done it incorrectly. Sometimes the user forgets to call release on an interface pointer and did not use a smart pointer. These kinds of things are what we need to look for but cannot spot because of the hundreds of falsepositives that get generated.

If I filter the inspector to only look at the calling code, how then can I see if my code inside the COM object leaked? Or how can I tell if I screwed up using a BSTR than I got control of but did not release? In essense, how can I unit test my implementation in a COM object for memory leaks?

I've taken to doing my best to make proxies for COM interfaces, seperating out COM class implementations into seperate classes, but it is really making things difficult and some of them arejust impossible to test at all.I can't just turn to my boss and say, "Well, let's not use COM/ATL anymore and spend the next few years replacing it, because we will never find a memory profiler that works with it."

BTW, how do I include more than an executable in the module filter? I've only seen examples limited to the .exe itself. How can I cover the .exe and a .dll or two? Say I have a "utility.dll" with some math functions in it or something.

[SergeyKostrov]

Quoting Christopher Pisz

Here is one small example and it doesn't even instantiate a COM object yet. It causes inspector to generate 1 missing allocation item,
2 kernal resource leak items, 2 invalid memory acces items...
...
I think there is something wrong with analysis inside of Inspector.

...
Does CoInitialize really leak? Probably not.

Absolutely correct. It isimpossible to believe that Microsoft's developers missed these "leaks" in
almost 20-year-old production code of fundamental COM functions!

I woulddo a real test with a modified version of your little Test-Case:

[cpp]#include #include void main( void ) { for( int t = 0; t < 1073741824; t++ ) { ::CoInitialize( NULL ); ::Sleep( 1 ); ::CoUninitialize(); } } [/cpp]

Best regards,
Sergey

[SergeyKostrov]

Hi Christopher,

Quoting Christopher Pisz

...Sometimes the user forgets to call release on an interface pointer and did not use a smart pointer. These kinds of things are what we need to look for but cannot spot because of the hundreds of falsepositives that get generated...

Let's assume that your application or SDK consists of several softwaresubsystems or domains. Than, forall software
subsystems or domainsyou should have a set ofstandalone test cases and all these test cases should pass.

In another words, all components must be certified for a production environment and a certification process
could consist of:

- Testing
- Stress-Testing
- Verification ( some FunctionAis implemented as described inRequirements Document )
- Validation ( some FunctionAreturns absolutely expected results )

If you don't have such a certification process in-placeit is not toolate to adoptit.

Of course, Inspector could help, but you need to look at all components anyway in order to fix these problems.

Best regards,
Sergey

[Peter_W_Intel]

Quoting Christopher Pisz

BTW, how do I include more than an executable in the module filter? I've only seen examples limited to the .exe itself. How can I cover the .exe and a .dll or two? Say I have a "utility.dll" with some math functions in it or something.

C:\temp>inspxe-cl -collect mi3 -module-filter-mode=include -module-filter=comtest.exe,mscrt.dll -- comtest.exe
Used suppression file(s): []

0 new problem(s) found

[Christopher_Pisz]

This doesn't seem to work. Making a list seperated by commas runs, but it doesn't yield any results.

I went and purposely put a memory leak in my unit test and ran it with the following:
inspxe-cl -collect mi3 -module-filter-mode=include -module-filter=DevTest_StatusClient.exe,CommonLibd.dll -- DevTest_StatusClient.exe

0 problems Found.

Then I ran it with:
inspxe-cl -collect mi3 -module-filter-mode=include -module-filter=DevTest_StatusClient.exe -- DevTest_StatusClient.exe

1 New Problem Found


Both should have found the problem. the spellings of the modules are correct. I verified with process explorer that the dll name, spelling, and location is correct. Even if it wasn't, it shouldn't run as normal and report 0 problems.

[Peter_W_Intel]

This is very interesting - if you added dll file as interest of module there was no error report.Did it imply some arguments (variables) inexport function(s) was declared indll module, butreturned to the caller in executable?

Please submit a ticket to https://premier.intel.com with your test case for investigating. Thank you!

Regards, Peter

[SergeyKostrov]

Quoting Christopher Pisz

...I went and purposely put a memory leak in my unit test...

How did you do this? Could you post a piece of code that simulates memory leaks for review?
Thanks.

PS: I domemory leaks simulationas follows:

[cpp] #ifdef _RTSIMULATE_MEMORYLEAKS char *pchBuffer = NULL; pchBuffer = new char[ sizeof( char ) * 777 ]; #endif [/cpp]

[Christopher_Pisz]

//-------------------------

// main.cpp

// Project Includes

#include "precompiled-header.h"

#include "StatusClientTest.h"

#include "StatusSubscriptionTest.h"

#include "CachedDBDataTest.h"

int main()

{

int return_code = 0;

// return_code = StatusClientTest01();

// return_code = StatusSubscriptionTest01();

return_code = CachedDBDataTest01();

return return_code;

}

// CachedDBDataTest.h

#ifndef CACHEDDBDATATEST_H

#define CACHEDDBDATATEST_H

/// Creates CachedDBData object, fetches the data from the database, and repeatidly calls the accessors to the data

int CachedDBDataTest01();

#endif

//CachedDBDataTest.cpp

// Project Includes

#include "precompiled-header.h"

#include "ComtracifCallbackProxy.h"

#include "TssifProxy.h"

// Ssm Core Lib

#include "CachedDBData.h"

#include "GlobalStatusClient.h"

#include "SsmToComtracifImpl.h"

#include "SsmToTssif.h"

#include "StatusClientImpl.h"

#include "StatusUpdate.h"

#include "StatusSubscription.h"

// Shared Utility Includes

#include "SharedUtility/Configuration.h"

// Utility Library Includes

#include "Report.h"

// Boost Includes

#include

//------------------------------------------------------------------------------

int CachedDBDataTest01()

{

/*

try

{

// Initialize COM

CoInitialize(NULL);

// Create instances needed in specific order, due to circular dependencies

ATMS::ReportThreadSafeC * reporter = new ATMS::ReportThreadSafeC(L"DevTest");

ATMS::pReporter = reporter;

ConfigurationData::SharedPtr configurationData(new ConfigurationData());

ATMS::CachedDBData::SharedPtr cachedDBData = ATMS::CachedDBData::Create(reporter, configurationData);

cachedDBData->FetchAllPersistentData();

ATMS::pEnumConverter = &(cachedDBData->enumConverter_);

// Idle for 10 seconds to let the threads for objects we have created do their thing

// and the memory difference reading we get really represents what we do after this point

Sleep(10000);

// Get the initial count if base objects created by the DBLib

unsigned int numLeaked = CommonBaseC::GetNumInstances();

for(ATMS::RIDt rid = 0; rid < 1000; ++rid)

{

cachedDBData->lock_.Shared();

ATMS::DBVehicleDetectorC * dataItem = cachedDBData->vehicleDetectors_.FindByRID(rid);

cachedDBData->lock_.Release();

// We must call Release() manually on anything we obtain from the DBLib

// Bug 2706, 2710, 2711

if( dataItem )

{

dataItem->Release();

}

}

// Get the count if base objects created by the DBLib

numLeaked = CommonBaseC::GetNumInstances();

cachedDBData.reset();

ATMS::pEnumConverter = NULL;

// Get the count if base objects created by the DBLib

numLeaked = CommonBaseC::GetNumInstances();

configurationData.reset();

if( reporter )

{

delete reporter;

reporter = NULL;

ATMS::pReporter = NULL;

}

// Release COM

CoUninitialize();

}

catch(...)

{

// Error something threw something something dark side

std::wcout << "Error - Unhandled Exception";

return 1;

}

*/

int * p = new int(6);

return 0;

}

//------------------------

//DevTest_StatusClient.bat

inspxe-cl -collect mi3 -module-filter-mode=include -module-filter=DevTest_StatusClient.exe,CommonLibd.dll -- DevTest_StatusClient.exe

//-------------------------
//DevTest_StatusClient02.bat
inspxe-cl -collect mi3 -module-filter-mode=include -module-filter=DevTest_StatusClient.exe -- DevTest_StatusClient.exe

[SergeyKostrov]

>>...
>>int *p = new int(6);
>>...

Your version allocates4 bytesthen initializesthe memory with a value '6' and nothing is wrongwith it.Could you
try to allocate a bigger chunk of memory, like:

...
int *p = new int[ sizeof( int ) *1024 ]; // In total 4KB ( 4x1024 bytes )
...
int *p = new int[1024 ]; // In total 4KB ( 4x1024 bytes )
...

Best regards,
Sergey

EDITED: Thank you for catching my error.

[Christopher_Pisz]

Umm, you've actually just allocated 4 * 1024 ints, each being 4 bytes on 32 bit Windows = 16KB. I really don't get your use of the sizeof operator there. You are allocating a quanitity of ints, not bytes.

Anyway, it doesn't matter if I allocate 1 or 5000, I get the same result.

[SergeyKostrov]

Quoting Christopher Pisz

Umm, you've actually just allocated 4 * 1024 ints, each being 4 bytes on 32 bit Windows = 16KB. I really don't get your use of the sizeof operator there. You are allocating a quanitity of ints, not bytes.
...

Thank you for catching my error. I've corrected it.

Best regards,
Sergey

[SergeyKostrov]

Christopher,

What type of application are you trying to monitor? Is it a Win32 Console or aGUI application?

If your application has some GUI I would recommend to investigate what is going on with WindowsUSER and GDI objects ( using
the Windows Task Manager ). Please take a look at a screenshot:



USER or GDI objects leaks ( UI-Resources Leaks )could lead to an application crash.

Best regards,
Sergey