Category: Memory

What’s new with Memory Validator

By , December 10, 2020 6:26 pm

There are many changes coming to Memory Validator.

I’m going to describe the various changes and the reasons behind them.

Name Change

The first one is the name change. C++ Memory Validator becomes Memory Validator.

Because Memory Validator will be capable of handling multiple technologies and languages having language specific designators prefixing Memory Validator doesn’t make any sense.

New UX Theme

A new UX theme which is has less visual clutter and is calmer to look at has been introduced. We’ve written about that in New UX Theme.


.Net Support

Memory Validator now supports .Net languages. C#, VB.Net, C++.Net, etc.

.Net Memory Validator is discontinued, all of it’s functionality moving into Memory Validator.

x64 and x86 support

C++ Memory Validator shipped in two versions, a 32 bit version and a 64 bit version that could also process 32 bit executables.

Memory Validator ships in a 64 bit version that can also process 32 bit executables. On a 32 bit machine the 32 bit version installs, on a 64 bit machine both 64 bit and 32 bit versions install, because occasionally there is a 32 bit native bug that you only deal with from the 32 bit version of the tool.

The reasons for this change are

  • .Net applications can be built in 32 bit, 64 bit, and Any CPU versions. An Any CPU version launched on a 64 bit machine will run as 64 bit. To provide full .Net support we couldn’t support Any CPU on 64 bit from 32 bit Memory Validator. The sensible option is to only support Memory Validator in a form that can support both 32 bit and 64 bit architectures.
  • 64 processors are the dominant processors in the market. We should support these by default.

New menu items

To support the new .Net functionality there are some additional launch options for working with ASP.Net applications (IIS and Web Development Server) as well as .Net applications and .Net services.


Launch ASP.Net application using IIS.


Launch ASP.Net application using Web Development Server.


New settings options

The settings dialog has four new panels to allow you to configure what types of .Net data are collected, whether to enable stale .Net object detection, if .Net heap dumps should be collected, and how and when .Net memory snapshots should be made.

.Net Collect


.Net Stale Object Detection


.Net Heap Dump


.Net Snapshots


New launch option

With the ability to process both native and .Net applications and mixed mode applications comes the desire to sometimes restrict memory monitoring to just native code, or just .Net code, or to allow any code (mixed mode) to be monitored. To handle this we’ve added a simple combo dropdown on the various launch dialogs that allows you to choose how memory monitoring is handled at a very high level.


New Summary Display


The summary display provides panels for a particular summary topic with key statistics for that topic displayed in the panel. Panels can display bar graphs, circular graphs and timeline graphs as well as data in table format. Most summaries allow you to click on a data item and be taken to a main tab for the data that was clicked on. This allows the summary display to function as a jumping off point to explore data throughout Memory Validator.

New Memory tab

The new memory tab performs the same function as the Memory tab in previous versions of Memory Validator, but now it contains two sub-tabs. One for viewing native allocations (memory and handles), and one for viewing .Net allocations (memory and handles).

New Statistics tab

The Statistics tab is where all the main statistics data is grouped together. Previously this data was in 3 main tabs: Types, Sizes and Locations. Those 3 tabs are now sub-tabs under the statistics tabs. They are joined by two .Net specific tabs: Generations and Ages, which report information about which generation has how many objects and what ages given objects are. The Ages tab can also report information about stale objects if that functionality has been turned on (off by default as CPU intensive).


New .Net tab

A new .Net tab is where you can find 3 new sub-tabs dedicated to understanding .Net specific memory behaviour. The three sub-tabs are .Net Snapshots, .Net Heap Dumps and .Net Leak Analysis.

New Snapshots tab

The Snapshots tab allows you to take memory snapshots and compare them to other memory snapshots creating memory comparisons. Both snapshots and comparisons can be viewed to identify the location where an object was allocated. You can also use the context menu to then jump to that object in a heap dump reference graph.


New Heap Dumps tab

The Heap Dump tab allows you to view heap dumps provided when a garbage collection takes place. Using the reference graph you can see which objects hold a reference to an object, and which objects an object is referencing. Plenty of options are provided to allow you to restrict the amount of data to wade through (so that you don’t end up with a graph full objects you didn’t build – from 3rd party software).


New .Net Leak Analysis tab

The Leak Analysis tab provides you with a powerful query functionality that allows to query the .Net allocation data to find objects that may be leaked, loitering unused in memory due to references that should have been set to null. We provide many predefined queries, but you can easily build your own queries. Combine with stale object detection data for even more power.


New Analysis tab

The old Analysis tab has been renamed Query and is now a sub-tab of the new Analysis tab. This sub-tab is joined by previous main tabs Coverage, Hotspots, Pages and Virtual.

The Hotspots tab has been updated to include hotspots for .Net memory and .Net handles. The hotspots tab has been moved to a sub-tab on the main Analysis tab.

Timeline updates

The Timeline tab has been updated to include data for .Net memory, .Net handles and .Net all allocations to provide an equivalent to the native timeline views.

Availability

These changes to Memory Validator will be available after 13 December 2020.

Memory leak in CPngImage

By , June 1, 2020 9:37 am

A memory leak in a surprising place

We’ve recently been doing some work switching our resources in our programs from BMP (using CBitmap) to PNG (using CPngImage).

At some point we got around to dog-fooding, which we do with our tools all the time, and we were surprised to see memory leaks (in the form of HGLOBAL handles) in our tools being reported by C++ Memory Validator.


We took a look and found they were coming from CPngImage::LoadFromBuffer(). Here’s the code, with the leaking lines highlighted.

BOOL CPngImage::LoadFromBuffer(LPBYTE lpBuffer, UINT uiSize)
{
        ASSERT(lpBuffer != NULL);

        HGLOBAL hRes = ::GlobalAlloc(GMEM_MOVEABLE, uiSize); // this line leaks
        if (hRes == NULL)
        {
                return FALSE;
        }

        IStream* pStream = NULL;
        LPVOID lpResBuffer = ::GlobalLock(hRes);             // this line leaks
        ASSERT (lpResBuffer != NULL);

        memcpy(lpResBuffer, lpBuffer, uiSize);

        HRESULT hResult = ::CreateStreamOnHGlobal(hRes, FALSE, &pStream);

        if (hResult != S_OK)
        {
                return FALSE;
        }

        if (CMFCToolBarImages::m_bMultiThreaded)
        {
                CMFCToolBarImages::m_CriticalSection.Lock();
        }

        if (m_pImage == NULL)
        {
                m_pImage = new CImage;
                ENSURE(m_pImage != NULL);
        }

        m_pImage->Load(pStream);
        pStream->Release();

        BOOL bRes = Attach(m_pImage->Detach());

        if (CMFCToolBarImages::m_bMultiThreaded)
        {
                CMFCToolBarImages::m_CriticalSection.Unlock();
        }

        return bRes;
}

Verifying the memory leak

At first we thought C++ Memory Validator might have made a mistake, as it seemed unlikely that such a well used class would contain a mistake like a memory leak.

To check if this was correct memory leak report or a FALSE positive we created a test program where we can repeatedly create and destroy images in rapid succession. If the leak is real the toy program will soon fail to allocate memory. If the leak is a false positive by C++ Memory Validator, the toy program will run forever with no problems. The toy program demonstrated the leak is real – after just over 65,000 loads of an image with CPngImage, all GlobalAlloc() allocations fail.

The test program allows you to repeatedly load a BMP, a PNG into CPngImage and a PNG into an svlPngImage. The image is destroyed after loading. You can specify how many times to perform the load, 1, 10, 100, 1000, 10000, 64K and 100,000 times.


We have verified that his memory leak is present in all versions of CPngImage that ship with all versions of Visual Studio (up to and including VS2019, we haven’t looked at VS2021 yet).

Fixing the memory leak

The fix is to add two lines just before the return bRes; statement.

        ::GlobalUnlock(hRes);
        ::GlobalFree(hRes);

Unfortunately CPngImage loads it data from methods that are not virtual, so we couldn’t replace them in the implementation. We created a drop in replacement for CPngImage called svlPngImage. It’s identical to the CPngImage class with the exception that the CMFCToolBarImages calls are commented out, and the two additional lines to prevent the memory leak are added. If you’d like to use svlPngImage the source code for this drop in replacement is in the download.

Test Program Source Code

You can download the test program source code and project files.

Update, after response from Microsoft

I was surprised when I got a reply saying this couldn’t happen because the second parameter to ::CreateStreamOnHGlobal() was TRUE. I searched my machine again, and it was FALSE. I then searched a different machine and it was TRUE. So most likely this bug did exist (and we’re using the version of Visual Studio that has it) but has been fixed in a more recent service pack. My claim that the bug was in all versions of Visual Studio was incorrect because I was checking for the absence of the memory freeing calls without checking the value passed to ::CreateStreamOnHGlobal() was FALSE. The drop in replacement class is still valid.

If you’re using CPngImage, check CPngImage::LoadFromBuffer() and if the second parameter passed to ::CreateStreamOnHGlobal() is FALSE, use our svlPngImage class instead. Otherwise carry on as you are.

Update 2, after further response from Microsoft

Microsoft has modified this function to improve the error handling to fix a memory leak if ::CreateStreamOnHGlobal() fails, in response to this bug report, which prompted them to reexamine the function. Ironic in that the report I made, although demonstrating a real bug, that bug had already been fixed, but it still prompted a fresh look at the function behaviour.

Detecting memory leaks in ISAPI extensions

By , April 21, 2020 11:51 am

Three weeks ago I wrote about how to setup IIS for use with ISAPI extensions.

Today I’m going to show you how easy it is to monitor memory and handle allocations in IIS, and use that information to detect any memory and handle leaks that may be present in your ISAPI extension.

IIS is a very secure Windows service. The account security for IIS mean that it can’t access most parts of the Windows filesystem or even Windows objects. CGI and ISAPI extensions will only execute if you’ve configured IIS properly (see the above linked article for details). You can’t launch IIS from a tool like Memory Validator, and you can’t inject into IIS from a tool like Memory Validator because of the security constraints. That leaves you the option of using an API to interact with IIS from your tool of choice.

But I don’t want to use an API!

“But I don’t want to use an API” I hear you say. You’re concerned about having to build multiple versions of your ISAPI. One for use with Memory Validator and one for production. That’s a valid concern, but it’s not the case. You can just build one version of your ISAPI that uses Memory Validator and use that. If you haven’t done “Monitor ISAPI…” from the Launch menu prior to loading your ISAPI the Memory Validator dlls won’t be loaded. There is no dependency on Software Verify DLLs, you can ship your DLL without any need to ship the Memory Validator DLLs.

But should you need to examine memory allocation behaviour you can just fire up Memory Validator and get to work without making a special build.

ISAPI API

ISAPI extensions provide 3 APIs for IIS to use:

BOOL WINAPI GetExtensionVersion(HSE_VERSION_INFO *pVer);

BOOL WINAPI TerminateExtension(DWORD	dwFlags);

DWORD WINAPI HttpExtensionProc(EXTENSION_CONTROL_BLOCK *pECB);

GetExtensionVersion() is called when the ISAPI is first loaded. We’ll use this to load Memory Validator into IIS.

TerminateExtension() is called when the ISAPI is unloaded. We’ll use this to tell Memory Validator that the work with IIS is done.

HttpExtensionProc() is used to process any requests made to the ISAPI extension.

Memory Validator API

To add the Memory Validator API to the ISAPI we do the following to the ISAPI source code:


  1. Add two header files:

    #include "svlMVStubService.h"
    #include "svlServiceError.h"
    

  2. We add the following code to GetExtensionVersion(). This includes logging success and failure so that we can identify if anything has gone wrong. Without the logging determining failure inside of IIS is very hard (a debugger and a special build of the Memory Validator DLL is required).

        // load Validator here
    
        svlMVStub_setLogFileName(L"C:\\testISAPIWebsite\\svl_MV_log.txt");
        svlMVStub_deleteLogFile();
    
        SVL_SERVICE_ERROR   errCode;
    #ifdef IS6432
        // x86 with x64 GUI
        errCode = svlMVStub_LoadMemoryValidator6432();
    #else   //#ifdef IS6432
        // x86 with x86 GUI
        // x64 with x64 GUI
        errCode = svlMVStub_LoadMemoryValidator();
    #endif   //#ifdef IS6432
        if (errCode != SVL_OK)
        {
            DWORD   lastError;
    
            lastError = GetLastError();
            svlMVStub_writeToLogFileW(L"C++ Memory Validator load failed. \r\n");
            svlMVStub_writeToLogFileLastError(lastError);
            svlMVStub_writeToLogFile(errCode);
    
            svlMVStub_dumpPathToLogFile();
        }
        else
        {
            svlMVStub_writeToLogFileW(L"C++ Memory Validator load success. \r\n");
    
            errCode = svlMVStub_StartMemoryValidatorForIIS();
    	if (errCode != SVL_OK)
    	{
                DWORD   lastError;
    
                lastError = GetLastError();
                svlMVStub_writeToLogFileW(L"Starting C++ Memory Validator failed. \r\n");
                svlMVStub_writeToLogFileLastError(lastError);
                svlMVStub_writeToLogFile(errCode);
    	}
    
            svlMVStub_writeToLogFileW(L"Finished starting C++ Memory Validator\r\n");
        }
    

    You’ll need to edit the location of the logfile to match the name of your website directory.

  3. We add the following code to TerminateExtension().

        // unload Validator here
    
        svlMVStub_UnloadMemoryValidator();
    


Monitoring memory allocations

Start Memory Validator.

From the Launch menu choose the IIS sub menu and then Monitor ISAPI….

The monitor ISAPI dialog is displayed. Any settings from a previous launch are displayed. Edit the settings appropriately.

You need to identify the ISAPI dll that is being processed, the website directory, the IIS process (choose Any IIS if you don’t know or don’t care), the web browser (the default is Microsoft Edge as it seems more tolerant of data format errors than Chrome) and finally the URL you wish to use to test the ISAPI.

Click OK.

Memory Validator copies some DLLs to the directory containing the ISAPI, sets up various variables that will be used inside the IIS process, resets IIS, and launches the web browser to load the specified URL that will load the ISAPI.

When you’ve finished interacting with the ISAPI, you’ll need to stop IIS. Go to the Launch menu, choose the IIS sub menu then Stop IIS.

When IIS has stopped executing Memory Validator displays “Ready” on the status bar, the Memory tab will show any memory leaks and handle leaks that are present.

A short video of this process is shown below.

Conclusion

We’ve completely reworked our ISAPI support so that you have very little to do, just use the API as shown above and launch using the Monitor ISAPI command.

In comparison, here are some instructions for using Boundschecker with IIS. These are out of date and will no longer work. But look how much you have to do manually.

Monitoring a service with the NT Service API

By , February 11, 2020 5:21 pm

Debugging services is a pain. There is a lot that can go wrong and very little you can do to find out what went wrong. Perfect! Just what you need for an easy day at work. Services run in a restricted environment, these days you also need to be Administrator to do anything with them, and getting your favourite software tool which isn’t a debugger working with them is hard. I remember years ago seeing the list of things you needed to do to get NuMega’s BoundsChecker to work with services. It was a couple of web pages of instructions, each line containing a detailed step. You had to do all of the actions correctly in order to set things up to work with services.

These days Microsoft have changed the security landscape and it’s no longer possible to launch your data monitoring software tool from a service as that ability is correctly regarded as a security vulnerability. It’s also pretty much impossible to inject into a service from a GUI application. As a result the correct way to work with services is to add a few lines of glue code, in the form of calls to an API that setup communications with an already running user interface.

We’ve described our updated NT Service API in a previous article, so in this article I’m going to talk about the using the API to track errors in the service code calling the API and also describe how you use the user interface to work with services. This article will focus on C++ Memory Validator, but the techniques described here will also work for C++ Coverage Validator, C++ Performance Validator and C++ Thread Validator. If you’re using a .Net service, or a mixed mode service with a .Net entry point you don’t need to use the API, but the GUI parts of this article will still apply to you. If you using a native service or a mixed mode service with a native entry point all of this article applies to you.

Monitoring a Service

Before we get into the error codes and error handling in the GUI, let’s first take a tour of how things should work if everything goes to plan. This will provide some context for the errors I’m going to describe later. I’m going to assume you’ve built both the example service and the example service client, and that you’ve installed the service (serviceMV.exe -install in an Administrator mode command prompt). The service client passes a string to the service, which reverses it and passes it back to the client. The service also deliberately leaks some memory for testing purposes.

Here’s a video of the process.

From the Launch menu, choose Monitor a Service.


The Monitor a Service dialog is displayed.


Enter the full path to your service and click OK to start monitoring. The Validator will now setup some environment variables and some data in the registry that will be used by the service API. After a few seconds the Start your Service dialog appears.


Click OK, then start your service (you’ll need to do this from an Administrator command prompt).

serviceMV -start

The Validator attaches to the service and after a few moments various status information in the Validator title bar and the Validator status bar updated.

It is possible that you may get a debug information informational dialog displayed. You can dismiss this (it can be viewed from the Validator Tools menu). To change how symbols are found you’ll need to look at the Symbol Server and File Locations parts of the Validator settings dialog.


Next a dialog is displayed informing you that Administrator Privileges may be required.


For some services you may find that the Validator gets better data, or sends data to the GUI faster if the Validator is run in Administrator mode. If that is the case you’ll need to restart the Validator with Administrator privileges (and also stop and restart the service, etc).

For this particular example service, we don’t need Administrator privileges so we’ll continue without them.

Now we can interact with the service from the service client by sending a string to the service. The service reverses it and sends it back.

serviceClient "Hello World"


Once we’re done working with the service we can stop it (you’ll need to do this from an Administrator command prompt).

serviceMV -stop

The Validator disconnects from the service and displays all the data it has collected from the service.


That’s how it looks when everything goes according to plan.

What happens when things go wrong? That’s what the next section is about.

Tracking errors in the service

The various API functions return a SVL_SERVICE_ERROR error code. We’ve extended this code so that you can detect when the user has forgotten to do something prior to starting the service, or you can detect if various other error conditions have occurred. Some of these error codes are internal error codes and should never be seen by a customer, but we’re documenting them here for completeness.


  • SVL_FAIL_PATHS_DO_NOT_MATCH. Internal error. Looks like you forgot to Monitor a Service from the Launch Menu before starting the service.

  • SVL_FAIL_INCORRECT_PRODUCT_PREFIX. Internal error. Looks like you forgot to Monitor a Service from the Launch Menu before starting the service.

  • SVL_FAIL_X86_VALIDATOR_FOUND_EXPECTED_X64_VALIDATOR. Looks like you’re monitoring a 64 bit service with a 32 bit Validator. You need to use a 64 bit Validator.

  • SVL_FAIL_X64_VALIDATOR_FOUND_EXPECTED_X86_VALIDATOR. Looks like you’re monitoring a 32 bit service with a 64 bit Validator with the svl*VStubService.lib library. You need to use a 64 bit Validator with the svl*VStubService6432.lib.

  • SVL_FAIL_DID_YOU_MONITOR_A_SERVICE_FROM_VALIDATOR. Looks like you forgot to Monitor a Service from the Launch Menu before starting the service.

  • SVL_FAIL_ENV_VAR_NOT_FOUND. Internal error. Looks like you forgot to Monitor a Service from the Launch Menu before starting the service.

  • SVL_FAIL_VALIDATOR_ENV_VAR_NOT_FOUND. Internal error. Looks like you forgot to Monitor a Service from the Launch Menu before starting the service.

  • SVL_FAIL_VALIDATOR_ID_NOT_SPECIFIED. Internal error. Looks like you forgot to Monitor a Service from the Launch Menu before starting the service.

  • SVL_FAIL_VALIDATOR_ID_NOT_A_PROCESS. Internal error. Looks like you forgot to Monitor a Service from the Launch Menu before starting the service.

  • SVL_FAIL_VALIDATOR_NOT_FOUND. Internal error. Looks like you forgot to Monitor a Service from the Launch Menu before starting the service.

To aid in debugging we strongly recommend that you log all error codes (successful or failure) from Software Verify API calls. This will allow you to track down errors rapidly rather a series of trial error coding mistakes or a back and forth with support but with no information to help support. We added all of the above error codes after 3 customers all reported similar, but different problems with using the service API. All of their problems would be have been solved if these error codes had been available.

Error codes can be logged with this call.

void writeToLogFile(const wchar_t     *fileName,
                    SVL_SERVICE_ERROR errCode);

Helpful messages can be logged with this call.

void writeToLogFile(const wchar_t *fileName,
                    const wchar_t *text);

Error codes can be turned into human readable messages with this call.

const wchar_t *getTextForErrorCode(SVL_SERVICE_ERROR	errorCode);

And if you need to log Windows error codes, use this call.

void writeToLogFileLastError(const wchar_t *fileName,
                             DWORD         errCode);

See the help documentation for all the available API calls.

Tracking errors in the GUI

There are a couple of mistakes that can be made in the user interface. These are related to monitoring the wrong type of service, and the location of the service. Where it is possible to identify this error in the GUI, we will do so. Where it is not, the error codes described above will help you understand the mistake that has been made.

64 bit Service, 32 bit GUI

If you try to monitor a 64 bit service with a 32 bit GUI that will fail. We can detect this and prevent this. When this error happens you will be shown an error dialog similar to this.


Note that monitoring a 32 bit service with a 64 bit GUI is OK, but you need to use the svl*VStubService6432.lib not the svl*VStubService.lib. We can’t detect this from the GUI, which is why the SVL_FAIL_X64_VALIDATOR_FOUND_EXPECTED_X86_VALIDATOR error code exists – you will get this if you are linked to svl*VStubService.lib when you should be linked to svl*VStubService6432.lib.

Service on a network share

Windows won’t let you start a service on a network share. And yet I’ve lost count of the number of times I’ve tried to do this. This is typically because I have the solution working on machine X (where I wrote it) and wish to test on machine Y, and I just use a network share to map it across. This works for applications and fails for services. This can be a real time waster and Windows isn’t exactly helpful about this, and of course it’s in a service’s startup code, so fun debugging that.

To make this failure easier to detect we check the path of the service you specify in the Monitor a Service dialog and determine if the service is on a network share. If it is we tell you we can’t work with it. This then alerts you to the fact you’ll need to copy that service locally to run tests on it. Probably and hour or two of your time saved, right there.


Conclusion

Working with services can be fraught with problems, but if you log your error codes you can easily and quickly identify any errors made configuring your use of the NT Service API that we were unable to catch with the Validator user interface.

There’s more than one way to leak a GDI object

By , September 4, 2018 10:05 am

Working with GDI in Windows, whether you’re using Win32 calls or MFC, you’re concerned with pens, brushs, fonts, bitmaps and regions for drawing. You may also be concerned with Palettes, although with our full colour displays these days working with palettes is something of a rarity.

The typical way to work with a GDI object, for example, a pen, is shown below. Create the pen, select it into the DC, do the drawing, select the original object back into the DC, delete the pen.

void CtestGDISelectObjectDlg::drawSomethingWithARedPen(HDC hDC)
{
	HPEN		hPen;
	HPEN		hPenOld;
	HGDIOBJ		retVal;

	hPen = ::CreatePen(PS_SOLID, 0, RGB(255, 0, 0));

	hPenOld = (HPEN)SelectObject(hDC, hPen);

	doDrawing1(hDC);

	retVal = SelectObject(hDC, hPenOld);

	DeleteObject(hPen);
}

Leaking GDI objects, type 1

The most common way people leak GDI objects is because they simply forget to delete them. Here’s the previous example, modified to leak the pen. (Don’t do this!).

void CtestGDISelectObjectDlg::drawSomethingWithARedPen(HDC hDC)
{
	HPEN		hPen;
	HPEN		hPenOld;
	HGDIOBJ		retVal;

	hPen = ::CreatePen(PS_SOLID, 0, RGB(255, 0, 0));

	hPenOld = (HPEN)SelectObject(hDC, hPen);

	doDrawing1(hDC);

	retVal = SelectObject(hDC, hPenOld);
}

The author of the code forgot to delete the pen using DeleteObject(hPen); This leak looks like this in Memory Validator.


Leaking GDI objects, type 2

There is another way to leak GDI objects, even when you think you’ve deleted all the objects you created. Take a look at this code. Does it leak?

	HPEN		hPen1;
	HPEN		hPen2;
	HPEN		hPenOld;
	HGDIOBJ		retVal;

	hPen1 = ::CreatePen(PS_SOLID, 0, RGB(255, 0, 0));
	hPen2 = ::CreatePen(PS_DASHDOTDOT, 0, RGB(255, 0, 0));

	hPenOld = (HPEN)SelectObject(hDC, hPen1);

	doDrawing1(hDC);

	retVal = SelectObject(hDC, hPen2);
	
	doDrawing2(hDC);

	DeleteObject(hPen1);
	DeleteObject(hPen2);

A quick examination shows that two pens are created, some work is done with each pen, then the pens are deleted.

No leaks, right?

Wrong! hPen2 was selected into the DC for use with doDrawing2() but was not deselected from the DC prior to DeleteObject(hPen2);. This means that the call to DeleteObject() will fail as the pen is still in use. hPen2 has been leaked.

Memory Validator can detect this (as of V7.38, released today). Here’s what that looks like:


Expanding the source and you can easily see the failed DeleteObject() call and the SelectObject() call that mean the object was still in use.


Here’s what non-leaking the code should look like:

	HPEN		hPen1;
	HPEN		hPen2;
	HPEN		hPenOld;
	HGDIOBJ		retVal;

	hPen1 = ::CreatePen(PS_SOLID, 0, RGB(255, 0, 0));
	hPen2 = ::CreatePen(PS_DASHDOTDOT, 0, RGB(255, 0, 0));

	hPenOld = (HPEN)SelectObject(hDC, hPen1);

	doDrawing1(hDC);

	retVal = SelectObject(hDC, hPen2);
	
	doDrawing2(hDC);

	SelectObject(hDC, hPenOld);

	DeleteObject(hPen1);
	DeleteObject(hPen2);

To ensure your object is not still selected into the DC you can select the value that was returned to you by the first call (hPenOld in the code above), or you can select a stock object into the DC instead. For example:

	SelectObject(hDC, GetStockObject(BLACK_PEN));

Conclusion

When working with GDI objects you need to keep track of two things:


  1. Creation and Deletion of GDI objects. For every pen, brush, font, bitmap, palette that you create you must delete those objects when you are finished with them. Delete objects using DeleteObject().

  2. You need to ensure that none of the objects that you create are selected into a DC when you try to delete them.

Detecting memory leaks in Visual Test unit tests

By , June 6, 2018 5:40 pm

Introduction

We recently had a request asking if C++ Memory Validator could detect memory leaks in unit tests managed by Microsoft’s Visual Test and Visual Test Explorer. They told us what they’d tried to do and that it had failed. Our response was to find out what was failing, fix it and then describe what you need to do to use our tools with Visual Test. That’s what this article is about.

There were a few bugs specific to working with Visual Test, plus a very novel environment variable data corruption that can only happen in very unusual circumstances (you almost certainly would not hit these in normal usage of our tools). We fixed these tools, then experimented with several ways of working with Visual Test. We’re only going to talk about C++ Memory Validator here, but this article also applies to our Coverage, Performance and Thread tools.

You can run Visual Test from the command line, and also via Visual Test Explorer, which is a component of Visual Studio.

Monitoring Visual Test from the command line

The full details of how to work with Visual Test from the command line are documented in this article from Microsoft.

Using this information we know that we need to launch vstest.console.exe and pass the unit test DLL to that as an argument. For example:

vstest.console.exe unitTest.dll 

Using this information we can launch vstest.console.exe from C++ Memory Validator and pass it the appropriate DLL. You can set the startup directory to whatever you like. We’ve chose to set it to the same directory as the unit test DLL.

Unit Test Code

namespace UnitTest1
{		
	TEST_CLASS(UnitTest1)
	{
	public:
		
		TEST_METHOD(TestMethod1)
		{
			// pretend this is a real unit test, 
			// exercising a real target class/function
			// and that it leaks some memory

			char *ptr;

			ptr = new char[123];
			strcpy_s(ptr, 123, "Excellent Adventure");

			ptr = (char *)malloc(456);
			strcpy_s(ptr, 456, "Bogus Journey");

			ptr = (char *)malloc(789);
			strcpy_s(ptr, 789, "Face the Music");

			Assert::AreNotEqual("Bill", "Ted");
		}

	};
}

You’ll notice that you don’t need to specify the TEST_MODULE_INITIALIZE(moduleInit) function, or the TEST_MODULE_CLEANUP(moduleCleanup) function. You only need to specify the unit tests you want tested. C++ Memory Validator does the rest.

Monitoring Visual Test Explorer

When working with Visual Test Explorer, Visual Test is launched from Dev Studio, and then loads the unit tests for testing from a DLL. Visual Test then stays running in the background and does not shutdown. This means the “program has ended” signal that C++ Memory Validator needs doesn’t get sent. It also means that subsequent tests run with Visual Test Explorer won’t cause Visual Test to startup, meaning that the “program has started” signal that C++ Memory Validator needs doesn’t get sent.

To get around these problems we need to use the NT Service API (see help file, or online documentation for details) to contact C++ Memory Validator at the start of the unit tests, and also at the end of the unit tests. We do that using the TEST_MODULE_INITIALIZE(moduleInit) function, or the TEST_MODULE_CLEANUP(moduleCleanup) functions.

Functions

We use svlMVStub_LoadMemoryValidator() to load C++ Memory Validator into the unit test, then svlMVStub_StartMemoryValidator() to start it monitoring the unit test and communicating with the user interface. At the end of the tests we use svlMVStub_UnloadMemoryValidatorAndTerminateProcess(1000) to shutdown C++ Memory Validator and set a thread running that will after a delay of 1000ms terminate the vstest.executionengine.x86.exe (Visual Test) process. You may need to experiment with this delay on your machine.

Header Files

We need to include two header files from C++ Memory Validator. svlMVStubService.h and svlServiceError.h. You’ll find these in the svlMVStubService folder in the C++ Memory Validator install directory.

Libraries

We also need to link to svlMVStubService.lib (32 bit builds) or svlMVStubService_x64.lib (64 bit builds). You’ll find versions of these libraries in the svlMVStubService folder (one library per version of Visual Studio).

Unit Test Code

#include "svlMVStubService.h"
#include "svlServiceError.h"

namespace UnitTest1
{		
	TEST_MODULE_INITIALIZE(moduleInit)
	{
		SVL_SERVICE_ERROR	sse;

		sse = svlMVStub_LoadMemoryValidator();

		sse = svlMVStub_StartMemoryValidator();
	}

	TEST_MODULE_CLEANUP(moduleCleanup)
	{
		SVL_SERVICE_ERROR	sse;
		
		sse = svlMVStub_UnloadMemoryValidatorAndTerminateProcess(1000);
	}

	TEST_CLASS(UnitTest1)
	{
	public:
		
		TEST_METHOD(TestMethod1)
		{
			// pretend this is a real unit test, 
			// exercising a real target class/function
			// and that it leaks some memory

			char *ptr;

			ptr = new char[123];
			strcpy_s(ptr, 123, "Excellent Adventure");

			ptr = (char *)malloc(456);
			strcpy_s(ptr, 456, "Bogus Journey");

			ptr = (char *)malloc(789);
			strcpy_s(ptr, 789, "Face the Music");

			Assert::AreNotEqual("Bill", "Ted");
		}

	};
}

Having reworked the unit tests to support the NT Service API, we now need to launch devenv.exe from C++ Memory Validator, but with instructions to ignore devenv.exe and monitor Visual Test (vstest.executionengine.x86.exe). We do that from the launch dialog/wizard.

First choose devenv.exe to monitor using the Browse… button next to the Application to launch field. In this example we chose C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\Common7\IDE\devenv.exe


Next we need to setup the applications to monitor. Click the Edit… button next to the Application to monitor field. The applications to monitor dialog is displayed.


Choose devenv.exe using the Browse… button next to the Application to launch field.

Click the Add… button and add the vstest.executionenginex86.exe that corresponds with the devenv.exe you selected. In this example we chose C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstestexecutionengine.x86.exe. Be sure to choose the correct one – there are several with similar names to choose from.


Click OK to accept the application to monitor.


Click OK to accept the new definitions.

Now on the launch dialog, change the Application to monitor combo to select the vstest.execution.x86.exe entry.

Enter the name of your unit test dll(s) in the Arguments field, separated by spaces. If you specify the directory containing the DLLs as the startup directory, you can specify the DLL names without paths.

Running Your Tests

Now that we’ve setup C++ Memory Validator to monitor Visual Test when started from devenv, let’s get to work.

To start devenv, click the Go! button. Devenv will start. Load the solution that contains your unit tests. When you choose Run Selected Tests or Debug Selected Tests from Visual Test Explorer, the tests will run and will be monitored by C++ Memory Validator at the same time.

Fix for FALSE positive memory leak report VS 2015 Update 1

By , December 18, 2015 12:39 pm

We’ve just released a V6.54 of C++ Memory Validator.

This contains a bug fix for FALSE positive memory leak reports when working with Release mode builds of programs built using Visual Studio 2015 Update 1.

Existing customers with valid software maintenance have been emailed about this release. The download is available from the customer login and also from the Software Updates menu in the software.

Marmalade game SDK support

By , December 16, 2015 12:02 pm

We’ve recently added support for the Marmalade game SDK to C++ Memory Validator.

This article will show you how to configure a Marmalade project for use with C++ Memory Validator, how to setup C++ Memory Validator for use with Marmalade and how to launch a Marmalade game from C++ Memory Validator.

Configuring your project settings

To work with C++ Memory Validator you need to build the x86 Debug configuration and/or the x86 Release configuration of your Marmalade project using Visual Studio.

These configurations need to be built so that they create debug information and so that a PDB file containing debug information is created. The example projects that ship with Marmalade do not do this – you will need to edit them to make the linker stage create debug information.

Editing the compile stage debug information

marmalade-vs-compile


Editing the link stage debug information

marmalade-vs-linker


You must ensure that both compile and link stages have the correct settings set. If only compile or only link is set you will not get debugging symbols.

Debugging symbols are important for two reasons:

  • Without symbols C++ Memory Validator cannot find the Marmalade memory allocators and will not be able to track the Marmalade memory allocations your game makes.
  • Without symbols C++ Memory Validator will not be able to turn callstack addresses into class names, function names, filenames and line numbers.

Configuring C++ Memory Validator

In order to work correctly with Marmalade we need to make sure we’re only going to track the memory allocation your game makes with Marmalade and not any of the work that the Marmalade game engine is doing itself. We need to make a few simple changes to C++ Memory Validator.

  • Open the settings dialog. Click Reset. This will reset all C++ Memory Validator settings to the default.
  • Go to the Collect tab, disable the check boxes in the top two groups of check boxes, then enable the single Marmalade check box. The settings should look like shown below.
  • Click OK.

settings-collect-marmalade


Launching the game

To launch a Marmalade game with C++ Memory Validator we launch the Marmalade simulator and specify the game to run using the Marmalade -via command line argument.

If Marmalade is installed in c:\marmalade then the path to the simulator is

c:/marmalade/8.0/s3e\win32\s3e_simulator_release.exe

If an example game (shipped with Marmalade) is found at this location

c:\Marmalade\8.0\examples\GameTutorial\Lua\Stage9\build_temp\build_stage9_vc14\Stage9_vc14_release.via

then the command line is

-via:"c:\Marmalade\8.0\examples\GameTutorial\Lua\Stage9\build_temp\build_stage9_vc14\Stage9_vc14_release.via"

and the startup directory is

c:\Marmalade\8.0\examples\GameTutorial\Lua\Stage9\build_temp\build_stage9_vc14\

We leave the Application to monitor unchanged. It should have the same value as Application to launch.

This is how the launch dialog looks when you are launching this game.

launch-marmalade


Click Go! to launch the Marmalade game. C++ Memory Validator will monitor all memory allocations, reallocations and deallocations made via the s3eMalloc, s3eRealloc, s3eFree functions. Run your game as normal, then close your game. Wait for C++ Memory Validator to show you any leaks on the Memory tab. Additional statistics can be views on the Objects, Sizes, Timeline and Hotspots tabs.

How to speed up Memory Validator by changing DbgHelp version

By , September 6, 2015 9:26 am

Recently we’ve had a few customers contact to tell us they have experienced a dramatic reduction in speed when using C++ Memory Validator.

We found this puzzling as we hadn’t really noticed this. We investigate and found that some parts of our code were hitting the disk a bit too much. To address this we implemented a buffered file read/write system so that we hit the disk once rather than many times. For our test case (which was a substantial program being monitored) this worked wonders. Performance improved enormously. Smiles all round.

But our customers still reported problems. This was puzzling. We started logging everything in one particular code path (which we knew was the problem). Nothing obvious. Next step, start timing all the logging. But just before we got to that we did a simple test. We iterated through each version of DbgHelp.dll that C++ Memory Validator can supply – if you remember we let you specify which Visual Studio version you used and we supply a version of DbgHelp.dll that ships and works for that (not all DbgHelp.dll are equal!).

Imagine our surprise when we found that DbgHelp.dll (6.11.1.404) shipped prior to Visual Studio 2013 are blazingly fast and the DbgHelp.dll we supply for use with Visual Studio 2013/2015 (6.3.9431.0) are slow. If you’re paying attention you’ll also notice the DbgHelp.dll version number has decreased rather than increased – ask Microsoft, we have no idea why they decreased the version number with more recent releases.

For now, until we can get a new release out to address this anomaly we recommend that you ignore choosing the Visual Studio you are using and deliberately choose Visual Studio 2012. This will select DbgHelp 6.11.1.404 and you should find the usual blazing speeds you are used to are restored.

To change the version of DbgHelp used, open the settings dialog, go to Symbol Lookup then change the version in the combo box. Click OK.

Any problems, as usual, please contact support.

64 bit C++ software tool Beta Tests are complete.

By , January 9, 2014 1:33 pm

We recently closed the beta tests for the 64 bit versions of C++ Coverage Validator, C++ Memory Validator, C++ Performance Validator and C++ Thread Validator.

We launched the software on 2nd January 2014. A soft launch, no fanfare, no publicity. We just wanted to make the software available and then contact all the beta testers so that we could honour our commitments made at the start of the beta test.

Those commitments were to provide a free single user licence to any beta tester that provided feedback, usage reports, bugs reports, etc about the software. This doesn’t include anyone that couldn’t install the software because they used the wrong licence key!

We’ve written a special app here that we can use to identify all email from beta test participants and allow us to evaluate that email for beta test feedback criteria. It’s saved us a ton of time and drudge work even though writing this extension to the licence manager software took a few days. It was interesting using the tool and seeing who provided feedback and how much.

We’ve just sent out the licence keys and download instructions to all those beta testers that were kind enough to take the time to provide feedback, bug reports etc. to us. A few people went the extra mile. These people bombarded us with email containing huge bugs, trivial items and everything in between. Two of them, we were on the verge of flying out to their offices when we found some useful software that allowed to us to remotely debug their software. Special mentions go to:

Bengt Gunne (Mimer.com)
Ciro Ettorre (Mechworks.com)
Kevin Ernst (Bentley.com)

We’re very grateful for everyone taking part in the beta test. Thank you very much.

Why didn’t I get a free licence?

If you didn’t receive a free licence and you think you did provide feedback, please contact us. It’s always possible that a few people slipped through our process of identifying people.

Dang! I knew I should’ve provided feedback

If you didn’t provide us with any feedback, check your inbox. You’ll find a 50% off coupon for the tool that you tested.

Panorama Theme by Themocracy