Rss Feed
Tweeter button
Facebook button
Technorati button
Reddit button
Myspace button
Linkedin button
Webonews button
Delicious button
Digg button
Flickr button
Stumbleupon button
Newsvine button

Category: Test Setup

Detecting memory leaks in Visual Test unit tests

By , June 6, 2018 5:40 pm

Introduction

We recently had a request asking if C++ Memory Validator could detect memory leaks in unit tests managed by Microsoft’s Visual Test and Visual Test Explorer. They told us what they’d tried to do and that it had failed. Our response was to find out what was failing, fix it and then describe what you need to do to use our tools with Visual Test. That’s what this article is about.

There were a few bugs specific to working with Visual Test, plus a very novel environment variable data corruption that can only happen in very unusual circumstances (you almost certainly would not hit these in normal usage of our tools). We fixed these tools, then experimented with several ways of working with Visual Test. We’re only going to talk about C++ Memory Validator here, but this article also applies to our Coverage, Performance and Thread tools.

You can run Visual Test from the command line, and also via Visual Test Explorer, which is a component of Visual Studio.

Monitoring Visual Test from the command line

The full details of how to work with Visual Test from the command line are documented in this article from Microsoft.

Using this information we know that we need to launch vstest.console.exe and pass the unit test DLL to that as an argument. For example:

vstest.console.exe unitTest.dll 

Using this information we can launch vstest.console.exe from C++ Memory Validator and pass it the appropriate DLL. You can set the startup directory to whatever you like. We’ve chose to set it to the same directory as the unit test DLL.

Unit Test Code

namespace UnitTest1
{		
	TEST_CLASS(UnitTest1)
	{
	public:
		
		TEST_METHOD(TestMethod1)
		{
			// pretend this is a real unit test, 
			// exercising a real target class/function
			// and that it leaks some memory

			char *ptr;

			ptr = new char[123];
			strcpy_s(ptr, 123, "Excellent Adventure");

			ptr = (char *)malloc(456);
			strcpy_s(ptr, 456, "Bogus Journey");

			ptr = (char *)malloc(789);
			strcpy_s(ptr, 789, "Face the Music");

			Assert::AreNotEqual("Bill", "Ted");
		}

	};
}

You’ll notice that you don’t need to specify the TEST_MODULE_INITIALIZE(moduleInit) function, or the TEST_MODULE_CLEANUP(moduleCleanup) function. You only need to specify the unit tests you want tested. C++ Memory Validator does the rest.

Monitoring Visual Test Explorer

When working with Visual Test Explorer, Visual Test is launched from Dev Studio, and then loads the unit tests for testing from a DLL. Visual Test then stays running in the background and does not shutdown. This means the “program has ended” signal that C++ Memory Validator needs doesn’t get sent. It also means that subsequent tests run with Visual Test Explorer won’t cause Visual Test to startup, meaning that the “program has started” signal that C++ Memory Validator needs doesn’t get sent.

To get around these problems we need to use the NT Service API (see help file, or online documentation for details) to contact C++ Memory Validator at the start of the unit tests, and also at the end of the unit tests. We do that using the TEST_MODULE_INITIALIZE(moduleInit) function, or the TEST_MODULE_CLEANUP(moduleCleanup) functions.

Functions

We use svlMVStub_LoadMemoryValidator() to load C++ Memory Validator into the unit test, then svlMVStub_StartMemoryValidator() to start it monitoring the unit test and communicating with the user interface. At the end of the tests we use svlMVStub_UnloadMemoryValidatorAndTerminateProcess(1000) to shutdown C++ Memory Validator and set a thread running that will after a delay of 1000ms terminate the vstest.executionengine.x86.exe (Visual Test) process. You may need to experiment with this delay on your machine.

Header Files

We need to include two header files from C++ Memory Validator. svlMVStubService.h and svlServiceError.h. You’ll find these in the svlMVStubService folder in the C++ Memory Validator install directory.

Libraries

We also need to link to svlMVStubService.lib (32 bit builds) or svlMVStubService_x64.lib (64 bit builds). You’ll find versions of these libraries in the svlMVStubService folder (one library per version of Visual Studio).

Unit Test Code

#include "svlMVStubService.h"
#include "svlServiceError.h"

namespace UnitTest1
{		
	TEST_MODULE_INITIALIZE(moduleInit)
	{
		SVL_SERVICE_ERROR	sse;

		sse = svlMVStub_LoadMemoryValidator();

		sse = svlMVStub_StartMemoryValidator();
	}

	TEST_MODULE_CLEANUP(moduleCleanup)
	{
		SVL_SERVICE_ERROR	sse;
		
		sse = svlMVStub_UnloadMemoryValidatorAndTerminateProcess(1000);
	}

	TEST_CLASS(UnitTest1)
	{
	public:
		
		TEST_METHOD(TestMethod1)
		{
			// pretend this is a real unit test, 
			// exercising a real target class/function
			// and that it leaks some memory

			char *ptr;

			ptr = new char[123];
			strcpy_s(ptr, 123, "Excellent Adventure");

			ptr = (char *)malloc(456);
			strcpy_s(ptr, 456, "Bogus Journey");

			ptr = (char *)malloc(789);
			strcpy_s(ptr, 789, "Face the Music");

			Assert::AreNotEqual("Bill", "Ted");
		}

	};
}

Having reworked the unit tests to support the NT Service API, we now need to launch devenv.exe from C++ Memory Validator, but with instructions to ignore devenv.exe and monitor Visual Test (vstest.executionengine.x86.exe). We do that from the launch dialog/wizard.

First choose devenv.exe to monitor using the Browse… button next to the Application to launch field. In this example we chose C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\Common7\IDE\devenv.exe


Next we need to setup the applications to monitor. Click the Edit… button next to the Application to monitor field. The applications to monitor dialog is displayed.


Choose devenv.exe using the Browse… button next to the Application to launch field.

Click the Add… button and add the vstest.executionenginex86.exe that corresponds with the devenv.exe you selected. In this example we chose C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstestexecutionengine.x86.exe. Be sure to choose the correct one – there are several with similar names to choose from.


Click OK to accept the application to monitor.


Click OK to accept the new definitions.

Now on the launch dialog, change the Application to monitor combo to select the vstest.execution.x86.exe entry.

Enter the name of your unit test dll(s) in the Arguments field, separated by spaces. If you specify the directory containing the DLLs as the startup directory, you can specify the DLL names without paths.

Running Your Tests

Now that we’ve setup C++ Memory Validator to monitor Visual Test when started from devenv, let’s get to work.

To start devenv, click the Go! button. Devenv will start. Load the solution that contains your unit tests. When you choose Run Selected Tests or Debug Selected Tests from Visual Test Explorer, the tests will run and will be monitored by C++ Memory Validator at the same time.

Share

Problems building Firefox

By , July 20, 2010 8:35 am

I’ve been building Firefox on Windows recently. Without problems. But then I wanted a build with symbols and that is when the problems started. It is meant to be simple, and it is, so long as you don’t do a

make -f client.mk clean

If you do that, then a subsequent build will fail when it comes to updater.exe.manifest.

Setup

Assuming you have downloaded and installed Mozilla build and have also installed a suitable version of Visual Studio and the latest Microsoft Web SDK, you can start the build environment by running c:\mozilla-build\start_msvc9.bat (change the number for the version of Visual Studio you are using).

Read these instructions first.

Download the appropriate source tree with a mercurial command like this:

hg clone http://hg.mozilla.org/releases/mozilla-1.9.2/ src192

Basic Build

I’m building on drive M: and have downloaded the source into folder src192.

cd M:/src192

Configure the build with:

echo '. $topsrcdir/browser/config/mozconfig' > mozconfig

Build the build with

make -f client.mk build

This gives you a firefox release build. Strangely the build is created without debug symbols. I don’t understand this as debug symbols are really useful even with release builds. The symbols don’t have to ship with the executable code, so there is no good reason for not creating them.

Given that you’ve downloaded the source to build yourself the image its fair to assume that you are probably curious about the internal works or that you may be interested in modifying the source and/or writing an extension for it. Either way, you are going to want the symbols so that you can see what is happening in a debugger. Thus I don’t understand the lack of symbols by default.

Creating symbols

You can create symbols in several ways, by adding the following lines to your mozconfig.

echo 'ac_add_options --enable-debug' >> mozconfig
echo 'export MOZ_DEBUG_SYMBOLS=1' >> mozconfig
echo 'ac_add_options --enable-debugger-info-modules=yes' >> mozconfig

Getting a working build

At this point you’ve already done a build and have created executables. Thus it seems the appropriate thing to do is a make -f client.mk clean followed by a make -f client.mk build. This won’t work. The problem is that the clean deletes the manifest files that are present in the original mercurial source download.

In the end I wiped the whole source tree, downloaded from the source control again, modified mozconfig with the debug symbol options and did a build. That works.

This took me several attempts to understand – I thought that my various attempts at configuring the debug symbol options were breaking something, so I would change the options, clean then build. Each time taking several hours. I had this working in a virtual machine while I worked on other tasks. I probably spent about a day, including the build times before I decided to try again by starting from scratch.

Solution

I don’t know if this is true for non-Windows firefox builds, but if you want your builds to succeed don’t do a make -f client.mk clean prior to make -f client.mk build as it will prevent you from successfully building firefox.

I hope this information helps prevent you from wasting your time with this particular build problem.

Share

Creating a JNI example for testing

By , May 8, 2010 11:28 am

The Java language can be extended using C and C++ to write extensions that are called using the Java Native Interface (JNI).
The JNI extensions is created by creating a Java class definition that defines the class methods as native and loads a native DLL to do that.

class HelloWorld
{
    public native void displayMessage();

    static
    {
        System.loadLibrary("HelloWorldImp");  // load HelloWorldImp.dll to implement the native interfaces
    }
}

A full description of this process can be found on Matthew Mead’s JNI page.

Example JNI code for test

We’ve created an example based on this which also allocates some memory, deallocates some memory and causes a memory leak. You can
download the example code. To build the example follow these steps:

  1. Open a command prompt and change the current directory to the testJavaJNI directory in the example.
  2. Edit buildJava.bat, createJNIHeaders.bat, runJava.bat to point to your JAVA SDK directory.
  3. Run buildJava.bat
  4. Run createJNIHeaders.bat
  5. Build the HelloWorldImp.dll. The Visual Studio 6 Project (*.dsp) and Visual Studio Solution (*.sln) files are in the HelloWorld directory.

After the DLL has been built the DLL and its PDB file will be copied to the same location as the Java class files so that the DLL can be found by the Java Virtual Machine when it attempts to load the native implementation of HelloWorld.

To run the example, run runJava.bat. The example prints “Hello World!” twice. The first time does not leak memory, the second time does leak memory.

Command prompt building and testing JNI interface

Share

Panorama Theme by Themocracy