Rss Feed
Tweeter button
Facebook button
Technorati button
Reddit button
Myspace button
Linkedin button
Webonews button
Delicious button
Digg button
Flickr button
Stumbleupon button
Newsvine button

Posts tagged: NT Service API

Detecting memory leaks in Visual Test unit tests

By , June 6, 2018 5:40 pm

Introduction

We recently had a request asking if C++ Memory Validator could detect memory leaks in unit tests managed by Microsoft’s Visual Test and Visual Test Explorer. They told us what they’d tried to do and that it had failed. Our response was to find out what was failing, fix it and then describe what you need to do to use our tools with Visual Test. That’s what this article is about.

There were a few bugs specific to working with Visual Test, plus a very novel environment variable data corruption that can only happen in very unusual circumstances (you almost certainly would not hit these in normal usage of our tools). We fixed these tools, then experimented with several ways of working with Visual Test. We’re only going to talk about C++ Memory Validator here, but this article also applies to our Coverage, Performance and Thread tools.

You can run Visual Test from the command line, and also via Visual Test Explorer, which is a component of Visual Studio.

Monitoring Visual Test from the command line

The full details of how to work with Visual Test from the command line are documented in this article from Microsoft.

Using this information we know that we need to launch vstest.console.exe and pass the unit test DLL to that as an argument. For example:

vstest.console.exe unitTest.dll 

Using this information we can launch vstest.console.exe from C++ Memory Validator and pass it the appropriate DLL. You can set the startup directory to whatever you like. We’ve chose to set it to the same directory as the unit test DLL.

Unit Test Code

namespace UnitTest1
{		
	TEST_CLASS(UnitTest1)
	{
	public:
		
		TEST_METHOD(TestMethod1)
		{
			// pretend this is a real unit test, 
			// exercising a real target class/function
			// and that it leaks some memory

			char *ptr;

			ptr = new char[123];
			strcpy_s(ptr, 123, "Excellent Adventure");

			ptr = (char *)malloc(456);
			strcpy_s(ptr, 456, "Bogus Journey");

			ptr = (char *)malloc(789);
			strcpy_s(ptr, 789, "Face the Music");

			Assert::AreNotEqual("Bill", "Ted");
		}

	};
}

You’ll notice that you don’t need to specify the TEST_MODULE_INITIALIZE(moduleInit) function, or the TEST_MODULE_CLEANUP(moduleCleanup) function. You only need to specify the unit tests you want tested. C++ Memory Validator does the rest.

Monitoring Visual Test Explorer

When working with Visual Test Explorer, Visual Test is launched from Dev Studio, and then loads the unit tests for testing from a DLL. Visual Test then stays running in the background and does not shutdown. This means the “program has ended” signal that C++ Memory Validator needs doesn’t get sent. It also means that subsequent tests run with Visual Test Explorer won’t cause Visual Test to startup, meaning that the “program has started” signal that C++ Memory Validator needs doesn’t get sent.

To get around these problems we need to use the NT Service API (see help file, or online documentation for details) to contact C++ Memory Validator at the start of the unit tests, and also at the end of the unit tests. We do that using the TEST_MODULE_INITIALIZE(moduleInit) function, or the TEST_MODULE_CLEANUP(moduleCleanup) functions.

Functions

We use svlMVStub_LoadMemoryValidator() to load C++ Memory Validator into the unit test, then svlMVStub_StartMemoryValidator() to start it monitoring the unit test and communicating with the user interface. At the end of the tests we use svlMVStub_UnloadMemoryValidatorAndTerminateProcess(1000) to shutdown C++ Memory Validator and set a thread running that will after a delay of 1000ms terminate the vstest.executionengine.x86.exe (Visual Test) process. You may need to experiment with this delay on your machine.

Header Files

We need to include two header files from C++ Memory Validator. svlMVStubService.h and svlServiceError.h. You’ll find these in the svlMVStubService folder in the C++ Memory Validator install directory.

Libraries

We also need to link to svlMVStubService.lib (32 bit builds) or svlMVStubService_x64.lib (64 bit builds). You’ll find versions of these libraries in the svlMVStubService folder (one library per version of Visual Studio).

Unit Test Code

#include "svlMVStubService.h"
#include "svlServiceError.h"

namespace UnitTest1
{		
	TEST_MODULE_INITIALIZE(moduleInit)
	{
		SVL_SERVICE_ERROR	sse;

		sse = svlMVStub_LoadMemoryValidator();

		sse = svlMVStub_StartMemoryValidator();
	}

	TEST_MODULE_CLEANUP(moduleCleanup)
	{
		SVL_SERVICE_ERROR	sse;
		
		sse = svlMVStub_UnloadMemoryValidatorAndTerminateProcess(1000);
	}

	TEST_CLASS(UnitTest1)
	{
	public:
		
		TEST_METHOD(TestMethod1)
		{
			// pretend this is a real unit test, 
			// exercising a real target class/function
			// and that it leaks some memory

			char *ptr;

			ptr = new char[123];
			strcpy_s(ptr, 123, "Excellent Adventure");

			ptr = (char *)malloc(456);
			strcpy_s(ptr, 456, "Bogus Journey");

			ptr = (char *)malloc(789);
			strcpy_s(ptr, 789, "Face the Music");

			Assert::AreNotEqual("Bill", "Ted");
		}

	};
}

Having reworked the unit tests to support the NT Service API, we now need to launch devenv.exe from C++ Memory Validator, but with instructions to ignore devenv.exe and monitor Visual Test (vstest.executionengine.x86.exe). We do that from the launch dialog/wizard.

First choose devenv.exe to monitor using the Browse… button next to the Application to launch field. In this example we chose C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\Common7\IDE\devenv.exe


Next we need to setup the applications to monitor. Click the Edit… button next to the Application to monitor field. The applications to monitor dialog is displayed.


Choose devenv.exe using the Browse… button next to the Application to launch field.

Click the Add… button and add the vstest.executionenginex86.exe that corresponds with the devenv.exe you selected. In this example we chose C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstestexecutionengine.x86.exe. Be sure to choose the correct one – there are several with similar names to choose from.


Click OK to accept the application to monitor.


Click OK to accept the new definitions.

Now on the launch dialog, change the Application to monitor combo to select the vstest.execution.x86.exe entry.

Enter the name of your unit test dll(s) in the Arguments field, separated by spaces. If you specify the directory containing the DLLs as the startup directory, you can specify the DLL names without paths.

Running Your Tests

Now that we’ve setup C++ Memory Validator to monitor Visual Test when started from devenv, let’s get to work.

To start devenv, click the Go! button. Devenv will start. Load the solution that contains your unit tests. When you choose Run Selected Tests or Debug Selected Tests from Visual Test Explorer, the tests will run and will be monitored by C++ Memory Validator at the same time.

Changes to the NT Service API

By , February 3, 2018 10:25 am

All of our Validator tools have an NT Service API.

We’ve just made some changes to the NT Service API. This article documents those changes and why we made them.

The changes to the NT Service API, break the existing API behaviour, and improve the existing API behaviour.

If you are using the NT Service API, you should read this article.

For the purposes of this article I’m going to describe the NT Service API for C++ Memory Validator. The changes apply to the NT Service APIs for each C++ Validator tool, the only thing that is different is the function name prefix of svlBVStub_, svlCVStub_, svlMVStub_, svlPVStub_, svlTVStub_. If you can’t directly use the NT Service API due to compiler/linker issues, please consult the documentation for your Validator to see an alternative way of getting the same result.

Changes to the API

  • We changed the name of an enumeration and extended the values in the enumeration.
  • We split some functions into two parts so that they could be reused with other functions to achieve a better result.
  • We added some debugging functions to help you debug any failures when using the API.

SVL_SERVICE_ERROR

The SVL_ERROR enumeration has been renamed to SVL_SERVICE_ERROR.

The definition for SVL_SERVICE_ERROR is now in it’s own include file svlServiceError.h rather than defined in the NT Service API header file (svlMVStubService.h for C++ Memory Validator).

typedef enum _svlServiceError
{
   SVL_OK,                              // Normal behaviour
   SVL_ALREADY_LOADED,                  // Stub DLL already loaded into service
   SVL_LOAD_FAILED,                     // Failed to load stub DLL into service
   SVL_FAILED_TO_ENABLE_STUB_SYMBOLS,   // Loaded DLL, but failed to enable stub symbols because couldn't find function
   SVL_NOT_LOADED,                      // Couldn't unload DLL because DLL not loaded
   SVL_FAIL_UNLOAD,                     // Couldn't unload DLL because couldn't find function
   SVL_FAIL_TO_CLEANUP_INTERNAL_HEAP,   // Couldn't get the internal stub heap and thus couldn't clean it up
   SVL_FAIL_MODULE_HANDLE               // Couldn't get the stub DLL handle so couldn't continue
   SVL_FAIL_SETSERVICECALLBACK,         // Couldn't call the set service callback
   SVL_FAIL_COULD_NOT_FIND_ENTRY_POINT, // Couldn't find the DLL entry point to start the validator
   SVL_FAIL_TO_START                    // Failed to start the Validator
} SVL_SERVICE_ERROR;

Split functionality

The previous implementation of svlMVStub_LoadMemoryValidator() and svlMVStub_LoadMemoryValidator6432() would load the Validator DLL, then start the Validator profiling the service.

The new implementation of svlMVStub_LoadMemoryValidator() and svlMVStub_LoadMemoryValidator6432() just loads the Validator DLL. It does not start the Validator profiling the service.

The reason for this change is that we wanted to allow the ability to set a service manager callback so that the service control manager could be informed about a long running instrumentation phase as the Validator is started. This is important because a delay of more than 10 seconds will cause the service control manager to kill the service.

To set the service callback call the svlMVStub_SetServiceCallback() function.

Then start the Validator with svlMVStub_StartMemoryValidator().

Loading, Starting, Unloading the Validator

To control the loading, starting and unloading of the Validator into the service, use these functions.

extern "C" SVL_SERVICE_ERROR svlMVStub_LoadMemoryValidator();

extern "C" SVL_SERVICE_ERROR svlMVStub_LoadMemoryValidator6432();

extern "C" SVL_SERVICE_ERROR svlMVStub_StartMemoryValidator();

extern "C" SVL_SERVICE_ERROR svlMVStub_UnloadMemoryValidator();

extern "C" SVL_SERVICE_ERROR svlMVStub_SetServiceCallback(serviceCallback_FUNC callback,
                                                          void*                userParam);

svlMVStub_LoadMemoryValidator

extern "C" SVL_SERVICE_ERROR svlMVStub_LoadMemoryValidator();

To load the Validator DLL into a 32 bit service when you are working with a 32 bit Validator, or to load the Validator DLL into a 64 bit service when you are working with a 64 bit Validator, use svlMVStub_LoadMemoryValidator().

svlMVStub_LoadMemoryValidator6432

extern "C" SVL_SERVICE_ERROR svlMVStub_LoadMemoryValidator6432();

To load the Validator DLL into a 32 bit service when you are working with a 64 bit Validator, use svlMVStub_LoadMemoryValidator6432().

svlMVStub_SetServiceCallback

extern "C" SVL_SERVICE_ERROR svlMVStub_SetServiceCallback(serviceCallback_FUNC callback,
                                                          void*                userParam);

Once you have successfully loaded the Validator DLL you can setup a service callback so that the service control manager can be kept updated during the process of starting the Validator.

When a service is starting, Windows requires the service to inform the Service Control Manager (SCM) that is starting at least every ten seconds. Failure to do so results in Windows concluding that the service has failed to start, and the service is terminated. Instrumenting your service may take more than 10 seconds, depending on the complexity and size of your service.

The solution is for the Validator to periodically call a user supplied callback from which you can regularly inform the SCM of the appropriate status.

Here is an example callback which ignores the userParam.

   void serviceCallback(void   *userParam)
   {
       static DWORD dwCheckPoint = 1;
   
       ssStatus.dwServiceType = SERVICE_WIN32_OWN_PROCESS;
       ssStatus.dwServiceSpecificExitCode = 0;

       ssStatus.dwControlsAccepted = 0;
      
       ssStatus.dwCurrentState = dwCurrentState;
       ssStatus.dwWin32ExitCode = dwWin32ExitCode;
       ssStatus.dwWaitHint = dwWaitHint;
      
       ssStatus.dwCheckPoint = dwCheckPoint++;
      
       // Report the status of the service to the service control manager.

       return SetServiceStatus(sshStatusHandle, &ssStatus);
   }

svlMVStub_StartMemoryValidator

extern "C" SVL_SERVICE_ERROR svlMVStub_StartMemoryValidator();

Once you have successfully loaded the Memory Validator DLL you can start the Validator inspecting the service by calling svlMVStub_StartMemoryValidator().

svlMVStub_UnloadMemoryValidator

extern "C" SVL_SERVICE_ERROR svlMVStub_UnloadMemoryValidator();

To unload the Validator DLL from your service call svlMVStub_UnloadMemoryValidator(), do not call FreeLibrary(). svlMVStub_UnloadMemoryValidator() shuts down various communications threads and removes any hooks that the Validator may have installed.

Debugging functions

We have added the following debugging functions to the NT Service API.

Before using these functions you must first set the log file name, otherwise the other debugging functions will do nothing.

The log file is opened for each write to the log file then closed afterwards. As such these log file functions are slow. Do not use these functions in any performance sensitive task. They are intended to allow you get debugging information out of a service when you’re trying to work out why something isn’t working, should you have problems.

extern "C" void svlMVStub_setLogFileName(const wchar_t* fileName);

extern "C" void svlMVStub_deleteLogFile();

extern "C" void svlMVStub_writeToLogFileA(const char* text);

extern "C" void svlMVStub_writeToLogFileW(const wchar_t* text);

extern "C" void svlMVStub_writeToLogFile(SVL_SERVICE_ERROR errCode);

extern "C" void svlMVStub_writeToLogFileLastError(DWORD errCode);

extern "C" void svlMVStub_dumpPathToLogFile();

svlMVStub_setLogFileName

To use any of the following logging functions you first need to set the name of the filename used for logging.
Setting this filename also sets the filename used by some of these API functions – you will find additional logging data from those functions that will help debug any issues with the service.

extern "C" void svlMVStub_setLogFileName(const wchar_t* fileName);

svlMVStub_deleteLogFile

extern "C" void svlMVStub_deleteLogFile();

This function deletes the log file.

svlMVStub_writeToLogFileA

extern "C" void svlMVStub_writeToLogFileA(const char* text);

Call this function to write a standard ANSI character string to the log file.

svlMVStub_writeToLogFileW

extern "C" void svlMVStub_writeToLogFileW(const wchar_t* text);

Call this function to write a unicode character string to the log file. The characters are cast to ANSI prior to writing. As such you can’t write Korean to the log file. This is simply a convenience function to write wide characters as simply as ANSI characters.

svlMVStub_writeToLogFile

extern "C" void svlMVStub_writeToLogFile(SVL_SERVICE_ERROR errCode);

Call this function to write a human readable description of the SVL_SERVICE_ERROR error code to the log file.

svlMVStub_writeToLogFileLastError

extern "C" void svlMVStub_writeToLogFileLastError(DWORD errCode);

Call this function to write a human readable description of the Windows error code to the log file.

svlMVStub_dumpPathToLogFile

extern "C" void svlMVStub_dumpPathToLogFile();

Call this function to write the contents of the PATH environment variable to the log file. This can be useful if you want to know what the search path is when trying to debug why a DLL wasn’t found during an attempt to load the Validator DLL.

NT Service API for each Validator

C++ Bug Validator

extern "C" SVL_SERVICE_ERROR svlBVStub_LoadBugValidator();

extern "C" SVL_SERVICE_ERROR svlBVStub_LoadBugValidator6432();

extern "C" SVL_SERVICE_ERROR svlBVStub_StartBugValidator();

extern "C" SVL_SERVICE_ERROR svlBVStub_UnloadBugValidator();

extern "C" SVL_SERVICE_ERROR svlBVStub_SetServiceCallback(serviceCallback_FUNC callback,
                                                          void*                userParam);

// debugging functions

extern "C" void svlBVStub_setLogFileName(const wchar_t* fileName);

extern "C" void svlBVStub_deleteLogFile();

extern "C" void svlBVStub_writeToLogFileA(const char* text);

extern "C" void svlBVStub_writeToLogFileW(const wchar_t* text);

extern "C" void svlBVStub_writeToLogFile(SVL_SERVICE_ERROR errCode);

extern "C" void svlBVStub_writeToLogFileLastError(DWORD errCode);

extern "C" void svlBVStub_dumpPathToLogFile();

C++ Coverage Validator

extern "C" SVL_SERVICE_ERROR svlCVStub_LoadCoverageValidator();

extern "C" SVL_SERVICE_ERROR svlCVStub_LoadCoverageValidator6432();

extern "C" SVL_SERVICE_ERROR svlCVStub_StartCoverageValidator();

extern "C" SVL_SERVICE_ERROR svlCVStub_UnloadCoverageValidator();

extern "C" SVL_SERVICE_ERROR svlCVStub_SetServiceCallback(serviceCallback_FUNC callback,
                                                          void*                userParam);

extern "C" void svlCVStub_setLogFileName(const wchar_t* fileName);

extern "C" void svlCVStub_deleteLogFile();

extern "C" void svlCVStub_writeToLogFileA(const char* text);

extern "C" void svlCVStub_writeToLogFileW(const wchar_t* text);

extern "C" void svlCVStub_writeToLogFile(SVL_SERVICE_ERROR errCode);

extern "C" void svlCVStub_writeToLogFileLastError(DWORD errCode);

extern "C" void svlCVStub_dumpPathToLogFile();

C++ Memory Validator

extern "C" SVL_SERVICE_ERROR svlMVStub_LoadMemoryValidator();

extern "C" SVL_SERVICE_ERROR svlMVStub_LoadMemoryValidator6432();

extern "C" SVL_SERVICE_ERROR svlMVStub_StartMemoryValidator();

extern "C" SVL_SERVICE_ERROR svlMVStub_UnloadMemoryValidator();

extern "C" SVL_SERVICE_ERROR svlMVStub_SetServiceCallback(serviceCallback_FUNC callback,
                                                          void*                userParam);

extern "C" void svlMVStub_setLogFileName(const wchar_t* fileName);

extern "C" void svlMVStub_deleteLogFile();

extern "C" void svlMVStub_writeToLogFileA(const char* text);

extern "C" void svlMVStub_writeToLogFileW(const wchar_t* text);

extern "C" void svlMVStub_writeToLogFile(SVL_SERVICE_ERROR errCode);

extern "C" void svlMVStub_writeToLogFileLastError(DWORD errCode);

extern "C" void svlMVStub_dumpPathToLogFile();

C++ Performance Validator

extern "C" SVL_SERVICE_ERROR svlPVStub_LoadPerformanceValidator();

extern "C" SVL_SERVICE_ERROR svlPVStub_LoadPerformanceValidator6432();

extern "C" SVL_SERVICE_ERROR svlPVStub_StartPerformanceValidator();

extern "C" SVL_SERVICE_ERROR svlPVStub_UnloadPerformanceValidator();

extern "C" SVL_SERVICE_ERROR svlPVStub_SetServiceCallback(serviceCallback_FUNC callback,
                                                          void*                userParam);

extern "C" void svlPVStub_setLogFileName(const wchar_t* fileName);

extern "C" void svlPVStub_deleteLogFile();

extern "C" void svlPVStub_writeToLogFileA(const char* text);

extern "C" void svlPVStub_writeToLogFileW(const wchar_t* text);

extern "C" void svlPVStub_writeToLogFile(SVL_SERVICE_ERROR errCode);

extern "C" void svlPVStub_writeToLogFileLastError(DWORD errCode);

extern "C" void svlPVStub_dumpPathToLogFile();

C++ Thread Validator

extern "C" SVL_SERVICE_ERROR svlTVStub_LoadThreadValidator();

extern "C" SVL_SERVICE_ERROR svlTVStub_LoadThreadValidator6432();

extern "C" SVL_SERVICE_ERROR svlTVStub_StartThreadValidator();

extern "C" SVL_SERVICE_ERROR svlTVStub_UnloadThreadValidator();

extern "C" SVL_SERVICE_ERROR svlTVStub_SetServiceCallback(serviceCallback_FUNC callback,
                                                          void*                userParam);

extern "C" void svlTVStub_setLogFileName(const wchar_t* fileName);

extern "C" void svlTVStub_deleteLogFile();

extern "C" void svlTVStub_writeToLogFileA(const char* text);

extern "C" void svlTVStub_writeToLogFileW(const wchar_t* text);

extern "C" void svlTVStub_writeToLogFile(SVL_SERVICE_ERROR errCode);

extern "C" void svlTVStub_writeToLogFileLastError(DWORD errCode);

extern "C" void svlTVStub_dumpPathToLogFile();

Panorama Theme by Themocracy