Rss Feed
Tweeter button
Facebook button
Technorati button
Reddit button
Myspace button
Linkedin button
Webonews button
Delicious button
Digg button
Flickr button
Stumbleupon button
Newsvine button

Posts tagged: api

Stdout redirection and capture

By , July 11, 2019 10:34 am

We were recently asked if Memory Validator could handle monitoring a program that took it’s input from a file and wrote its’ output to a file. As shown in the following example.

redirect.exe < input.txt > output.txt

Our tools could handle this, but it wasn’t obvious that they could. Also for interactive use there was no easy way to do this via the launch dialog, unless you were using Coverage Validator. We had to make changes to most of our tools so that they could do what Coverage Validator could do. All tools had a new diagnostic sub-tab added so that stdout data that was captured (an option) could be viewed.

In this article I’m going to explain how to launch a program that reads from stdin and/or writes to stdout using our tools. Although I’m talking about Memory Validator, these techniques apply to all our Validator tools.

There are four ways of doing this.

  1. Start your program from a batch file, putting the redirect of stdin and stdout in the batch file.
  2. Start your program from the launch dialog/wizard, specifying the input and output files on the launch wizard.
  3. Start your program from the command line, specifying the input and output files on the command line.
  4. Use the Memory Validator API to start Memory Validator from inside the target program.

Batch File

Using a batch file to do this is easy. Simply create your batch file in the form shown in the example below.

e:\redirect\release\redirect.exe < e:\test\input.txt > e:\test\output.txt

Save the batch file with a known name. In this example I’ll save the batch file in e:\test\redirectTest.bat. Then launch the batch file from Memory Validator (or another Validator tool). The first program launched by the batch file will be monitored.


Launch Dialog/Wizard

We modified the launch wizard and the launch dialog to include fields for an optional input file and an optional output file. We also added an option that would capture stdout so that you could view the output on the diagnostic tab.

This example shows the program testAppTheReadsFromStdinAndWritesToStdout.exe being launched, reading from e:\test\input.txt, and writing to reading from e:\test\output.txt.


The stdout capture checkbox has been selected. This will mean a copy of stdout will be captured and displayed on the diagnostic sub-tab stdout.


Command Line

For command line operation we need two new options -stdin and -stdout, each of which takes a filename for an argument.

There are two additional arguments that you can supply to tell Memory validator to ignore missing input files and missing output files: -ignoreMissingStdin and -ignoreMissingStdout.

memoryValidator.exe -program e:\redirect\release\redirect.exe 
                    -directory e:\redirect\release 
                    -stdin e:\test\input.txt 
                    -stdout e:\test\output.txt
                    -showErrorsWithMessageBox

API

You can use the Memory Validator API to start Memory Validator each time the target program is run. In that case just running the following command on the command prompt would cause Memory Validator monitor the target program redirect.exe.

e:\redirect\release\redirect.exe < e:\test\input.txt > e:\test\output.txt

To use the Memory Validator API with a particular application, the following steps outline the minimum steps required.

  1. Link to svlMemoryValidatorStub.lib (_x64.lib for x64)
  2. Link to svlMemoryValidatorStubLib.lib (_x64.lib for x64)
  3. #include “stublib.h”
  4. call startProfiler(); at the start of your program
  5. See documentation in the help file for more details.

For other Validator tools the library names will change. See the documentation (topic API) for the particular tool for details.

Conclusion

There are four ways you can work with stdin and stdout with our Validator tools.

You can work with batch files, launch interactively using the launch dialog/wizard, work from the command line, and use the Validator API.

Thread naming

By , June 19, 2019 7:21 pm

Multi-threading is becoming quite common these days. It’s a useful way to provide a responsive user interface while performing work at the same time. Our tools report data per thread where that is warranted (per-thread code coverage doesn’t seem to be a thing – no one has requested it in 17 years). In this article I’m going to discuss thread naming, OS support for thread naming, and additional support for thread naming that our tools automatically provide.

The Default Thread Display

Threads are represented by a thread handle and a thread id. Typically the thread id is what will be used to represent the thread in a user interface reporting thread related data. Thread ids are numeric. For example: 341. For trivial programs you can often infer which thread is which by looking at the data allocated on each thread. However that doesn’t scale very well to more complex applications. You can end up with thread displays like this:


Microsoft Thread Naming Exception

The WIN32 API does not provide any functions to allow you to name a thread. To handle this oversight, Microsoft use a convention that allows a program to communicate a thread name with it’s debugger. This is done via means of an exception that the program throws (and then catches to prevent it’s propagation terminating the application). The debugger also catches this exception and with the help of ReadProcessMemory() can retrieve the exception name from the program. Here’s how that works.

In your program

The exception code that identifies a thread naming exception is 0x406D1388. To pass the thread name to the debugger you need to create a struct of type THREADNAME_INFO (definition shown int the code sample), populate it with the appropriate data then raise an exception with the specified exception code. You’ll need to use SEH __try/__except to surround the RaiseException() call to prevent crashing the program.

typedef struct tagTHREADNAME_INFO
{
	DWORD	dwType;		// must be 0x1000
	LPCSTR	szName;		// pointer to name (in user addr space) buffer must be 8 chars + 1 terminator
	DWORD	dwThreadID;	// thread ID (-1 == caller thread)
	DWORD	dwFlags;	// reserved for future use, must be zero
} THREADNAME_INFO;

#define MS_VC_EXCEPTION 0x406D1388

void nameThread(const DWORD	threadId,
                const char	*name)
{
	// You can name your threads by using the following code. 
	// Thread Validator will intercept the exception and pass it along (so if you are also running
	// under a debugger the debugger will also see the exception and read the thread name

	// NOTE: this is for 'unmanaged' C++ ONLY!

	#define BUFFER_LEN		16

	THREADNAME_INFO	ThreadInfo;
	char		szSafeThreadName[BUFFER_LEN];	// buffer can be any size, just make sure it is large enough!
	
	memset(szSafeThreadName, 0, sizeof(szSafeThreadName));	// ensure all characters are NULL before
	strncpy(szSafeThreadName, name, BUFFER_LEN - 1);	// copying name
	//szSafeThreadName[BUFFER_LEN - 1] = '\0';

	ThreadInfo.dwType = 0x1000;
	ThreadInfo.szName = szSafeThreadName;
	ThreadInfo.dwThreadID = threadId;
	ThreadInfo.dwFlags = 0;

	__try
	{
		RaiseException(MS_VC_EXCEPTION, 0, 
                               sizeof(ThreadInfo) / sizeof(DWORD_PTR), 
                               (DWORD_PTR *)&ThreadInfo); 
	}
	__except(EXCEPTION_EXECUTE_HANDLER)
	{
		// do nothing, just catch the exception so that you don't terminate the application
	}
}

In the debugger

The THREADNAME_INFO struct address is communicated to the debugger in event.u.Exception.ExceptionRecord.ExceptionInformation[0]. Cast this pointer to a THREADNAME_INFO *, check the number of parameters are correct (4 on x86, 3 on x64), check the dwType and dwFlags are correct, then use the pointer in the szName field. This is a pointer inside the other process, which means you can’t read it directly you need to use ReadProcessMemory(). Once the debugger continues and this event goes out of scope, the ability to read the thread name no longer exists. If you want to read the thread name you must read it immediately, storing it for future use.

	if ((de->dwDebugEventCode == EXCEPTION_DEBUG_EVENT) &&
	    (de->u.Exception.ExceptionRecord.ExceptionCode == SVL_THREAD_NAMING_EXCEPTION))
	{
#ifdef _WIN64
		if (de->u.Exception.ExceptionRecord.NumberParameters == 3)
#else	//#ifdef _WIN64
		if (de->u.Exception.ExceptionRecord.NumberParameters == 4)
#endif	//#ifdef _WIN64
		{
			THREADNAME_INFO	*tni;

			tni = (THREADNAME_INFO *)&de->u.Exception.ExceptionRecord.ExceptionInformation[0];
			if (tni->dwType == 4096 &&
			    tni->dwFlags == 0)
			{
				void	*ptr = (void *)tni->szName;
				DWORD	threadId = tni->dwThreadID;

				if (ptr != NULL)
				{
					char	buffer[1000];
					int		bRet;
					SIZE_T	dwRead = 0;

					memset(buffer, 0, 1000);
					bRet = ReadProcessMemory(hProcess,
								 ptr,
								 buffer,
								 1000 - 1,	// remove one so there is always a terminating zero
								 &dwRead);

					if (buffer[0] != '\0')
						setThreadName(threadId, buffer);
				}
			}
		}
	}

Our tools support intercepting this mechanism so that we can name your threads if you use this mechanism.

The problem with this mechanism is that it puts the onus for naming the threads on the creator of the application. If that involves 3rd party components that spawn their own threads, and OS threads then almost certainly not all threads will be named, which results in some threads having a name and other threads being represented by a thread id.


CreateThread() And Symbols

In an attempt to automatically name threads to provide useful naming for threads without having to rely on the author of a program using thread naming exceptions we started monitoring CreateThread() to get the thread start address, then resolve that address into a symbol name (assuming debugging symbols are available) and use that symbol name to represent the thread. After some tests (done with our own tools – which are heavily multithreaded – dog-fooding) we concluded this was a useful way to name threads.

However this doesn’t solve the problem as many threads are now named _threadstartex().


_beginthread(), _beginthreadex() And Symbols

The problem with the CreateThread() approach is that any threads created by calls to the C runtime functions _beginthread() or _beginthreadex() will result in the thread being called _threadstart() or _threadstartex(). This is because these functions pass their own thread function to CreateThread(). The way to solve this problem is to also monitor _beginthread() and _beginthreadex() functions to get the thread start address, then resolve that address into a symbol name.


SetThreadDescription

Starting with Windows 10, a name can be associated with a thread handle via the API SetThreadDescription(). The name associated with a thread handle can be retrieved using GetThreadDescription(). We use these functions to provide names for your threads if you have used these functions.

Un-named Threads

Some threads still end up without names – typically these are transient threads created by the OS to do a short amount of work. If the call to create the thread was internal to Kernel32.dll then CreateThread() will not be called by an IAT call and will not be monitored, resulting in the thread not getting named automatically. This isn’t ideal, but ultimately you don’t control these threads, which means you can’t affect the data reported by these threads, so it’s not that important.

Thread Naming Priority

We’ve made these changes to all our Validator tools and updated the user interfaces to represent threads by both thread id and thread name.

If a thread is named by a thread naming exception, SetThreadDescription(), or via a Software Verify API call that name will take precedence over any automatically named thread. This is useful in cases where the same function is used by many threads – you can if you wish give each thread a unique name to prevent multiple threads having the same name.

Here are some of the displays that have benefited from named threads.

Bug Validator


Memory Validator


Performance Validator


Thread Validator



Detecting memory leaks in Visual Test unit tests

By , June 6, 2018 5:40 pm

Introduction

We recently had a request asking if C++ Memory Validator could detect memory leaks in unit tests managed by Microsoft’s Visual Test and Visual Test Explorer. They told us what they’d tried to do and that it had failed. Our response was to find out what was failing, fix it and then describe what you need to do to use our tools with Visual Test. That’s what this article is about.

There were a few bugs specific to working with Visual Test, plus a very novel environment variable data corruption that can only happen in very unusual circumstances (you almost certainly would not hit these in normal usage of our tools). We fixed these tools, then experimented with several ways of working with Visual Test. We’re only going to talk about C++ Memory Validator here, but this article also applies to our Coverage, Performance and Thread tools.

You can run Visual Test from the command line, and also via Visual Test Explorer, which is a component of Visual Studio.

Monitoring Visual Test from the command line

The full details of how to work with Visual Test from the command line are documented in this article from Microsoft.

Using this information we know that we need to launch vstest.console.exe and pass the unit test DLL to that as an argument. For example:

vstest.console.exe unitTest.dll 

Using this information we can launch vstest.console.exe from C++ Memory Validator and pass it the appropriate DLL. You can set the startup directory to whatever you like. We’ve chose to set it to the same directory as the unit test DLL.

Unit Test Code

namespace UnitTest1
{		
	TEST_CLASS(UnitTest1)
	{
	public:
		
		TEST_METHOD(TestMethod1)
		{
			// pretend this is a real unit test, 
			// exercising a real target class/function
			// and that it leaks some memory

			char *ptr;

			ptr = new char[123];
			strcpy_s(ptr, 123, "Excellent Adventure");

			ptr = (char *)malloc(456);
			strcpy_s(ptr, 456, "Bogus Journey");

			ptr = (char *)malloc(789);
			strcpy_s(ptr, 789, "Face the Music");

			Assert::AreNotEqual("Bill", "Ted");
		}

	};
}

You’ll notice that you don’t need to specify the TEST_MODULE_INITIALIZE(moduleInit) function, or the TEST_MODULE_CLEANUP(moduleCleanup) function. You only need to specify the unit tests you want tested. C++ Memory Validator does the rest.

Monitoring Visual Test Explorer

When working with Visual Test Explorer, Visual Test is launched from Dev Studio, and then loads the unit tests for testing from a DLL. Visual Test then stays running in the background and does not shutdown. This means the “program has ended” signal that C++ Memory Validator needs doesn’t get sent. It also means that subsequent tests run with Visual Test Explorer won’t cause Visual Test to startup, meaning that the “program has started” signal that C++ Memory Validator needs doesn’t get sent.

To get around these problems we need to use the NT Service API (see help file, or online documentation for details) to contact C++ Memory Validator at the start of the unit tests, and also at the end of the unit tests. We do that using the TEST_MODULE_INITIALIZE(moduleInit) function, or the TEST_MODULE_CLEANUP(moduleCleanup) functions.

Functions

We use svlMVStub_LoadMemoryValidator() to load C++ Memory Validator into the unit test, then svlMVStub_StartMemoryValidator() to start it monitoring the unit test and communicating with the user interface. At the end of the tests we use svlMVStub_UnloadMemoryValidatorAndTerminateProcess(1000) to shutdown C++ Memory Validator and set a thread running that will after a delay of 1000ms terminate the vstest.executionengine.x86.exe (Visual Test) process. You may need to experiment with this delay on your machine.

Header Files

We need to include two header files from C++ Memory Validator. svlMVStubService.h and svlServiceError.h. You’ll find these in the svlMVStubService folder in the C++ Memory Validator install directory.

Libraries

We also need to link to svlMVStubService.lib (32 bit builds) or svlMVStubService_x64.lib (64 bit builds). You’ll find versions of these libraries in the svlMVStubService folder (one library per version of Visual Studio).

Unit Test Code

#include "svlMVStubService.h"
#include "svlServiceError.h"

namespace UnitTest1
{		
	TEST_MODULE_INITIALIZE(moduleInit)
	{
		SVL_SERVICE_ERROR	sse;

		sse = svlMVStub_LoadMemoryValidator();

		sse = svlMVStub_StartMemoryValidator();
	}

	TEST_MODULE_CLEANUP(moduleCleanup)
	{
		SVL_SERVICE_ERROR	sse;
		
		sse = svlMVStub_UnloadMemoryValidatorAndTerminateProcess(1000);
	}

	TEST_CLASS(UnitTest1)
	{
	public:
		
		TEST_METHOD(TestMethod1)
		{
			// pretend this is a real unit test, 
			// exercising a real target class/function
			// and that it leaks some memory

			char *ptr;

			ptr = new char[123];
			strcpy_s(ptr, 123, "Excellent Adventure");

			ptr = (char *)malloc(456);
			strcpy_s(ptr, 456, "Bogus Journey");

			ptr = (char *)malloc(789);
			strcpy_s(ptr, 789, "Face the Music");

			Assert::AreNotEqual("Bill", "Ted");
		}

	};
}

Having reworked the unit tests to support the NT Service API, we now need to launch devenv.exe from C++ Memory Validator, but with instructions to ignore devenv.exe and monitor Visual Test (vstest.executionengine.x86.exe). We do that from the launch dialog/wizard.

First choose devenv.exe to monitor using the Browse… button next to the Application to launch field. In this example we chose C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\Common7\IDE\devenv.exe


Next we need to setup the applications to monitor. Click the Edit… button next to the Application to monitor field. The applications to monitor dialog is displayed.


Choose devenv.exe using the Browse… button next to the Application to launch field.

Click the Add… button and add the vstest.executionenginex86.exe that corresponds with the devenv.exe you selected. In this example we chose C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstestexecutionengine.x86.exe. Be sure to choose the correct one – there are several with similar names to choose from.


Click OK to accept the application to monitor.


Click OK to accept the new definitions.

Now on the launch dialog, change the Application to monitor combo to select the vstest.execution.x86.exe entry.

Enter the name of your unit test dll(s) in the Arguments field, separated by spaces. If you specify the directory containing the DLLs as the startup directory, you can specify the DLL names without paths.

Running Your Tests

Now that we’ve setup C++ Memory Validator to monitor Visual Test when started from devenv, let’s get to work.

To start devenv, click the Go! button. Devenv will start. Load the solution that contains your unit tests. When you choose Run Selected Tests or Debug Selected Tests from Visual Test Explorer, the tests will run and will be monitored by C++ Memory Validator at the same time.

Perl Profiling API

By , August 6, 2011 3:48 pm

For some time we have had tools for use with Perl in the beta stage. The reason the tools never left beta is because the instrumentation part of the tools was tool brittle. It relied up scanning the Perl binary for suitable insertion points then modifying the binary to insert a suitable profiling callback. The callback would then ask the interpreter for filename, line number and function name information.

This worked fine so long as the version of Perl being used had data stored in the same format as the version we had inspected when we wrote the tools. As soon as the internal design of the Perl runtime changed our tools stopped working.

The Perl interpreter has some rudimentary profiling capabilities built in. The are only suitable for gathering timing statistics and cannot be used to provide code coverage flow tracing or call graph timing results. The only was to get statistics for these uses is to build your own profiling API. That is what we have done with the Perl Profiling API.

The Perl Profiling API is inspired by similar functionality in the Python interpreter, the Ruby interpreter and the Mozilla JavaScript interpreter. The API consists of four functions:

  • void Perl_set_do_profiling(int enable);
  • int Perl_get_do_profiling();
  • void Perl_set_coverage_callback(PERL_PROFILER_CALLBACK callback, void *userData);
  • void Perl_set_profiling_callback(PERL_PROFILER_CALLBACK callback, void *userData);

PERL_PROFILER_CALLBACK

PERL_PROFILER_CALLBACK is defined as

void (*PERL_PROFILER_CALLBACK)(PERL_PROFILER_EVENT event,
                               const char          *fileName,
                               const int            lineNumber,
                               const char          *functionName,
                               const char          *packageName,
                               void                *userData);

The PERL_PROFILER_EVENT event can receive values of PPE_LINE, PPE_CALL, PPE_RETURN. PPE_LINE is used for line visits, PPE_CALL is used when a function is called, PPE_RETURN is used when a function returns. The fileName and lineNumber specify which part of which fileName the code is executing, the functionName specifies the current function and the packageName specifies which package/namespace the function is part of (if any). The userData value is the value passed when setting the callback using Perl_set_coverage_callback() and/or Perl_set_profiling_callback.

void Perl_set_do_profiling(int enable);

Perl_set_do_profiling() is used to enable or disable profiling with either of the callbacks. Pass TRUE to enable and FALSE to disable.

int Perl_get_do_profiling();

Perl_get_do_profiling() is used to determine if profiling is enabled or not. The function returns TRUE for enabled and FALSE for disabled.

void Perl_set_coverage_callback(PERL_PROFILER_CALLBACK callback, void *userData);

Perl_set_coverage_callback() is used to set the callback that will be called for line visit events. The callback function will be called for all lines the Perl program visits. The userData value specified will be passed to the callback.

void Perl_set_profiling_callback(PERL_PROFILER_CALLBACK callback, void *userData);

Perl_set_profiling_callback() is used to set the callback that will be called for function call and function return events. The callback function will be called for all functions the Perl program visits. The userData value specified will be passed to the callback.

Using the Perl Profiling API

By default the Perl Profiling API is not enabled and no callback values are set.

To use the Perl Profiling API we need to define a callback to do the work.

void callback(PERL_PROFILER_EVENT event,
              const char          *fileName,
              const int            lineNumber,
              const char          *functionName,
              const char          *packageName,
              void                *userData)
{
    switch(event)
    {
    case PPE_LINE:
        storeLineVisitInfo(fileName, lineNumber, userData);
        break;

    case PPE_CALL:
        startFunctionCallTiming(fileName, lineNumber, functionName, packageName, userData);
        break;

    case PPE_RETURN:
        endFunctionCallTiming(fileName, lineNumber, functionName, packageName, userData);
        break;
    }
}

The implementation of storeLineVisitInfo(), startFunctionCallTiming() and endFunctionCallTiming() are down to the profiler writer. You can use these to implement a timing profiler, code coverage, flow tracing or some other tool.

We also need to tell the Perl to use the Perl Profiling API.

	Perl_set_coverage_callback(callback, userData);
	Perl_set_profiling_callback(callback, userData);
	Perl_set_dp_profiling(TRUE);

Conclusion

As you can see using the Perl Profiling API is easy.

Adding the Perl Profiling API to the version of Perl you are using is also straightforward – there are two files that implement the API and two simple modifications to the runtime loop in dump.c and run.c. The overhead of the API is trivial when not in use and when in use the overhead is defined by what the callback writer chooses to do.

Learn more about the sources and binaries for the Perl Profiling API.

Ruby Memory Tracking API

By , December 1, 2006 5:15 pm

To provide the memory tracking, heap dump and object referencing facilities in Ruby Memory Validator we had to reverse engineer some hooks into the Ruby executable. This is not the easiest task in the world and beyond the abilities of anyone with less than a cursory knowledge of x86 assembly and Win32 hooking techniques.

To make it easier for this work to be done, to make it more reliable and to make these hooks available for those that just want to mainly work with Ruby and write a little C (as opposed to getting immersed in PE formats, x86 assembly and so on) we have created an extension to Ruby that provides for memory allocation and deallocation tracking, heap dumping and object reference determination. The extension is in the form of a simple C API.

You can read about this work, download the source code and download a working Ruby DLL with the extra code here.

http://www.softwareverify.com/ruby/customBuild/index.html

If you have any comments, please contact support at the usual address.

Panorama Theme by Themocracy