Rss Feed
Tweeter button
Facebook button
Technorati button
Reddit button
Myspace button
Linkedin button
Webonews button
Delicious button
Digg button
Flickr button
Stumbleupon button
Newsvine button

Category: Development

Stdout redirection and capture

By , July 11, 2019 10:34 am

We were recently asked if Memory Validator could handle monitoring a program that took it’s input from a file and wrote its’ output to a file. As shown in the following example.

redirect.exe < input.txt > output.txt

Our tools could handle this, but it wasn’t obvious that they could. Also for interactive use there was no easy way to do this via the launch dialog, unless you were using Coverage Validator. We had to make changes to most of our tools so that they could do what Coverage Validator could do. All tools had a new diagnostic sub-tab added so that stdout data that was captured (an option) could be viewed.

In this article I’m going to explain how to launch a program that reads from stdin and/or writes to stdout using our tools. Although I’m talking about Memory Validator, these techniques apply to all our Validator tools.

There are four ways of doing this.

  1. Start your program from a batch file, putting the redirect of stdin and stdout in the batch file.
  2. Start your program from the launch dialog/wizard, specifying the input and output files on the launch wizard.
  3. Start your program from the command line, specifying the input and output files on the command line.
  4. Use the Memory Validator API to start Memory Validator from inside the target program.

Batch File

Using a batch file to do this is easy. Simply create your batch file in the form shown in the example below.

e:\redirect\release\redirect.exe < e:\test\input.txt > e:\test\output.txt

Save the batch file with a known name. In this example I’ll save the batch file in e:\test\redirectTest.bat. Then launch the batch file from Memory Validator (or another Validator tool). The first program launched by the batch file will be monitored.

Launch Dialog/Wizard

We modified the launch wizard and the launch dialog to include fields for an optional input file and an optional output file. We also added an option that would capture stdout so that you could view the output on the diagnostic tab.

This example shows the program testAppTheReadsFromStdinAndWritesToStdout.exe being launched, reading from e:\test\input.txt, and writing to reading from e:\test\output.txt.

The stdout capture checkbox has been selected. This will mean a copy of stdout will be captured and displayed on the diagnostic sub-tab stdout.

Command Line

For command line operation we need two new options -stdin and -stdout, each of which takes a filename for an argument.

There are two additional arguments that you can supply to tell Memory validator to ignore missing input files and missing output files: -ignoreMissingStdin and -ignoreMissingStdout.

memoryValidator.exe -program e:\redirect\release\redirect.exe 
                    -directory e:\redirect\release 
                    -stdin e:\test\input.txt 
                    -stdout e:\test\output.txt


You can use the Memory Validator API to start Memory Validator each time the target program is run. In that case just running the following command on the command prompt would cause Memory Validator monitor the target program redirect.exe.

e:\redirect\release\redirect.exe < e:\test\input.txt > e:\test\output.txt

To use the Memory Validator API with a particular application, the following steps outline the minimum steps required.

  1. Link to svlMemoryValidatorStub.lib (_x64.lib for x64)
  2. Link to svlMemoryValidatorStubLib.lib (_x64.lib for x64)
  3. #include “stublib.h”
  4. call startProfiler(); at the start of your program
  5. See documentation in the help file for more details.

For other Validator tools the library names will change. See the documentation (topic API) for the particular tool for details.


There are four ways you can work with stdin and stdout with our Validator tools.

You can work with batch files, launch interactively using the launch dialog/wizard, work from the command line, and use the Validator API.

Thread naming

By , June 19, 2019 7:21 pm

Multi-threading is becoming quite common these days. It’s a useful way to provide a responsive user interface while performing work at the same time. Our tools report data per thread where that is warranted (per-thread code coverage doesn’t seem to be a thing – no one has requested it in 17 years). In this article I’m going to discuss thread naming, OS support for thread naming, and additional support for thread naming that our tools automatically provide.

The Default Thread Display

Threads are represented by a thread handle and a thread id. Typically the thread id is what will be used to represent the thread in a user interface reporting thread related data. Thread ids are numeric. For example: 341. For trivial programs you can often infer which thread is which by looking at the data allocated on each thread. However that doesn’t scale very well to more complex applications. You can end up with thread displays like this:

Microsoft Thread Naming Exception

The WIN32 API does not provide any functions to allow you to name a thread. To handle this oversight, Microsoft use a convention that allows a program to communicate a thread name with it’s debugger. This is done via means of an exception that the program throws (and then catches to prevent it’s propagation terminating the application). The debugger also catches this exception and with the help of ReadProcessMemory() can retrieve the exception name from the program. Here’s how that works.

In your program

The exception code that identifies a thread naming exception is 0x406D1388. To pass the thread name to the debugger you need to create a struct of type THREADNAME_INFO (definition shown int the code sample), populate it with the appropriate data then raise an exception with the specified exception code. You’ll need to use SEH __try/__except to surround the RaiseException() call to prevent crashing the program.

typedef struct tagTHREADNAME_INFO
	DWORD	dwType;		// must be 0x1000
	LPCSTR	szName;		// pointer to name (in user addr space) buffer must be 8 chars + 1 terminator
	DWORD	dwThreadID;	// thread ID (-1 == caller thread)
	DWORD	dwFlags;	// reserved for future use, must be zero

#define MS_VC_EXCEPTION 0x406D1388

void nameThread(const DWORD	threadId,
                const char	*name)
	// You can name your threads by using the following code. 
	// Thread Validator will intercept the exception and pass it along (so if you are also running
	// under a debugger the debugger will also see the exception and read the thread name

	// NOTE: this is for 'unmanaged' C++ ONLY!

	#define BUFFER_LEN		16

	char		szSafeThreadName[BUFFER_LEN];	// buffer can be any size, just make sure it is large enough!
	memset(szSafeThreadName, 0, sizeof(szSafeThreadName));	// ensure all characters are NULL before
	strncpy(szSafeThreadName, name, BUFFER_LEN - 1);	// copying name
	//szSafeThreadName[BUFFER_LEN - 1] = '\0';

	ThreadInfo.dwType = 0x1000;
	ThreadInfo.szName = szSafeThreadName;
	ThreadInfo.dwThreadID = threadId;
	ThreadInfo.dwFlags = 0;

		RaiseException(MS_VC_EXCEPTION, 0, 
                               sizeof(ThreadInfo) / sizeof(DWORD_PTR), 
                               (DWORD_PTR *)&ThreadInfo); 
		// do nothing, just catch the exception so that you don't terminate the application

In the debugger

The THREADNAME_INFO struct address is communicated to the debugger in event.u.Exception.ExceptionRecord.ExceptionInformation[0]. Cast this pointer to a THREADNAME_INFO *, check the number of parameters are correct (4 on x86, 3 on x64), check the dwType and dwFlags are correct, then use the pointer in the szName field. This is a pointer inside the other process, which means you can’t read it directly you need to use ReadProcessMemory(). Once the debugger continues and this event goes out of scope, the ability to read the thread name no longer exists. If you want to read the thread name you must read it immediately, storing it for future use.

	if ((de->dwDebugEventCode == EXCEPTION_DEBUG_EVENT) &&
	    (de->u.Exception.ExceptionRecord.ExceptionCode == SVL_THREAD_NAMING_EXCEPTION))
#ifdef _WIN64
		if (de->u.Exception.ExceptionRecord.NumberParameters == 3)
#else	//#ifdef _WIN64
		if (de->u.Exception.ExceptionRecord.NumberParameters == 4)
#endif	//#ifdef _WIN64

			tni = (THREADNAME_INFO *)&de->u.Exception.ExceptionRecord.ExceptionInformation[0];
			if (tni->dwType == 4096 &&
			    tni->dwFlags == 0)
				void	*ptr = (void *)tni->szName;
				DWORD	threadId = tni->dwThreadID;

				if (ptr != NULL)
					char	buffer[1000];
					int		bRet;
					SIZE_T	dwRead = 0;

					memset(buffer, 0, 1000);
					bRet = ReadProcessMemory(hProcess,
								 1000 - 1,	// remove one so there is always a terminating zero

					if (buffer[0] != '\0')
						setThreadName(threadId, buffer);

Our tools support intercepting this mechanism so that we can name your threads if you use this mechanism.

The problem with this mechanism is that it puts the onus for naming the threads on the creator of the application. If that involves 3rd party components that spawn their own threads, and OS threads then almost certainly not all threads will be named, which results in some threads having a name and other threads being represented by a thread id.

CreateThread() And Symbols

In an attempt to automatically name threads to provide useful naming for threads without having to rely on the author of a program using thread naming exceptions we started monitoring CreateThread() to get the thread start address, then resolve that address into a symbol name (assuming debugging symbols are available) and use that symbol name to represent the thread. After some tests (done with our own tools – which are heavily multithreaded – dog-fooding) we concluded this was a useful way to name threads.

However this doesn’t solve the problem as many threads are now named _threadstartex().

_beginthread(), _beginthreadex() And Symbols

The problem with the CreateThread() approach is that any threads created by calls to the C runtime functions _beginthread() or _beginthreadex() will result in the thread being called _threadstart() or _threadstartex(). This is because these functions pass their own thread function to CreateThread(). The way to solve this problem is to also monitor _beginthread() and _beginthreadex() functions to get the thread start address, then resolve that address into a symbol name.


Starting with Windows 10, a name can be associated with a thread handle via the API SetThreadDescription(). The name associated with a thread handle can be retrieved using GetThreadDescription(). We use these functions to provide names for your threads if you have used these functions.

Un-named Threads

Some threads still end up without names – typically these are transient threads created by the OS to do a short amount of work. If the call to create the thread was internal to Kernel32.dll then CreateThread() will not be called by an IAT call and will not be monitored, resulting in the thread not getting named automatically. This isn’t ideal, but ultimately you don’t control these threads, which means you can’t affect the data reported by these threads, so it’s not that important.

Thread Naming Priority

We’ve made these changes to all our Validator tools and updated the user interfaces to represent threads by both thread id and thread name.

If a thread is named by a thread naming exception, SetThreadDescription(), or via a Software Verify API call that name will take precedence over any automatically named thread. This is useful in cases where the same function is used by many threads – you can if you wish give each thread a unique name to prevent multiple threads having the same name.

Here are some of the displays that have benefited from named threads.

Bug Validator

Memory Validator

Performance Validator

Thread Validator

Support for Menus in Visual Studio 2017, 2015, 2013, 2012, 2010

By , May 12, 2018 10:10 am

Visual Studio menu support

We’ve just added support for integrating tool menus into Visual Studio. This integration supports most of our tools. The menu only shows tools that are installed. Supported versions of Visual Studio are 2017, 2015, 2013, 2012, 2010.

Here’s what it looks like in Visual Studio 2017 (if you have all of our tools installed).

This means that most of our tools can be launched from within Visual Studio. If a solution is loaded, the tool you select will be started with appropriate options to process the startup project that is selected. For tools that support both x64 and x86 installation, on x86 platforms the x86 tool will be selected, on x64 platforms the x64 tool will be selected (if it was installed), otherwise the x86 tool will be selected.

For Thread Lock Checker and OutputDebugString checker the tools are given a list of the source files in the project. The source files are scanned for errors.

For DbgHelp Browser and PE File Browser the tools are given the executable that the project builds. The tools then display data about the executable and/or it’s debug information as appropriate.

For the various C++ Validators and .Net Validators, VM Validator, Exception Tracer and Page Fault Monitor, the tools are passed the same information that the Visual Studio debugger uses to start a program (program name, program arguments, startup directory). The tools then start the target program and monitor data according to which tool has been launched.

Exception Tracer

The eagle eyed will have noticed that in this list is the name of a tool that isn’t on our website (yet). Exception Tracer. This is a tool that has been in genesis for some time. It has transformed from an experimental debugger, into an experimental code coverage tool and now it’s an interesting view on the internal life of an application, from the point of view of the exceptions it throws. We’ve used Exception Tracer to analyse some unusual bugs that were almost impossible to analyse using a traditional debugger. We’ll be releasing this tool later this year. It will add to our growing list of free debugging and analysis tools on our website.

Cleaning up Visual Studio

By , March 19, 2018 12:33 pm

Why would you ever need a tool to clean up Visual Studio? Doesn’t Visual Studio have a built in “clean” mode that cleans your build directories?

Yes, it does, but that only cleans the file created by compiling and linking your software. Visual Studio also creates other files. How do you get rid of those files? Look them up, then search for them manually? That’s going to be time consuming.

Why would you even want to delete these extra files?

We’ve also seen Visual Studio crash from time to time. Sometimes it’s a plugin or extension gone astray, but other times it’s when you do something harmless like load a project or workspace or solution. What we found was that in many cases, just deleting the extra files would prevent future crashes.

So we wrote Visual Studio Project Cleaner to do that. The first version of the tool was written to work with Visual Studio 6, but we’ve updated it to also cleanup so of the files created by the newer versions of Visual Studio, Visual Lint and Visual Studio Project Fixer.

What type of file can we clean?

  • aps. Last Resource editor state.
  • bak. Backup file.
  • bsc. Source browser database.
  • dbg. Debugging information.
  • exp. Exports file.
  • idb. Minimum rebuild linker file.
  • ilk. Incremental linker file.
  • ipch. Incremental precompiled header file.
  • lastbuildstate. Build helper.
  • log. Build log.
  • map. Linker MAP file.
  • ncb. Intellisense database.
  • obj. Object file.
  • opt. Visual Studio options.
  • pch. Pre-compiled header file.
  • res. Compiled resource script.
  • sbr. Source browser information.
  • sdf. Intellisense database.
  • tlog. Depdenency file information.

We can also clean the following file types (these are disabled by default).

  • lib. Library file.
  • dll. Dynamic Link Library.
  • exe. Executable.
  • pdb. Program database.
  • old. Visual Studio upgrade files.
  • Visual Lint files in .visualint directories.
  • Visual Studio Project Fixer backup files (*.vcxproj-YYY-MM-DD-HH-MM-SS)

Clean your Visual Studio

We’ve been using Visual Studio Project Cleaner for years to clean unwanted files. It turns a tedious job into a simple and easy job.

Clean your files today with Visual Studio Project Cleaner.

Improving how we build projects and solutions

By , March 12, 2018 12:01 pm

The problem with Solutions

If you’re working with just one solution and the projects it contains Microsoft Visual Studio is a good tool for managing the builds. Except that it won’t necessarily build the projects in the right order, forcing you to make multiple build passes to get a built solution.

But what if you’re working with many solutions?

What if some of the projects in the solutions have identical project names, even though they are different projects? You can’t merge these into one mega solution so that you can build everything at once because the project names will clash.

Another problem with solutions is that the projects in the solution do not necessarily get built in the optimum order. A project that depends on another project may be built before that project dependency is satisfied. This forces you to build the solution multiple times until all dependencies are satisfied.

What if there was a better way to manage your builds? What if there was a better way to automate your builds?

Born from a real need

At Software Verify we have a code base of several million lines of C++, plus some C, assembler and C#. Looking at the Visual Studio Project Builder display I can see we have 28 solutions, with 217 projects, giving us a total of 962 different project configurations to complete a build. (The actual number is higher than this because it doesn’t include the legacy support solution and projects for Visual Studio 2002, because that can’t be installed on a 64 bit machine.).

This large number of projects and solutions includes every version of Visual Studio from Visual Studio 2017 to Visual Studio 6.

We needed to have a build tool that would allow us to reliably build a project, or solution, or a specific configuration of a project, regardless of version of Visual Studio or programming language.

All this would be done with one simple to use software tool that could be used interactively with a GUI or from the command line. The tool would provide visual feedback so that you could tell from 50 feet if the build was going pear shaped. The tool would also provide notifications so that you could forget about it and go do something else instead.

Visual Studio Project Builder

I’ve seen various solutions to this problem, lots of scripts, batch files, programs interacting. It works, kind of, but it’s all a bit of a mess, held together with string and duct tape in the best “make it work” seat of the pants “we haven’t got enough time” developer fashion.

That isn’t what I wanted. We’d already finished Visual Studio Project Fixer and I was very pleased with the results from that. I wanted the same “one tool” approach. We had the requirements, we knew what we needed to be able to build. We knew we had a decent test case (build our own tools) and we knew what a good tool would look like. As simple as possible.

That’s how Visual Studio Project Builder was born.

The basic capabilities are:

  • Automatically discover solutions and projects.
  • Optimise the build order.
  • Build, rebuild, clean project for any Visual Studio version from 2017 to 6.
  • Build individual project configurations, or variations based on project, solution and configuration.
  • Easily identify warnings and errors.
  • Easily filter warnings and errors.
  • Interactive GUI with real time build progress updating.
  • Can be run from a command line if desired.
  • Works on any Windows OS from Windows 10 through Windows XP.

A simple user interface

When performing a full build, full rebuild or full clean, a progress dialog is displayed, allowing the progress of the build to assessed at a glance, but with extra detail if you want it. You can tell from the colours on this dialog if a build is good, or bad, or has warnings, from 50 feet.

Behind the progress dialog is the main display, showing the status of every project configuration that can be built. Colours indicate status (good, warning, compiler error, linker error). A context menu provides access to options related to that project configuration, project or solution. For example:

  • Build all projects named “stubNonMFC” that have the configuration “Win32|Debug”.
  • Rebuild all projects that have the configuration “Win32|Release”.
  • Rebuild all projects in this solution that have the configuration “Win32|Release”.
  • Clean all projects named “tabserv”.
  • View build logfile.
  • View build command line passed to Visual Studio.

Some of the build options from the context menu cannot be done with Visual Studio, except by manual intervention for each project. When you span this across many solutions and projects this becomes very powerful and an incredible time saver.

Build notifications

Software developers perform a mentally taxing job. The last thing you want to add to their list is “Is the build done yet?”. To remedy this problem we have provided several (optional) notification methods:

  • Email when build is complete.
  • Text message when build is complete. Uses Twilio.
  • Speech message when build is complete.

My favourite is the last one. I can go and do something else, read a book, investigate a bug, work on something else, talk to someone, and when the build is done my computer talks to me: “Build Complete, Success”, or “Build Failure. 3 Errors”.

The email and text message notifications are intended for use when you are out of range of the computer talking to you.

We have delayed implementing carrier pigeon functionality until there is a proven market for it.

Does this compete with existing build solutions?

Does this compete with existing build solutions such as IncrediBuild?

No. Incredibuild speeds up builds of projects and solutions by distributing them on your local network.

Visual Studio Project Builder is all about getting projects built, in the right order, even though many toolsets may be in use. It all happens on the same machine. We’re hoping that Visual Studio Project Builder can work with build accelerators like IncrediBuild to speed things up.

Build your Visual Studio Projects

We’ve been using Visual Studio Project Builder in house for some time. We thought it was about time everyone else got a chance to use it. We’re making this available for free. Learn more about Visual Studio Project Builder.

Fixing Visual Studio project problems

By , March 5, 2018 2:51 pm

The productivity sink

Visual Studio is a very productive development environment. However, you can get Visual Studio into a state (for C++ and C developers, at least) where the compile and link both succeed, but when you go to run the executable Visual Studio annoyingly tells you that the build is out of date and would you like to build it?

This is a very frustrating experience. You go around and around in circles trying to work out what is wrong with the project that is causing this warning message to be displayed.

After some head scratching you’ll Google it and find solutions on numerous websites that tell you to do this, or that, or the other. The advice on some sites is consistent with the advice on other sites, some advice is not consistent. Then you find that some advice is for one version of Visual Studio, but you need to do something differently for a different version of Visual Studio.

It’s a mess. But you work out what you need to do, set about it and after tweaking a registry setting, running a tool to grab the output from OutputDebugString (or writing one if you couldn’t find a tool!), you now know which file is causing Visual Studio to tell you the build is out of date. Great!

Er, not so fast. The problem with this is Visual Studio only tells you about the first file that causes it to need to be rebuilt. Any other files aren’t listed, so you have to make the change then repeat the whole cycle. Slow!

“Crikey, that was a lot of effort, I hope that doesn’t happen again”, you think, or perhaps something a bit less polite. And then you remember you’ve got 10 more projects that behave like this, but you’d been ignoring that for a long time. But now you know how to fix it, but you can’t imagine putting in the effort for those 10 projects.

That was the situation we were in, except that just about every project we had (over 200 projects) had this problem. After tackling just one project I knew we had a choice:

  • Ignore the warnings.
  • Fix the warnings manually. Very slow. Lots of typing. Boring. Morale killer.
  • Find a way to automate the fixing of these problems.

Our engineering needs drove the decision.

Visual Studio Project Fixer

We decided to write a tool to make the hard bits of the above easy and to automatically discover what was wrong with a project file, then allow the user to choose which items to fix and which to leave alone. Plus an automatic mode that just fixes everything for you.

Things that can affect a project file are:

  • Missing files. The file exists on disk, but is not specified in the project.
  • Incorrect files. The file exists on disk, but not in the location specified in the project
  • Nonexistent files. The file doesn’t exist on disk, but is specified in the project.
  • Obsolete files. Files from older versions of Visual Studio.
  • .tlog files. Visual Studio file dependency files.

Some examples…

Missing file. You have a class defined in class.h and implemented in class.cpp. Only class.cpp is listed in the project file.

Incorrect file. A file is listed in the project as ..\svlCommonUI\class.h but is actually located at ..\svlCommon\class.h.

Nonexistent file. A resource file is specified that doesn’t exist. .\images\nonexistent.bmp.

Obsolete file. An old format help file from an early version of Visual Studio, carried over in Visual Studio project conversion.

A simple user interface

The image above shows the user interface after a scan of many projects has been performed. The summary display is shown, which lists an overview of the state of the projects and provides an option to automatically fix all the projects. If you click an individual project, the display changes to provided detailed information about that project.

There are four ways of scanning projects, providing for maximum flexibility in automating fixes or manually choosing each fix to apply.

  • Open a project.
  • Open a solution.
  • Open all projects in a directory (and sub-directories).
  • Open all projects in solutions in a directory (and sub-directories).

Having scanned some projects you can examine the results and choose to fix individual projects, or to fix all the projects that need fixing.

The default settings are to backup all project files before modifying them.

Diagnostic Information

It is possible that even with all the information provided by Visual Studio Project Fixer you may still wish to view the output from Visual Studio. Visual Studio 2015 and 2017 provide an option to show you this information. However for Visual Studio 2010, 2012 and 2013 you can enable diagnostic information to be sent via OutputDebugString(). This means that Visual Studio Project Fixer can intercept this information. All you need to do is enable it (from the Edit menu).

Then every time Visual Studio checks the status of a build, it’s output can be seen on the Visual Studio OutputDebugString tab.

Fix your Visual Studio Projects

We’ve been using Visual Studio Project Fixer in house for some time. We thought it was about time everyone else got a chance to use it. We’re making this available for free. Learn more about Visual Studio Project Fixer.

Changes to the first run wizard

By , February 15, 2018 2:41 pm

When you run one of our tools for the first time it displays the first run wizard.

The first run wizard takes you through the steps of configuring a few settings that are important to the tool your a using. Most of these settings are the same for each tool, although some of the first run wizards have settings pages that are specific to that tool.

We’ve just add a new page to the first run wizard. We’ve added the ability to configure the behaviour of the launch, inject and wait dialogs. The default behaviour is wizard. The behaviour can be modified from the Settings menu (formerly, Configure menu). Now this setting can also be set on the first run wizard.

The reason for adding this option to the first run wizard is because when we’ve done interactive debugging with customers we often notice that the user still has their Validator configured to use the launch wizard, rather than the more concise launch dialog. We thought there was a chance that some people don’t realise they can change the style of these dialogs/wizards.

New tools and old tools

By , February 6, 2018 11:16 am

Last week we released a versions of our tools that no longer install DLLs in the Windows system32 directory. We documented these changes in an article Changing How We Install Our Software.

In hindsight, I think we rushed that release. Which is a strange thing to write given that the work that went into it was nearly a month in duration and the last two weeks of testing every use case for the update NT Service API and the new DLL locations, that was very detailed and deliberate.

But where we let ourselves down was how we communicated about how the old versions of the tools and the new versions of the tools can live side by side. This blog post is about clearing up any misunderstanding. We’ve also improved the user interface for informing you about the old DLLs and choosing what action to take if old DLLs are found.

How do old tools and new tools co-exist?

Old Tools. The old tools always copy the DLLs they need to System32 every time they start. Because of this, even if a new tool removes the DLLs, the old tools will reinstall the DLLs automatically.

New tools. The new tools don’t expect to find DLLs in System32 and thus may not work correctly if DLLs are found there (because of how Windows search paths work). To deal with this we’ve provided a helper utility system32 cleanup which you can use to automatically delete any DLLs from old tools.

If you set the cleanup utility to automatically delete any dlls from old tools, then the new tools will always cleanup any dlls from the old tools, while the old tools will always replace them. This system will work unless you have an anti-virus tool that starts deleting anything copied into System32 – if that is happening you need to start using the new versions of our tools.

System32 Cleanup

The system32 cleanup user interface is only displayed if there are DLLs from old versions of the tools in the Windows system32 directory.

Please note that on 64 bit machines the 32 bit versions of our tools will be looking in SysWow64 (although the path will be reported as system32 due to the Windows file redirection mechanism).

We’ve replaced the message box with a dedicated dialog that allows you to choose how you want the old DLLs treated.

The dialog provides a list of DLLs that are from old versions of our software tools.

Below the list is a combo box allowing you to choose what action is taken when any subsequent check for DLLs is run (regardless of which Validator tool starts the check for old DLLs). The options are:

  • Always ask to delete files.
  • Always delete files.
  • Never delete files.

Each option results in different behaviour.

Always Ask To Delete Files

If Software Verify DLLs are found in system32 the dialog is displayed. The user of the software gets to choose if the DLLs are deleted or not. The DLLs can be deleted by clicking the Delete Files button. The future behaviour of the dialog can be changed by choosing one of the options in the combo box. If Always delete files is chosen the DLLs will be deleted when the dialog is closed.

Always Delete Files

If Software Verify DLLs are found in system32 the DLLs are deleted without informing the user. The dialog is never displayed.

Never Delete Files

No DLLs are deleted. The dialog is never displayed.


Because only one of the above behaviours results in the dialog being displayed it is possible to set the dialog up so that you can never interact with it again. But what if you wanted to change the behaviour once later on? To do that, run the svlSystem32Cleanup.exe (or svlSystem32Cleanup_x64.exe as appropriate) with the /reset command line argument. This will reset the behaviour to the default state and cause the user interface to display even if there are no DLLs to delete.

Changing how we install our software

By , February 3, 2018 10:46 am

With the next release of our software (later today) we are changing how we install our software.

The installers stay the same, the software (as you see it) stays the same, but under the hood, things are different. This article describes that change, why we made that change, the consequences of that change, and the work we did to ensure the software keeps functioning.

First, some history.


Once upon a time, software developers routinely placed their DLLs in the Windows system directory, and then when Windows 95 came along, they placed their DLLs in the Windows system32 directory. This eventually resulted in a mess, as far too many DLLs from different software vendors ended up stored in one place.

To solve this Microsoft stated that an application’s DLLs should be stored with the application. Microsoft also made installing their redistributables easier and made various other changes that made “DLL Hell” go away (or manifest in subtly different forms). Over time most software vendors changed how they distributed their software.

Debugging Tools

When we started Software Verify Limited (originally as Software Verification Limited, which no-one could spell!), we noticed that for certain types of problem we needed to investigate, it was quite hard to get the appropriate DLLs to load into the target application unless the DLLs were stored in the system32 directory.

We took the view, rightly or wrongly, that because we were providing tools that do unusual things with applications we could also do non-standard things with how we installed the software. We also added code to the user interface so that each time it started it would copy it’s DLLs to where it thought they needed to be (usually system32, sometimes SysWow64 – the 32 bit system32 on a 64 bit machine).

This behaviour was so ingrained and so core to how the software worked, that even when we spent 3 months rewriting the software many years ago to work nicely in the administrator mode enabled world of Windows Vista/7/8/10, it never occurred to us to change our abuse of the system32 directory.


Then one day, a developer researching our tools emailed us saying that he liked our tools but he was constantly fighting his anti-virus tool (BitDefender) that was flagging our tools as viruses because we copied files into the Windows system32 directory. He had disabled this feature but really wanted our software modifying so that we didn’t trigger the Anti-Virus software.

We’d also had reports from other developers that other anti-virus tools had flagged our tools as viruses – which we had assumed was a reference to the fact that our software uses CreateRemoteThread(). The function CreateRemoteThread() can be used for good and for ill. The Toolhelp and PSAPI libraries use this function to query data from running executables. So does Process Explorer from SysInternals (now part of Microsoft). We also use it. So do Bad Guys ™.

These various reports indicated to us that we had a problem, so we started investigating what needed to change to get the many moving parts of our software to work correctly without us touching Windows system32 or Windows sysWow64.

The change to what we install

Things we don’t do any more:

  • We no longer install any Software Verify DLLs or EXEs to system32 or sysWow64.
  • We no longer copy any Software Verify DLLs or EXEs to system32 or sysWow64 when the user interface runs.

So how do the DLLs get found?

One solution to this would be to place the path to the installed tool in the PATH environment variable. We considered this for a short while. But we have a lot of tools and the paths to them will be long if stored in the default location of c:\program files etc. We didn’t like the idea of someone installing C++ Developer Suite x64 / x86 and suddenly being faced with 8 new, very long paths added to their PATH environment variable. We had to come up with a better way.

We decided that we’d update the PATH environment variable to point to the tool in use, at the time the tool being used launched an application, launched a batch file, attached to a running application, or started working with a service. This way, only the application being monitored has it’s PATH updated, and it only has it’s PATH updated with the path to the tool in use, not all tools that have been installed. It’s elegant. It results in path isolation, so incompatible DLLs for one tool can’t affect another tool. And the user’s PATH environment variable doesn’t get polluted with lots of PATHs to our tools.

OK, so that sounds easy to do. Is it? Not exactly. There are quite a few use cases we need to cover. Some are trivial to implement, other quite involved. For all of the cases listed below we need to be able to handle:

  • 32 bit tool working with 32 bit application.
  • 64 bit tool working with 32 bit application.
  • 64 bit tool working with 64 bit application.

We made a grid of everything we needed to test, in every combination of tool (32 bit / 64 bit) and launch/batch/inject/service (32 bit and 64 bit). It came to 140 separate tests cases, each with it’s own test peculiarities, often requiring new and inventive ways to address the problem. When working through the services section we had to reboot a few times as we managed to get the service manager into a rather confused state :-).

Why does the PATH matter so much?

The reason the PATH matters so much is because when you’re injecting a DLL into another process so that you can gather information (flow tracing, code coverage, memory allocations, profiling, thread analysis) the DLL you’re injecting almost certainly has dependencies on other DLLs. If those dependencies can’t be satisfied by being found on the DLL loader search path, the DLL won’t load.

The reason we used to store the DLLs in system32 (or sysWow64) is because it’s an easy (and lazy) way to get the DLL dependencies found.

However, if we can modify the PATH suitably, we can ensure the DLLs can still be found.

Launching native (unmanaged) applications

This is when we start a typical application built using C, C++, Delphi, Visual Basic etc.

We modify the PATH in the tool prior to launching the target application. The target application inherits the environment variables of the parent process (the tool launching it).

Launching .Net (managed) applications

This is when we start a typical application built using C#, Visual Basic .Net etc.

We modify the PATH in the tool prior to launching the target application. The target application inherits the environment variables of the parent process (the tool launching it).

Launching applications from batch files

This is when we start an application from a batch file.

We modify the PATH in the tool prior to launching the target application. The target application inherits the environment variables of the parent process (the tool launching it).

This method will work for most use cases. If only life were that simple…

Launching applications from batch files with no path

This is when we start an application from a batch file but the creator of the batch file has explicitly modified the PATH environment variable, nullifying any changes we made to the PATH environment variable when we launched the command processor that is executing the batch file.

We need to intercept where the command processor starts the process and then modify the PATH in the environment block passed to the target application.

Launching applications from other applications

This is where we monitor the Nth application launched by another application.

We need to intercept where the first application starts the process we wish to monitor and then modify the PATH in the environment block passed to the target application.

Attaching to running applications

This is where we need to inject our DLL (which is in a known location that is not in the PATH) into the address space of a running process that already has the PATH environment variable set, most likely without including a path to where our DLL (and it’s dependencies) is located. The key is the fact that the injected DLL has dependencies. If the dependencies can’t be found, the DLL load will fail.

We had to invent a way to modify the PATH in the already running application before we inject our DLL with all it’s dependencies.

Working with services

This is where we need to get our DLL loaded by the tool’s NT Service API (each Validator has it’s own NT Service API).

This isn’t as straightforward as it seems. The path to the DLL isn’t known (the user could have installed the Validator anywhere). We have to find a way to get the PATH information into the service so that the various NT Service APIs can load the DLL from the correct location. To do this we also updated all the NT Service APIs with additional functions to help debug any NT Service API failures. A separate blog post covers this topic. Services run in quite locked down environments, getting this information into each service was quite an interesting adventure.

Changes to installer and Validators

The way our software tools are written you can normally install one version over the top of a previous version without needing to uninstall the previous version. For this next release, with the new DLL layout, we recommend uninstalling the previous version. However this is not essential as we have added some code to the licence key installer and also to the main Validator user interface, code that will check for the existence of Software Verify DLLs in the Windows system32 directory. If Software Verify DLLs are found in this directory, you will be informed and offered the ability to remove these DLLs.

We recommend removing the DLLs, as we cannot predict the interaction between DLLs left in the system32 directory and the new version of the Validator software. It is possible that for most, if not all, use cases that the software will function correctly. This depends on the way the Windows DLL Loader searches for DLLs and how it loads DLLs. If the system32 directory DLLs are found first then the wrong DLLs will be loaded. This is why we recommend removing them.

“But if I delete the svl*.* files in system32, won’t my older tools stop working?”

Older versions of our software tools will continue to work. Every time you start the GUI for an older tool, it will copy the DLLs it needs back into system32 so that it can function. Then, the next time you start a modern tool, you’ll get a prompt about the svl*.* files and have the option to delete them. With both solutions, the tools keep working. However, if you have an anti-virus tool (for example, BitDefender) installed, that will then delete those files, preventing the tool from working. That’s why the new mechanism is better, the tools work and anti-virus tools are happy.

The above scenario isn’t ideal, but it will allow older and newer tools to work together. If this is a serious issue for some people we may add an “always delete the files” option so that subsequent starts of the newer tools automatically delete the svl*.* files from system32. If you need this functionality, please let us know.

We believe the current versions of our tools are better than the older versions and that upgrading to the current versions will provide a better experience, both in functionality, better user experience, and better documentation. This is the 2nd time in our company history (nearly 16 years) that we’ve chosen to make a non-backwards compatible change with previous behaviour (excluding changes to saved data file formats).


We’ve changed how we install our software. We no longer do things that might alarm anti-virus tools. As a user of our software, you shouldn’t notice any change in functionality. Everything should continue working as normal.

Changes to the NT Service API

By , February 3, 2018 10:25 am

All of our Validator tools have an NT Service API.

We’ve just made some changes to the NT Service API. This article documents those changes and why we made them.

The changes to the NT Service API, break the existing API behaviour, and improve the existing API behaviour.

If you are using the NT Service API, you should read this article.

For the purposes of this article I’m going to describe the NT Service API for C++ Memory Validator. The changes apply to the NT Service APIs for each C++ Validator tool, the only thing that is different is the function name prefix of svlBVStub_, svlCVStub_, svlMVStub_, svlPVStub_, svlTVStub_. If you can’t directly use the NT Service API due to compiler/linker issues, please consult the documentation for your Validator to see an alternative way of getting the same result.

Changes to the API

  • We changed the name of an enumeration and extended the values in the enumeration.
  • We split some functions into two parts so that they could be reused with other functions to achieve a better result.
  • We added some debugging functions to help you debug any failures when using the API.


The SVL_ERROR enumeration has been renamed to SVL_SERVICE_ERROR.

The definition for SVL_SERVICE_ERROR is now in it’s own include file svlServiceError.h rather than defined in the NT Service API header file (svlMVStubService.h for C++ Memory Validator).

typedef enum _svlServiceError
   SVL_OK,                              // Normal behaviour
   SVL_ALREADY_LOADED,                  // Stub DLL already loaded into service
   SVL_LOAD_FAILED,                     // Failed to load stub DLL into service
   SVL_FAILED_TO_ENABLE_STUB_SYMBOLS,   // Loaded DLL, but failed to enable stub symbols because couldn't find function
   SVL_NOT_LOADED,                      // Couldn't unload DLL because DLL not loaded
   SVL_FAIL_UNLOAD,                     // Couldn't unload DLL because couldn't find function
   SVL_FAIL_TO_CLEANUP_INTERNAL_HEAP,   // Couldn't get the internal stub heap and thus couldn't clean it up
   SVL_FAIL_MODULE_HANDLE               // Couldn't get the stub DLL handle so couldn't continue
   SVL_FAIL_SETSERVICECALLBACK,         // Couldn't call the set service callback
   SVL_FAIL_COULD_NOT_FIND_ENTRY_POINT, // Couldn't find the DLL entry point to start the validator
   SVL_FAIL_TO_START                    // Failed to start the Validator

Split functionality

The previous implementation of svlMVStub_LoadMemoryValidator() and svlMVStub_LoadMemoryValidator6432() would load the Validator DLL, then start the Validator profiling the service.

The new implementation of svlMVStub_LoadMemoryValidator() and svlMVStub_LoadMemoryValidator6432() just loads the Validator DLL. It does not start the Validator profiling the service.

The reason for this change is that we wanted to allow the ability to set a service manager callback so that the service control manager could be informed about a long running instrumentation phase as the Validator is started. This is important because a delay of more than 10 seconds will cause the service control manager to kill the service.

To set the service callback call the svlMVStub_SetServiceCallback() function.

Then start the Validator with svlMVStub_StartMemoryValidator().

Loading, Starting, Unloading the Validator

To control the loading, starting and unloading of the Validator into the service, use these functions.

extern "C" SVL_SERVICE_ERROR svlMVStub_LoadMemoryValidator();

extern "C" SVL_SERVICE_ERROR svlMVStub_LoadMemoryValidator6432();

extern "C" SVL_SERVICE_ERROR svlMVStub_StartMemoryValidator();

extern "C" SVL_SERVICE_ERROR svlMVStub_UnloadMemoryValidator();

extern "C" SVL_SERVICE_ERROR svlMVStub_SetServiceCallback(serviceCallback_FUNC callback,
                                                          void*                userParam);


extern "C" SVL_SERVICE_ERROR svlMVStub_LoadMemoryValidator();

To load the Validator DLL into a 32 bit service when you are working with a 32 bit Validator, or to load the Validator DLL into a 64 bit service when you are working with a 64 bit Validator, use svlMVStub_LoadMemoryValidator().


extern "C" SVL_SERVICE_ERROR svlMVStub_LoadMemoryValidator6432();

To load the Validator DLL into a 32 bit service when you are working with a 64 bit Validator, use svlMVStub_LoadMemoryValidator6432().


extern "C" SVL_SERVICE_ERROR svlMVStub_SetServiceCallback(serviceCallback_FUNC callback,
                                                          void*                userParam);

Once you have successfully loaded the Validator DLL you can setup a service callback so that the service control manager can be kept updated during the process of starting the Validator.

When a service is starting, Windows requires the service to inform the Service Control Manager (SCM) that is starting at least every ten seconds. Failure to do so results in Windows concluding that the service has failed to start, and the service is terminated. Instrumenting your service may take more than 10 seconds, depending on the complexity and size of your service.

The solution is for the Validator to periodically call a user supplied callback from which you can regularly inform the SCM of the appropriate status.

Here is an example callback which ignores the userParam.

   void serviceCallback(void   *userParam)
       static DWORD dwCheckPoint = 1;
       ssStatus.dwServiceType = SERVICE_WIN32_OWN_PROCESS;
       ssStatus.dwServiceSpecificExitCode = 0;

       ssStatus.dwControlsAccepted = 0;
       ssStatus.dwCurrentState = dwCurrentState;
       ssStatus.dwWin32ExitCode = dwWin32ExitCode;
       ssStatus.dwWaitHint = dwWaitHint;
       ssStatus.dwCheckPoint = dwCheckPoint++;
       // Report the status of the service to the service control manager.

       return SetServiceStatus(sshStatusHandle, &ssStatus);


extern "C" SVL_SERVICE_ERROR svlMVStub_StartMemoryValidator();

Once you have successfully loaded the Memory Validator DLL you can start the Validator inspecting the service by calling svlMVStub_StartMemoryValidator().


extern "C" SVL_SERVICE_ERROR svlMVStub_UnloadMemoryValidator();

To unload the Validator DLL from your service call svlMVStub_UnloadMemoryValidator(), do not call FreeLibrary(). svlMVStub_UnloadMemoryValidator() shuts down various communications threads and removes any hooks that the Validator may have installed.

Debugging functions

We have added the following debugging functions to the NT Service API.

Before using these functions you must first set the log file name, otherwise the other debugging functions will do nothing.

The log file is opened for each write to the log file then closed afterwards. As such these log file functions are slow. Do not use these functions in any performance sensitive task. They are intended to allow you get debugging information out of a service when you’re trying to work out why something isn’t working, should you have problems.

extern "C" void svlMVStub_setLogFileName(const wchar_t* fileName);

extern "C" void svlMVStub_deleteLogFile();

extern "C" void svlMVStub_writeToLogFileA(const char* text);

extern "C" void svlMVStub_writeToLogFileW(const wchar_t* text);

extern "C" void svlMVStub_writeToLogFile(SVL_SERVICE_ERROR errCode);

extern "C" void svlMVStub_writeToLogFileLastError(DWORD errCode);

extern "C" void svlMVStub_dumpPathToLogFile();


To use any of the following logging functions you first need to set the name of the filename used for logging.
Setting this filename also sets the filename used by some of these API functions – you will find additional logging data from those functions that will help debug any issues with the service.

extern "C" void svlMVStub_setLogFileName(const wchar_t* fileName);


extern "C" void svlMVStub_deleteLogFile();

This function deletes the log file.


extern "C" void svlMVStub_writeToLogFileA(const char* text);

Call this function to write a standard ANSI character string to the log file.


extern "C" void svlMVStub_writeToLogFileW(const wchar_t* text);

Call this function to write a unicode character string to the log file. The characters are cast to ANSI prior to writing. As such you can’t write Korean to the log file. This is simply a convenience function to write wide characters as simply as ANSI characters.


extern "C" void svlMVStub_writeToLogFile(SVL_SERVICE_ERROR errCode);

Call this function to write a human readable description of the SVL_SERVICE_ERROR error code to the log file.


extern "C" void svlMVStub_writeToLogFileLastError(DWORD errCode);

Call this function to write a human readable description of the Windows error code to the log file.


extern "C" void svlMVStub_dumpPathToLogFile();

Call this function to write the contents of the PATH environment variable to the log file. This can be useful if you want to know what the search path is when trying to debug why a DLL wasn’t found during an attempt to load the Validator DLL.

NT Service API for each Validator

C++ Bug Validator

extern "C" SVL_SERVICE_ERROR svlBVStub_LoadBugValidator();

extern "C" SVL_SERVICE_ERROR svlBVStub_LoadBugValidator6432();

extern "C" SVL_SERVICE_ERROR svlBVStub_StartBugValidator();

extern "C" SVL_SERVICE_ERROR svlBVStub_UnloadBugValidator();

extern "C" SVL_SERVICE_ERROR svlBVStub_SetServiceCallback(serviceCallback_FUNC callback,
                                                          void*                userParam);

// debugging functions

extern "C" void svlBVStub_setLogFileName(const wchar_t* fileName);

extern "C" void svlBVStub_deleteLogFile();

extern "C" void svlBVStub_writeToLogFileA(const char* text);

extern "C" void svlBVStub_writeToLogFileW(const wchar_t* text);

extern "C" void svlBVStub_writeToLogFile(SVL_SERVICE_ERROR errCode);

extern "C" void svlBVStub_writeToLogFileLastError(DWORD errCode);

extern "C" void svlBVStub_dumpPathToLogFile();

C++ Coverage Validator

extern "C" SVL_SERVICE_ERROR svlCVStub_LoadCoverageValidator();

extern "C" SVL_SERVICE_ERROR svlCVStub_LoadCoverageValidator6432();

extern "C" SVL_SERVICE_ERROR svlCVStub_StartCoverageValidator();

extern "C" SVL_SERVICE_ERROR svlCVStub_UnloadCoverageValidator();

extern "C" SVL_SERVICE_ERROR svlCVStub_SetServiceCallback(serviceCallback_FUNC callback,
                                                          void*                userParam);

extern "C" void svlCVStub_setLogFileName(const wchar_t* fileName);

extern "C" void svlCVStub_deleteLogFile();

extern "C" void svlCVStub_writeToLogFileA(const char* text);

extern "C" void svlCVStub_writeToLogFileW(const wchar_t* text);

extern "C" void svlCVStub_writeToLogFile(SVL_SERVICE_ERROR errCode);

extern "C" void svlCVStub_writeToLogFileLastError(DWORD errCode);

extern "C" void svlCVStub_dumpPathToLogFile();

C++ Memory Validator

extern "C" SVL_SERVICE_ERROR svlMVStub_LoadMemoryValidator();

extern "C" SVL_SERVICE_ERROR svlMVStub_LoadMemoryValidator6432();

extern "C" SVL_SERVICE_ERROR svlMVStub_StartMemoryValidator();

extern "C" SVL_SERVICE_ERROR svlMVStub_UnloadMemoryValidator();

extern "C" SVL_SERVICE_ERROR svlMVStub_SetServiceCallback(serviceCallback_FUNC callback,
                                                          void*                userParam);

extern "C" void svlMVStub_setLogFileName(const wchar_t* fileName);

extern "C" void svlMVStub_deleteLogFile();

extern "C" void svlMVStub_writeToLogFileA(const char* text);

extern "C" void svlMVStub_writeToLogFileW(const wchar_t* text);

extern "C" void svlMVStub_writeToLogFile(SVL_SERVICE_ERROR errCode);

extern "C" void svlMVStub_writeToLogFileLastError(DWORD errCode);

extern "C" void svlMVStub_dumpPathToLogFile();

C++ Performance Validator

extern "C" SVL_SERVICE_ERROR svlPVStub_LoadPerformanceValidator();

extern "C" SVL_SERVICE_ERROR svlPVStub_LoadPerformanceValidator6432();

extern "C" SVL_SERVICE_ERROR svlPVStub_StartPerformanceValidator();

extern "C" SVL_SERVICE_ERROR svlPVStub_UnloadPerformanceValidator();

extern "C" SVL_SERVICE_ERROR svlPVStub_SetServiceCallback(serviceCallback_FUNC callback,
                                                          void*                userParam);

extern "C" void svlPVStub_setLogFileName(const wchar_t* fileName);

extern "C" void svlPVStub_deleteLogFile();

extern "C" void svlPVStub_writeToLogFileA(const char* text);

extern "C" void svlPVStub_writeToLogFileW(const wchar_t* text);

extern "C" void svlPVStub_writeToLogFile(SVL_SERVICE_ERROR errCode);

extern "C" void svlPVStub_writeToLogFileLastError(DWORD errCode);

extern "C" void svlPVStub_dumpPathToLogFile();

C++ Thread Validator

extern "C" SVL_SERVICE_ERROR svlTVStub_LoadThreadValidator();

extern "C" SVL_SERVICE_ERROR svlTVStub_LoadThreadValidator6432();

extern "C" SVL_SERVICE_ERROR svlTVStub_StartThreadValidator();

extern "C" SVL_SERVICE_ERROR svlTVStub_UnloadThreadValidator();

extern "C" SVL_SERVICE_ERROR svlTVStub_SetServiceCallback(serviceCallback_FUNC callback,
                                                          void*                userParam);

extern "C" void svlTVStub_setLogFileName(const wchar_t* fileName);

extern "C" void svlTVStub_deleteLogFile();

extern "C" void svlTVStub_writeToLogFileA(const char* text);

extern "C" void svlTVStub_writeToLogFileW(const wchar_t* text);

extern "C" void svlTVStub_writeToLogFile(SVL_SERVICE_ERROR errCode);

extern "C" void svlTVStub_writeToLogFileLastError(DWORD errCode);

extern "C" void svlTVStub_dumpPathToLogFile();

Panorama Theme by Themocracy