Category: Porting to Win64

64 bit tools leaving beta this month

By , December 5, 2013 1:14 pm

For those of you keeping a keen eye on the version numbers of the various C++ 64 bit betas we have will have noticed that the betas now have the same version number as their 32 bit counterparts. The reason for this is that we are getting ready for these tools to leave beta and to be released. We expect this to be during December 2013, if all goes well in the next week or so.

Before we can do this we need to make some modifications to the website, the shopping cart, test it all still works correctly, etc.

Once released the 32 bit and 64 bit C++ tools will have the same version number.

The 32 bit tools will continue to work with 32 bit executables on both 32 bit and 64 bit operating systems.

The 64 bit tools will only work with 64 bit executables on 64 bit operating systems.

The 64 bit tools will come bundled with the 32 bit version of the tool so that you can work with 32 bit or 64 bit executables on 32 bit and 64 bit operating systems. We anticipate a future version of the 64 bit tools that can launch and monitor 32 bit executables. This will allow you to collect more data and examine much larger datasets when testing 32 bit executables. We have prototypes of these tools now, but they are not ready for public release yet.

Why so long?

I know its been a very long time for these tools to have been in beta. We could have released some of them some time ago but we wanted to release all four at the same time so that we could also offer suites of tools and the also the combined 32+64 bundles. The problem was that C++ Memory Validator x64 had some problems with a few products being beta tested and C++ Performance Validator x64 also had some problems with SolidWorks 64 bit. We fixed the C++ Performance Validator x64 problem a few months ago and have been working on nailing all the remaining C++ Memory Validator x64 bugs.

Two beta testers in particular deserve a special mention for really helping us with these last bugs. Ciro Ettorre and Kevin Ernst. Ciro provided many logs of data for me to stare at and Kevin provided us with a machine that we could use to repeatedly try new fixes on.

I guess I didn’t really follow the advice I give to many people: “Get your product out there”. But that’s because these are new versions of existing products and I wanted them to be as good as the existing 32 bit tools. I hope we do not disappoint you.


I’d also like to recommend TeamViewer as a very useful product for remote working. We couldn’t have done the work with Kevin without this excellent tool. Best used with a headset microphone with the Voice over IP part of TeamViewer turned on.

What next?

As always we’ll keep you updated with what is happening with the betas via the email address associated with your beta test account.

x64 Porting gotcha #3. Serializing collections

By , April 22, 2012 1:21 pm

When Microsoft ported MFC to 64 bits they also changed the return type for the GetSize() and GetCount() methods in the collection classes. They changed the return type from the 32 bit DWORD to the 64 bit DWORD_PTR on x64. This has implications if you write your own collection serialization methods. For example if you use a CMap<> to map a thread id to an object you will want to write your own serialization code.

For example (error checking removed for simplification), consider the serialization of this collection.

	CMap		threadObjectStatistics;


	CSingleLock	lock(&threadObjectStatisticsSect, TRUE);

	ar << threadObjectStatistics.GetCount();

	pos = threadObjectStatistics.GetStartPosition();
	while(pos != NULL)
		runningObjectManager	*rom = NULL;
		DWORD			threadID;
		threadObjectStatistics.GetNextAssoc(pos, threadID, rom);
		ar << threadID;


	CSingleLock	lock(&threadObjectStatisticsSect, TRUE);
	DWORD		i, count;

	ar >> count;
	for(i = 0; i < count; i++)
		runningObjectManager	*rom = new runningObjectManager();
		DWORD			threadID;
		ar >> threadID;
		threadObjectStatistics.SetAt(threadID, rom);

In the above code the first item saved/loaded is the number of objects in the CMap. After that the thread id and the complex object associated with the type is saved/loaded for each pair of objects in the CMap. The code above uses a DWORD to load the size. This won’t work for x64 because the count of objects is taken directly from the GetCount() method (or GetSize() for some collection types).

	ar << threadObjectStatistics.GetCount();

x86, return type is DWORD, count is saved as DWORD (32 bit)

x64, return type is DWORD_PTR, count is saved as DWORD_PTR (64 bit)

This is a problem because the loading code is expecting a DWORD.

	DWORD		i, count;

	ar >> count;

Update (23/4/2012): Turns out the same issue affects the STL collections as well. If you are directly serializing the result from the size() method in an STL collection you will be faced with the same problem as I describe for MFC.

Solution 1

One solution is simply to change the type being loaded from a DWORD to a DWORD_PTR.

	DWORD_PTR	i, count;

	ar >> count;

Solution 2

An alternative solution is to always save the size as a DWORD.

	DWORD		count;

	count = (DWORD)threadObjectStatistics.GetCount();
	ar << count;


You may think this is a trivial issue and why write a blog post about it? I agree the actual problem is trivial and the fix for it is also trivial. However the fact that you have a mismatch in datatypes being serialized because you directly serialized the return value from GetCount() is not so obvious. So much so that this particular issue escaped our attention (and got past a static analyser) until today.

So yes, its a trivial problem but its a hidden problem and it will cause all sorts of issues during serialization and when you go looking for it you'll probably look straight past it for a while. Hopefully I've just saved you a few hours of banging your head on a brick wall, or more likely your desk.

64 bit porting gotcha #2! x64 Register preservation

By , March 9, 2012 8:01 pm

In a previous article on x64 development I mentioned the problem of aligning the callstack on 16 byte boundaries and what happens if you do not do this.

Why 16 bytes?

At the time it seemed odd to me that the stack had to be 16 byte aligned. Why? All the parameters are 8 bytes (64 bits) wide and the first four are passed in registers. Everything else spills onto the stack and anything larger is passed as a reference. Floating point values are passed in dedicated floating point registers.

And there lies the key. That last sentence. The floating point registers.

Floating point on x64 is not done using the floating point coprocessor instructions. Instead the SSE instruction sets (and its extensions) are used.

If everything floats, whats the point?

If you are just hooking x64 functions and possibly collecting callstack you may never know need to know about floating point register preservation. We managed to get all four of our C++ tools (coverage, memory, profiler and deadlock detector) functional without knowing. Why would we need to know? Floating point preservation was never important for x86 because we could never damage the floating point without trying to do so.

But when we got into the details of the last bugs that were killing us we noticed seemingly random crashes. Closer investigation showed that calls to Win32 API functions had a tendency to wipe out the floating point registers. And that is when we got interested in what happens if we preserved the x64 floating point registers.

How to view the registers?

At first glance this wasn’t obvious to me. The Registers window in Visual Studio just shows the registers from RAX through R15 etc. However if you right click this window there is a helpful context menu that allows you to choose just how much information you display in this window.

Once you have the registers in view things get a lot easier inside the debugger. You can step through your code ensuring that nothing is getting trampled on until Viola! the floating point registers get totally hosed. A bit more investigation and you realise that seemingly innocent call you had in your code contains a call to a Win32 function (for example VirtualProtect) and that that function is responsible for the register death.

OK, so how do we preserve registers on x64? Its nothing like on x86.

x64 floating point preservation

The x64 designers in their infinite wisdom took away two very useful instructions (pushad and popad). As a result x64 hook writers now have to push lots of registers and pop lots of registers at the start and end of each hook. You can even see this in parts of the Windows operating system DLLs. So much simpler just to push everything and pop everything.

However what the Lord taketh away he can give back. And the x64 designers did that by providing two dedicated instructions for saving and restoring floating point. fxsave and fxrstor. These instructions take one operand each. The operand must point to a 512 byte chunk of memory which is 16 byte aligned.

A common usage would be as shown below although you can use any register as the destination location. It just so happens that the stack pointer (rsp) is the most common usage.

	sub	rsp, 200h;
	fxsave	[rsp];

	.. do your task that damages the floating point registers

	fxrstor	[rsp;]
	add	rsp, 200h;

When you see the above usage you can see why there is the requirement for the stack to be 16 byte aligned. Why 16 bytes? I suspect it is because it is the start of a cache line and that makes executing the instruction *SO* much quicker.


So now you know why the x64 callstack is 16 byte aligned. Its all to do with ensuring your code executes as fast as possible, especially when executing a memory intensive register copy when saving and restoring the x64 floating point registers. I’ve also shown you how to preserve and restore the floating point registers.

Removing annoying ‘ceil’ : attributes not present on previous declaration warning C4985

By , September 15, 2010 5:36 pm

A bug in Visual Studio 2008 when compiling for 64 bit code results in a rather odd warning message. The message is typically shown for the ‘ceil’ mathematical function. The message looks a bit like this:

C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\math.h(136) : warning C4
985: 'ceil': attributes not present on previous declaration.
        C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\intrin.h(142) :
see declaration of 'ceil'

The message occurs when you include both intrin.h and math.h into the same file (either directly or indirectly).
The message is harmless but annoying if you rely on warning free builds as one of your indicators of build quality.

How do you get rid of this warning?

Solution #1: Service Pack

Install a service pack for Visual Studio 2008.
I am not sure which service pack has the fix for this particular bug.
Depending upon which version of Visual Studio 2008 you are using there are many results.

Solution #2: #pragma in stdafx.h

If installing a service pack is not an option you can use, you can simply disable the warning using a #pragma directive in your application’s stdafx.h file.
Add the following line to the stdafx.h. You should find that the warnings are suppressed for any files including that stdafx.h

#pragma warning (disable: 4985)

Solution #3: #pragma in appropriate files

If you don’t want the one-size fits all approach offered by using stdafx.h to suppress this warning you can find where math.h is being included (if intrin.h is included first) or where intrin.h is included (if math.h is included first) and suppress the message during the include. Here is how you do it:

Assume I’ve already included intrin.h

#pragma warning (push)
#pragma warning (disable: 4985) // disable the warning message during the include
#include <math.h>               // this is where I would normally get the warning message
#pragma warning (pop)

Assume I’ve already included math.h

#pragma warning (push)
#pragma warning (disable: 4985) // disable the warning message during the include
#include <intrin.h>             // this is where I would normally get the warning message
#pragma warning (pop)

Thread Validator x64 enters BETA

By , August 6, 2010 8:57 am

Thread Validator x64 is now available for beta testing.

Thread locking history

Thread Validator x64 is the 64 bit version of our successful 32 bit Thread Validator software tool that runs on Microsoft Windows operating systems. Thread Validator x64 is a deadlock detection and thread analysis software tool, running on Windows 7 64 bit, Windows Vista 64 bit and Windows XP 64 bit.

Thread Validator has multiple displays to provide you with different perspectives onto the data you have collected.

What does Thread Validator do?

Thread Validator x64 identifies thread deadlocks, potential deadlocks and locks with a high contention rate.

Thread deadlocks usually mean that one or more threads can no longer function correctly because they are waiting on a lock that will never be released. This is an error condition and usually manifests as an unresponsive computer program.

Potential deadlocks are locking sequences that have not triggered a deadlock but may lead to a deadlock under slightly different conditions.

High contention rate locks result in your program spending too much time waiting for access to a lock. A different program design can often reduce a high contention rate to a less demanding contention rate.

How does Thread Validator work?

Thread Validator instruments your computer program so that it can monitor the appropriate synchronization APIs used to control access to locks, mutexes, semaphores and wait conditions. Using the information gained from monitoring these APIs, Thread Validator can calculate deadlock conditions, potential deadlock conditions and detect locks with high contention rates.

Thread Validator gathers data for all locks, all threads, all mutexes, all semaphores and all wait conditions. The data is organised into various displays allowing you to view information:

  • All active locks.
  • All active locks, organized by thread.
  • All locks that are locked at a given time.
  • Allocation information for all allocated synchronization objects, showing callstack and source code.
  • Thread locking history. View all threads, see what each threads is doing and when.
  • Thread lock order. View the order locks are acquired across threads for a given lock sequence.
  • List of all application objects that can be used in wait conditions.
  • How Thread Validator helps you be more productive

    Thread Validator x64 can help you:

    • Identify deadlocks in your application – quickly identify and fix hard threading problems.
    • Identify potential deadlocks in your application – prevent problems before they get serious.
    • Identify busy contended critical sections in your application – improve performance.
    • View thread locking behaviour in real time.
    • Improve your software quality by modifying your threading behaviour.
    • View all open handles that your application can wait on.

    Join the beta test

    If you are developing 64 bit software and have some multi-threading problems you would like to analyze, please join the beta, analyze your multi-threading problems and let us know your thoughts.

64 bit porting gotcha #1! x64 Datatype misalignment.

By , June 17, 2010 11:10 am

Datatype misalignment, there is a topic so interesting you’d probably prefer to watch paint dry.

But! There are serious consequences for getting it wrong. So perhaps you’d better read about it after all 🙂

The problem that wasted my time

Why am I writing about datatype misalignment? Because its just eaten two days of my time and if what I share with you helps save you from such trouble, all the better.

The problem I was chasing was that three calls to CreateThread() were failing. All calls were failing with ERROR_NOACCESS. They would only fail if called from functions in the Thread Validator x64 profiling DLL injected into the target x64 application. If the same functions were called later in the application (via a Win32 API hook or directly from the target application) the functions would work. That meant that the input parameters were correct.

Lots of head scratching and trying many, many variations of input parameters and asking questions on Stack Overflow and we were stuck. I could only think it was to do with the callstack but I had no idea why. So I started investigating the value of RSP during the various calls. The investigating what happened if I pushed more data onto the stack to affect the stack pointer. After some trial and error I found a combination that worked. Then I experimented with that combination to determine if it was the values being pushed that were important or the actual value of the stack pointer that was important.

At this point I was confused, as I didn’t know about any stack alignment requirements, I only knew about data alignment requirements. I then went searching for appropriate information about stack alignments and found this handy document from Microsoft clarifies that.

What is datatype misalignment?

Datatype alignment is when the data read by the CPU falls on the natural datatype boundary of the datatype. For example, when you read a DWORD and the DWORD is aligned on a 4 byte boundary.

In the following code examples, let us assume that data points to a location aligned on a four byte boundary.

void aligned(BYTE	*data)
	DWORD	dw;

	dw = *(DWORD *)&bp[4];

Datatype misalignment is when the data read by the CPU does not fall on the natural datatype boundary of the datatype. For example, when you read a DWORD and the DWORD is not aligned on a 4 byte boundary.

void misaligned(BYTE	*data)
	DWORD	dw;

	dw = *(DWORD *)&bp[5];

Why should I care about datatype misalignment?

Aligned data reads and data writes happen at the maximum speed the memory subsystem and processor can provide. For example to read an aligned DWORD, one 32 bit data read needs to be performed.

	BYTE	[ 1];
	BYTE	[ 2];
	BYTE	[ 3];
DWORD	BYTE	[ 4];	// ignored
	BYTE	[ 5];	// read
	BYTE	[ 6];	// read
	BYTE	[ 7];	// read
DWORD	BYTE	[ 8];	// read
	BYTE	[ 9];	// ignored
	BYTE	[10];	// ignored
	BYTE	[11];	// ignored
	BYTE	[13];
	BYTE	[14];
	BYTE	[15];

Misligned data reads and data writes do not happen at the maximum speed the memory subsystem and processor can provide. For example to read a misaligned DWORD the processor has to fetch data for the two 32 bit words that the misaligned data straddles.

In the misaligned example shown above, the read happens at offset 5 in the input array. I’ve shown the input array first 16 bytes, marking where each DWORD starts and showing which bytes are read and which are ignored. If we assume the input array is aligned then the DWORD being read has 3 byte2 in the first DWORD and 1 bytes in the second DWORD. The processor has to read both DWORDs, then shuffle the bytes around, discarding the first byte from the first DWORD and discarding the last 3 bytes from the last DWORD, then combining the remaining bytes to form the requested DWORD.

Performance tests on 32 bit x86 processors shown performance drops of between 2x and 3x. Thats quite a hit. On some other architectures, the performance hit can be much worse. This largely depends on if the processor does the rearrangement (as on x86 and x64 processors) or if an operating system exception handler handles it for you (much slower).

I’ve shown the example with DWORDs, because the they are short enough to be easily shown in a diagram whereas 8 byte or larger values would be unweildy.

The above comments also apply to 8 byte values such as doubles, __int64, DWORD_PTR (on x64), etc.

Clearly, getting your datatype alignments optimized can be very handy in performance terms. Niave porting from 32 bit to 64 bit will not necessarily get you there. You may need to reorganise the order of some data members in your structures. We’ve had to do that with Thread Validator x64.

Not just performance problems either!

In addition to the performance problems mentioned above there is another, more important consideration to be aware of: On x64 Windows operating systems you must have the stackframe correctly aligned. Correct stack alignment on x64 systems means that the stack frame must be aligned on a 16 byte boundary.

Failure to ensure that this is the case will mean that a few Windows API calls will fail with the cryptic error code ERROR_NOACCESS (hex: 0x3E6, decimal: 998). This means “Invalid access to memory location”.

The problem with the error code ERROR_NOACCESS in this case is that the real error code that gets converted into ERROR_NOACCESS is STATUS_DATATYPE_MISALIGNMENT, which tells you a lot more. I spent quite a bit of time digging around until I found the true error code for the bug I was chasing that lead me to write this article.

If you are writing code using a compiler, the compiler will sort the stack alignment details out for you. However if you are writing code in assembler, or writing hooks using dynamically created machine language, you need to be aware of the 16 byte stack alignment requirement.

x64 datatype alignment requirements

Correct stack alignment on x64 systems means that the stack frame must be aligned on a 16 byte boundary.


Size Alignment
1 1
2 2
4 4
8 8
10 10
16 16

Anything larger than 8 bytes is aligned on the next power of 2 boundary.


Correct stack frame alignment is essential to ensure calling functions works reliably.

Correct datatype alignment is essential for maximum speed when accessing data.

Failure to align stack frames correctly could lead to Win32 API calls failing and or program failure or lack of correct behaviour.

Failure to align data correctly will lead to slow speed when accessing data. This could be disasterous for your application, depending upon what it is doing.


Microsoft x86/x64/IA64 alignment document.

First steps to 64 bit

By , June 10, 2010 6:23 pm

Today Thread Validator 64 bit took it’s first steps on its 64 bit training wheels.

Interesting problems including

  • DLL Hooking
  • DLL Import structure walking
  • Injecting
  • PE 64 bit DLL inspection

The first attempt at launching a 64 bit application and injecting into it worked. Awesome! First time out and something that hard to do worked first time. We’ve got a few datastructure size mismatches and a stack walking problem to look at, but looks like our first 64 bit port is going well.

Check back on the blog for progress or follow us on twitter. We’ll be announcing the progress of our various 64 bit ports on both the blog and twitter.

How to do cpuid and rdtsc on 32 bit and 64 bit Windows

By , June 4, 2010 3:57 pm

With the introduction of WIN64 the C++ compiler has many improvements and certain restrictions. One of those restrictions is no inline assembly code. For those few of us that write hooking software this is a real inconvenience. Inline assembly is also useful for adding little snippets of code to access hardware registers that are not so easy to access from C or C++.

The 64 bit compiler also introduces some intrinsics which are defined in intrinsic.h. These intrinsics allow you to add calls to low level functionality in your 64 bit code. Such functionality includes setting breakpoints, getting CPU and hardware information (cpuid instruction) and reading the hardware timestamp counter.

In this article I’ll show you how you can use the same code for both 32 bit and 64 bit builds to have access to these intrinsics on both platforms.


The 64 bit compiler provides a convenient way for you to hard code breakpoints into your code. Often very useful for putting breakpoints in your code during testing. The __debugbreak() intrinsic provides this functionality.

There is no 32 bit __debugbreak();

For 32 bit systems you have to know 80386 assembly. The breakpoint instruction as opcode 0xcc. The inline assembly for this is __asm int 3;

	#define __debugbreak()				__asm { int 3 }

cpuid – 32 bit

On 32 bit systems there is no cpuid assembly instruction so you have to use the emit directive.

	#define cpuid	__asm __emit 0fh __asm __emit 0a2h

and then you can use cpuid anywhere you need to.

	void doCpuid()
		__asm pushad;		// save all the registers - cpuid trashes EAX, EBX, ECX, EDX
		__asm mov eax, 0; 	// get simplest cpuid data
		cpuid;			// call cpuid, results returned in EAX, EBX, ECX, EDX
		// read cpuid results here before restoring the registers...

		__asm popad;		// restore registers

cpuid – 64 bit

On 64 bit systems you have to use the intrinsic __cpuid(registers, 0) provided in the intrinsic.h file.

	void doCpuid()
	    int registers[4];

	    __cpuid(registers, 0);

rdtsc – 32 bit

On 32 bit systems there is no rdtsc assembly instruction so you have to use the emit directive.

	#define rdtsc	__asm __emit 0fh __asm __emit 031h

and then you can use rdtsc anywhere you need to.

	__int64 getTimeStamp()


	    __asm	mov	li.LowPart, eax;
	    __asm	mov	li.HighPart, edx;
	    return li.QuadPart;

rdtsc – 64 bit

On 64 bit systems you have to use the intrinsic __rdtsc() provided in the intrinsic.h file.

	__int64 getTimeStamp()
	    return __rdtsc();

Thats not very portable is it?

The problem with the above approach is that you end up having two implementations for these functions – one for your 32 bit build and one for your 64 bit build. It would be much more elegant to have a drop in replacement that you can use in your 32 bit code that will compile in the same manner as the 64 bit code that uses the intrinsics defined in intrinsic.h

Here is how you do it. Put all of the code below into a header file and #include that header file wherever you need access to __debugbreak(), __cpuid() or rdtsc().

#ifdef _WIN64
#else	// _WIN64
	// x86 architecture

	// __debugbreak()

	#if     _MSC_VER >= 1300
		// Win32, __debugbreak defined for VC2005 onwards
	#else	//_MSC_VER >= 1300
		// define for before VC 2005

		#define __debugbreak()				__asm { int 3 }
	#endif	//_MSC_VER >= 1300

	// __cpuid(registers, type)
	//		registers is int[4], 
	//		type = 0

	// DO NOT add ";" after each instruction - it screws up the code generation

	#define rdtsc	__asm __emit 0fh __asm __emit 031h
	#define cpuid	__asm __emit 0fh __asm __emit 0a2h

	inline void __cpuid(int	cpuInfo[4], 
						int	cpuType)
		__asm pushad;
		__asm mov	eax, cpuType;

		if (cpuInfo != NULL)
			__asm mov	cpuInfo[0], eax;
			__asm mov	cpuInfo[1], ebx;
			__asm mov	cpuInfo[2], ecx;
			__asm mov	cpuInfo[3], edx;

		__asm popad;

	// __rdtsc()

	inline unsigned __int64 __rdtsc()


		__asm	mov	li.LowPart, eax;
		__asm	mov	li.HighPart, edx;
		return li.QuadPart;

#endif	// _WIN64

Now you can just the the 64 bit style intrinsics in your code and not have to worry about any messy condition code for doing the 32 bit inline assembly or the 64 bit intrinsics. Much neater, elegant, readable and more maintainable.

Additional Information

If you wish to know more about __cpuid(), the parameters it takes and the values returned in the registers array, Microsoft have a __cpuid() instrinsic description which explains everything in great detail.

Cupid instruction

When I wrote this article I kept typing cupid instead of cpuid. I’m sure my mind was on something else :-). How would the cupid instruction be implemented…

Panorama Theme by Themocracy