x64 Porting gotcha #3. Serializing collections

By , April 22, 2012 1:21 pm

When Microsoft ported MFC to 64 bits they also changed the return type for the GetSize() and GetCount() methods in the collection classes. They changed the return type from the 32 bit DWORD to the 64 bit DWORD_PTR on x64. This has implications if you write your own collection serialization methods. For example if you use a CMap<> to map a thread id to an object you will want to write your own serialization code.

For example (error checking removed for simplification), consider the serialization of this collection.

	CMap		threadObjectStatistics;


	CSingleLock	lock(&threadObjectStatisticsSect, TRUE);

	ar << threadObjectStatistics.GetCount();

	pos = threadObjectStatistics.GetStartPosition();
	while(pos != NULL)
		runningObjectManager	*rom = NULL;
		DWORD			threadID;
		threadObjectStatistics.GetNextAssoc(pos, threadID, rom);
		ar << threadID;


	CSingleLock	lock(&threadObjectStatisticsSect, TRUE);
	DWORD		i, count;

	ar >> count;
	for(i = 0; i < count; i++)
		runningObjectManager	*rom = new runningObjectManager();
		DWORD			threadID;
		ar >> threadID;
		threadObjectStatistics.SetAt(threadID, rom);

In the above code the first item saved/loaded is the number of objects in the CMap. After that the thread id and the complex object associated with the type is saved/loaded for each pair of objects in the CMap. The code above uses a DWORD to load the size. This won’t work for x64 because the count of objects is taken directly from the GetCount() method (or GetSize() for some collection types).

	ar << threadObjectStatistics.GetCount();

x86, return type is DWORD, count is saved as DWORD (32 bit)

x64, return type is DWORD_PTR, count is saved as DWORD_PTR (64 bit)

This is a problem because the loading code is expecting a DWORD.

	DWORD		i, count;

	ar >> count;

Update (23/4/2012): Turns out the same issue affects the STL collections as well. If you are directly serializing the result from the size() method in an STL collection you will be faced with the same problem as I describe for MFC.

Solution 1

One solution is simply to change the type being loaded from a DWORD to a DWORD_PTR.

	DWORD_PTR	i, count;

	ar >> count;

Solution 2

An alternative solution is to always save the size as a DWORD.

	DWORD		count;

	count = (DWORD)threadObjectStatistics.GetCount();
	ar << count;


You may think this is a trivial issue and why write a blog post about it? I agree the actual problem is trivial and the fix for it is also trivial. However the fact that you have a mismatch in datatypes being serialized because you directly serialized the return value from GetCount() is not so obvious. So much so that this particular issue escaped our attention (and got past a static analyser) until today.

So yes, its a trivial problem but its a hidden problem and it will cause all sorts of issues during serialization and when you go looking for it you'll probably look straight past it for a while. Hopefully I've just saved you a few hours of banging your head on a brick wall, or more likely your desk.

Leave a Reply

Panorama Theme by Themocracy