Rss Feed
Tweeter button
Facebook button
Technorati button
Reddit button
Myspace button
Linkedin button
Webonews button
Delicious button
Digg button
Flickr button
Stumbleupon button
Newsvine button

Category: Hints and tips

How to output to stdout from an MFC program

By , January 31, 2017 1:00 pm

If you’ve ever developed an MFC program with a graphical user interface and then later thought that it would be really nice to also provide a command line version with output to stdout you’ve probably bumped into the problem that there is no stdout for these programs. The program isn’t attached to a console.

So how do you do it?

The secret to this is a Win32 API “AttachConsole” which is available from Windows XP onwards.
AttachConsole takes one argument, a DWORD identifying the process to which to attach. In our case we want to attach to the parent process, so we pass ATTACH_PARENT_PROCESS which is defined as (DWORD)-1.

AllocConsole(ATTACH_PARENT_PROCESS);

When you need to print to this console use _cprintf(), which is defined in conio.h.

But we’re still working with legacy systems!

If you need your code to work on old systems as well as modern systems you’ll need to use GetProcAddress() as shown below.

#ifndef ATTACH_PARENT_PROCESS
#define ATTACH_PARENT_PROCESS	((DWORD)-1)
#endif	//ATTACH_PARENT_PROCESS

typedef BOOL (WINAPI *AttachConsole_FUNC)(DWORD);

//-NAME---------------------------------
//.DESCRIPTION..........................
//.PARAMETERS...........................
//.RETURN.CODES.........................
//--------------------------------------

BOOL attachToProcessConsole(DWORD	processId)
{
	HMODULE				hMod;

	hMod = GetModuleHandle(_T("Kernel32.dll"));
	if (hMod != NULL)
	{
		AttachConsole_FUNC	func;

		func = (AttachConsole_FUNC)GetProcAddress(hMod, "AttachConsole");
		if (func != NULL)
		{
			// valid for Windows XP onwards

			return (*func)(processId);
		}
	}

	return FALSE;
}
Share

Working with Dev C++

By , December 14, 2016 5:57 pm

We’ve had a few people asking how to configure C++ Memory Validator to work with programs built using Dev C++. Dev C++ is an IDE used for developing programs with the MingW compiler.

We tested using this download of Dev C++.

Debug information

Any program built using the default Dev C++ settings will generate a binary image that contains debugging information that is in a format that our tools cannot read. The MingW compiler can create debug information in many formats, including COFF and STABS, both of which our tools support. You can turn these debugging formats on using the -gCoff and -gstabs flags. We recommend using STABS symbols.

Configuring Dev C++

Open the Project Options… dialog from the Project menu.
devcppprojectmenu

Choose the Parameters tab. Add the option -gstabs to all three columns. Click OK.
devcpp_debugoptions

Now that you have configured the debug options all you need to do is to rebuild your project to ensure the debug information is present.

Share

The correct way to determine if a file is a directory.

By , November 30, 2016 1:41 pm

After writing for Microsoft’s Windows platform for 20 years I thought I knew all I could know about GetFileAttributes(). Until I found a rather odd and subtle bug in some code that interacted with data supplied by the user of the software. A call would succeed that I expected to fail. Naturally this meant the software didn’t make the right choices and instead of being presented with an helpful dialog explaining what had failed, the software sat silently in a corner humming to itself waiting for the user to work out what had happened. The failure was that I was presenting incorrect data to GetFileAttributes() assuming that it would always fail for bad input. How wrong I was!

I thought I’d write up what can go wrong with GetFileAttributes().

It’s tempting to test if a file is a directory by writing code like this:

if ((GetFileAttributes(fileName) & FILE_ATTRIBUTE_DIRECTORY) != 0)
{
    // file is a directory
}

The above looks logically correct. But there are problems with it.

First, a refresher on file attribute values…

File Attributes

The list of defined file attributes is in WinNT.h. The values are shown below.

#define FILE_ATTRIBUTE_READONLY             0x00000001  
#define FILE_ATTRIBUTE_HIDDEN               0x00000002  
#define FILE_ATTRIBUTE_SYSTEM               0x00000004  
#define FILE_ATTRIBUTE_DIRECTORY            0x00000010  
#define FILE_ATTRIBUTE_ARCHIVE              0x00000020  
#define FILE_ATTRIBUTE_DEVICE               0x00000040  
#define FILE_ATTRIBUTE_NORMAL               0x00000080  
#define FILE_ATTRIBUTE_TEMPORARY            0x00000100  
#define FILE_ATTRIBUTE_SPARSE_FILE          0x00000200  
#define FILE_ATTRIBUTE_REPARSE_POINT        0x00000400  
#define FILE_ATTRIBUTE_COMPRESSED           0x00000800  
#define FILE_ATTRIBUTE_OFFLINE              0x00001000  
#define FILE_ATTRIBUTE_NOT_CONTENT_INDEXED  0x00002000  
#define FILE_ATTRIBUTE_ENCRYPTED            0x00004000  
#define FILE_ATTRIBUTE_VIRTUAL              0x00010000  

Rather strangely, the invalid attributes flag is defined in a different file, WinBase.h.

#define INVALID_FILE_ATTRIBUTES ((DWORD)-1)

Problem 1

What if GetFileAttributes() fails? If the file doesn’t exist, the call fails. If the filename specifies a computer name, the call fails. See GetFileAttributes() documentation for more informtion. When GetFileAttributes() fails it returns INVALID_FILE_ATTRIBUTES. This error status passes the above test. OK, so add an additional check and the code becomes

DWORD attribs;

attribs = GetFileAttributes(fileName);
if ((attribs != INVALID_FILE_ATTRIBUTES) &&
    ((attribs & FILE_ATTRIBUTE_DIRECTORY) != 0))
{
    // file is a directory
}

Problem 2

Even with the above file-does-not-exist problem solved there is another problem. The file could be a directory, but it could be a directory that you don’t want. For example what if you’ve allowed the user to specify the directory name and they typed _T(“/”), or what if your filename creation code has a bug in it that fails when passed an empty name, resulting in a calculated filename of _T(“\”). What then?

In these cases the following calls all return 0x16.

GetFileAttributes(_T("\\")); 
GetFileAttributes(_T("/"));

0x16 means hidden (0x02), system (0x04), directory (0x10).

It’s a reasonable bet that in your code, any code looking for a directory to use is probably not looking for a hidden directory and almost certainly not intending to use a system directory. OK, time for a new implementation.

DWORD attribs;

attribs = GetFileAttributes(fileName);
if ((attribs != INVALID_FILE_ATTRIBUTES) &&         // check if a valid file
    ((attribs & FILE_ATTRIBUTE_DIRECTORY) != 0) &&  // file is a directory
    ((attribs & FILE_ATTRIBUTE_HIDDEN) == 0) &&     // file is not hidden
    ((attribs & FILE_ATTRIBUTE_SYSTEM) == 0))       // file is not system
{
    // file is a directory that isn't hidden and isn't system
}

What about files, rather than directories?

It’s natural to think about implementing checks for if a filename identifies a file rather than a directory. You test for this in exactly the same way but looking for different attributes. You’ll want to exclude FILE_ATTRIBUTE_DIRECTORY and then depending on the job your code is doing you’ll want to consider excluding files depending upon the following attributes:

FILE_ATTRIBUTE_DEVICE
FILE_ATTRIBUTE_INTEGRITY_STREAM
FILE_ATTRIBUTE_OFFLINE
FILE_ATTRIBUTE_REPARSE_POINT
FILE_ATTRIBUTE_SPARSE_FILE
FILE_ATTRIBUTE_TEMPORARY
FILE_ATTRIBUTE_VIRTUAL

and of course, you might also want to consider FILE_ATTRIBUTE_HIDDEN and FILE_ATTRIBUTE_SYSTEM.

Additional reading

Microsoft documentation on GetFileAttributes().

Why is GetFileAttributes the way old-timers test file existence? Old New Thing.

Share

How to make your MFC (or non MFC) program support high DPI monitors. The easy way.

By , November 29, 2016 10:35 am

A few years ago Microsoft introduced support for very high resolution monitors. Such monitors might have nearly 4000 pixels horizontally and over 2000 pixels vertically. This provides wonderful scope for editing photographs and providing great graphics. But at your normal scaling the text on your displays starts looking a bit small. This is typical if you’ve specified your text height in pixels. Which from what I’ve seen is how a lot of people (most?) do it. “Oh, 14 look looks OK, I’ll go with that”. Looks fab for a 21 year. No so good for your customers over 40 though, most of them are wearing glasses, and so will you be when you get there (or contacts, you know what I mean).

This approach isn’t going to work with high DPI displays. You need to think in terms of point sizes for text, like you would if you are printing a document. Once you have an appropriate point size for your normal non-scaled (non high-DPI) display you can then just get the correct font height in pixels (based on the point size) and multiply by a scale factor to determine the correct column width for any columns in grids, or any graphics that need scaling.

Getting the scale factor

Non scaled monitors are deemed to be monitors that display at 96 DPI. From this the following table follows:

DPI scaling
96 100%
120 125%
144 150%
192 200%

You can calculate a scale factor for the whole system, or a scale factor per monitor – if you have monitors with different display facilities. The ability to calculate a scale factor per monitor only works on Windows 8.1 and Windows 10 onwards. You can find out more about this using the GetDpiForMonitor() function. For the purposes of this discussion we will only cover getting the system scale factor.

If you are thinking of doing per-monitor scaling, you might be well served by first doing system wide scaling, because it is simpler and will introduce you to most of the problems you will face, but in an easier to manage problem space than per-monitor scaling. Stepping from a working system wide scaling to per monitor scaling should be a lot easier than going from non-scaling to per-monitor scaling in one go.

double getSystemScaleFactor()				// returns 1.0 for no scaling
{
	// some simple caching to speed things up

	static int		calculatedScale = FALSE;
	static double	scale = 0;

	if (calculatedScale)
		return scale;

	// get DPI for the system

	HDC		hdc;

	hdc = ::GetDC(NULL);

	UINT	dpiX = 96;		// default DPI

	dpiX = GetDeviceCaps(hdc, LOGPIXELSX);

	scale = (double)dpiX / 96.0;
	if (scale < 1.0)
		scale = 1.0;		// prevent funny business with super large monitors returning very low DPI values

	::ReleaseDC(NULL, hdc);

	calculatedScale = TRUE;
	return scale;
}

We also need some supporting functions that will assist us during the following phases.


  • We need to be able to calculate a font point size based on a font pixel height.

  • We need to be able to calculate a font pixel height based on a font point size.

Getting a font point size

Win32

double getFontPointSize(HDC		hDC,
			DWORD	pixelSize)
{
	LONG	pixPerInch = GetDeviceCaps(hDC, LOGPIXELSY);
	double	points = (pixelSize * 72.0) / pixPerInch;

	return points;
}

MFC

double getFontPointSize(CDC		*dc,
			DWORD	pixelSize)
{
	if (dc == NULL)
	{
		return getFontPointSizeForDefaultDC(pixelSize);
	}
	else
	{
		return getFontPointSize(dc->GetSafeHdc(), pixelSize);
	}
}

Getting a font pixel height

LONG getFontHeightForDefaultDC(double	pointSize)		// returns font height for the appropriate screen
{
	// this function can be optimized by using a simple cache on the point size (worth it in some situations!)

	HDC		hdc;

	hdc = ::GetDC(NULL);
	height = getFontHeight(hdc, pointSize);
	::ReleaseDC(NULL, hdc);
}
LONG getFontHeight(HDC		hDC,
		   double	pointSize)		// returns font height for the appropriate screen
{
	LONG	pixPerInch = GetDeviceCaps(hDC, LOGPIXELSY);
	LONG	lfHeight = (LONG)((pointSize * (double)pixPerInch) / 72.0);

	return lfHeight;
}
LONG getFontHeight(CDC		*dc,			// can be NULL for default screen
		   double	pointSize)		// returns font height for the appropriate screen
{
	if (dc == NULL)
	{
		return getFontHeightForDefaultDC(pointSize);
	}
	else
	{
		return getFontHeight(dc->GetSafeHdc(), pointSize);
	}
}

Updating your code

Versions of MFC since Visual Studio 2010 support MFC scaling, although nothing happens by default. You still have to supply the scale factor for the fonts in use. These fonts will mainly be for any text drawn on graphics displays and for any owner drawn controls such as lists and grid controls which sit on top of huge amounts of data that can't be reliably stored inside the control. You need to find the places where you create fonts and set the font height. Often this will be when you create a font or when you set the lfHeight member of a LOGFONT structure. Once we have these locations we can update the code.

The sequence of operations to update your code is as follows:


  • You need to find each location where a font height is set.

  • For each location you need to calculate the point height and replace the integer height with a call to calculate a height based on a point height.

  • For each location verify that the calculated height is correct for a non-scaled display.

  • For each location verify that the font now scales with the display.

Because the new height calculated by getFontHeight() is based on a point size we don't need to worry about scaling the returned value with getSystemScaleFactor(), the point size is relative to how we view the display, so is automatically the correct scale.

CreateFont

Search for uses of the MFC method CFont::CreateFont(), or the Win32 function ::CreateFont(). The first parameter is the height of the font.

#define FONT_SIZE		16

...

font.CreateFont(FONT_SIZE,		// default height
		0,			// default width
		0,
		0,
		FW_NORMAL,
		FALSE,
		FALSE,
		FALSE,
		DEFAULT_CHARSET,
		OUT_TT_PRECIS,
		CLIP_DEFAULT_PRECIS,
		PROOF_QUALITY,
		FF_DONTCARE,
		_T("Arial"));

We need to know what point size of the specified font equates to our chosen (above) font height of 16 pixels. The font height may change according to the font face, italics, weight etc. So you do need to check this for each one and not just rely on it being what you think it is. This is why I haven't provided a table of values for you to plug in.

We can calculate the font point size using the following code, passing in the device context of the appropriate display.

double testPointSize;

testPointSize = getFontPointSize(hDC, FONT_SIZE);

Just add the code before the call to CreateFont, build it, put a breakpoint there and when it stops in the debugger make a note of the value testPointSize. In our tests, a point size of 11.0 gave us a pixel height of 16 (FONT_SIZE).

Once we know testPointSize we can update the code, as shown below. We notated our code with info about how it was tested to help anyone that ends up at this code wondering why these sizes are used. People won't always search the version history and disk space is cheap. Comments help. Use them.

LONG	lfHeight;

lfHeight = getFontHeight(GetDC(), 11.0);		// aiming for lfHeight == 16 (at "small" 1920x1080);

// set font for tab control

font.CreateFont(lfHeight,		// default height (used to be hard coded at 16)
		0,			// default width
		0,
		0,
		FW_NORMAL,
		FALSE,
		FALSE,
		FALSE,
		DEFAULT_CHARSET,
		OUT_TT_PRECIS,
		CLIP_DEFAULT_PRECIS,
		PROOF_QUALITY,
		FF_DONTCARE,
		_T("Arial"));

If the above font is going to be created many times, caching the value of getSystemScaleFactor() and/or caching the fonts created will be a good idea.

CreateFontIndirect

Search for uses of the MFC method CFont::CreateFontIndirect(), or the Win32 function ::CreateFontIndirect(). With this usage you are interested in the lfHeight value of the LOGFONT structure passed to CreateFontIndirect. The procedure is identical as for CreateFont() described above.

Before scaling:

pDispInfo->item.lfFont.lfWeight = FW_NORMAL;
pDispInfo->item.lfFont.lfHeight = -11;

After scaling:

pDispInfo->item.lfFont.lfWeight = FW_NORMAL;
pDispInfo->item.lfFont.lfHeight = -getFontHeightForDefaultDC(8.5);		// aiming for lfHeight == -11 (at "small" 1920x1080);

Alternative implementation

Ignore point sizes and just multiply your existing font pixel heights by getSystemScaleFactor(). This is less work to implement, but results in lots of calls to getSystemScaleFactor() throughout your code base. If you eventually decide to implement per-monitor scaling I think you will regret choosing this implementation method.

pDispInfo->item.lfFont.lfWeight = FW_NORMAL;
pDispInfo->item.lfFont.lfHeight = -11 * getSystemScaleFactor();

Setting column widths

If you are setting column widths in a grid control you'll need to scale these appropriately.

Before scaling:

m_Grid.SetColumnWidth(0, 200);
m_Grid.SetColumnWidth(1, 400);

After scaling:

 double scaleFactor;

scaleFactor = getSystemScaleFactor();
m_Grid.SetColumnWidth(0, (UINT)(200 * scaleFactor));
m_Grid.SetColumnWidth(1, (UINT)(400 * scaleFactor));

Implementation note: For per-monitor scaling you'll need to update these calls to get the scale factor for the appropriate monitor.

How to test

To test you don't need to purchase a high DPI monitor. Although if you do have a high DPI monitor you will be more aware of the issues your high DPI using users have.


  • Change the text scaling size from 100% to another value.

  • Right click on the Windows desktop.

  • Choose Screen Resolution from the context menu.
    dpicontextmenu


  • Click the link Make text and other items larger or smaller.
    dpiscreenresolution


  • Change the text size slider larger or smaller as appropriate. Notice that as you change it a warning notice is displayed about different scaling results. Click Apply. Now close the dialog box.
    dpichangetextsize


When you click Apply you should notice the text and graphics scale up or down in relation to the choice you made. "Aha! I've done it!" you think. Not so fast. This is just Windows pretending everything is OK. You'll notice some of the items that have been scaled up don't look so perfect, a bit blurry. This is because this is Windows virtualising the scaling. This also means that many of the fonts you need to change will have been automatically changed for you - for this session only! Don't let this fool you into thinking your job is done. If you want to see what they'll look like for your customers you need to log out and then log back in. You don't need to shutdown the computer or reboot it. Just log out and then log in (not lock the screen and log in!).

When you log in, start your application. Now look at the text you are updating to be responsive to scaling. Has it scaled? Yes? Great! No? You must have missed something. Failed to update it to work from point sizes? Columns too narrow? Forgot to add the multiply by scaling factor? Debug as necessary. Just add developer.

When you've got this working, you're going to have to go back to normal font size (same procedure as before, but just move the text slider all the way to the left), then log out and log in. Now do your tests for the next font. Rinse and repeat. You are going to be logging out and logging back in a lot during this work. Or purchase a modern laptop with a QHD screen or a nice QHD monitor (or better!).

Optimisations and improvements

Namespaces.
We've placed all the helper functions I described in their own namespace to separate them from the rest of the code.

Caching.
We've also used some caches to improve the query time of some requests. We didn't find this bothersome until we had a grid with a lot of items (about a million) that wanted to do a specific query. We thought the delay was unacceptable. Adding a cache fixed that quite simply.

Per Monitor DPI.
You could look into supporting different DPI values for each monitor. See the GetDpiForMonitor() link we mentioned earlier in the article.

Icons.
You may also need to scale icons (in toolbars etc) accordingly. In practice we haven't found this to be a huge problem so long as your non-scaled icons are reasonably large. Ideally you could scale your icons, but we've found scaling icons to be the least important thing for users of our software. What they really want is legible text, hence our focus on that in this article.

X and Y scaling.
If you've been paying attention you've noticed that getSystemScaleFactor() returns the horizontal scale factor. We've used this everywhere, for both scaling of font heights (vertical) and scaling of column widths (horizontal). You could choose to calculate a vertical scale factor and use that for the scaling of font heights.

The vertical scale factor should be the same (barring any hardware restrictions) to the horizontal scale factor in order to preserve image integrity - so we haven't bothered to calculate a vertical scaling factor. In practice we've been happy with the results returned from using one scale factor.

Conclusion

Now you know everything you need to know to get your Win32 or MFC program displaying well on high DPI displays. There's a lot of great information about this on the web, but reading it can make it seem quite daunting, when in actual fact it's very straight forward. We recommend changing one font at a time, then test, then check with a scaled display, then move on to the next font. Don't try to do them all at once.

If you're testing without a high DPI display, once you think it's all good to go, show it to some of your customers that have high DPI displays that have been bending your ear about supporting high DPI displays. They'll be pleased to tell you what isn't quite right. Across all our applications, we missed on thing that a customer spotted.

I'd also like to extend a big Thank You to Anna Jayne Metcalfe at Riverblade for some helpful pointers when we started this work. Riverblade make the excellent Visual Lint static analysis tool analyser, an excellent complement to our dynamic analysis tools.

Further Reading

Writing Win32 applications and high DPI.

MFC applications default to being DPI aware.

Getting the DPI per Monitor.

Getting the Logical Monitor Width.

High DPI Tutorial.

Share

Don’t try this at home – custom control time sink

By , November 16, 2010 11:44 am

There are time when writing a custom control will waste your time like nothing on earth.

I’m going to share with you a particularly painful timewaster than bit my behind last night.

The custom control
I’m writing a custom control that will take data from an arbitrary data source via a data provider and then display this data as a graph. The screenshots in this article are not from the finished control, just some mockups to test various ideas I am experimenting with. As you can see it is a graph with a pastel block colour and a harder solid outline. All configurable of course.

Graph custom control

The problem
The problem was first noticed when I wanted to change some settings in the host program. I clicked on the toolbar button to open the settings dialog and nothing happened. Button depressed, but nothing. Clicking elsewhere on the screen and I get a beep – hmmm, not allowed. GUI is totally frozen.

It felt like a deadlock. But all the code I’d written had its multithreading code locked down and everything I’m doing is on the GUI thread. No chance of a deadlock. Still I go investigating, several times in the debugger, use Thread Validator and Thread Status Monitor. No sign of a deadlock. Thats odd.

Try a different tack. Comment out the subclassing of the graph controls – effectively disabling them. Does the settings dialog display? Yes it does. OK, so it is something to do with the custom control. But what? Identify by process of elimination. Comment out all message map handlers except erase background and paint. Do I still get the problem? Yes, OK, so the problem is with erase background or paint. Turns out the problem is in paint. I’ll show you the code below.

void CSvlGraphCtrl::OnPaint()
{
    if (GetSafeHwnd() == 0)
        return;

    CRect    rCl;

    GetClientRect(&rCl);

    CClientDC    dc(this);
    CMemDC     dcMem(&dc, rCl);

    OnDraw(&dcMem);
}

Can you spot the problem? I’ve forgotten to call CWnd::OnPaint() which sets the I’ve been drawn bits inside the class and causes various other drawing related operations to happen. Without this call, the settings dialog I’ve been trying interact with (which is displayed on top of this graph custom control in this case) does not get drawn properly – or at all. So the settings dialog which is nothing to do with the custom control is displayed but doesn’t get drawn due to an incorrect cascade of events caused by failing to call CWnd::OnPaint() from the custom control’s OnPaint() handler.

The code should look like this (added line in bold)

void CSvlGraphCtrl::OnPaint()
{
    CWnd::OnPaint();

    if (GetSafeHwnd() == 0)
        return;

    CRect    rCl;

    GetClientRect(&rCl);

    CClientDC    dc(this);
    CMemDC     dcMem(&dc, rCl);

    OnDraw(&dcMem);
}

Conclusion
Sometimes the bug isn’t what you think it is. I started thinking this was a deadlock as the GUI appeared to freeze (pressing ALT caused the display to fix itself, but I found that by accident several hours later) but the fix was a missing call to a base class method. I’ve been writing MFC for years and yet this trivial bug eluded me for some time.

Hopefully sharing this with you will help prevent you from making the same mistake.

Share

How to read embedded data from a resource

By , August 8, 2010 9:48 am

In the previous article I showed you how to embed data into a custom resource in your executable.

In this article I’m going to show you how to extract the same data using the Win32 API for use in your executable at runtime.

To extract data from a resource in an executable we need some information:

  • Executable name.
  • Custom resource type name.
  • Custom resource name.

In our previous example, the executable name was mvJavaDetective.dll, the custom resource type name was “CLASSFILE” and the custom resource name was “myJavaSpy”.

The API

FindResource

    HRSRC FindResource(HMODULE hModule,
                       LPCTSTR lpName,
                       LPCTSTR lpType);

Call FindResource() to find a resource in an executable and return a resource handle. The executable is specified using a module handle that represents a module loaded in the current program. If the module is not currently loaded you can load it with LoadLibrary(). The resource is identified by its custom resource name and custom resource type.

LoadResource

    HGLOBAL LoadResource(HMODULE hModule,
                         HRSRC   hResInfo);

Call LoadResource() to load the resource specified by the module handle and the resource handle. The returned handle should not be passed to any Global memory function for deallocation.

LockResource

    LPVOID LockResource(HGLOBAL hResData);

Call LockResource() to lock the resource in memory. Pass the handle returned by LoadResource() as the input parameter. If the call succeeds a pointer to the data represented by the handle is returned.

SizeofResource

    DWORD SizeofResource(HMODULE hModule,
                         HRSRC   hResInfo);

Call SizeofResource() to determine the size of a resource. Pass the module handle and the handle returned from FindResource() as input parameters.

Putting it together

In the previous example our example DLL myJavaDetective.dll had a class myJavaSpy.class embedded into a resource with the type “CLASSFILE” and name “myJavaSpy”. I will now show you how to extract the myJavaSpy.class byte codes from the resource.

First we need to get the module handle of the executable (myJavaDetective.dll) containing the myJavaSpy.class. For this example we will assume that myJavaDetective.dll is already loaded into memory.

	HMODULE	hModJavaDetective;

	hModJavaDetective = GetModuleHandle("myJavaDetective.dll");

Once we have the module handle we can attempt to find the resource in the executable. We don’t need to check for a NULL module handle as FindResource() handles and will return a NULL resource handle (just as it will if the resource is not embedded in the executable).

	jbyte	*classBytes = NULL;
	DWORD	classBytesLength = 0;
	HRSRC	hResource;

	hResource = FindResource(hModJavaDetective,
							 _T("myJavaSpy"),
							 _T("CLASSFILE"));
	if (hResource != NULL)
	{

If FindResource() returns a non NULL handle the resource has been found. Now we must load the resource using a LoadResource().

		HGLOBAL	hResourceMemory;

		hResourceMemory = LoadResource(hModInjectedJVMTI, hResource);
		if (hResourceMemory != NULL)
		{

If LoadResource() returns a non NULL handle the resource has been correctly loaded from the executable. This returns a handle of type HGLOBAL. Caution you must not pass this handle to any HGLOBAL related functions such as GlobalFree() or GlobalRealloc() as this handle does not represent a memory allocation. This type is used for backward compatibility with earlier versions of the Windows API.

Before we can use the data we must convert the returned handle into a pointer to the data by calling LockResource(). We also want to know the size of the data in the resource so we call SizeofResource() to determine the size. The pointer returned by LockResource() must not be passed to any memory deallocation functions – it does not need to be deallocated or unlocked.

			void	*ptr;
			DWORD	size;

			ptr = LockResource(hResourceMemory);
			size = SizeofResource(hModInjectedJVMTI, hResource);
			if (ptr != NULL)
			{

If LockResource() returns a non NULL pointer the pointer represents the data embedded in the executable.

Now we have the data we make a copy for our own use and continue as normal. This step is optional, you can use the data directly from the returned pointer if you wish.

				classBytes = new jbyte [size];
				if (classBytes != NULL)
				{
					memcpy(classBytes, ptr, size);
					classBytesLength = size;
				}
			}
		}

		// CAUTION! LoadResource() and LockResource() DO NOT allocate handles or locks, 
		// read the documentation
	}

Now that we have extracted the data from the resource embedded into the executable we can use the data as normal. For this example I will conclude by using the extracted Java class bytescodes to define a Java class in a Java Virtual Machine.

	if (classBytes != NULL)
	{
		// define our class, must have same name as class file bytes
		// pass NULL for the class loader - use default class loader

		jclass		klass = 0;

		klass = jniEnv->DefineClass(SVL_COVERAGE_CLASS, NULL, classBytes, classBytesLength);
		if (klass != 0)
		{
                    // class defined correctly
		}

		// tidy up

		delete [] classBytes;
	}

Wrap up

Now you know how to embed data in an executable at runtime (and after the fact with the utility presented in the previous article) and how to extract data from an executable at runtime. The techniques are quite straightforward to master and allow you to easily embed data for you to use at runtime without worrying about distributing and locating extra data files.

Share

How to embed data into a resource

By , August 7, 2010 2:44 pm

In this article I will demonstrate how you can embed data into a Windows PE format executable (EXE or DLL). At the end I will also provide a working example which you can use to embed data into your executable as custom resources.

The problem

Often software requires ancillary data to support the software we write. This data can reside in files on your hard disk, on a network computer or on a computer accessed across the Internet. Or the data can be embedded in your executable. There is no correct solution for all cases. You have to choose the correct solution for the task at hand. I’ll briefly describe the four methods, outlining the potential pitfalls involved.

  • Loading the data from disk. You need to locate the file and read the contents of the file. What happens if the file is missing? If the file is present and readable has it been modified by accident or has been deliberately tampered with? You will need a mechanism to detect this if appropriate.
  • Loading the data from a network computer. This is similar to loading the file from the disk except that you need to know the network computer name.
  • Loading the data from the a computer on the Internet. This is more complex, now you need engage in some protocol to download the file. What if the Internet connection is not available or is refused?
  • Embedding the data in your executable. Embedding the data is harder than creating a file, and reading the data is harder than reading a file. However, the data will always be available. If you application uses checksums (MD5, etc) or is digitally signed then you will know if the embedded data has been modified or tampered with.

Embedding data

Sometimes it would be more convenient if the data was embedded right into the executable we are creating.

There may be no convenient method for embedding the data. Visual Studio provides a means to embed data. You could transcribe the data by hand. But that would be time consuming, expensive, error prone and tedious. Alternatively you can add a custom resource, then edit the properties for the custom resource and identify the file that contains the data you wish to embed into the executable. We have tried this but there are no error messages for when the file cannot be found (you made a typing error typing the filename) and there is no way to conditionally change which custom resource is embedded depending on the build.

Fortunately, Windows provides an API for adding data to the resource section of an executable (.exe or .dll). The API also provides mechanisms for finding this data. With the use of the API we can create a helper application to embed as many custom resources as you want after you have built your executable.

For this example I will assume the data we adding to the executable is not data you would normally find in a resource. This means we will be adding a custom resource.

Let us say we want to add a Java class file to our executable so that we can find this class file at runtime without knowing anything about the current Java CLASSPATH or the file system. Once we’ve extracted the class file we could use it to define a class that would then be used by the Java Virtual Machine to do the work we want (presumably somewhere else we’ll be instrumenting Java class files so they know about the Java class we just defined).

We need a few things first, which we will also need when we come to extract the resource from the executable.

  • Executable to add the resource to.
  • Type name for the custom resource.
  • Name for the custom resource.
  • Data for the custom resource.

For our Java class file example, type could be “CLASSFILE”, name could be “myJavaSpy” and data would be the byte code for the class myJavaSpy which we would load from the file myJavaSpy.class (having previously compiled it from myJavaSpy.java).

The API

BeginUpdateResource

    HANDLE BeginUpdateResource(const TCHAR *executableName,
                               BOOL        fDeleteExistingResources);

Call BeginUpdateResource() to open the specified executable and return a resource handle. Pass TRUE for the second argument to erase all existing resources, pass FALSE to keep any existing resources in the executable.

UpdateResource

    BOOL UpdateResource(HANDLE  hUpdate,
                        LPCTSTR lpType,
                        LPCTSTR lpName,
                        WORD    wLanguage,
                        LPVOID  lpData,
                        DWORD   cbData);

Call UpdateResource() to update a resource in the executable represented by the handle hUpdate. Specify the type, name, language (locale) and the data with the remaining arguments. For our example above lpType would be “CLASSFILE” and lpName would be “myJavaSpy”. Pass MAKELANGID(LANG_NEUTRAL, SUBLANG_NEUTRAL) for language. Pass the java byte code and the lenght of the byte code for the last two arguments.

EndUpdateResource

    EndUpdateResource(HANDLE hUpdate,
                      BOOL   fDiscard);

Call EndUpdateResource() to finish updating the resource. If you wish to discard your changes, pass TRUE as the second argument. If you wish to keep your changes, pass FALSE as the second argument.

Putting it together

    HANDLE hUpdateRes;

    // Open the file to which you want to add the dialog box resource. 
	
    hUpdateRes = BeginUpdateResource(executableName, 
                                     FALSE);          // do not delete existing resources
    if (hUpdateRes != NULL) 
    {  
        BOOL   result; 
	
        // Add the dialog box resource to the update list. 
	
        result = UpdateResource(hUpdateRes,
                                customType,
                                customName,
                                MAKELANGID(LANG_NEUTRAL, SUBLANG_NEUTRAL),
                                bytes,
                                numBytes);
	if (result) 
	{ 
            // Write changes to the input DLL and then close it
	
            EndUpdateResource(hUpdateRes, FALSE);
        }
    } 

First we call BeginUpdateResource() to open the executable for resource updating. We pass FALSE as the second argument to make sure we keep the existing resources and only add our new resource. This calls returns an update handle.

If the call to BeginUpdateResource() is successful we received a non NULL update handle. We use to call UpdateResource() passing the type and name of resource data we wish to update along with the data to update and its length. In this example we have specified a neutral locale.

Finally we call EndUpdateResource() to finish updating the resource and to write the results back to the executable (pass FALSE as the second argument).

addResourceToDLL

addResourceToDLL.exe is command line program that you can add to your post-build process to embed custom resources into your EXE/DLL as you build. It has a quiet mode so that you can suppress any information and/or error messages it may emit. I don’t use the quiet mode, I like to see the confirmation message that it succeeded embedding data into the DLL. Run without arguments to get the help message.

Help summary

All arguments are mandatory unless noted otherwise.

  • -moduleName pathToDLL (or EXE)
  • -customResource pathToCustomResource
  • -customType type
  • -customName name
  • -quiet (optional)
  • Example:

    addResourceToDLL.exe -moduleName c:\myJavaDetective\myJavaDetective.dll -customResource c:\myJavaDetective\myJavaSpy.class -customType CLASSFILE -customName myJavaSpy

    The example above embeds the myJavaSpy.class file into myJavaDetective.dll with the type “CLASSFILE” and name “myJavaSpy”.

    Download

    Download the addResourceToDLL source code.

    Download the addResourceToDLL executable.

    In the next article I will show you how to read the embedded data from the resource.

    Share

Don’t use srand(clock()), use srand((unsigned)time(NULL)) instead

By , July 9, 2010 9:42 am

Typically you use srand() when you need to start the random number generator in a random place. You may do this because you are going to generate some keys or coupons and want them to start in an unpredictable place.

From time to time we provide special offers to customers in the form of a unique coupon code that can be used at purchase to get a specific discount. These coupons are also used to provide discounts to customers upgrading from say Performance Validator to C++ Developer Suite so that they do not pay for Performance Validator twice.

When the coupon management system was written, we used srand(clock()) thinking that would be an acceptable random value for generating coupons. The thinking was the management system would be running all the time and thus clock() would return a value that was unlikely to be hit twice for the number of valid coupons at any one time. However, the way the system is used is that users close the coupon management system when not in use and thus clock() will return values close to the starting time (start the app, navigate to the appropriate place, generate a coupon).

Result: Sooner or later a duplicate coupon is created. And that is when we noticed this problem.

This resulted in a confused customer (“My coupon has already been used”), a confused member of customer support (“That shouldn’t be possible!”) followed by some checking of the coupon files and then the code to see how it happened. Easy to fix, but better selection of the seed in the first place would have prevented the problem.

So if you want better random numbers don’t use clock() to seed srand().

Better seeds

  • Use time(NULL) to get the time of day and cast the result to seed srand().
    time(NULL) returns the number of seconds elapsed since midnight January 1st, 1970.
  • Use rdtsc() to get the CPU timestamp and cast the result to seed srand().
    rdtsc() is unlikely to return duplicate values as it returns the number of instructions executed by the processor since startup.
Share

Panorama Theme by Themocracy