Rss Feed
Tweeter button
Facebook button
Technorati button
Reddit button
Myspace button
Linkedin button
Webonews button
Delicious button
Digg button
Flickr button
Stumbleupon button
Newsvine button

How to read embedded data from a resource

By , August 8, 2010 9:48 am

In the previous article I showed you how to embed data into a custom resource in your executable.

In this article I’m going to show you how to extract the same data using the Win32 API for use in your executable at runtime.

To extract data from a resource in an executable we need some information:

  • Executable name.
  • Custom resource type name.
  • Custom resource name.

In our previous example, the executable name was mvJavaDetective.dll, the custom resource type name was “CLASSFILE” and the custom resource name was “myJavaSpy”.

The API

FindResource

    HRSRC FindResource(HMODULE hModule,
                       LPCTSTR lpName,
                       LPCTSTR lpType);

Call FindResource() to find a resource in an executable and return a resource handle. The executable is specified using a module handle that represents a module loaded in the current program. If the module is not currently loaded you can load it with LoadLibrary(). The resource is identified by its custom resource name and custom resource type.

LoadResource

    HGLOBAL LoadResource(HMODULE hModule,
                         HRSRC   hResInfo);

Call LoadResource() to load the resource specified by the module handle and the resource handle. The returned handle should not be passed to any Global memory function for deallocation.

LockResource

    LPVOID LockResource(HGLOBAL hResData);

Call LockResource() to lock the resource in memory. Pass the handle returned by LoadResource() as the input parameter. If the call succeeds a pointer to the data represented by the handle is returned.

SizeofResource

    DWORD SizeofResource(HMODULE hModule,
                         HRSRC   hResInfo);

Call SizeofResource() to determine the size of a resource. Pass the module handle and the handle returned from FindResource() as input parameters.

Putting it together

In the previous example our example DLL myJavaDetective.dll had a class myJavaSpy.class embedded into a resource with the type “CLASSFILE” and name “myJavaSpy”. I will now show you how to extract the myJavaSpy.class byte codes from the resource.

First we need to get the module handle of the executable (myJavaDetective.dll) containing the myJavaSpy.class. For this example we will assume that myJavaDetective.dll is already loaded into memory.

	HMODULE	hModJavaDetective;

	hModJavaDetective = GetModuleHandle("myJavaDetective.dll");

Once we have the module handle we can attempt to find the resource in the executable. We don’t need to check for a NULL module handle as FindResource() handles and will return a NULL resource handle (just as it will if the resource is not embedded in the executable).

	jbyte	*classBytes = NULL;
	DWORD	classBytesLength = 0;
	HRSRC	hResource;

	hResource = FindResource(hModJavaDetective,
							 _T("myJavaSpy"),
							 _T("CLASSFILE"));
	if (hResource != NULL)
	{

If FindResource() returns a non NULL handle the resource has been found. Now we must load the resource using a LoadResource().

		HGLOBAL	hResourceMemory;

		hResourceMemory = LoadResource(hModInjectedJVMTI, hResource);
		if (hResourceMemory != NULL)
		{

If LoadResource() returns a non NULL handle the resource has been correctly loaded from the executable. This returns a handle of type HGLOBAL. Caution you must not pass this handle to any HGLOBAL related functions such as GlobalFree() or GlobalRealloc() as this handle does not represent a memory allocation. This type is used for backward compatibility with earlier versions of the Windows API.

Before we can use the data we must convert the returned handle into a pointer to the data by calling LockResource(). We also want to know the size of the data in the resource so we call SizeofResource() to determine the size. The pointer returned by LockResource() must not be passed to any memory deallocation functions – it does not need to be deallocated or unlocked.

			void	*ptr;
			DWORD	size;

			ptr = LockResource(hResourceMemory);
			size = SizeofResource(hModInjectedJVMTI, hResource);
			if (ptr != NULL)
			{

If LockResource() returns a non NULL pointer the pointer represents the data embedded in the executable.

Now we have the data we make a copy for our own use and continue as normal. This step is optional, you can use the data directly from the returned pointer if you wish.

				classBytes = new jbyte [size];
				if (classBytes != NULL)
				{
					memcpy(classBytes, ptr, size);
					classBytesLength = size;
				}
			}
		}

		// CAUTION! LoadResource() and LockResource() DO NOT allocate handles or locks, 
		// read the documentation
	}

Now that we have extracted the data from the resource embedded into the executable we can use the data as normal. For this example I will conclude by using the extracted Java class bytescodes to define a Java class in a Java Virtual Machine.

	if (classBytes != NULL)
	{
		// define our class, must have same name as class file bytes
		// pass NULL for the class loader - use default class loader

		jclass		klass = 0;

		klass = jniEnv->DefineClass(SVL_COVERAGE_CLASS, NULL, classBytes, classBytesLength);
		if (klass != 0)
		{
                    // class defined correctly
		}

		// tidy up

		delete [] classBytes;
	}

Wrap up

Now you know how to embed data in an executable at runtime (and after the fact with the utility presented in the previous article) and how to extract data from an executable at runtime. The techniques are quite straightforward to master and allow you to easily embed data for you to use at runtime without worrying about distributing and locating extra data files.

Share

How to embed data into a resource

By , August 7, 2010 2:44 pm

In this article I will demonstrate how you can embed data into a Windows PE format executable (EXE or DLL). At the end I will also provide a working example which you can use to embed data into your executable as custom resources.

The problem

Often software requires ancillary data to support the software we write. This data can reside in files on your hard disk, on a network computer or on a computer accessed across the Internet. Or the data can be embedded in your executable. There is no correct solution for all cases. You have to choose the correct solution for the task at hand. I’ll briefly describe the four methods, outlining the potential pitfalls involved.

  • Loading the data from disk. You need to locate the file and read the contents of the file. What happens if the file is missing? If the file is present and readable has it been modified by accident or has been deliberately tampered with? You will need a mechanism to detect this if appropriate.
  • Loading the data from a network computer. This is similar to loading the file from the disk except that you need to know the network computer name.
  • Loading the data from the a computer on the Internet. This is more complex, now you need engage in some protocol to download the file. What if the Internet connection is not available or is refused?
  • Embedding the data in your executable. Embedding the data is harder than creating a file, and reading the data is harder than reading a file. However, the data will always be available. If you application uses checksums (MD5, etc) or is digitally signed then you will know if the embedded data has been modified or tampered with.

Embedding data

Sometimes it would be more convenient if the data was embedded right into the executable we are creating.

There may be no convenient method for embedding the data. Visual Studio provides a means to embed data. You could transcribe the data by hand. But that would be time consuming, expensive, error prone and tedious. Alternatively you can add a custom resource, then edit the properties for the custom resource and identify the file that contains the data you wish to embed into the executable. We have tried this but there are no error messages for when the file cannot be found (you made a typing error typing the filename) and there is no way to conditionally change which custom resource is embedded depending on the build.

Fortunately, Windows provides an API for adding data to the resource section of an executable (.exe or .dll). The API also provides mechanisms for finding this data. With the use of the API we can create a helper application to embed as many custom resources as you want after you have built your executable.

For this example I will assume the data we adding to the executable is not data you would normally find in a resource. This means we will be adding a custom resource.

Let us say we want to add a Java class file to our executable so that we can find this class file at runtime without knowing anything about the current Java CLASSPATH or the file system. Once we’ve extracted the class file we could use it to define a class that would then be used by the Java Virtual Machine to do the work we want (presumably somewhere else we’ll be instrumenting Java class files so they know about the Java class we just defined).

We need a few things first, which we will also need when we come to extract the resource from the executable.

  • Executable to add the resource to.
  • Type name for the custom resource.
  • Name for the custom resource.
  • Data for the custom resource.

For our Java class file example, type could be “CLASSFILE”, name could be “myJavaSpy” and data would be the byte code for the class myJavaSpy which we would load from the file myJavaSpy.class (having previously compiled it from myJavaSpy.java).

The API

BeginUpdateResource

    HANDLE BeginUpdateResource(const TCHAR *executableName,
                               BOOL        fDeleteExistingResources);

Call BeginUpdateResource() to open the specified executable and return a resource handle. Pass TRUE for the second argument to erase all existing resources, pass FALSE to keep any existing resources in the executable.

UpdateResource

    BOOL UpdateResource(HANDLE  hUpdate,
                        LPCTSTR lpType,
                        LPCTSTR lpName,
                        WORD    wLanguage,
                        LPVOID  lpData,
                        DWORD   cbData);

Call UpdateResource() to update a resource in the executable represented by the handle hUpdate. Specify the type, name, language (locale) and the data with the remaining arguments. For our example above lpType would be “CLASSFILE” and lpName would be “myJavaSpy”. Pass MAKELANGID(LANG_NEUTRAL, SUBLANG_NEUTRAL) for language. Pass the java byte code and the lenght of the byte code for the last two arguments.

EndUpdateResource

    EndUpdateResource(HANDLE hUpdate,
                      BOOL   fDiscard);

Call EndUpdateResource() to finish updating the resource. If you wish to discard your changes, pass TRUE as the second argument. If you wish to keep your changes, pass FALSE as the second argument.

Putting it together

    HANDLE hUpdateRes;

    // Open the file to which you want to add the dialog box resource. 
	
    hUpdateRes = BeginUpdateResource(executableName, 
                                     FALSE);          // do not delete existing resources
    if (hUpdateRes != NULL) 
    {  
        BOOL   result; 
	
        // Add the dialog box resource to the update list. 
	
        result = UpdateResource(hUpdateRes,
                                customType,
                                customName,
                                MAKELANGID(LANG_NEUTRAL, SUBLANG_NEUTRAL),
                                bytes,
                                numBytes);
	if (result) 
	{ 
            // Write changes to the input DLL and then close it
	
            EndUpdateResource(hUpdateRes, FALSE);
        }
    } 

First we call BeginUpdateResource() to open the executable for resource updating. We pass FALSE as the second argument to make sure we keep the existing resources and only add our new resource. This calls returns an update handle.

If the call to BeginUpdateResource() is successful we received a non NULL update handle. We use to call UpdateResource() passing the type and name of resource data we wish to update along with the data to update and its length. In this example we have specified a neutral locale.

Finally we call EndUpdateResource() to finish updating the resource and to write the results back to the executable (pass FALSE as the second argument).

addResourceToDLL

addResourceToDLL.exe is command line program that you can add to your post-build process to embed custom resources into your EXE/DLL as you build. It has a quiet mode so that you can suppress any information and/or error messages it may emit. I don’t use the quiet mode, I like to see the confirmation message that it succeeded embedding data into the DLL. Run without arguments to get the help message.

Help summary

All arguments are mandatory unless noted otherwise.

  • -moduleName pathToDLL (or EXE)
  • -customResource pathToCustomResource
  • -customType type
  • -customName name
  • -quiet (optional)
  • Example:

    addResourceToDLL.exe -moduleName c:\myJavaDetective\myJavaDetective.dll -customResource c:\myJavaDetective\myJavaSpy.class -customType CLASSFILE -customName myJavaSpy

    The example above embeds the myJavaSpy.class file into myJavaDetective.dll with the type “CLASSFILE” and name “myJavaSpy”.

    Download

    Download the addResourceToDLL source code.

    Download the addResourceToDLL executable.

    In the next article I will show you how to read the embedded data from the resource.

    Share

Thread Validator x64 enters BETA

By , August 6, 2010 8:57 am

Thread Validator x64 is now available for beta testing.

Thread locking history

Thread Validator x64 is the 64 bit version of our successful 32 bit Thread Validator software tool that runs on Microsoft Windows operating systems. Thread Validator x64 is a deadlock detection and thread analysis software tool, running on Windows 7 64 bit, Windows Vista 64 bit and Windows XP 64 bit.

Thread Validator has multiple displays to provide you with different perspectives onto the data you have collected.

What does Thread Validator do?

Thread Validator x64 identifies thread deadlocks, potential deadlocks and locks with a high contention rate.

Thread deadlocks usually mean that one or more threads can no longer function correctly because they are waiting on a lock that will never be released. This is an error condition and usually manifests as an unresponsive computer program.

Potential deadlocks are locking sequences that have not triggered a deadlock but may lead to a deadlock under slightly different conditions.

High contention rate locks result in your program spending too much time waiting for access to a lock. A different program design can often reduce a high contention rate to a less demanding contention rate.

How does Thread Validator work?

Thread Validator instruments your computer program so that it can monitor the appropriate synchronization APIs used to control access to locks, mutexes, semaphores and wait conditions. Using the information gained from monitoring these APIs, Thread Validator can calculate deadlock conditions, potential deadlock conditions and detect locks with high contention rates.

Thread Validator gathers data for all locks, all threads, all mutexes, all semaphores and all wait conditions. The data is organised into various displays allowing you to view information:

  • All active locks.
  • All active locks, organized by thread.
  • All locks that are locked at a given time.
  • Allocation information for all allocated synchronization objects, showing callstack and source code.
  • Thread locking history. View all threads, see what each threads is doing and when.
  • Thread lock order. View the order locks are acquired across threads for a given lock sequence.
  • List of all application objects that can be used in wait conditions.
  • How Thread Validator helps you be more productive

    Thread Validator x64 can help you:

    • Identify deadlocks in your application – quickly identify and fix hard threading problems.
    • Identify potential deadlocks in your application – prevent problems before they get serious.
    • Identify busy contended critical sections in your application – improve performance.
    • View thread locking behaviour in real time.
    • Improve your software quality by modifying your threading behaviour.
    • View all open handles that your application can wait on.

    Join the beta test

    If you are developing 64 bit software and have some multi-threading problems you would like to analyze, please join the beta, analyze your multi-threading problems and let us know your thoughts.

    Share

Give up caffeine, improve productivity

By , August 5, 2010 9:12 am

Give up caffeine, improve productivity – yes I am serious.

Just recently I found out that I was allergic to several things, one of them being caffeine.

My history with caffeine

I really like tea, but dislike coffee, having announced to my mother at age 5 that I didn’t like coffee. Seems to have stuck with me. I drink tea with no milk and have done for years. A little bit of sugar to take the edge off the black tea and its fine. And I loved the caffeine. I could never see the point of caffeine free tea. Until I found out I was allergic to caffeine.

Giving up caffeine

At the same time, I had noticed that a lot of the time I was distracted, unable to relax, always casting about for something to do. Granted, folks with active minds are like this a bit, I guess thats why I like to write software. But this was different, even when too tired to write software I’d still be this coiled spring.

Then I gave up drinking caffeine in any drinks. Apparently if you drink more than a few cups of tea a day you are classed as addicted to caffeine. I guess you could say I was easily addicted to caffeine. According to Wikipedia there are caffeine withdrawal symptoms but I can’t say I noticed any.

A few days after I stopped drinking caffeinated tea, my distracted state of mind went away. Easier to focus on software, on bugs, reading books, watching movies, whatever the thing was.

Caffeine also affects your blood sugar levels, causing a boost. This in turn can lead to up and down swings in your blood sugar with a possible change of mood.

The problem with energy drinks and software

Its not uncommon to see physically active people consuming lots of calories, either in the form of food or drink. Or even drinking an energy drink which may also contain caffeine. That is fine because the physical activity will consume the calories and burn them leaving your blood sugar levels relatively normal.

However if you are sitting at your desk (or in your car) then an energy drink or high carbohydrate food is just going to put a big spike into your blood sugar to which your body will have to react with some insulin to regulate it. Not so long later (hour or two) you’ll feel lethargic as you get the counter effects of the insulin kicking in.

As such I’ve never understood the idea of consuming energy drinks if you are writing software – you are winding yourself up, and also setting yourself up for a blood sugar trough after the spike. If you are taking an energy drink so you can stay awake and code that is a sign you are too tired anyway. You should take 20 minutes out and have a short sleep. Drink half a pint to a pint of water before you go to sleep. It surprising how much this short break can help. The water is to rehydrate you while you sleep – tiredness is a sign of being dehydrated.

It is not uncommon for me to wake from a short nap with the solution to a problem and the also the correct approach to implementing the solution. Try it for yourself.

You can have a similar problem with food

The same problem with energy drinks applies to fast acting carbohydrate foods. Basically anything filled with sugar (energy bars, cakes, sweets…). You’ll get a blood sugar spike followed by a trough as your body overcompensates with insulin. These foods are great if you are active and on the go and need a boost but totally counter productive if you are not physically active (typing does not count!).

You will be much better served eating something that is more slowly processed by your body. Namely protein. Vegetable protein (beans, pulses) or meat protein, it does not matter which. Protein takes time for your body to convert into energy. As a result the energy is released in a much slower, more controlled manner, supplying you with energy without any blood sugar spikes or troughs.

What do I drink instead of caffeinated tea?

I now drink the redbush caffeine free tea, various herbal fruit teas and water. I drink water because a 5% drop in your body hydration leads to a significant drop in your ability to concentrate.

Recap

  • Avoid caffeine and other stimulants. More focus, less distractedness, better productivity.
  • Drink water, do not work dehydrated.
  • Drinking caffeinated drinks will dehydrate you – caffeine is a directic.
  • Keep your blood sugar even, improve your productivity.
Share

Panorama Theme by Themocracy