Rss Feed
Tweeter button
Facebook button
Technorati button
Reddit button
Myspace button
Linkedin button
Webonews button
Delicious button
Digg button
Flickr button
Stumbleupon button
Newsvine button

Posts tagged: Java

Getting code coverage for a child process?

By , May 31, 2017 5:43 pm

In this blog post I’m going to explain how to collect code coverage for a process that is launched by another process. We’ll be using C++ Coverage Validator to collect the code coverage.

For example you may have a control process that launches helper programs to do specific jobs and you wish to collect code coverage data for one of the helper programs. I’m first going to show how you do this with the GUI, then I’ll show you how to do this with the command line.

For the purposes of this blog post I’m going to use a test program called testAppFromOtherProcess.exe as the child program and testAppOtherProcessCpp.exe as the parent process. Once I’ve explained this for C++, I’ll also provide examples for programs launched from Java and for programs launched from Python.

The test program

The test program is simple. It takes two numbers and calculates the sum of all the products. If less than two arguments are supplied they default to 10.

int _tmain(int argc, _TCHAR* argv[])
{
	int	nx, ny;
	int	x, y;
	int	v;

	nx = 10;
	ny = 10;
	v = 0;

	if (argc == 2)
	{
		nx = _tcstol(argv[1], NULL, 10);
	}
	else if (argc >= 3)
	{
		nx = _tcstol(argv[1], NULL, 10);
		ny = _tcstol(argv[2], NULL, 10);
	}

	for(y = 0; y < ny; y++)
	{
		for(x = 0; x < nx; x++)
		{
			v += (x + 1) * (y + 1);
		}
	}

	return v;
}

The parent C++ program

The parent C++ program is a simple MFC dialog that collects two values and launches the test program. The code for launching the child process looks like this:

void CtestAppOtherProcessCppDlg::OnBnClickedOk()
{
	// get data values

	CString	str1, str2;
	DWORD	v = 0;

	GetDlgItemText(IDC_EDIT_COUNT1, str1);
	GetDlgItemText(IDC_EDIT_COUNT2, str2);

	// create command line

	CString	commandline;

	commandline += _T("testAppFromOtherProcess.exe");
	commandline += _T(" ");
	commandline += str1;
	commandline += _T(" ");
	commandline += str2;

	// run child process

	STARTUPINFO         stStartInfo;
	PROCESS_INFORMATION stProcessInfo;

	memset(&stStartInfo, 0, sizeof(STARTUPINFO));
	memset(&stProcessInfo, 0, sizeof(PROCESS_INFORMATION));

	stStartInfo.cb = sizeof(STARTUPINFO);
	stStartInfo.dwFlags = STARTF_USESHOWWINDOW;
	stStartInfo.wShowWindow = SW_HIDE;

	int	bRet;

	bRet = CreateProcess(NULL,
			(TCHAR *)(const TCHAR *)commandline,
			NULL,
			NULL,
			FALSE,
			0,
			NULL,
			NULL,
			&stStartInfo,
			&stProcessInfo);
	if (bRet)
	{
		// wait until complete then get exit code

		WaitForSingleObject(stProcessInfo.hProcess, INFINITE);

		GetExitCodeProcess(stProcessInfo.hProcess, &v);

		// tidy up

		CloseHandle(stProcessInfo.hProcess);
		CloseHandle(stProcessInfo.hThread);
	}

	// display result

	SetDlgItemInt(IDC_STATIC_VALUE, v, FALSE);
}

Configuring the target C++ program

Before we can collect code coverage we need to tell C++ Coverage Validator about the target program and the program that is going to launch it. We do this from the launch dialog (or launch wizard). From the launch dialog, select the program to launch using the Browse... button and selecting the file with the File dialog. Once a file has been chosen a default value will be selected for the Application to Monitor. This is the same program as you just selected with the File dialog.

CVLaunchDialogApplicationToMonitor

To allow us to monitor other programs we need to edit the list of applications we can monitor. Click the Edit... button to the right of the Application to monitor combo box. The Applications To Monitor dialog is displayed.

CVApplicationsToMonitorDialog

We need to add our target program to the list of programs to monitor. Click Add.... The Application To Monitor dialog is displayed. Choose our launch program testAppOtherProcessCpp.exe using Browse.... C++ Coverage Validator will identify any other executables in the same folder and add these to the list of target programs you may want to monitor. You can remove any programs you don't want to monitor with the Remove and Remove All buttons. Your dialog should look like the one shown below.

CVApplicationToMonitorDialog

Click OK to close the Application To Monitor dialog.

Click OK to close the Applications To Monitor dialog.

The Application to monitor combo will now have additional entries in it. Select testAppFromOtherProcess.exe in the Application to monitor combo. Leave the launch count set to 1. The first time testAppFromOtherProcess.exe is launched it will be monitored. Click Go! to start the parent process.

CVApplicationToMonitorParentProcess

You will notice that C++ Coverage Validator is not collecting data. Now click on the Launch Child Process button. The child process is launched, C++ Coverage Validator recognises the parent process is launching a child process that is configured to be monitored and has the correct launch count (this is the first time it is being launched and the launch count is set to "1") - the child process is instrumented for code coverage. You can see the instrumentation progress in the title bar and pretty soon code coverage statistics are being displayed by C++ Coverage Validator.

CVCodeCoverageResults

Command Line, example for C++

OK, that's wonderful, we can collect code coverage using the GUI to launch one program and collect data from a child process. All without any coding. Super. So how do we do that from the command line? Glad you asked!

"c:\C++ Coverage Validator\coverageValidator.exe" 
-program "e:\test\release\testAppOtherProcessCpp.exe"
-directory "e:\test\release" 
-programToMonitor "e:\test\release\testAppFromOtherProcess.exe" 

How does this work?

  • -directory. Specify the startup directory.
  • -program. Specify the program to launch.
  • -programToMonitor. Specify the program to that will be monitored for code coverage.

Very straightforward and simple. Paths must have quotes if they contain spaces. If in doubt always use quotes. Note also that where you've installed C++ Coverage Validator will be different, most likely in C:\Program Files (x86)\Software Verification. We shortened it for the example to make it fit the page.

Java

The parent program in Java is very simple. It takes any arguments passed to it and passes them to the target program.

import java.io.IOException;
import java.lang.ProcessBuilder;
import java.util.ArrayList;

public class testAppFromOtherProcessJava 
{
    public static void main(String[] args) throws IOException, InterruptedException
	{
		String			target = "e:\\om\\c\\testApps\\testAppFromOtherProcess\\Release\\testAppFromOtherProcess.exe";
        	ProcessBuilder	p = new ProcessBuilder();

		// add the args to be passed to the target program, unlike C/C++, args[0] is not the program name

		ArrayList	targetArgs;

		targetArgs = new ArrayList();
		targetArgs.add(target);
		for(int i = 0; i < args.length; i++)
		{
			targetArgs.add(args[i]);
		}

		p.command(targetArgs);

		// run the process, wait for it to complete and report the value calculated

		Process			proc;

	        proc = p.start();
		proc.waitFor();

		System.out.println("Result: " + proc.exitValue()); 
    }
}

You can compile this program with this simple command line. This assumes you have a Java Development Kit installed and javac.exe on the command line.

javac testAppFromOtherProcessJava.java

Configuring the target Java program

As with the C++ target program we need to tell C++ Coverage Validator about the target program and the program that is going to launch it. We're running a Java program so the executable to launch is the Java runtime. Click the Browse... button and select the Java runtime you are using.

CVLaunchDialogJava

The launch directory is automatically configured to be the same as the launch program. In the case of a Java program, that is almost certainly incorrect. We're going to choose the directory where our Java class is located. Click the Dir... button and choose that directory.

CVLaunchDialogJavaDirectory

We also need to tell the Java runtime what class to execute. This is provided as an argument to the program being run (the Java rutnime). In the arguments field, type the name of the class. In this case testAppFromOtherProcessJava (without the .class extension).

CVLaunchDialogJavaArguments

To allow us to monitor other programs we need to edit the list of applications we can monitor. Click the Edit... button to the right of the Application to monitor combo box. The Applications To Monitor dialog is displayed.

CVApplicationsToMonitorDialog

We need to add our target program to the list of programs to monitor. Click Add.... The Application To Monitor dialog is displayed. Choose the Java runtime java.exe using Browse.... C++ Coverage Validator will identify any other executables in the same folder and add these to the list of target programs you may want to monitor. You can remove any programs you don't want to monitor with the Remove and Remove All buttons. We now need to add the target program to the list of programs we want to monitor. Click Add... and select testAppFromOtherProcess.exe. Your dialog should look like the one shown below.

CVApplicationToMonitorDialogJava

Select testAppFromOtherProcess.exe in the Application to monitor combo. Leave the launch count set to 1. The first time testAppFromOtherProcess.exe is launched it will be monitored. Click Go! to start the parent process.

CVLaunchDialogApplicationToMonitorJava

The Java process launches testAppFromOtherProcess.exe immediately. As such you will notice that C++ Coverage Validator starts collecting code coverage almost instantly because it has recognised the Java process is launching a child process that is configured to be monitored and has the correct launch count.

CVCodeCoverageResultsJava

Command Line, example for Java

As you can see, it's slightly more complicated for Java than for C++, but only because the Java runtime is located in a different folder than the test executable and because we also have to specify a Java class to execute. We still managed to collect code coverage for a child process of a just in time compiled language without any coding.

Of course, you now want to know how to do this for the command line. Is this any more complicated than for the C++ example? No! Just as easy. Here's how you do it:

"c:\C++ Coverage Validator\coverageValidator.exe" 
-program "c:\program files\java\jdk1.8.0_121\bin\java.exe"
-directory "e:\test\release" 
-arg testAppFromOtherProcessJava
-programToMonitor "e:\test\release\testAppFromOtherProcess.exe"

How does this work?

  • -arg. Specify an argument to the program to launch. In this example this specifies the Java class to execute.
  • -directory. Specify the startup directory.
  • -program. Specify the program to launch. In this example this specifies the Java runtime.
  • -programToMonitor. Specify the program to that will be monitored for code coverage.

Use as many -arg options as you need. We only used one because that's all we need for the example.

Python

The parent program in Python is very simple.

import sys
import subprocess

cmdLine = r"E:\om\c\testApps\testAppFromOtherProcess\Release\testAppFromOtherProcess.exe"
for arg in range(1, len(sys.argv)):
  cmdLine += " "
  cmdLine += sys.argv[arg]
  
subprocess.call(cmdLine, stdin=None, stdout=None, stderr=None, shell=False)

Configuring the target Python program

As with the C++ target program we need to tell C++ Coverage Validator about the target program and the program that is going to launch it. We're running a Python program so the executable to launch is the Python interpreter. Click the Browse... button and select the Python interpreter you are using.

CVLaunchDialogPython

The launch directory is automatically configured to be the same as the launch program. In the case of a Python program, that is almost certainly incorrect. We're going to choose the directory where our Python script is located. Click the Dir... button and choose that directory.

CVLaunchDialogPythonDirectory

We also need to tell Python what script to launch. This is provided as an argument to the program being run (the Python interpreter). In the arguments field, type the name of the script. In this case testAppFromOtherProcess.py.

CVLaunchDialogPythonArguments

To allow us to monitor other programs we need to edit the list of applications we can monitor. Click the Edit... button to the right of the Application to monitor combo box. The Applications To Monitor dialog is displayed.

CVApplicationsToMonitorDialog

We need to add our target program to the list of programs to monitor. Click Add.... The Application To Monitor dialog is displayed. Choose the Python interpreter python.exe using Browse.... C++ Coverage Validator will identify any other executables in the same folder and add these to the list of target programs you may want to monitor. You can remove any programs you don't want to monitor with the Remove and Remove All buttons. We now need to add the target program to the list of programs we want to monitor. Click Add... and select testAppFromOtherProcess.exe. Your dialog should look like the one shown below.

CVApplicationToMonitorDialogPython

Select testAppFromOtherProcess.exe in the Application to monitor combo. Leave the launch count set to 1. The first time testAppFromOtherProcess.exe is launched it will be monitored. Click Go! to start the parent process.

CVLaunchDialogPythonAppToMonitor

The Python process launches testAppFromOtherProcess.exe immediately. As such you will notice that C++ Coverage Validator starts collecting code coverage almost instantly because it has recognised the Python process is launching a child process that is configured to be monitored and has the correct launch count.

CVCodeCoverageResultsPython

Command Line, example for Python

As you can see, it's slightly more complicated for Python than for C++, but only because the Python interpreter is located in a different folder than the test executable and because we also have to specify a Python script. We still managed to collect code coverage for a child process of a scripted language without any coding.

Of course, you now want to know how to do this for the command line. Is this any more complicated than for the C++ example? No! Just as easy. Here's how you do it:

"c:\C++ Coverage Validator\coverageValidator.exe" 
-program "c:\python36-32\python.exe"
-directory "e:\test\release" 
-arg testAppFromOtherProcess.py
-programToMonitor "e:\test\release\testAppFromOtherProcess.exe"

How does this work?

  • -arg. Specify an argument to the program to launch. In this example this specifies the Python script to run.
  • -directory. Specify the startup directory.
  • -program. Specify the program to launch. In this example this specifies the Python interpreter.
  • -programToMonitor. Specify the program to that will be monitored for code coverage.

Use as many -arg options as you need. We only used one because that's all we need for the example.

Conclusion

We've demonstrated how to monitor code coverage in a target program launched from C++, Java and Python, using both the GUI and the command line. Each example is slightly different, showing you the changes required for each situation. If you have any questions please email support@softwareverify.com

You can download the C++, Java and Python code used in these examples here.

Share

How to read embedded data from a resource

By , August 8, 2010 9:48 am

In the previous article I showed you how to embed data into a custom resource in your executable.

In this article I’m going to show you how to extract the same data using the Win32 API for use in your executable at runtime.

To extract data from a resource in an executable we need some information:

  • Executable name.
  • Custom resource type name.
  • Custom resource name.

In our previous example, the executable name was mvJavaDetective.dll, the custom resource type name was “CLASSFILE” and the custom resource name was “myJavaSpy”.

The API

FindResource

    HRSRC FindResource(HMODULE hModule,
                       LPCTSTR lpName,
                       LPCTSTR lpType);

Call FindResource() to find a resource in an executable and return a resource handle. The executable is specified using a module handle that represents a module loaded in the current program. If the module is not currently loaded you can load it with LoadLibrary(). The resource is identified by its custom resource name and custom resource type.

LoadResource

    HGLOBAL LoadResource(HMODULE hModule,
                         HRSRC   hResInfo);

Call LoadResource() to load the resource specified by the module handle and the resource handle. The returned handle should not be passed to any Global memory function for deallocation.

LockResource

    LPVOID LockResource(HGLOBAL hResData);

Call LockResource() to lock the resource in memory. Pass the handle returned by LoadResource() as the input parameter. If the call succeeds a pointer to the data represented by the handle is returned.

SizeofResource

    DWORD SizeofResource(HMODULE hModule,
                         HRSRC   hResInfo);

Call SizeofResource() to determine the size of a resource. Pass the module handle and the handle returned from FindResource() as input parameters.

Putting it together

In the previous example our example DLL myJavaDetective.dll had a class myJavaSpy.class embedded into a resource with the type “CLASSFILE” and name “myJavaSpy”. I will now show you how to extract the myJavaSpy.class byte codes from the resource.

First we need to get the module handle of the executable (myJavaDetective.dll) containing the myJavaSpy.class. For this example we will assume that myJavaDetective.dll is already loaded into memory.

	HMODULE	hModJavaDetective;

	hModJavaDetective = GetModuleHandle("myJavaDetective.dll");

Once we have the module handle we can attempt to find the resource in the executable. We don’t need to check for a NULL module handle as FindResource() handles and will return a NULL resource handle (just as it will if the resource is not embedded in the executable).

	jbyte	*classBytes = NULL;
	DWORD	classBytesLength = 0;
	HRSRC	hResource;

	hResource = FindResource(hModJavaDetective,
							 _T("myJavaSpy"),
							 _T("CLASSFILE"));
	if (hResource != NULL)
	{

If FindResource() returns a non NULL handle the resource has been found. Now we must load the resource using a LoadResource().

		HGLOBAL	hResourceMemory;

		hResourceMemory = LoadResource(hModInjectedJVMTI, hResource);
		if (hResourceMemory != NULL)
		{

If LoadResource() returns a non NULL handle the resource has been correctly loaded from the executable. This returns a handle of type HGLOBAL. Caution you must not pass this handle to any HGLOBAL related functions such as GlobalFree() or GlobalRealloc() as this handle does not represent a memory allocation. This type is used for backward compatibility with earlier versions of the Windows API.

Before we can use the data we must convert the returned handle into a pointer to the data by calling LockResource(). We also want to know the size of the data in the resource so we call SizeofResource() to determine the size. The pointer returned by LockResource() must not be passed to any memory deallocation functions – it does not need to be deallocated or unlocked.

			void	*ptr;
			DWORD	size;

			ptr = LockResource(hResourceMemory);
			size = SizeofResource(hModInjectedJVMTI, hResource);
			if (ptr != NULL)
			{

If LockResource() returns a non NULL pointer the pointer represents the data embedded in the executable.

Now we have the data we make a copy for our own use and continue as normal. This step is optional, you can use the data directly from the returned pointer if you wish.

				classBytes = new jbyte [size];
				if (classBytes != NULL)
				{
					memcpy(classBytes, ptr, size);
					classBytesLength = size;
				}
			}
		}

		// CAUTION! LoadResource() and LockResource() DO NOT allocate handles or locks, 
		// read the documentation
	}

Now that we have extracted the data from the resource embedded into the executable we can use the data as normal. For this example I will conclude by using the extracted Java class bytescodes to define a Java class in a Java Virtual Machine.

	if (classBytes != NULL)
	{
		// define our class, must have same name as class file bytes
		// pass NULL for the class loader - use default class loader

		jclass		klass = 0;

		klass = jniEnv->DefineClass(SVL_COVERAGE_CLASS, NULL, classBytes, classBytesLength);
		if (klass != 0)
		{
                    // class defined correctly
		}

		// tidy up

		delete [] classBytes;
	}

Wrap up

Now you know how to embed data in an executable at runtime (and after the fact with the utility presented in the previous article) and how to extract data from an executable at runtime. The techniques are quite straightforward to master and allow you to easily embed data for you to use at runtime without worrying about distributing and locating extra data files.

Share

How to embed data into a resource

By , August 7, 2010 2:44 pm

In this article I will demonstrate how you can embed data into a Windows PE format executable (EXE or DLL). At the end I will also provide a working example which you can use to embed data into your executable as custom resources.

The problem

Often software requires ancillary data to support the software we write. This data can reside in files on your hard disk, on a network computer or on a computer accessed across the Internet. Or the data can be embedded in your executable. There is no correct solution for all cases. You have to choose the correct solution for the task at hand. I’ll briefly describe the four methods, outlining the potential pitfalls involved.

  • Loading the data from disk. You need to locate the file and read the contents of the file. What happens if the file is missing? If the file is present and readable has it been modified by accident or has been deliberately tampered with? You will need a mechanism to detect this if appropriate.
  • Loading the data from a network computer. This is similar to loading the file from the disk except that you need to know the network computer name.
  • Loading the data from the a computer on the Internet. This is more complex, now you need engage in some protocol to download the file. What if the Internet connection is not available or is refused?
  • Embedding the data in your executable. Embedding the data is harder than creating a file, and reading the data is harder than reading a file. However, the data will always be available. If you application uses checksums (MD5, etc) or is digitally signed then you will know if the embedded data has been modified or tampered with.

Embedding data

Sometimes it would be more convenient if the data was embedded right into the executable we are creating.

There may be no convenient method for embedding the data. Visual Studio provides a means to embed data. You could transcribe the data by hand. But that would be time consuming, expensive, error prone and tedious. Alternatively you can add a custom resource, then edit the properties for the custom resource and identify the file that contains the data you wish to embed into the executable. We have tried this but there are no error messages for when the file cannot be found (you made a typing error typing the filename) and there is no way to conditionally change which custom resource is embedded depending on the build.

Fortunately, Windows provides an API for adding data to the resource section of an executable (.exe or .dll). The API also provides mechanisms for finding this data. With the use of the API we can create a helper application to embed as many custom resources as you want after you have built your executable.

For this example I will assume the data we adding to the executable is not data you would normally find in a resource. This means we will be adding a custom resource.

Let us say we want to add a Java class file to our executable so that we can find this class file at runtime without knowing anything about the current Java CLASSPATH or the file system. Once we’ve extracted the class file we could use it to define a class that would then be used by the Java Virtual Machine to do the work we want (presumably somewhere else we’ll be instrumenting Java class files so they know about the Java class we just defined).

We need a few things first, which we will also need when we come to extract the resource from the executable.

  • Executable to add the resource to.
  • Type name for the custom resource.
  • Name for the custom resource.
  • Data for the custom resource.

For our Java class file example, type could be “CLASSFILE”, name could be “myJavaSpy” and data would be the byte code for the class myJavaSpy which we would load from the file myJavaSpy.class (having previously compiled it from myJavaSpy.java).

The API

BeginUpdateResource

    HANDLE BeginUpdateResource(const TCHAR *executableName,
                               BOOL        fDeleteExistingResources);

Call BeginUpdateResource() to open the specified executable and return a resource handle. Pass TRUE for the second argument to erase all existing resources, pass FALSE to keep any existing resources in the executable.

UpdateResource

    BOOL UpdateResource(HANDLE  hUpdate,
                        LPCTSTR lpType,
                        LPCTSTR lpName,
                        WORD    wLanguage,
                        LPVOID  lpData,
                        DWORD   cbData);

Call UpdateResource() to update a resource in the executable represented by the handle hUpdate. Specify the type, name, language (locale) and the data with the remaining arguments. For our example above lpType would be “CLASSFILE” and lpName would be “myJavaSpy”. Pass MAKELANGID(LANG_NEUTRAL, SUBLANG_NEUTRAL) for language. Pass the java byte code and the lenght of the byte code for the last two arguments.

EndUpdateResource

    EndUpdateResource(HANDLE hUpdate,
                      BOOL   fDiscard);

Call EndUpdateResource() to finish updating the resource. If you wish to discard your changes, pass TRUE as the second argument. If you wish to keep your changes, pass FALSE as the second argument.

Putting it together

    HANDLE hUpdateRes;

    // Open the file to which you want to add the dialog box resource. 
	
    hUpdateRes = BeginUpdateResource(executableName, 
                                     FALSE);          // do not delete existing resources
    if (hUpdateRes != NULL) 
    {  
        BOOL   result; 
	
        // Add the dialog box resource to the update list. 
	
        result = UpdateResource(hUpdateRes,
                                customType,
                                customName,
                                MAKELANGID(LANG_NEUTRAL, SUBLANG_NEUTRAL),
                                bytes,
                                numBytes);
	if (result) 
	{ 
            // Write changes to the input DLL and then close it
	
            EndUpdateResource(hUpdateRes, FALSE);
        }
    } 

First we call BeginUpdateResource() to open the executable for resource updating. We pass FALSE as the second argument to make sure we keep the existing resources and only add our new resource. This calls returns an update handle.

If the call to BeginUpdateResource() is successful we received a non NULL update handle. We use to call UpdateResource() passing the type and name of resource data we wish to update along with the data to update and its length. In this example we have specified a neutral locale.

Finally we call EndUpdateResource() to finish updating the resource and to write the results back to the executable (pass FALSE as the second argument).

addResourceToDLL

addResourceToDLL.exe is command line program that you can add to your post-build process to embed custom resources into your EXE/DLL as you build. It has a quiet mode so that you can suppress any information and/or error messages it may emit. I don’t use the quiet mode, I like to see the confirmation message that it succeeded embedding data into the DLL. Run without arguments to get the help message.

Help summary

All arguments are mandatory unless noted otherwise.

  • -moduleName pathToDLL (or EXE)
  • -customResource pathToCustomResource
  • -customType type
  • -customName name
  • -quiet (optional)
  • Example:

    addResourceToDLL.exe -moduleName c:\myJavaDetective\myJavaDetective.dll -customResource c:\myJavaDetective\myJavaSpy.class -customType CLASSFILE -customName myJavaSpy

    The example above embeds the myJavaSpy.class file into myJavaDetective.dll with the type “CLASSFILE” and name “myJavaSpy”.

    Download

    Download the addResourceToDLL source code.

    Download the addResourceToDLL executable.

    In the next article I will show you how to read the embedded data from the resource.

    Share

Monitoring memory use in a JNI DLL called from Java

By , May 8, 2010 1:35 pm

Java is a garbage collected language that allows you to extend the language with code written in C and C++.

Given that the extensions are written in C or C++ the normal tools for monitoring Java memory usage will not report the C/C++ memory usage. So how do you monitor the memory usage of these JNI extensions when called from Java? This article is going to explain how to do this task.

Building an example JNI test

The first thing to do is to create an example to work with. The post Creating a JNI example for testing describes how to create a JNI example (both Java code and JNI C/C++ code, with downloadable source and project files). Please download the example code and build the example.

Monitoring JNI Memory

Monitoring memory in JNI extensions is straightforward. Java programs are executed by running a Java Virtual Machine (JVM). These are typically named java.exe, javaw.exe, jre.exe, jrew.exe. We can just launch java.exe with a C/C++ memory leak detection software tool and monitor the results. For this example we are going to monitor java with C++ Memory Validator. The image below shows the launch dialog (note if you are using Memory Validator for the first time you will be using the launch wizard, which is slightly different)

Memory Validator launch dialog showing launch of Java application

Items to note:

  • Application is set to the path to java.exe. C:\Program Files (x86)\Java\jdk1.5.0_07\bin\java.exe
  • Arguments is set to the name of the class to execute. Main
  • Startup directory is set to the directory containing the class to execute (and the native DLL to monitor). E:\om\c\memory32\testJavaJNI
  • If you wish to set the CLASSPATH you can set it in the Environment Variables part of the launch dialog. If you do not set the CLASSPATH here (as in this example) the CLASSPATH will be taken from the inherited environment variables of Memory Validator’s environment. For this example CLASSPATH is set in the global environment variables and thus does not need to be set on the launch dialog.

Click Go! to start the Java program. Java is started, Memory Validator monitors the application and records all memory allocations and deallocations. Any memory not deallocated by the end of the program is a leak.

JNI Leaks with no filtering

As you can see from the screenshot above, there is a quite a bit of memory left over after a simple run of this example Java program which does very little except load a native extension that prints Hello World! twice and deliberately leaks one 20 byte chunk of memory. The example image indicates there are 857 items, some of which are handles, the remainder C/C++ memory allocations. There are 3493 events. The memory leak we are interested in occurs at event 2985.

Clearly this is inefficient. To find memory leaks in your code you are going to have to wade through all the noise of the memory allocations made by Java and the JVM. There must be a better way!

There is. We’ll focus only on the native DLL that prints the Hello World! messages. Open the settings dialog and go to the Hooked DLLs section.

Setting up DLL filters to focus on the JNI DLL

  • Select the second radio box to indicate that only the DLLs we list will be monitored.
  • Now click Add Module… and select the HelloWorldImp.dll.
  • Click OK.

Memory Validator is now configured to monitor only HelloWorldImp.dll for memory and handle allocations.
Relauch the java application with these new settings.

JNI Leaks with DLL filtering

As you can see from the picture above much less data is collected. A total of 54 events compared to the previous session’s 3493 events. This is much more managable.

The list of items Memory Validator reports for this run contains only 11 events. This reduced amount makes it very easy to identify errors in the DLL.

  • 8 events are DLL loads (MV always reports DLL loads regardless of settings)
  • A one time, per-thread, buffer allocation inside printf
  • A memory leak that is deliberately present in the example JNI code
  • The final event is the status information for the application at the end of its run

If you don’t wish to see the DLL loads you can filter them out with a global, session or local filter.

Detail view of source code of memory leak in JNI code

The image above shows the source code of the location of the leaking memory in the JNI extension.

Conclusion

Monitoring memory allocations in JNI Dlls called from Java is a straightforward task.

Things to remember:

  • Ensure your JNI DLL is compiled with debugging information and linked with debugging information.
  • Ensure your JNI DLL debug information is present (put the PDB file in the same directory as the DLL).
  • Ensure your CLASSPATH is set correctly so that when Memory Validator starts your Java application the correct CLASSPATH is used.
Share

Creating a JNI example for testing

By , May 8, 2010 11:28 am

The Java language can be extended using C and C++ to write extensions that are called using the Java Native Interface (JNI).
The JNI extensions is created by creating a Java class definition that defines the class methods as native and loads a native DLL to do that.

class HelloWorld
{
    public native void displayMessage();

    static
    {
        System.loadLibrary("HelloWorldImp");  // load HelloWorldImp.dll to implement the native interfaces
    }
}

A full description of this process can be found on Matthew Mead’s JNI page.

Example JNI code for test

We’ve created an example based on this which also allocates some memory, deallocates some memory and causes a memory leak. You can
download the example code. To build the example follow these steps:

  1. Open a command prompt and change the current directory to the testJavaJNI directory in the example.
  2. Edit buildJava.bat, createJNIHeaders.bat, runJava.bat to point to your JAVA SDK directory.
  3. Run buildJava.bat
  4. Run createJNIHeaders.bat
  5. Build the HelloWorldImp.dll. The Visual Studio 6 Project (*.dsp) and Visual Studio Solution (*.sln) files are in the HelloWorld directory.

After the DLL has been built the DLL and its PDB file will be copied to the same location as the Java class files so that the DLL can be found by the Java Virtual Machine when it attempts to load the native implementation of HelloWorld.

To run the example, run runJava.bat. The example prints “Hello World!” twice. The first time does not leak memory, the second time does leak memory.

Command prompt building and testing JNI interface

Share

Panorama Theme by Themocracy