Instrumenting OSGi Bundles Through Equinox Adaptor Hooks

Since Eclipse 3.2 the Equinox OSGi implementation offers so called adaptor hooks. This paper explains how code coverage can easily be added to the Equinox OSGi runtime via on-the-fly code instrumentation employing these hooks. A working example based on EMMA is provided that allows measuring code coverage of any Eclipse application.

Marc R. Hoffmann, Mountainminds GmbH & Co. KG, April 2007 (updated March 2008)

This paper describes a technique how to integrate the EMMA code coverage library into OSGi/Equinox applications. Please also consider our new code coverage library JaCoCo which greatly simplifies the integration as it provides on-the-fly coverage data recording for any Java application (including OSGi frameworks).

The plug-in based architecture of Eclipse is the foundation of modular design and extensibility. EclEmma for example inserts additional menu items to the workbench or adds the new launch mode Coverage to the existing debug infrastructure. The component model underneath making this possible is based on the OSGi standard which manages the lifecycle and contracts between all the bundles forming an application. OSGi Release 4 introduced so called extension bundles (chapter 3.14) which can be configured to become a part of the OSGi runtime itself and contribute implementation specific functionality.

Eclipse comes with its own OSGi implementation called Equinox, which itself offers several hooks that can be used by extension bundles to modify the behaviour of the OSGi platform. At EclipseCon 2007 in Santa Clara I became aware of so called adaptor hooks offered by Equinox. A impressive demonstration showed how these hooks can be used to dynamically apply aspect-oriented techniques: Little flashing planets for each plug-in graphically visualized the plug-ins currently executing code. So why shouldn't the same mechanism allow imposing code coverage analysis on any Eclipse application?

Hooking into Equinox

Framework adaptor hooks are available since Eclipse 3.2 and specified as call-back interfaces in the org.eclipse.osgi.baseadaptor.hooks package of the org.eclipse.osgi plug-in. By providing implementations of these hook interfaces one can inject the OSGi runtime with new or additional behavior in respect of framework lifecycle, bundle storage or class loading. Unfortunately there is no JavaDoc for this package included with the online help. To explore the interfaces simply import the org.eclipse.osgi into your Eclipse workspace. Enabling your hooks requires the following steps:

• Implement a org.eclipse.osgi.baseadaptor.HookConfigurator that registers your adaptor hook implementations with the hook registry in its addHooks() method.
• Place a file hookconfigurators.properties in your extension bundle containing a property definition hook.configurators that points to your HookConfigurator implementation class.
• To make your bundle a framework extension it must be enlisted in the system property osgi.framework.extensions when launching Eclipse. This can be achieved by specifying a -D JVM parameter or by adding an entry to the ./configuration/config.ini file.

Note that Equinox requires extension bundles to be placed in the same directory as the org.eclipse.osgi plug-in. For a installed Eclipse instance this means extension bundles have to be placed directly in the plugins/ directory. If you want to debug extension bundles within PDE, you need to import the source version of the org.eclipse.osgi plug-in into your workspace.

Code Coverage via Class File Instrumentation

There are different possible techniques to determine code coverage during a program run. The one used by several coverage tools like EMMA is byte code instrumentation: In this approach probes are added to the original Java class files. These extra instructions record which parts of the program are executed. While the instrumentation process must be performed before the class files are loaded by the JVM, the instrumented code typically needs a extra runtime library where the coverage information is collected and written e.g. to a local file when the program terminates.

The instrumentation process can either be performed on the class files before the program is launched or during the class loading process. The latter requires hooking into the class loading mechanism of the application – which is easily possible for OSGi applications using Equinox adaptor hooks .

Byte Code Instrumentation Hook

To hide the complexity of the sparely documented EMMA API we abstract a code coverage analyzer with a simple interface that may also work for any other code coverage technologies. For implementation details you may study the EMMAAnalyzer implementation.

public interface ICoverageAnalyzer {

  /**
   * Called when the OSGi framework is started. Can be used for initialization
   * tasks.
   */
  public void start();

  /**
   * Called when the OSGi framework shuts down. Here we can e.g. write a
   * coverage report.
   */
  public void stop();

  /**
   * For each class definition loaded from a bundle this method is called. The
   * method may return a instrumented version of the class or null, if the
   * class should not be modified.
   * 
   * @param bundleid
   *            symbolic name of the bundle
   * @param classname
   *            full qualified VM class name
   * @param bytes
   *            original class file bytes
   * @return instrumented class file bytes or null
   */
  public byte[] instrument(String bundleid, String classname, byte[] bytes);

  /**
   * Class file instrumentation might introduce dependencies on a vendor
   * specific runtime library. The list of Java packages returned by this
   * method will be made available to the instrumented plug-ins.
   * 
   * @return comma separated list of Java package names
   */
  public String getRuntimePackages();

}

There are two adapter hook interfaces that we can implement to drive the coverage analyzer: With the AdaptorHook we get involved with the framework lifecycle (start/stop), the ClassLoadingHook allows us to modify raw class files before the classes get defined.

public class InstrumentationHook implements AdaptorHook, ClassLoadingHook {

  private final ICoverageAnalyzer analyzer;

  public InstrumentationHook(ICoverageAnalyzer analyzer) {
    this.analyzer = analyzer;
  }

  public void frameworkStart(BundleContext context) throws BundleException {
    analyzer.start();
  }
  
  public void frameworkStop(BundleContext context) throws BundleException {
    analyzer.stop();
  }
  
  public byte[] processClass(String name, byte[] classbytes,
      ClasspathEntry classpathEntry, BundleEntry entry, ClasspathManager manager) {
    return analyzer.instrument(manager.getBaseData().getSymbolicName(), name, classbytes);
  }

  // ... stubs for remaining interface methods
  
}

Providing Additional Runtime Classes to Instrumented Bundles

When executing EMMA instrumented Java classes additional runtime classes are required, basically providing functionality for holding and saving collected coverage data. These few runtime classes are contained within the emma.jar library itself. Instrumenting loaded Java classes with EMMA adds a new runtime dependency that is neither explicitly declared in OSGi bundles under inspection nor available within to the bundles during a standard Eclipse launch. As a consequence all instrumented bundles will fail with a NoClassDefFoundError.

To solve this the adapter hooks come into picture again: If the coverage extension exports the coverage runtime classes, e.g. has the following manifest header

  Export-Package: com.vladium.emma.rt

and each instrumented bundle would have a DynamicImport-Package declaration for this package everything would work. Of course we don't want to modify the manifest files of all our bundles under test. After I published the first version of this paper Thomas Watson pointed me to some internal Equinox implementation classes that could do the job on-the-fly. For this we populate another call-back of the ClassLoadingHook:

public class InstrumentationHook implements AdaptorHook, ClassLoadingHook {

  // ...

  public BaseClassLoader createClassLoader(ClassLoader parent,
      ClassLoaderDelegate delegate, BundleProtectionDomain domain,
      BaseData data, String[] bundleclasspath) {
    BundleLoader loader = (BundleLoader) delegate;
    try {
      loader.addDynamicImportPackage(ManifestElement.parseHeader(
        Constants.DYNAMICIMPORT_PACKAGE, analyzer.getRuntimePackages()));
    } catch (BundleException be) {
      throw new RuntimeException(be);
    }
    return null;
  }

}

Making assumptions about implementation details of the OSGi framework implementation looks like bad coding style. But as an adaptor hooks you're anyway closely tight to Equinox implementation. And maybe one day Equinox will offer official APIs to modify bundle headers within adaptor hooks.

Running the Code Coverage Framework Extension

So now let's try to get the example provided with this paper running. Make sure you use at least Eclipse 3.2, the version that introduced adaptor hooks in Equinox.

The Equinox framework extension com.mountainminds.eclemma.osgihook measures Java code coverage for all bundles of a Equinox based OSGi system, for example a Eclipse RCP application. If the extension is installed code coverage data is automatically recorded and written out when the OSGi system terminates. In the current working directory a folder coverage-nnn will be created and populated with a HTML report and a *.es file for further analysis (e.g. import into EclEmma). The framework extension can be used for standalone OSGi instances as well as for program launches from the Eclipse plug-in development environment (PDE).

Standalone Mode

In case you start your application directly with eclipse.exe perform these steps to add the code coverage extension:

• Get the compiled bundle com.mountainminds.eclemma.osgihook_2.0.0.v20080311.zip from the Sourceforge download page.
• Unzip the file in your Eclipse installation root. The expanded bundle com.mountainminds.eclemma.osgihook_2.0.0.v20080311 has to be located in the plugins folder. This must be the same directory where the org.eclipse.osgi bundle is located. Using e.g. an extension site will not work! The com.mountainminds.eclemma.osgihook must not be jared, as it contains another JAR wich can not be loaded from a packaged bundles in case of framework extensions.
• Insert the following line into the ./configuration/config.ini file of your Eclipse installation:

  osgi.framework.extensions=com.mountainminds.eclemma.osgihook

PDE Mode

If you want to study or modify the extension bundle you may directly import it to your PDE workspace and launch a Eclipse application in development mode. The extension works for any OSGi based launch type, i.e. Eclipse application OSGi framework and JUnit plug-in test.

• Import the latest version of the extension bundle from the SVN repository at https://eclemma.svn.sourceforge.net/svnroot/eclemma/research/com.mountainminds.eclemma.osgihook.
• Due to the fact that Equinox needs any framework extension co-located with the OSGi framework bundle you need to also import org.eclipse.osgi into your workspace (from the Plug-Ins view's context menu select Import As → Source Project).
• Take a existing launch configuration or create a new one. Make sure that the com.mountainminds.eclemma.osgihook and the org.eclipse.osgi bundle are enabled.
• On the Arguments tab insert the following definition in the VM arguments field:

  -Dosgi.framework.extensions=com.mountainminds.eclemma.osgihook

Advantages and Limitations

The obvious advantage of the technique described here is that code coverage can be determined for any OSGi bundle respectively Eclipse plug-in without pre-processing the bundles, i.e. physically modifying them. Using this technique for test runs in automated build environments makes extra build targets for instrumented versions of the bundles superfluous. While the application start-up is slightly delayed due to the required byte code instrumentation, there is no significant slow-down during execution.

Java classes required are instrumented on-the-fly, classes that are not loaded don't need to be processed. While this looks like an advantage at first glance it comes with the drawback that there is also no meta information recorded for these classes. They don't appear on the coverage report; completely untested classes will not be shown. A simple workaround here would be to collect the complete meta data "offline" on all involved plug-ins without actually instrumenting them.

Outlook

While the provided example is kept as simple as possible just to demonstrate the principles, a more evolved version may add additional functionality like filtering the analyzed bundles. Another interesting topic for investigation would be how the framework extension can be used in automated build environments for coverage reports.

This adaptor bundle technique would be a great benefit for EclEmma. In-place instrumentation is no longer required for Eclipse launches. As a consequence any plug-in (not only the ones in the workspace) could be analyzed. Anyhow there are some issues for investigation:

• Due to limited resources there is a single development branch of EclEmma only which supports all Eclipse versions from 3.1 on. Adopting framework extensions would mean to drop Eclipse 3.1 support or create a second development branch.
• Equinox framework extensions must be placed in the same directory than the OSGi runtime. This requires adding the coverage extension to your Eclipse installation – a undesirable modification that may also be restricted by access right. Another option would be to put the extension along with a copy of org.eclipse.osgi to a third place, which may interfere with other installed extensions.

References

• Eclipse Equinox
  http://www.eclipse.org/equinox/
• wiki.eclipse.org, Adaptor Hooks, 2006
  http://wiki.eclipse.org/index.php/Adaptor_Hooks
• wiki.eclipse.org, Equinox Boot Delegation, 2006
  http://wiki.eclipse.org/index.php/Equinox_Boot_Delegation
• OSGi Alliance, OSGi Service Platform Release 4, 2006
  http://www.osgi.org/osgi_technology/download_specs.asp?section=2#Release4
• EMMA - a free Java code coverage tool, Vlad Roubtsov, 2006
  http://emma.sourceforge.net/
• EclEmma - Java Code Coverage for Eclipse
  http://www.eclemma.org/

Resources

• Compiled bundle com.mountainminds.eclemma.osgihook_2.0.0.v20080311.zip
  https://sourceforge.net/projects/eclemma/files/05_Research/Examples/
• SVN URL (browse repository)
  https://eclemma.svn.sourceforge.net/svnroot/eclemma/research/com.mountainminds.eclemma.osgihook
• JaCoCo and EclEmma Users group for any remarks about this paper
  https://groups.google.com/d/forum/jacoco

Change Log

• April 2007: Initial Draft, the example was based on an ugly hack to inject the code coverage runtime into the instrumented plug-ins. As already stated in the first version of the paper, this workaround will not work for proper OSGi runtimes, where the boot class loader is not visible to the plug-ins. (first version)
• March 2008: New version solving the problem stated above with DynamicImport-Package manifest headers added to the plug-ins through the adapter hook. This solution was proposed by Thomas Watson, thanks!