View source for Context Class Loader Enhancements

== Overview ==
This design defines the behavior of the context class loader in the Equinox OSGi Framework.  Many java libraries exist that use the context class loader.  In an OSGi Framework environment the context class loader is not defined.  This leads to class loading issues when trying to package such libraries into bundles.  This document describes the solutions offered by the Equinox OSGi Framework to solve some of these issues.

== Terminology ==

=== Context Class Loader ===
A context class loader is associated with a Thread.  This class loader is provided by the thread creator to code running in the thread for the purpose loading classes and resources.  This class loader is useful for loading classes and resources that are not always available to the class loader which loaded the code running in the thread.

=== Context Switch ===
For the purpose of this design a Context Switch is defined as the point in an execution sequence when a component boundary is crossed from one component to the next.

=== Class forName ===
A classic Java mechanism for dynamic class discovery and loading. It uses the current classloader to look for and load the requested class. The current classloader is the classloader that loaded the class containing the method executing the forName(String) call.

=== Context Finder ===
A special class loader that finds that first Bundle classloader on that stack and delegates load requests to that classloader.

=== Buddy Policy ===
A policy that allows a bundle to declare it needs help from other bundles to load classes.  This type of policy does not cause a hard dependency wire to be used to load the classes from a buddy.

== Problem Description ==
Most OSGi-biased developers will concede that the majority of java libraries out there are not currently shipped as OSGi bundles and are not aware of the Modular Layer in the OSGi Framework.  Many existing java libraries are designed to run inside a container (J2EE container, Applet container etc).  Such containers explicitly define execution boundaries between the various components running within the container.  The container controls the execution boundaries and knows when a boundary is being crossed from one component to the next.

This level of boundary control allows a container to switch the context of a thread when a component boundary is crossed.  Typically when a container detects a context switch it will set the context class loader on the thread to a class loader associated with the component which is being entered.  When the component is exited then the container will switch the context class loader back to the previous context class loader.

The OSGi Framework specification does not define what the context class loader should be set to and does not define when it should be switched.  Part of the problem is the Framework is not always aware of when a component boundary is crossed.  For some operations the Framework is aware of component boundaries, for example, when calling out to BundleActivators and event listeners.  Switching the context class loader when calling out to these types of objects will not solve a large number of usecases.

Consider the following example:

*Bundle W exports a package “some.foo.library”.  This package contains code which uses the context class loader to load additional classes.  Imagine it has some SPI like behavior where it loads classes based on some properties file similar to how javax.xml.parsers package works.
*Bundle X exports a package “some.foo.stuff”.  This package contains a service interface “some.foo.stuff.BarService”.
*Bundle Y imports the “some.foo.stuff” package and registers a “BarService” implementation with the service registry.
*Bundle Y also imports the “some.foo.library” and expects the exporter to be able to load classes from bundle Y.  Bundle Y uses the “some.foo.library” package in its implementation of the “BarService”.
*Bundle Z imports the “some.foo.stuff” package and gets the “BarService” from the service registry and uses it.

In this scenario imagine the framework sets the context class loader to Bundle Z’s class loader before calling its BundleActivator to start the bundle.  In the BundleActivator of Z it gets the BarService if it is available and start making calls to it.  At this point code in Bundle Y will get executed because it contains the implementation of the BarService.  The implementation of the BarService then uses the package “some.foo.library”.  This will cause the code in Bundle W to run that uses the context class loader.  At this point the context class loader will be set to Bundle Z’s class loader.  This class loader does not have access to the content in Bundle Y and will result in Bundle W’s library code not being able to load classes from Bundle Y.

As a work around to this issue Bundle Y could wrap all calls to code in the package “some.foo.library” with context class loader switches, but this puts a great burden on any developer using the package “some.foo.library”.  This also calls into question the purpose of the Framework switching the context class loader at all (to Bundle Z’s class loader in the example above).  Any time more than one component boundary is crossed the original context class loader will likely not be the one that is needed.

This design specifies a behavior for the context class loader that should help solve some of these issues.

== Requirements ==

# The solution MUST reduce or illiminate the need for code packaged in a bundle to call Thread.setContextClassLoader while calling library code
# Library bundles MUST be able to use the context class loader to access classes needed by the library
# other reqs ...

== Technical Solution ==
To solve the requirements this design introduces the buddy class loading and context finder concepts
----
=== Buddy Class Loading ===
Buddy class loading offers an integration strategy that does not require modifying the Java code of existing libraries that use the Class.forName(String) approach to dynamically discover and load classes.  The mechanism works as follows:

*A Bundle declares that it needs the help of other bundles to load classes
*The Bundle identifies the kind of help they need by specifying a buddy policy.  The policy defines what kind of bundles will be considered to be buddies.
*When a bundle class loader fails to find a desired class through the normal OSGi delegation model (i.e. Import-Package, Require-Bundle, and local classpath), its buddy policy is invoked
*The invoked buddy policy discovers a set of buddies and consults each one in turn until either the class is found or the list is exhausted.

==== Search Order ====

The following is a modified class/resource search order:

Frameworks must adhere to the following rules for class or resource loading.  When a bundle’s class loader is requested to load a class or find a resource, the search must be performed in the following order:
#If the class or resource is in a java.* package, the request is delegated to the parent class loader; otherwise, the search continues with the next step. If the request is delegated to the parent class loader and the class or resource is not found, then the search terminates and the request fails.
#If the class or resource is from a package included in the boot delegation list (org.osgi.framework.bootdelegation), then the request is delegated to the parent class loader. If the class or resource is found there, the search ends.
#If the class or resource is in a package that is imported using Import-Package or was imported dynamically in a previous load, then the request is delegated to the exporting bundle’s class loader; otherwise the search continues with the next step. If the request is delegated to an exporting class loader and the class or resource is not found, then the search terminates and the request fails.
#If the class or resource is in a package that is imported from one or more other bundles using Require-Bundle, the request is delegated to the class loaders of the other bundles, in the order in which they are specified in this bundle’s manifest. If the class or resource is not found, then the search continues with the next step.
#The bundle’s own internal bundle class path is searched. If the class or resource is not found, then the search continues with the next step.
#Each attached fragment’s internal bundle class path is searched. The fragments are searched in ascending bundle ID order. If the class or resource is not found, then the search continues with the next step.
#If the class or resource is in a package that is exported by the bundle or the package is imported by the bundle (using Import-Package or Require-Bundle), then the search ends and the class or resource is not found.
#Otherwise, if the class or resource is in a package that is imported using DynamicImport-Package, then a dynamic import of the package is now attempted. An exporter must conform to any implied package constraints. If an appropriate exporter is found, a wire is established so that future loads of the package are handled in Step 3. If a dynamic wire is not established, then the search continues to step 10.
#If the dynamic import of the package is established, the request is delegated to the exporting bundle’s class loader. If the request is delegated to an exporting class loader and the class or resource is not found, then the search terminates and the request fails.
#If the bundle has declared a buddy policy then the request is delegated to each buddy bundle class loader until either the class or resource is found or all of the buddy bundle class loaders have been delegated too.
When delegating to another bundle class loader, the delegated request enters this algorithm at step 4 and exits at step 9.  The buddy policy is not used when delegating to another bundle class loader.

==== Buddy Policy ====
A buddy policy defines a policy for selecting buddies that will be used to for buddy class loading.  Currently Equinox defines a number of built-in Buddy policies.  The following are the buddy policies available in Equinox

*dependent - Consults all bundles that directly or indirectly depend on the bundle.  An indirect dependency can be introduced by bundles that use Require-Bundle with the visibility directive set to reexport.  Note that this casts a rather wide net and may introduce performance problems as the number of bundles increase.
*registered - Consults all dependent bundles that explicitly register themselves as buddies to the bundle.  This is similar to the ''dependent'' policy but scopes down the number of bundles that will be consulted as buddies.
*global - Consults the available packages exported in the global pool of exported packages.
*app - Consults the application classloader.
*ext - Consults the extension classloader.
*boot - Consults the boot classloader.

Note when using the above policies all packages available from the source buddy will be available.  In the case of '''dependent''', and '''registered''', all packages from the bundle are available even if the package is not exported.  There is no way to scope down the available packages from the buddy sources.

==== Eclipse-BuddyPolicy Header ====
The Eclipse-BuddyPolicy header allows a bundle to declare a comma-separated list of buddy loading policy names that are to be used for the bundle.

The syntax of the Eclipse-BuddyPolicy header is the following

 Eclipse-BuddyPolicy ::= buddy-policy ( ',' buddy-policy )*
  buddy-policy ::= ('dependent' | 'registered' | 'global' | 'app' | 'ext' | 'boot')


==== Eclipse-RegisterBuddy ====

The Eclipse-RegisterBuddy header is used to declare a comma-separated list of symbolic names of bundles that this bundle should be a registered buddy to.  The bundles with the specified symbolic names must use the '''registered''' buddy policy in order for this bundle to be a registered buddy.

Note that the following conditions must be met before a bundle '''X''' can become a registered buddy of another bundle '''Y''':

# The bundle '''Y''' must specify the '''registered''' buddy policy (i.e. Eclipse-BuddyPolicy: registered)
# The bundle '''X''' must specify the symbolic name of '''Y''' in the Eclipse-RegisterBuddy header (i.e Eclipse-RegisterBuddy: Y)
# The bundle '''X''' must be dependent on a package exported by bundle '''Y'''.  This can happen through either a Require-Bundle or Import-Package constraint.

The syntax of the Eclipse-RegisterBuddy header is the following:

 Eclipse-RegisterBuddy ::= bundle-symbolic-name ( ',' bundle-symbolic-name )*

----

=== Context Class Loader ===
Since Java 1.2, the Class.forName(String) mechanism has been largely superseded by context classloading. As such, most modern class libraries use a context classloader. This section describes how the use of a context class loader can be transparently converted into something equivalent to Class.forName(String). Doing this allows the buddy loading mechanism described above (and DynamicImport-Package) to be used to eliminate ClassNotFoundExceptions and NoClassDefFoundErrors.

Each Java Thread has an associated context classloader field that contains a classloader. The classloader in this field is set, typically by the application container, to match the context of this current execution. That is, the field contains a classloader that has access to the classes related to the current execution (e.g., Web request being processed). Libraries such as log4j access and use the context classloader with the updated AppenderHelper code pattern below:

<source lang="java">
public class AppenderHelper {
  private Appender createAppender(String appenderName) {
    ClassLoader loader = Thread.currentThread().getContextClassLoader();
    Class appenderClass = loader.loadClass(appenderName);
    return (Appender)appenderClass.newInstance();
  }
}
</source>

By default, the context classloader is set to be the normal Java application classloader. That is, the use of the context classloader in normal Java application scenarios is equivalent to using Class.forName(String) and there is only one classloader, the application classloader. When running inside the Framework, however, the code pattern outlined above fails because:

*By default, the Framework does not consult the application classloader. OSGi-based applications put their code in bundles and export particular packages instead of placing all the code on the normal Java application classpath.
*The Framework cannot detect bundle context switches and set the context classloader as required. That is, there is no way to tell when execution context shifts from one bundle to the next as is done in Web application servers.

These characteristics, combined with the compositional nature of OSGi, mean that the value of the context classloader field is seldom useful.

Clients can, however, explicitly set the context classloader before calling libraries that use the context classloader. The snippet below shows an example of calling log4j using this approach:

<source lang="java">
Thread thread = Thread.currentThread();
ClassLoader loader = thread.getContextClassLoader();
thread.setContextClassLoader(this.getClass().getClassLoader());
try {
  ... log4j library call that calls AppenderHelper.createAppender() ...
} finally {
  thread.setContextClassLoader(loader);
}
</source>

First the current context classloader is saved. The context classloader is then set to an appropriate value for the current execution and log4j is called. log4j’s AppenderHelper uses the context classloader, so in this case, it uses the client’s classloader (e.g., this.getClass().getClassLoader()). When the operation is finished, the original context classloader is restored.

The assumption here is that the client’s classloader is able to load all required classes. This may or may not be true. Even if it can, the coding pattern is cumbersome to use and hard to maintain for any significant number of library calls. Ideally, log4j would be able to dynamically discover the context relevant to a particular classloading operation. The 'context finder' enables this.

==== Context Finder ====
The context finder is a kind of ClassLoader that is installed by the Equinox Framework as the default context classloader. When invoked, it searches down the Java execution stack for a classloader other than the system classloader. In the AppenderHelper example above, it finds the log4j bundle’s classloader (the one that loaded AppenderHelper). The context finder then delegates the load request to the discovered classloader.

This mechanism transforms log4j’s call to getContextClassLoader().loadClass(String) to the equivalent Class.forName(String) call using log4j’s classloader to load the given class. Now the buddy classloading techniques can be applied to help log4j load the needed appender classes.

The net effect is that clients of log4j do not have to use the cumbersome coding
pattern outlined above even though the libraries they call use the context
classloader. This approach generalizes to other context classloading situations.