Jetty/Tutorial/Configuring the Jetty Overlay Deployer

Jetty Overlayed WebApp Deployer

The Jetty Overlay Deployer allows you to overlay multiple WAR files so that you can customize, configure, and deploy a web application without the need to unpack, modify and repack the WAR file. This has the following benefits:

• You can keep the WAR file immutable, even signed, so that it is clear which version you have deployed.
• All modifications you make to customise/configure the web application are separate WARs, and thus are easily identifiable for review and migration to new versions.
• You can create a parameterised template overlay that contains common customisations and configuration that apply to many instances of the web application (for example, for multi-tenant deployment).
• Because the layered deployment clearly identifies the common and instance specific components, Jetty is able to share classloaders and static resource caches for the template, greatly reducing the memory footprint of multiple instances.

This document describes how to configure Jetty to use the Overlay deployer, and how to deploy multiple instances of a web application, in this example, JTrac.

Overview

Customising, configuring and deploying a web application bundled as a WAR file frequently includes some or all of these steps:

• Editing the WEB-INF/web.xml file to set init parameters, add filters/servlets or to configure JNDI resources.
• Editing other application specific configuration files in WEB-INF/*.
• Editing container specific configuration files in WEB-INF/* (for example, jetty-web.xml or jboss-web.xml).
• Adding/modifying static content such as images and css to style/theme the web application.
• Adding Jars to the container classpath for Datasource and other resources.
• Modifying the container configuration to provide JNDI resources.

The result is that the customisations and configurations blend into both the container and the WAR file. If you upgrade either the container or the base WAR file to a new version, it can be a very difficult and error prone task to identify all the changes that you have made, and to reapply them to a new version.

Overlays

To solve the problems highlighted above, Jetty 7.4 introduces WAR overlays (a concept borrowed from the Maven WAR plugin). An overlay is basically just another WAR file, whose contents are merged on top of the original WAR so that you can add or replace files. Jetty overlays also allow mixin fragments of web.xml, so the you can modify the configuration without replacing it.

Jtrac Overlay Example

The jtrac issue tracking web application is a good example of a typical web application, as it uses the usual suspects of libs: spring, hibernate, dom4j, commons-*, wicket, etc. The files for this demonstration are available in overlays-demo.tar.gz. You can expand it on top of the jetty distribution, but tutorial expands it to /tmp and installs the components step-by-step:

cd /tmp
wget http://webtide.intalio.com/wp-content/uploads/2011/05/overlays-demo.tar.gz
tar xfvz overlays-demo.tar.gz
export OVERLAYS=/tmp/overlays

Configuring Jetty for Overlays

Overlays support is included in jetty distributions from 7.4.1-SNAPSHOT onwards, so you can download a distribution from oss.sonatype.org or maven central(once 7.4.1 is released) and unpack into a directory.    The start.ini file then needs to be edited so that it includes the overlay option and configuration file.   The resulting file should look like:

OPTIONS=Server,jsp,jmx,resources,websocket,ext,overlay
etc/jetty.xml
etc/jetty-deploy.xml
etc/jetty-overlay.xml

The smarts of this are in etc/jetty-deploy.xml files, which installs the OverlayedAppProvider into the DeploymentManager. You can then start Jetty normally:

java -jar start.jar

Jetty is now listening on port 8080, but with no webapp deployed. The rest of the tutorial should be conducted in another window with the JETTY_HOME environment set to the jetty distribution directory.

Installing the WebApp

You can download and deploy the WAR file for this demo using the following commands, which essentially downloads and extracts the WAR file to the $JETTY_HOME/overlays/webapps directory.

cd /tmp
wget -O jtrac.zip http://sourceforge.net/projects/j-trac/files/jtrac/2.1.0/jtrac-2.1.0.zip/download
jar xfv jtrac.zip jtrac/jtrac.war
mv jtrac/jtrac.war $JETTY_HOME/overlays/webapps

When you have run these commands (or equivalent), you see in the Jetty server window a message saying that the OverlayedAppProvider has extracted and loaded the WAR file:

2011-05-06 10:31:54.678:INFO:OverlayedAppProvider:Extract jar:file:/tmp/jetty-distribution-7.4.1-SNAPSHOT/overlays/webapps/jtrac-2.1.0.war!/ to /tmp/jtrac-2.1.0_236811420856825222.extract
2011-05-06 10:31:55.235:INFO:OverlayedAppProvider:loaded jtrac-2.1.0@1304641914666

Unlike the normal webapps dir, loading a war file from the overlays/webapp dir does not deploy the webapplication.  It simply makes it available to be used as the basis for templates and overlays.

Installing a Template Overlay

A template overlay is a WAR structured directory/archive that contains just the files that have been added or modified to customize/configure the webapplication for all instances that will be deployed.

The demo template can be installed from the downloaded files with the command:

mv $OVERLAYS/jtracTemplate\=jtrac-2.1.0 $JETTY_HOME/overlays/templates/

In the Jetty server window, you should see the template loaded with a message like:

2011-05-06 11:00:08.716:INFO:OverlayedAppProvider:loaded jtracTemplate=jtrac-2.1.0@1304643608715

The contents of the loaded template is as follows:

templates/jtracTemplate=jtrac-2.1.0
└── WEB-INF
    ├── classes
    │   └── jtrac-init.properties
    ├── log4j.properties
    ├── overlay.xml
    ├── template.xml
    └── web-overlay.xml

The name of the template directory (or it could be a war) uses the ‘=’ character in jtracTemplate=jtrac-2.1.0 to separates the name of the template from the name of the WAR file in webapps that it applies to.  If  = is a problem, then -- may also be used.

WEB-INF/classes/jtrac-init.properties – replaces the jtrac properties file with an empty file, as the properties contained within it are configured elsewhere

WEB-INF/log4j.properties – configures the logging for all instances of the template.

WEB-INF/overlay.xml – a Jetty XML formatted IoC file that is used to inject/configure the ContextHandler for each instances. In this case it just sets up the context path:

<?xml version="1.0"  encoding="ISO-8859-1"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure.dtd">
<Configure class="org.eclipse.jetty.webapp.WebAppContext">
  <Set name="contextPath">/</Set>
</Configure>

WEB-INF/template.xml – a Jetty XML formatted IoC file that is used to inject/configure the resource cache and classloader that is shared by all instances of the template. It is run only once per load of the template:

<?xml version="1.0"  encoding="ISO-8859-1"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure.dtd">
<Configure class="org.eclipse.jetty.overlays.TemplateContext">
  <Get name="resourceCache">
    <Set name="useFileMappedBuffer">true</Set>
    <Set name="maxCachedFileSize">10000000</Set>
    <Set name="maxCachedFiles">1000</Set>
    <Set name="maxCacheSize">64000000</Set>
  </Get>
</Configure>

WEB-INF/web-overlay.xml – a web.xml fragment that is overlayed on top of the web.xml from the base WAR file, that can set init parameters and add/modify filters and servlets. In this it sets the application home and springs rootKey:

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/j2ee"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd"
    version="2.4">
  <context-param>
    <param-name>jtrac.home</param-name>
    <param-value>/tmp/jtrac-${overlay.instance.classifier}</param-value>
  </context-param>
  <context-param>
    <param-name>webAppRootKey</param-name>
    <param-value>jtrac-${overlay.instance.classifier}</param-value>
  </context-param>
  <filter>
</web-app>

Note the use of parameterisation of values such as ${overlays.instance.classifier}, as this allows the configuration to be made in the template and not customised for each instance.

Without the overlayed deployer, all the configurations above would still need to have been made, but rather that being in a single clear structure they would have been either in the servers common directory, the servers webdefaults.xml (aka server.xml), or baked into the WAR file of each application instance using copied/modified files from the original. The overlayed deployer allows us to make all these changes in one structure, more over it allows some of the configuration to be parameterised to facilitate easy multi-tenant deployment.

Installing an Instance Overlay

Now that we have installed a template, we can install one or more instance overlays, which deploy the actual web applications:

mv /tmp/overlays/instances/jtracTemplate\=blue $JETTY_HOME/overlays/instances/
mv /tmp/overlays/instances/jtracTemplate\=red $JETTY_HOME/overlays/instances/
mv /tmp/overlays/instances/jtracTemplate\=blue $JETTY_HOME/overlays/instances/

As each instance is moved into place, you will see the jetty server window react and deploy that instance. Within each instance, there is the structure:

instances/jtracTemplate=red/
├── WEB-INF
│   └── overlay.xml
├── favicon.ico
└── resources
    └── jtrac.css

WEB-INF/overlay.xml – a Jetty XML format IoC file that injects/configures the context for the instance. In this case it sets up a virtual host for the instance:

<?xml version="1.0"  encoding="ISO-8859-1"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure.dtd">
<Configure class="org.eclipse.jetty.webapp.WebAppContext">
  <Set name="virtualHosts">
    <Array type="String">
      <Item>127.0.0.2</Item>
      <Item>red.myVirtualDomain.com</Item>
    </Array>
  </Set>
</Configure>

favicon.ico – replaces the icon in the base war with one themed for the instance colour.

resources/jtrac.css – replaces the style sheet from the base war with one themed for the instance colour

The deployed instances can now be viewed by pointing your browser at http://127.0.0.1:8080, http://127.0.0.2:8080 and http://127.0.0.3:8080. The default username/password for jtrac is admin/admin.

Things to know and notice

• Each instance is themed with images and styles sheets from the instance overlay.
• Each instance is running with it’s own application directory (eg. /tmp/jtrac-red), that is set templates web-overlay.xml.
• The instances are distinguished by virtual host that is set in the instance overlay.xml
• The static content from the base war and template are shared between all instances. Specifically there is a shared ResourceCache so only a single instance of each static content is loaded into memory.
• The classloader at the base war and template level is shared between all instances, so that only a single instance of common classes is loaded into memory. Classes with non shared statics can be configured to load in the instances classloader.
• All overlays are hot deployed and dependencies tracked. If an XML is touched in an instance, it is redeployed. If an XML is touched in a template, then all instances using it are redeployed. If a WAR file is touched, then all templates and all instances dependant on it are redeployed.
• New versions can easily be deployed. Eg when jtrac-2.2.0.war becomes available, it can just be dropped into overlays/webapps and then rename jtracTemplate\=jtrac-2.1.0 to jtracTemplate\=jtrac-2.2.0
• There is a fuller version of this demo in overlays-demo-jndi.tar.gz, that uses JNDI (needs options=jndi,annotations and jetty-plus.xml in start.ini) and shows how extra jars can be added in the overlays.