RAP/RWT Cluster

| RAP wiki home | RAP project home |

Introduction

Load balancing to distribute workload across multiple nodes in a cluster works with RAP out of the box since version 1.0. One thing to mind though when setting up a load balancer for RAP applications is that it must be configured to use sticky sessions (aka session affinity). This entails that once a session is started, the same server serves all subsequent requests for that session. For information on how to set up load balancing, refer to the servlet container documentation. E.g. Tomcat

If your application provides critical servies you may want to make it fail safe by running it in a high-availability cluster. A high availability cluster operates by having redundant nodes. So that if one node crashes, the service as a whole is unaffected as other nodes take over the work.

Throughout this document, the term cluster refers to high-availablility clusters.

With version 1.5 M1, RAP can be configured for transparent session failover so that applications can run in a high-availability cluster. This means that two or more servlet engines form a cluster, where a session, and its contents, can be moved from one node of the cluster to another node.

In §7.7.2 (Distributed Environments), the servlet specification requests for distributable applications that all session data must implement the Serializable interface. This results in two main obstacles to overcome with the current implementation:

• All RWT objects that live in the session scope (i.e. are directly or indirectly stored in a session attribute) must be serializable. This will mainly affect Display, Widget and derived classes and Resource and derived classes.
• The RWTLifeCycle which creates a UI Thread that lives until the session is terminated. The UI thread is necessary to enable blocking dialogs and so ensures compatibility with SWT. The downside of the UI thread of course is that sessions (which hold a reference to the UI thread) cannot be migrated as - naturally - a thread cannot be serialized.

Serializable RWT sessions can also be used to swap inactive sessions onto disk and thereby releasing the memory consumed by them. This allows for long to infinite session timeout settings that result in user friendly web applications that don't bother users with 'session has expried' messages and on the other hand don't consume more memory than actually needed. This feature is e.g. supported by Tomcat

State of Development

The core features that enable RWT and JFace to run in a high-availability cluster are implemented. bug 341761 is used as a meta-bug to capture the problem and track the solution.

JEE Compatible Operation Mode

An alternative operation mode is available that does not use an extra thread. As a consequence thereof blocking dialogs aren't possible with this operation mode. This operation mode also makes it easier to access security or transaction contexts and is conform to the JEE Specification. Even if you don't have a need for high availability clustering, if you don’t use blocking dialogs or workbench features we recommend to use this operation mode.

Serializable Session Data

The core classes with session scope are serializable. This includes Device and Display, all widgets, layouts and layout data, Dialog and its subclasses, Accessible and its associated classes, drag & drop classes, resources, data structures (e.g.Point, Rectangle, etc.), listeners and their respective events, ISessionStoreListener.

If a session attribute changes, the servlet engine must be told to replicate the change. Unfortunately the Servlet specs do not specify how this should be done. The most common way is to call HttpSession.setAttribute() to flag the object as changed (see J2EE clustering, Part 2, section Session-storage guidelines). In RWT, after each request the session attribute that holds the ISessionStore is re-set into the session. This means that as long as application code stores data in an attribute of the ISessionStore or with the help of SessionSingletonBase, all changes will be replicated.

Each serializable class has a version associated, which is used during deserialization to ensure that the loaded class is compatible with the serialized object with respect to serialization. A class can state the version of its serialized form explicitly by declaring a field named serialVersionUID that holds the verion number. If a class does not explicitly declare a serialVersionUID, then a version will be calculated at runtime. Even though it is strongly recommended to explicitly declare the serialVersionUID, we decided against for these reasons (bug 351641):

• The serialized form is stored for a short time only. Currently, it is hardly imaginable that in between serialization and deserialization, a new application is deployed (and with it the version of the serialized form changes)
• Maintaining an explicit serialVersionUID is hard. With each field that is added or removed from a serializable class, it must be considered whether the serial version is affected. Thre is no (automated) quality assurance established to ensure this and installing one would be a considerable amount of work.
• The clutter of serialVersionUID declarations reduces the readability of the source code.

If the future shows that explicit serialVersionUID become necessary, they can still be added.

Blocking Dialogs

For application code to still be able to use dialogs we introduced the DialogUtil helper class. It allows to open dialogs in a non-blocking way and informs you via the supplied DialogCallback when the dialog was closed.

MessageBox messageBox = new MessageBox( parent, SWT.YES | SWT.NO );
messageBox.setMessage( "Are you OK?" );
DialogUtil.open( messageBox, new DialogCallback() {
  public void dialogClosed( int returnCode ) {
    if( returnCode == SWT.YES ) {
      // ...
    } 
  }
} );

Please consult the JavaDoc for further details.

Background Threads and Display#(a)syncExec()

UICallback#activate() and #deactivate() - RWTs server push mechanism - is not affected when running in a cluster. They continue to work as before.

However, UICallback#activate() is usually used to propagate changes from background threads to the UI.. Display#(a)syncExec() can be used to propagate UI changes from background threads. The passed in runnable is stored in a queue of the Display and executed when the event loop is running for the next time.

In a cluster environment, this has several implications. Passing a runnable to (a)syncExec() effectively alters the session. But as this usually happens outside of a request, the servlet engine has no chance to replicate the changed session.

Each display has a Synchronizer that manages the queue of async runnables for that display. At application start, the default synchronizer for a display can be altered by calling Display#setSynchronizer(). Each Synchronizer is notified whenever a runnable is added by calling either asyncExec() or syncExec(). The ClusterSynchronizer is provided to replicate changes to the queue of async runnables of a display. It sends a notification-request to the session to which the display belongs that in turn gives the servlet engine the opportunity to replicate the session.

A further problem and one that cannot be solved by RWT is the background thread itself. Threads - naturally - aren't serializable and thus cannot be replicated among cluster nodes. This means that the application is responsible to implement the background thread in a fail-safe way.

JFace

The RAP port of JFace was altered to work with transparent session failover. All classes from org.eclipse.rap.jface except those from the org.eclipse.jface.bindings.* packages are serializable.

OSGi

Running a clustered RWT application in OSGi is currently not possible. See bug 353117 for more details and a list of blocking issues.

Known Issues

Images that were created with one of the Graphics#getImage() methods cannot be serialized (bug 357769).

The graphics context (GC) is not meant to be serialized because it should only be used within the scope of a request. I.e. it is created and destroyed within the paintControl() method of a PaintListener.

As a consequence of the JEE compatible life cycle, Display#sleep() cannot be implemented and throws an UnsupportedOperationException. This affects several other places:

• Browser#execute() and evaluate() will not work, these methods call sleep() to wait for the result of the Javascript execution.
• Attempts to call Windows#open() when setBlockOnOpen() was set to true (which is the default) will not work. Blocking dialogs use a nested event loop to block execution flow. See the Blocking Dialogs section on how to work around this issue.
• Entry points that whish to use the JEE compatible life cycle must not contain the usual event loop.

Test Infrastructure

To be able to run tests from JUnit, there are helper classes with which an arbitrary number of embedded serlvet engines can be created. Either standalone or clustered. Currently the tests can be run with Jetty 7.4 and Tomcat 7.

The tests reside the source code repository under org.eclipse.rap.rwt.cluster.test and the fixture can be found in org.eclipse.rap.rwt.cluster.testfixture. Tests for the fixture in turn are in org.eclipse.rap.rwt.cluster.testfixture.test.

How to setup RWT in a cluster

Cluster support in RWT requires the servlet 3.0 API. Setup a cluster with a servlet engine that conforms to the servlet 3.0 specification. There are short tutorials that help to set up a cluster with Jetty or Tomcat.

The web.xml should look like the one below. The distributable element tells the servlet engine to replicate the session among the nodes in the cluster.

<web-app xmlns="http://java.sun.com/xml/ns/javaee" 
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
  version="2.5">
 
  <distributable/>
 
  <context-param>
    <param-name>org.eclipse.rwt.Configurator</param-name>
    <param-value>com.example.ExampleConfigurator</param-value>
  </context-param>
 
  <listener>
    <listener-class>org.eclipse.rwt.engine.RWTServletContextListener</listener-class>
  </listener>
 
  <servlet>
    <servlet-name>rwtDelegate</servlet-name>
    <servlet-class>org.eclipse.rwt.engine.RWTServlet</servlet-class>
  </servlet>
  <servlet-mapping>
    <servlet-name>rwtDelegate</servlet-name>
    <url-pattern>/example</url-pattern>
  </servlet-mapping>
 
</web-app>

The class specified in the org.eclipse.rwt.Configurator parameter must implement the org.eclipse.rwt.application.ApplicationConfigurator interface. Its configure() method is called just before the application starts. It should be used to add the entry point(s) that are available to the application and configure other aspects of the application. To work in a cluster environment, the ApplicationConfigurator implementation must also select the SESSION_FAILOVER operation mode. The ExampleConfigurator should look like this:

public class ExampleConfigurator implements ApplicationConfigurator {
  public void configure( ApplicationConfiguration configuration ) {
    configuration.setOperationMode( OperationMode.SESSION_FAILOVER );
    configuration.addEntryPoint( "default", ExampleEntryPoint.class );
  }
}

As the cluster-enabled life cycle does not support Display#sleep(), the event loop must be removed from the entry point. The ExampleEntryPoint should look like the one below.

public class ExampleEntryPoint implements IEntryPoint {
  public int createUI() {
    Display display = new Display();
    Shell shell = new Shell( display );
    shell.setText( "Hello World" );
    // create more widgets...
    shell.open();
    // omit the event loop
    return 0;
  }
}

If the application is going to use Display#(a)syncExec() to propagate UI changes from background threads, an appropriate Synchronizer must be attached to the Display.

Display display = new Display();
display.setSynchronizer( new ClusteredSynchronizer( display ) );

When deploying the application in the cluster, it must be ensured that each node receives the exact same version of the application.

Performance Comparison

To get an idea what the performance impact of clustering a web application means, we did basic load tests.

A simple web application that consists of a shell and several buttons was used. This application was deployed to a Tomcat 7.0 and a Jetty 7.4 servlet engine. Each once configured as a single engine and once clustered.

With the help of JMeter, a web session was recorded in which several of the buttons were selected. The entire recording consists of 24 requests. This session was then played back, simulating 500 concurrent users with a ramp up period of 60 Seconds that use the web application.

Even though the performance impact is significant, the response times are sufficiently fast for Ajax-driven web applications Manual sessions that were opened while the load tests were running, also proved that. The table below shows the response times for each of the scenarios in Milliseconds.

	Average	Median	90% Line
Jetty single	4.8 ms	1 ms	4 ms
Jetty cluster	13.3 ms	6 ms	29 ms
Tomcat single	1.5 ms	1 ms	2 ms
Tomcat cluster	3.1 ms	2 ms	6 ms

Development Snapshots

Development takes place in HEAD, if you are interested you may want to directly check out from the source code repository. Nightly builds are also available and can be obtained from the downloads page.

If you whish to run the cluster tests, you will have to add Jetty 8.x, the H2 database (version 1.1) and the Servlet API (version 3.0) to your target platform. The Eclipse Orbit project offers bundled versions of H2 (org.h2) and the Servlet API (javax.servlet). In addition Tomcat 7.0.12 is needed, please use the pre-bundled version from the RAP source code repository at org.apache.tomcat.git (browse, stats, fork on OrionHub) .

Please note that these tests (like all other RWT tests) are neither part of nightly builds nor milestone builds.