Revision as of 11:33, 29 October 2009

| RAP wiki home | RAP project home |

DRAFT Load testing / stress testing of RAP applications

This is a brief example on how to do load testing / stress testing of RAP applications with JMeter.

Recording a test

JMeter allows to record tests by adding a proxy to your browser and simply record the user interactions with the server. This Introduction (PDF) is very useful for getting started with JMeter.

Here is:

• a sample proxy configuration (download and unzip File:HTTP Proxy Server.zip), or
• a sample test recording (download and unzip File:Demo-testplan.zip)

that you can load into JMeter 2.3.2.

There are a couple of hints that you need to take into account for recording RAP application tests:

• Re-Start the browser before recording a test (or make otherwise sure that you are starting with a NEW session, e.g. by deleting the cookies)
• After recording your test you need to eliminate duplicate requests to the UICallbackServiceHandler (so that there are never two or more consecutive requests of this type). The callback requests do not have the ?nocache directive as url parameter.
• WRONG:

• RIGHT:

• It does not matter which callback request(s) you delete, as long as exactly one is remaining

Running a test

• When running the tests with multiple threads you need to include an HTTP Cookie Manager (Please note that there is a bug in JMeter 2.3.1 for clearing cookies for multiple runs / iterations; please use either JMeter 2.3.0 or 2.3.2 see [1])

This is how the test plan should look like:

If cookies are not working every request will create a new session, this is not what we want to test (and leads to massive server load).

Also note the Gaussian Timer that has been added. A timer adds wait time between the requests, leading to a better approximation of a real users behaviour. If you don't add a timer you are putting a much higher load on the server, as every response is immediately followed by the next request, something a real user is not able to do.

This is how we check if the playback of the session is working.

• Start testing with only ONE session and two loops (configured in the Thread Group)
• Add a "Save Responses" post-processor
• 
• Examine the response files, the patter should look like this
  • The first response is an HTML file (expanded size is approxametly one megabyte, the compressed size that is going over the wire is about 200KB)
  • The second response should be a .js or .javascript file with a size above 20KB, it contains the instructions for setting up the first user interface
  • You may have responses that are of type "unknown" as responses to the UICallBack requests (can also appear as .js), in our example the third response
  • The fourth response and all consecutive "non-UICallback" requests should have a response that contains some JavaScript instructions that do set some properties (e.g. w.setSpace of w.setItems in the example below)

org.eclipse.swt.EventUtil.suspendEventHandling();
var req = org.eclipse.swt.Request.getInstance();req.setRequestCounter( "7" );
var wm = org.eclipse.swt.WidgetManager.getInstance();
var w = wm.findWidgetById( "w44" );
w.setSpace( 0, 796, 19, 20 );
var w = wm.findWidgetById( "w81" );
w.setItems( [ "View I / Locate in browser view", "View I / Root", "View I / null"]);
qx.ui.core.Widget.flushGlobalQueues();
org.eclipse.swt.EventUtil.resumeEventHandling();

• • if all requests contain only like this, the test is not properly executed

org.eclipse.swt.Request.getInstance().send();
org.eclipse.swt.Request.getInstance().enableUICallBack( "rap",
"custom_service_handler",
"org.eclipse.rwt.internal.lifecycle.UICallBackServiceHandler" );

• • The second pass of the the test should lead to the EXACT same pattern of responses as the first one
  • If this is not the case you need to turn on "Custom IDs" for the RAP widgets, an instruction will follow shortly
  • After you verified the result pattern you should save only failed responses and turn up the volume of your load tests.

Configuration and Tuning Tips

• Tomcat / JVM
  • For best results monitor the memory consumption of your application. Turn on the garbage collector statistics to find out what is going on. If your application is leaking memory, performance will decrease very quickly!
  • Provide your application with enough heap. This depends on the number of active sessions and the memory consumption per session of your application. For realistic results, use an appropriate session time out setting
  • Choose an appropriate garbage collection strategy. We've found -XX:+UseConcMarkSweepGC to work best (on dual-core machines)
  • Start the JVM in server mode (-server)
  • RAP can ensure that widget id's are generated in the exact same way from session to session. This comes at a slight performance cost. It is useful for load testing because, if id's are different between sessions, this could result in a server response that does not match what the test expects. To activate the 'repeatable id numbering scheme', download the org.eclipse.rap.widgetidgenerator plug-in from the RAP Sandbox (/cvsroot/rt/org.eclipse.rap/sandbox/org.eclipse.rap.widgetidgenerator) and deploy it with your application. Use -Dorg.eclipse.rwt.enableUITests=true in tomcat's launch arguments
  • If you use tomcat, this servlet http://yourserver/manager/status/all provides interesting information
  • Obviously - don't run JMeter and the Tomcat on the same machine

• JMeter
  • When running JMeter with many worker threads, you must increase it's heap size. It is advisable to start with a large initial heap size, to avoid delays resulting from additional heap allocation. Refer to the HEAP parameter in the jmeter / jmeter.bat scripts
  • Running JMeter with a GUI could slow it down, since updating the GUI takes away CPU cycles. You can run jMeter without a UI, using the commandline below. It starts jmeter without a UI, runs testplan.jmx and saves the results to result.jtl. The result file can be opened with jmeter after the test run is finished. A large result file takes some time and memory to open - be patient and increase the heap. Warning: interrupting a test run in progress, will create an incomplete result file that cannot be opened!

./jmeter -n -t /home/elias/testplan.jmx -l /tmp/result.jtl

Troubleshooting

• Multiple Browser instances
  • Problem: The only response data you get looks like this:

qx.core.Init.getInstance().getApplication().reload( "Multiple browser-instances or browser-tabs per session are not\nsupported. You may click OK for restarting the session." )

• • Solution: You likely didn't get a new session on recording but rather reused an old one. This reflects in the parameter requestCounter that is sent with each HTTP request. Rather than reloading in an existing browser tab you need to load the page in a new browser tab or window.