Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "RAP/LoadTesting"

< RAP
m (Configuration and Tuning Tips)
m (newCounter has not been initialized at this pointl. Probably oldCounter should be used.)
(10 intermediate revisions by 5 users not shown)
Line 15: Line 15:
 
There are a couple of hints that you need to take into account for recording RAP application tests:
 
There are a couple of hints that you need to take into account for recording RAP application tests:
 
* Re-Start the browser before recording a test (or make otherwise sure that you are starting with a NEW session, e.g. by deleting the cookies)
 
* Re-Start the browser before recording a test (or make otherwise sure that you are starting with a NEW session, e.g. by deleting the cookies)
* After recording your test you need to eliminate duplicate requests to the UICallbackServiceHandler (so that there are never two or more consecutive requests of this type). The callback requests do not have the ?nocache directive as url parameter.
+
* RAP uses an increasing <tt>requestCounter</tt> to keep track of the requests. This can lead to problems when replaying requests when the number does not begin at zero or is not increasing.
* WRONG: [[Image:UICallback.png]]
+
* To get around this problem go to the Testplan and add a parameter '<tt>test</tt>' with the value 'requestCounter' before recording the test. When you record the test the <tt>requestCounter</tt> value will be replaced by the parameter <tt>${test}</tt>
* RIGHT: [[Image:OneCallback.png]]
+
* When you have finished recording, just change the value of the testplan parameter '<tt>test</tt>' to the value 'test'. This way you just send an additional parameter which does not influence the RAP internals. Change it back to 'requestCounter' if you do some additional recording
* It does not matter which callback request(s) you delete, as long as exactly one is remaining
+
* After recording your test you need to eliminate duplicate subsequent requests to the UICallbackServiceHandler (so that there are never two or more consecutive requests of this type). The callback requests do not have the ?nocache directive as url parameter.
 +
* <font color="red">WRONG: </font>
 +
  [[Image:UICallback.png]]
 +
* <font color="green">RIGHT:</font>
 +
  [[Image:OneCallback.png]]
 +
* It does not matter which callback request(s) you delete, as long as exactly one is remaining. Subsequent requests with a ?nocache directive have to remain untouched.
  
 
==Running a test==
 
==Running a test==
Line 31: Line 36:
 
Also note the Gaussian Timer that has been added. A timer adds wait time between the requests, leading to a better approximation of a real users behaviour. If you don't add a timer you are putting a much higher load on the server, as every response is immediately followed by the next request, something a real user is not able to do.
 
Also note the Gaussian Timer that has been added. A timer adds wait time between the requests, leading to a better approximation of a real users behaviour. If you don't add a timer you are putting a much higher load on the server, as every response is immediately followed by the next request, something a real user is not able to do.
  
 +
'''Generating <tt>requestCounter</tt>'''
 +
 +
To make sure that we don't get the erros related to a wrong '<tt>requestCounter</tt>' value, add the following
 +
* A 'Response Assertion in the Thread Group for 'Main sample and sub-samples' with a Pattern Matching Rule 'contains not'
 +
Multiple browser-instances
 +
 +
To generate the '<tt>requestCounter</tt>' parameter when replaying the request you have to add the following:
 +
* A 'User Defined Variable' in the Thread Group with the name '<tt>request_counter</tt>' and the value '0'
 +
* A 'Bean Shell Pre-Processor' in the Thread Group containing the following script
 +
  var sampler = ctx.getCurrentSampler();
 +
  if(sampler.getPath().contains("nocache")){
 +
    var doCount = vars.get("startCounter");
 +
    if(doCount == "OK") {
 +
      oldCounter = Integer.parseInt(vars.get("request_counter"));
 +
      sampler.addArgument("requestCounter", Integer.toString(oldCounter));
 +
      newCounter = oldCounter + 1;
 +
      vars.put("request_counter",Integer.toString(newCounter));
 +
    }
 +
  }
 +
* Underneath the first request with a <tt>?nocache</tt> directive add a 'Bean Shell PostProcessor' with the following script
 +
  vars.put("startCounter", "OK");
 +
 +
This is what it does:
 +
* After the first post we set the flag 'startCounter' to 'OK' to start incrementing the <tt>requestCounter</tt> with every subsequent request.
 +
* The Pre-Processor is called before every request. If it contains the <tt>?nocache</tt> directive it is a POST and must contain the '<tt>requestCounter</tt>' parameter
 +
* The parameter ist added to the request with the <tt>sampler.addArgument</tt> call.
 +
* The user variable is incremented for the next request
 +
 +
'''Check Playback'''
  
 
This is how we check if the playback of the session is working.
 
This is how we check if the playback of the session is working.
 
* Start testing with only ONE session and two loops (configured in the Thread Group)
 
* Start testing with only ONE session and two loops (configured in the Thread Group)
* Add a "Save Responses" post-processor
+
* Add a "Save Responses to a file" listener
 
* [[Image:JMeterSaveResponses.png]]
 
* [[Image:JMeterSaveResponses.png]]
 
* Examine the response files, the patter should look like this
 
* Examine the response files, the patter should look like this
Line 40: Line 74:
 
** The second response should be a .js or .javascript file with a size above 20KB, it contains the instructions for setting up the first user interface
 
** The second response should be a .js or .javascript file with a size above 20KB, it contains the instructions for setting up the first user interface
 
** You may have responses that are of type "unknown" as responses to the UICallBack requests (can also appear as .js), in our example the third response
 
** You may have responses that are of type "unknown" as responses to the UICallBack requests (can also appear as .js), in our example the third response
** The fourth response and all consecutive "non-UICallback" requests should have a response that contains some JavaScript instructions that do set some properties (e.g. w.setSpace of w.setItems in the example below)
+
** The fourth response and all consecutive "non-UICallback" requests should have a response that contains some JavaScript instructions that do set some properties (e.g. w.setSpace or w.setItems in the example below)
 
<pre>org.eclipse.swt.EventUtil.suspendEventHandling();
 
<pre>org.eclipse.swt.EventUtil.suspendEventHandling();
 
var req = org.eclipse.swt.Request.getInstance();req.setRequestCounter( "7" );
 
var req = org.eclipse.swt.Request.getInstance();req.setRequestCounter( "7" );
Line 50: Line 84:
 
qx.ui.core.Widget.flushGlobalQueues();
 
qx.ui.core.Widget.flushGlobalQueues();
 
org.eclipse.swt.EventUtil.resumeEventHandling();</pre>
 
org.eclipse.swt.EventUtil.resumeEventHandling();</pre>
** if all requests contain only like this, the test is not properly executed
+
** if all responses contain only code like this, the test is not properly executed
 
<pre>org.eclipse.swt.Request.getInstance().send();
 
<pre>org.eclipse.swt.Request.getInstance().send();
 
org.eclipse.swt.Request.getInstance().enableUICallBack( "rap",
 
org.eclipse.swt.Request.getInstance().enableUICallBack( "rap",
Line 67: Line 101:
 
** Choose an appropriate garbage collection strategy. We've found <tt>-XX:+UseConcMarkSweepGC</tt> to work best (on dual-core machines)
 
** Choose an appropriate garbage collection strategy. We've found <tt>-XX:+UseConcMarkSweepGC</tt> to work best (on dual-core machines)
 
** Start the JVM in server mode (<tt>-server</tt>)
 
** Start the JVM in server mode (<tt>-server</tt>)
** RAP can ensure that widget id's are generated in the exact same way from session to session. This comes at a slight performance cost. It is useful for load testing because, if id's are different between sessions, this could result in a server response that does not match what the test expects. To activate the 'repeatable id numbering scheme', download the org.eclipse.rap.widgetidgenerator plug-in from the RAP Sandbox (/cvsroot/technology/org.eclipse.rap/sandbox/org.eclipse.rap.widgetidgenerator) and deploy it with your application. Use <tt>-Dorg.eclipse.rwt.enableUITests=true</tt> in tomcat's launch arguments
+
** RAP can ensure that widget id's are generated in the exact same way from session to session. This comes at a slight performance cost. It is useful for load testing because, if id's are different between sessions, this could result in a server response that does not match what the test expects. To activate the 'repeatable id numbering scheme', download the <tt>org.eclipse.rap.widgetidgenerator</tt> plug-in from the RAP Sandbox (/cvsroot/rt/org.eclipse.rap/sandbox/org.eclipse.rap.widgetidgenerator) and deploy it with your application. Use <tt>-Dorg.eclipse.rwt.enableUITests=true</tt> in tomcat's launch arguments
 
** If you use tomcat, this servlet <tt>http://yourserver/manager/status/all</tt> provides interesting information
 
** If you use tomcat, this servlet <tt>http://yourserver/manager/status/all</tt> provides interesting information
 
** Obviously - don't run JMeter and the Tomcat on the same machine
 
** Obviously - don't run JMeter and the Tomcat on the same machine
Line 75: Line 109:
 
** Running JMeter with a GUI could slow it down, since updating the GUI takes away CPU cycles. You can run jMeter without a UI, using the commandline below. It starts jmeter without a UI, runs testplan.jmx and saves the results to result.jtl. The result file can be opened with jmeter after the test run is finished. A large result file takes some time and memory to open - be patient and increase the heap. Warning: interrupting a test run in progress, will create an incomplete result file that cannot be opened!
 
** Running JMeter with a GUI could slow it down, since updating the GUI takes away CPU cycles. You can run jMeter without a UI, using the commandline below. It starts jmeter without a UI, runs testplan.jmx and saves the results to result.jtl. The result file can be opened with jmeter after the test run is finished. A large result file takes some time and memory to open - be patient and increase the heap. Warning: interrupting a test run in progress, will create an incomplete result file that cannot be opened!
 
<pre>./jmeter -n -t /home/elias/testplan.jmx -l /tmp/result.jtl</pre>
 
<pre>./jmeter -n -t /home/elias/testplan.jmx -l /tmp/result.jtl</pre>
 +
 +
=== URL rewriting instead of cookies ===
 +
 +
From [http://www.eclipse.org/forums/index.php/mv/msg/261197/754685/#msg_754685]:
 +
<blockquote>
 +
I was able to run the test with the url rewriting. In JMeter there is something called regular expression extracter. I used this to extract the jsessionid from the first response from the server and then pass the same id in the all the other requests.
 +
</blockquote>
 +
 +
==Troubleshooting==
 +
* Multiple Browser instances
 +
** Problem: The only response data you get looks like this:
 +
<pre>
 +
qx.core.Init.getInstance().getApplication().reload( "Multiple browser-instances or
 +
browser-tabs per session are not\nsupported. You may click OK for restarting the session." )
 +
</pre>
 +
** Solution: You likely didn't get a new session on recording but rather reused an old one. This reflects in the parameter <tt>requestCounter</tt> that is sent with each HTTP request. When recording, instead of reloading the application in an existing browser tab you need to load the page in a new browser tab or window.
 +
** Follow the advices in the Recording section to be able to generate a fresh <tt>requestCounter</tt> value for each request
  
 
[[Category:RAP]]
 
[[Category:RAP]]

Revision as of 04:46, 13 March 2013

| RAP wiki home | RAP project home |

DRAFT Load testing / stress testing of RAP applications

This is a brief example on how to do load testing / stress testing of RAP applications with JMeter.

Recording a test

JMeter allows to record tests by adding a proxy to your browser and simply record the user interactions with the server. This Introduction (PDF) is very useful for getting started with JMeter.

Here is:

that you can load into JMeter 2.3.2.

There are a couple of hints that you need to take into account for recording RAP application tests:

  • Re-Start the browser before recording a test (or make otherwise sure that you are starting with a NEW session, e.g. by deleting the cookies)
  • RAP uses an increasing requestCounter to keep track of the requests. This can lead to problems when replaying requests when the number does not begin at zero or is not increasing.
  • To get around this problem go to the Testplan and add a parameter 'test' with the value 'requestCounter' before recording the test. When you record the test the requestCounter value will be replaced by the parameter ${test}
  • When you have finished recording, just change the value of the testplan parameter 'test' to the value 'test'. This way you just send an additional parameter which does not influence the RAP internals. Change it back to 'requestCounter' if you do some additional recording
  • After recording your test you need to eliminate duplicate subsequent requests to the UICallbackServiceHandler (so that there are never two or more consecutive requests of this type). The callback requests do not have the ?nocache directive as url parameter.
  • WRONG:
 UICallback.png
  • RIGHT:
 OneCallback.png
  • It does not matter which callback request(s) you delete, as long as exactly one is remaining. Subsequent requests with a ?nocache directive have to remain untouched.

Running a test

  • When running the tests with multiple threads you need to include an HTTP Cookie Manager (Please note that there is a bug in JMeter 2.3.1 for clearing cookies for multiple runs / iterations; please use either JMeter 2.3.0 or 2.3.2 see [1])

This is how the test plan should look like:

JMeterCookieManager.png

If cookies are not working every request will create a new session, this is not what we want to test (and leads to massive server load).

Also note the Gaussian Timer that has been added. A timer adds wait time between the requests, leading to a better approximation of a real users behaviour. If you don't add a timer you are putting a much higher load on the server, as every response is immediately followed by the next request, something a real user is not able to do.

Generating requestCounter

To make sure that we don't get the erros related to a wrong 'requestCounter' value, add the following

  • A 'Response Assertion in the Thread Group for 'Main sample and sub-samples' with a Pattern Matching Rule 'contains not'
Multiple browser-instances

To generate the 'requestCounter' parameter when replaying the request you have to add the following:

  • A 'User Defined Variable' in the Thread Group with the name 'request_counter' and the value '0'
  • A 'Bean Shell Pre-Processor' in the Thread Group containing the following script
 var sampler = ctx.getCurrentSampler();
 if(sampler.getPath().contains("nocache")){
   var doCount = vars.get("startCounter");
   if(doCount == "OK") {
     oldCounter = Integer.parseInt(vars.get("request_counter"));
     sampler.addArgument("requestCounter", Integer.toString(oldCounter));
     newCounter = oldCounter + 1;
     vars.put("request_counter",Integer.toString(newCounter));
   }
 }
  • Underneath the first request with a ?nocache directive add a 'Bean Shell PostProcessor' with the following script
 vars.put("startCounter", "OK");

This is what it does:

  • After the first post we set the flag 'startCounter' to 'OK' to start incrementing the requestCounter with every subsequent request.
  • The Pre-Processor is called before every request. If it contains the ?nocache directive it is a POST and must contain the 'requestCounter' parameter
  • The parameter ist added to the request with the sampler.addArgument call.
  • The user variable is incremented for the next request

Check Playback

This is how we check if the playback of the session is working.

  • Start testing with only ONE session and two loops (configured in the Thread Group)
  • Add a "Save Responses to a file" listener
  • JMeterSaveResponses.png
  • Examine the response files, the patter should look like this
    • The first response is an HTML file (expanded size is approxametly one megabyte, the compressed size that is going over the wire is about 200KB)
    • The second response should be a .js or .javascript file with a size above 20KB, it contains the instructions for setting up the first user interface
    • You may have responses that are of type "unknown" as responses to the UICallBack requests (can also appear as .js), in our example the third response
    • The fourth response and all consecutive "non-UICallback" requests should have a response that contains some JavaScript instructions that do set some properties (e.g. w.setSpace or w.setItems in the example below)
org.eclipse.swt.EventUtil.suspendEventHandling();
var req = org.eclipse.swt.Request.getInstance();req.setRequestCounter( "7" );
var wm = org.eclipse.swt.WidgetManager.getInstance();
var w = wm.findWidgetById( "w44" );
w.setSpace( 0, 796, 19, 20 );
var w = wm.findWidgetById( "w81" );
w.setItems( [ "View I / Locate in browser view", "View I / Root", "View I / null"]);
qx.ui.core.Widget.flushGlobalQueues();
org.eclipse.swt.EventUtil.resumeEventHandling();
    • if all responses contain only code like this, the test is not properly executed
org.eclipse.swt.Request.getInstance().send();
org.eclipse.swt.Request.getInstance().enableUICallBack( "rap",
"custom_service_handler",
"org.eclipse.rwt.internal.lifecycle.UICallBackServiceHandler" );
    • The second pass of the the test should lead to the EXACT same pattern of responses as the first one
    • If this is not the case you need to turn on "Custom IDs" for the RAP widgets, an instruction will follow shortly
    • After you verified the result pattern you should save only failed responses and turn up the volume of your load tests.

Configuration and Tuning Tips

  • Tomcat / JVM
    • For best results monitor the memory consumption of your application. Turn on the garbage collector statistics to find out what is going on. If your application is leaking memory, performance will decrease very quickly!
    • Provide your application with enough heap. This depends on the number of active sessions and the memory consumption per session of your application. For realistic results, use an appropriate session time out setting
    • Choose an appropriate garbage collection strategy. We've found -XX:+UseConcMarkSweepGC to work best (on dual-core machines)
    • Start the JVM in server mode (-server)
    • RAP can ensure that widget id's are generated in the exact same way from session to session. This comes at a slight performance cost. It is useful for load testing because, if id's are different between sessions, this could result in a server response that does not match what the test expects. To activate the 'repeatable id numbering scheme', download the org.eclipse.rap.widgetidgenerator plug-in from the RAP Sandbox (/cvsroot/rt/org.eclipse.rap/sandbox/org.eclipse.rap.widgetidgenerator) and deploy it with your application. Use -Dorg.eclipse.rwt.enableUITests=true in tomcat's launch arguments
    • If you use tomcat, this servlet http://yourserver/manager/status/all provides interesting information
    • Obviously - don't run JMeter and the Tomcat on the same machine
  • JMeter
    • When running JMeter with many worker threads, you must increase it's heap size. It is advisable to start with a large initial heap size, to avoid delays resulting from additional heap allocation. Refer to the HEAP parameter in the jmeter / jmeter.bat scripts
    • Running JMeter with a GUI could slow it down, since updating the GUI takes away CPU cycles. You can run jMeter without a UI, using the commandline below. It starts jmeter without a UI, runs testplan.jmx and saves the results to result.jtl. The result file can be opened with jmeter after the test run is finished. A large result file takes some time and memory to open - be patient and increase the heap. Warning: interrupting a test run in progress, will create an incomplete result file that cannot be opened!
./jmeter -n -t /home/elias/testplan.jmx -l /tmp/result.jtl

URL rewriting instead of cookies

From [2]:

I was able to run the test with the url rewriting. In JMeter there is something called regular expression extracter. I used this to extract the jsessionid from the first response from the server and then pass the same id in the all the other requests.

Troubleshooting

  • Multiple Browser instances
    • Problem: The only response data you get looks like this:
qx.core.Init.getInstance().getApplication().reload( "Multiple browser-instances or
browser-tabs per session are not\nsupported. You may click OK for restarting the session." )
    • Solution: You likely didn't get a new session on recording but rather reused an old one. This reflects in the parameter requestCounter that is sent with each HTTP request. When recording, instead of reloading the application in an existing browser tab you need to load the page in a new browser tab or window.
    • Follow the advices in the Recording section to be able to generate a fresh requestCounter value for each request

Copyright © Eclipse Foundation, Inc. All Rights Reserved.