Notice: This Wiki is now read only and edits are no longer possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Difference between revisions of "RAP/LoadTesting"
(Update to RAP 2.1 and later) |
(→Configuration and Tuning Tips) |
||
| (9 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
This article describes load testing / stress testing of RAP applications with Apache [http://jakarta.apache.org/jmeter/ JMeter]. | This article describes load testing / stress testing of RAP applications with Apache [http://jakarta.apache.org/jmeter/ JMeter]. | ||
| + | JMeter lets you record all HTTP(S) requests of a browser session and replay these requests in multiple parallel sessions. | ||
This [http://jakarta.apache.org/jmeter/usermanual/jmeter_proxy_step_by_step.pdf Introduction (PDF)] is very useful for getting started with JMeter. | This [http://jakarta.apache.org/jmeter/usermanual/jmeter_proxy_step_by_step.pdf Introduction (PDF)] is very useful for getting started with JMeter. | ||
| − | == | + | == Prepare a Test Plan == |
| − | + | * Download a recent version of JMeter and start it. | |
| − | + | [[File:JMeter-TestPlan.png]] | |
| − | + | ||
| − | + | There are two elements in the tree, called ''Test Plan'' and ''WorkBench''. Let's start with creating the Test Plan. | |
=== Add a Thread Group === | === Add a Thread Group === | ||
| Line 15: | Line 16: | ||
The Thread Group will contain your recorded requests. | The Thread Group will contain your recorded requests. | ||
| − | Select ''Add > Threads (Users) > Thread Group'' from the context menu. | + | * Select ''Add > Threads (Users) > Thread Group'' from the context menu. |
| + | * Configure at least these settings: | ||
| + | ** ''Number of Threads (users)'': this is the number of sessions that will in run in parallel | ||
| + | ** ''Ramp-up Period (in second)'': the time it takes until the last session is started | ||
| − | + | [[File:JMeter-ThreadGroup.png]] | |
| − | + | ||
| − | + | ||
=== Add HTTP Request Defaults === | === Add HTTP Request Defaults === | ||
| + | |||
| + | * ''Add > Config Element > HTTP Request Data'' | ||
| + | * Configure at least these settings: | ||
| + | ** ''Server Name or IP'': the host name or IP address of the server you run the tests against. | ||
| + | ** ''Path'': the base path of your application. | ||
| + | |||
| + | [[File:JMeter-RequestDefaults.png]] | ||
=== Add a Recording Controller === | === Add a Recording Controller === | ||
| Line 27: | Line 36: | ||
The recording controller will contain the recorded requests | The recording controller will contain the recorded requests | ||
| − | ''Add > Logic Controller > Recording Controller'' | + | * ''Add > Logic Controller > Recording Controller'' |
| + | |||
| + | [[File:JMeter-RecordingController.png]] | ||
=== Add a Cookie Manager === | === Add a Cookie Manager === | ||
| Line 34: | Line 45: | ||
If cookies are not working every request will create a new HTTP session and the server will return an HTTP error code. | If cookies are not working every request will create a new HTTP session and the server will return an HTTP error code. | ||
| − | ''Add > Config Element > HTTP Cookie Manager'' | + | * ''Add > Config Element > HTTP Cookie Manager'' |
The defaults are fine. | The defaults are fine. | ||
| + | |||
| + | [[File:JMeter-CookieManager.png]] | ||
Please note that there is a bug in JMeter 2.3.1 for clearing cookies for multiple runs / iterations; please use either JMeter 2.3.2 or later see [http://markmail.org/message/bis6om3wnpqtqin5]). | Please note that there is a bug in JMeter 2.3.1 for clearing cookies for multiple runs / iterations; please use either JMeter 2.3.2 or later see [http://markmail.org/message/bis6om3wnpqtqin5]). | ||
| Line 46: | Line 59: | ||
You might want to use a ''random'' timer to add some deviation to the interval. This will lead to a better approximation of a real users behavior. | You might want to use a ''random'' timer to add some deviation to the interval. This will lead to a better approximation of a real users behavior. | ||
| − | We recommend | + | We recommend the Gaussian random timer. |
| − | ''Add > Timer > Gaussian Random Timer'' | + | * ''Add > Timer > Gaussian Random Timer'' |
| + | * Configure: | ||
| + | ** ''Constant Delay Offset (in milliseconds)'': the time between two requests | ||
| + | ** ''Deviation (in milliseconds)'': the deviation of this interval | ||
| − | + | [[File:JMeter-RandomTimer.png]] | |
| − | + | == Prepare the Recording == | |
| − | + | === Add Test Script Recorder === | |
| − | Add | + | |
| − | + | JMeter contains a test script recorder. This is basically a proxy server that listens on a local port (8080 by default). | |
| + | Since the recorder is not part of the test plan, it is added to the ''Workbench'' node. | ||
| − | * Configure your browser to use a proxy. Use ''localhost'' and the port you | + | * Open the context menu on ''WorkBench'', and select ''Add > Non-Test Elements > HTTP(S) Test Script Recorder'' (in older versions this was called ''HTTP Proxy Server''). |
| − | * | + | * Things to configure: |
| − | * Open a new browser tab and load the URL of your application | + | ** ''Global Settings > Port'': the port that the proxy server should listen to. |
| + | ** ''URL Patterns to exclude'': to exclude static resources served by RAP, use the exclude pattern <code>.*\/rwt-resources\/.*</code> | ||
| + | |||
| + | Whether or not to exclude requests for static resources (images, script files, etc.) is up to you. | ||
| + | These resources are cached by the browser and only contribute a small portion to the entire server load. | ||
| + | However, you can safely keep static resources in the recording if you prefer. | ||
| + | |||
| + | [[File:JMeter-ScriptRecorder.png]] | ||
| + | |||
| + | === Configure Browser === | ||
| + | |||
| + | * Configure your browser to use a proxy. Use ''localhost'' and the port you configured in the test recorder (8080 by default). | ||
| + | |||
| + | == Record a test == | ||
| + | |||
| + | * Press the ''Start'' button at the bottom of the ''HTTP(S) Test Script Recorder'' page to start the recorder. | ||
| + | * Open a '''new browser tab''' and load the URL of your application. | ||
| + | * Now click through the application to record the test session. | ||
* Note: In RAP versions prior to 2.1, you had to re-start the browser before recording a test in order to ensure the parameter ''requestCounter'' started from zero. | * Note: In RAP versions prior to 2.1, you had to re-start the browser before recording a test in order to ensure the parameter ''requestCounter'' started from zero. | ||
| Line 70: | Line 103: | ||
These requests are not answered by the server and will lead to blocking. | These requests are not answered by the server and will lead to blocking. | ||
| + | |||
| + | === Save Test Plan === | ||
| + | |||
| + | At this point, you should save the recorded test plan to a file. | ||
== Running a test == | == Running a test == | ||
| Line 76: | Line 113: | ||
In the top right corner, you can observe the number of parallel sessions that are currently running. | In the top right corner, you can observe the number of parallel sessions that are currently running. | ||
| + | |||
| + | [[File:JMeter-AggregateReport.png]] | ||
== Checking the Results == | == Checking the Results == | ||
| Line 88: | Line 127: | ||
''Add > Listener > Safe Responses to a File'' | ''Add > Listener > Safe Responses to a File'' | ||
| − | [[ | + | [[File:JMeter-SaveResponses.png]] |
Note that the target directory has to be present, it will not be created by JMeter. | Note that the target directory has to be present, it will not be created by JMeter. | ||
| Line 106: | Line 145: | ||
* If you use Tomcat, this servlet <tt>http://yourserver/manager/status/all</tt> provides interesting information | * If you use Tomcat, this servlet <tt>http://yourserver/manager/status/all</tt> provides interesting information | ||
| − | * When running JMeter with many worker threads, you must increase | + | * When running JMeter with many worker threads, you must increase its heap size. It is advisable to start with a large initial heap size, to avoid delays resulting from additional heap allocation. Refer to the <tt>HEAP</tt> parameter in the <tt>jmeter</tt> script. |
| − | + | === Running JMeter in headless mode === | |
| + | |||
| + | Running JMeter with a GUI could slow it down. You can run JMeter in headless mode, using the command below. | ||
| + | This runs <tt>testplan.jmx</tt> and saves the results to <tt>result.jtl</tt>. | ||
<pre>./jmeter -n -t /home/elias/testplan.jmx -l /tmp/result.jtl</pre> | <pre>./jmeter -n -t /home/elias/testplan.jmx -l /tmp/result.jtl</pre> | ||
| + | |||
| + | The result file can be opened with jmeter after the test run is finished. A large result file takes some time and memory to open - be patient and increase the heap. Warning: interrupting a test run in progress, will create an incomplete result file that cannot be opened! | ||
| + | Note also that the command will immediately return and then jmeter will be executing in the background (as a separate javaw process). Wait for this to finish before opening the results file. | ||
=== URL rewriting instead of cookies === | === URL rewriting instead of cookies === | ||
| − | + | We don't recommend using URL rewriting. However, if you rely on it, [http://www.eclipse.org/forums/index.php/mv/msg/261197/754685/#msg_754685 this forum post] might help: | |
<blockquote> | <blockquote> | ||
I was able to run the test with the url rewriting. In JMeter there is something called regular expression extracter. I used this to extract the jsessionid from the first response from the server and then pass the same id in the all the other requests. | I was able to run the test with the url rewriting. In JMeter there is something called regular expression extracter. I used this to extract the jsessionid from the first response from the server and then pass the same id in the all the other requests. | ||
Latest revision as of 07:57, 28 February 2014
This article describes load testing / stress testing of RAP applications with Apache JMeter. JMeter lets you record all HTTP(S) requests of a browser session and replay these requests in multiple parallel sessions.
This Introduction (PDF) is very useful for getting started with JMeter.
Contents
Prepare a Test Plan
- Download a recent version of JMeter and start it.
There are two elements in the tree, called Test Plan and WorkBench. Let's start with creating the Test Plan.
Add a Thread Group
The Thread Group will contain your recorded requests.
- Select Add > Threads (Users) > Thread Group from the context menu.
- Configure at least these settings:
- Number of Threads (users): this is the number of sessions that will in run in parallel
- Ramp-up Period (in second): the time it takes until the last session is started
Add HTTP Request Defaults
- Add > Config Element > HTTP Request Data
- Configure at least these settings:
- Server Name or IP: the host name or IP address of the server you run the tests against.
- Path: the base path of your application.
Add a Recording Controller
The recording controller will contain the recorded requests
- Add > Logic Controller > Recording Controller
Add a Cookie Manager
The cookie manager is required to ensure the same HTTP session in each request. If cookies are not working every request will create a new HTTP session and the server will return an HTTP error code.
- Add > Config Element > HTTP Cookie Manager
The defaults are fine.
Please note that there is a bug in JMeter 2.3.1 for clearing cookies for multiple runs / iterations; please use either JMeter 2.3.2 or later see [1]).
Add a Timer
In order to configure the request interval, you have to add a timer. Without a timer, every response is immediately followed by the next request, something a real user is not able to do.
You might want to use a random timer to add some deviation to the interval. This will lead to a better approximation of a real users behavior. We recommend the Gaussian random timer.
- Add > Timer > Gaussian Random Timer
- Configure:
- Constant Delay Offset (in milliseconds): the time between two requests
- Deviation (in milliseconds): the deviation of this interval
Prepare the Recording
Add Test Script Recorder
JMeter contains a test script recorder. This is basically a proxy server that listens on a local port (8080 by default). Since the recorder is not part of the test plan, it is added to the Workbench node.
- Open the context menu on WorkBench, and select Add > Non-Test Elements > HTTP(S) Test Script Recorder (in older versions this was called HTTP Proxy Server).
- Things to configure:
- Global Settings > Port: the port that the proxy server should listen to.
- URL Patterns to exclude: to exclude static resources served by RAP, use the exclude pattern
.*\/rwt-resources\/.*
Whether or not to exclude requests for static resources (images, script files, etc.) is up to you. These resources are cached by the browser and only contribute a small portion to the entire server load. However, you can safely keep static resources in the recording if you prefer.
Configure Browser
- Configure your browser to use a proxy. Use localhost and the port you configured in the test recorder (8080 by default).
Record a test
- Press the Start button at the bottom of the HTTP(S) Test Script Recorder page to start the recorder.
- Open a new browser tab and load the URL of your application.
- Now click through the application to record the test session.
- Note: In RAP versions prior to 2.1, you had to re-start the browser before recording a test in order to ensure the parameter requestCounter started from zero.
Remove ServerPush Requests
After recording your test, you need to eliminate all requests with parameter servicehandler = org.eclipse.rap.serverpush. Usually, these are all requests without a cid parameter in the URL.
These requests are not answered by the server and will lead to blocking.
Save Test Plan
At this point, you should save the recorded test plan to a file.
Running a test
Once the test plan is configured and session is recorded, you can start the test by pressing the green Start button in the toolbar.
In the top right corner, you can observe the number of parallel sessions that are currently running.
Checking the Results
To ensure that your recording is OK, you may want to start the test with only ONE session (configured in the Thread Group).
Save Responses to Files
By adding this element to your test plan you can ensure that the responses contain real JSON (not just errors). The first response will be HTML, all other responses JSON.
Add > Listener > Safe Responses to a File
Note that the target directory has to be present, it will not be created by JMeter.
If you run the recording twice, the second pass should lead to the same pattern of responses as the first one.
Configuration and Tuning Tips
- Run JMeter and the server on different machines. JMeter causes significant CPU load as well.
- Provide your application with enough heap. This depends on the number of active sessions and the memory consumption per session of your application.
- Choose an appropriate garbage collection strategy. We've found -XX:+UseConcMarkSweepGC to work best (on multi-core machines). The default garbage collector may lead to longer GC periods and can delay some responses drastically.
- Start the JVM in server mode (-server)
- If you use Tomcat, this servlet http://yourserver/manager/status/all provides interesting information
- When running JMeter with many worker threads, you must increase its heap size. It is advisable to start with a large initial heap size, to avoid delays resulting from additional heap allocation. Refer to the HEAP parameter in the jmeter script.
Running JMeter in headless mode
Running JMeter with a GUI could slow it down. You can run JMeter in headless mode, using the command below. This runs testplan.jmx and saves the results to result.jtl.
./jmeter -n -t /home/elias/testplan.jmx -l /tmp/result.jtl
The result file can be opened with jmeter after the test run is finished. A large result file takes some time and memory to open - be patient and increase the heap. Warning: interrupting a test run in progress, will create an incomplete result file that cannot be opened! Note also that the command will immediately return and then jmeter will be executing in the background (as a separate javaw process). Wait for this to finish before opening the results file.
URL rewriting instead of cookies
We don't recommend using URL rewriting. However, if you rely on it, this forum post might help:
I was able to run the test with the url rewriting. In JMeter there is something called regular expression extracter. I used this to extract the jsessionid from the first response from the server and then pass the same id in the all the other requests.