Testing the ICE UI with Jubula

NiCE uses Jubula for UI testing. This article explains how to use Jubula and contains other reference sources.

Terminology

AUT: Application Under Test (The application that will have its GUI tested).

AUT Agent: The server component that allows the ITE to communicate with the AUT.

Category: provides convenient storage in the Test Suite Browser; it holds groups of Test Suites and Jobs (or even other categories).

ITE: Integrated Testing Environment

Jubula: A free ITE that provides rich testing features.

Object Mapping Testing: Tests by combining the Test Case library (input methods or actions to perform on the AUT) and identifying the individual components of the AUT to per form these on; this is much more flexible testing method and should be preferred by all professional development teams.

Observation Mode Testing: Tests by recording the input of the human and repeating it exactly unto the AUT; uses position coordinates and is not a flexible testing method.

Test Case: a container for Test Steps.

Test Job: a container for Test Suites; this is what should be selected for running tests on the AUT.

Test Step: the smallest unit of testing in Jubula.

Test Suite: a container for Test Cases.

Step-by-Step Guide to Running Your First Jubula Test

Here, we summarize the actions for executing tests on an AUT, but the steps will be covered in depth throughout the upcoming sections of this article.

• Activate the AUT agent by going to the jubula/server folder and executing the autagent-lin command. This opens the AUT Agent which uses port 60000 to allow communication between the AUT and the ITE.
• Connect the ITE to the AUT Agent by clicking the Connect to the AUT Agent in the toolbar.
• Start the AUT; it should appear in the Running AUTs View.
• Now all the testing features become available. Either create tests using observation mode or object mapping.
• Run the tests by creating a test job, and a test suite. Configure these to run the specified AUT Definition and Configuration.

Stage 1: Getting Jubula

• Download the version of Jubula corresponding to your platform.
• Install the zip file by following a wizard or starting the executable setup.sh.

The main folder contains: * the ability to run AUT from the command line. * RCP-support.zip - once unzipped, it has one plugin that needs to be installed in your AUT so the ITE may communicate with it.

Open the Jubula executable, and select an arbitrary workspace (the default should be acceptable).

Stage 2: Setting up the AUT and AUT Agent

Jubula is set up to automatically work with Swing, GEF, or HTML applications but requires a few extra steps in order to communicate with an RCP application. Jubula does not yet work with purely SWT applications. These steps allow the AUT Agent to communicate with your RCP AUT (this is the most difficult part of using Jubula).

• Extract the contents of the rcp-support.zip folder into the plug-ins directory for your RCP AUT.
• RCP applications generally have a configuration/config.ini file which contains the parameter osgi.bundle. It may need to be modified to allow the RCP remote control plugin to load on AUT startup.

The AUT and AUT Agent will always be on the same computer.

For Remote Testing Setup (Good luck here, this is tricky and requires that you learn object mapping, because observation tests do not run remotely.):

• Run the ITE on “Computer A”.
• Run the AUT Agent on “Computer B”.
• Activate the AUT on “Computer B” either from the command line locally, or from the ITE on “Computer A”.

Stage 3: Setting up the ITE (Jubula General)

• Begin by going to Test -> New, which creates a new testing project.
• Give the project a name, and specify the toolkit according to what type of application is being tested.
• Click Finish. The following tabs allow the setup of AUT; it can be done just as easily from the Test Suite Browser.
• Go to the Test Suite Browser, and access the properties of your new Test Project (Right-click or Ctrl+Shift+P). This defaults to having AUTs, selected on the left (the other tabs are just more advanced features).
• Click Add on the right to create a new AUT Definition.
• In this menu, enter the name of the application being tested (AUT), and the type of application (toolkit).
• Create an AUT configuration; this determines how the AUT is run. If there is one AUT that needs to be tested locally, and the same AUT needs to be tested remotely, just make another configuration in the same AUT Definition. If there are two separate applications to test, make two AUT Definitions.
• At a minimum, enter an arbitrary AUT configuration name, tell which computer to execute the tests on, assign an arbitrary AUT ID, and give the location of the executable for the application. More advanced options are available under the Expert View if needed.
• Right-click on the Testing project (in the Test Suite Browser), and select New -> Test Job.
• Repeat, but select New -> Test Suite. Double-click this Test Suite in the Test Suite Browser to open its editor. Here, it is possible to add Test Cases (using either observation or object mapping).
• Right-click the Test Case Browser and select New -> Test Case. Drag the test case from the Test Case from the Browser into the Test Suite Editor.

Manipulating anything in the hierarchy must be done by opening the editor to delete or add something a Test Object contains (The Test Object Editor is opened by “opening specification”, shortcut F3). The browsers only allow you arrange these Test Objects into Categories.

• Open the editor for the test case.
• Add a new test step with Shift+Enter, or right-click and select new est Step. The name is an arbitrary label for readability. The type identifies what the component is The action tells what to do with it. The component name is the actual link to the object map that must correspond to the component name you created and matched to the technical component (actual reference to button on the AUT) during object mapping.

So, if creating a component name now, it will appear in the unassigned component names box in the object mapping editor.

• Start Object Mapping Mode
• Collect the component from the application by pressing Ctrl+Shift+Q while hovering over the component. The selected component will be outlined in green.
• Return to the Mapping Editor (using the split view tab on the bottom is generally easiest), and drag the Unassigned Component Name onto the Unassigned Technical Name--this will move them both down to the assigned names view. This links the actions (Test Step) created to the actual AUT.

Stage 4: Running the Tests (The Test Executor)

• Run the Test Executor and select your Test Job. Make sure that all the Test Suite are configured from the Test Job Editor in the properties view to the current AUT ID that they will run on.
• Click yes to mark as relevant (relevant tests are exported with the project).
• Click yes to open the Test Results View when the test completes to evaluate the success of the tests.

Jubula as a default will minimize and shift focus to the application, then watch the test press buttons.

Stage 5: Review the Results

Upon completion of the test, Jubula will appear again and the Test Results View appears showing the success or failure of each Test Step.

Stage 6: Continuous Integration (CI) on Hudson

Everything needs to be launched from the command line in Hudson, and the needed args to be run in jubula/server run are:

Launch AUT Agent:

./autagent-lin  

Launch AUT:

./autrun -rcp -e /home/user/workspace/AppWithJubula/ -i AppAUTID  

In jubula(numbers)/jubula:

Launch Test Executor:

testexec.exe -project x -version x -autconfig x -autid x -testjob x  

(-s Stops screenshots from being stored)

Troubleshooting

I had the incorrect path to my AUT executable in the AUT Configuration twice, once to the wrong executable, and once to the containing folder both giving me strange results (such as the AUT launching but never arriving at the Running AUTs View, because I added the new config file but forgot to switch to the new address of my application that had this update). Start by double checking that the address here is correct.

Then make sure if using an RCP App, that you installed all the necessary plugins and configured it correctly (stage 2).

Next try running the AUT with the -clean Argument (Enter in the AUT Configuration).

An illegal position argument can be caused by moving the application or the mouse off of the active screen.

If none of these helped, try restarting Jubula and run it with the -clean arg. If you make an account at Eclipse Community Forums, someone will generally answer any questions you may have within a day or two.

Materials for Additional Assistance:

The most important Jubula reference is the Jubula User Manuel, which explains every detail of Jubula. Jubula has a built in tutorial that explains everything quite well, under HELP -> CHEAT SHEETS [This shows how to run your first test.]

jmhofer's insignificant insights Tutorial

Word Press Tutorial

http://www.rcp-vision.com/?p=1506&lang=en

http://www.subshell.com/en/subshell/blog/Extending-Jubula-For-Custom-SWT-Controls-In-RCP-Applications100.html