View source for DSDP/TM/JUnittests Framework Documentation

== Hitch-Hacker's Guide to Using and Writing DSDP-TM (aka RSE) JUnit tests ==

This page is giving a introduction to the usage of the DSDP-TM JUnit tests framework as well as providing ''How To's'' and short descriptions of the features available in the framework for JUnit test developers.

'''Note:''' The page is under on-going maintenance and will be updated if the framework is changed or extended!

== Using the DSDP-TM JUnit tests ==

Using means running the test cases either within a ''target platform'' Eclipse or within the ''currently executing'' Eclipse instance. There is basically only one question to distiguish between the two use cases:

''Do you only '''use''' RSE within your currently running Eclipse environment for remote system connectivity or do you '''develop''' plugin(s) based on RSE and/or using RSE API?''
* If you answer ''I develop plugin(s) using RSE API and/or I contribute/plan to contribute actively to RSE.'' '''-> see section [[#Launching_the_DSDP-TM_JUnit_tests_as_JUnit_Plug-in_Test|2.1]]''' 
* If you answer ''I only use RSE within my running Eclipse for accessing remote systems.'' '''-> see section [[#Launching_the_DSDP-TM_JUnit_tests_from_within_the_currently_running_Eclipse_instance|2.2]]'''
* If you cannot answer the above question clearly (because you may simple fall in both user groups at the same time), you may choose the use case with can serve your needs most efficiently.

=== Launching the DSDP-TM JUnit tests as JUnit Plug-in Test ===

'''Precondition:''' Both JDT and PDE needs to be installed and activated.

# Open the Eclipse Launch Configuration Dialog in Run mode (Run -> Run...).
# Expand the "''JUnit Plug-in test''" category if not expanded anyway.
# Select the pre-configured launch configuration named "''RSE Combined Test Suite''".
# Choose "''Duplicate''" either from the context menu or from the tree toolbar.
# Change the launch configuration name of the duplicate to identify your test suites launch configuration.
# If your test suite is in another project than "''org.eclipse.rse.tests''", click now on the "''Browse...''" right after the "''Project''" entry field and select the project which contains your test suite.
# Click on the "''Select...''" button right after the "''Test class''" and choose your test suite or test case to run.
# Switch to the "''Main''" tab and check all settings to be as you want them. Usely you can leave them with their defaults.
# Switch to the "''Arguments''" tab and check all settings to be as you want them. Usely you can leave them with their defaults.
# Switch to the "''Environment''" tab if you require to set a specific environment to enable your test suite to run.
# '''Switch to the "''Common''" tab and change the "''Save as''" setting to "''Local file''".'''
# Click on the "''Run''" button to execute the launch configuration.

The ''JUnit'' view will open, near to the ''Problems'' view, to show the progress, successes and failures in executing the single test cases. Inspect the failures within the failure trace part for analysing why a test case might have failed. You have real great navigation possibilities and comparisation support from there.

In case that the ''JUnit'' view is not opened automatically, you can open the view via ''Window -> Show View -> Other ... -> Java -> JUnit''.

=== Launching the DSDP-TM JUnit tests from within the currently running Eclipse instance ===

'''Precondition:''' The plugin ''org.eclipse.rse.tests.framework'' must be installed and activated.

# Switch to the "''Remote System Explorer''" perspective.
# Open the "''Test Suites''" view via ''Window -> Show View -> Other ... -> RSE Testing -> Test Suites''.
# Select your test suite (or all test suites if you want to run them all) and choose "''Run''" from the context menu.

Progress and results of the single running test cases are presented in the result pane of the view. Copy the content of the result pane and send it to the contact how have asked you to run a specific test suite (or all) within your running Eclipse instance.

Please note that the "''Test Suites''" view executes the test suites by default in a random Eclipse worker thread. You may enforce the test suite execution to the SWT UI thread by unselecting the green button in the view toolbar. However this will not have any effect either way for test suites or test cases how '''require''' to run in worker threads to succeed. As these test suites or test cases can (and should) flag the desire to run in worker threads and the test framework can (and will) asure that this desire is fulfilled on runtime, the green button may not show an noticable effect.

If you are using this way to launch the test cases for RSE development, you must carefully indentify the output part belonging to the failed test case and copy the failure stack trace from there to you RSE development environments ''Java Stack Trace Console''. As you loose any comfort of easy navigation and result comparisation this way, for RSE developers it is gently advised to follow section [[#Launching_the_DSDP-TM_JUnit_tests_as_JUnit_Plug-in_Test|2.1]].

== Writing DSDP-TM JUnit tests ==

=== Where to put a new test suite or test case? ===

* If you write test suites and test cases to test RSE components or parts, add the test suites and test cases to the ''org.eclipse.rse.tests'' plug-in.
* If you write test suites and test cases for your own implementation based on RSE and the RSE tests framework, create your own tests plug-in which depends on RSE and RSE tests framework plug-ins.
* If you want to add a new test case, always check if you can extend an existing test suite before creating a new one.
* If you have a set of test cases, probably linked to an specific RSE component or part, create a new test suite if there is not already one for the same RSE component or part.
* If creating a new test suite
** Follow the package naming scheme of the existing test suites and test cases.
** Each package must start with the prefix ''org.eclipse.rse.tests''.
** Place interfaces and implementations designated for use only in specific test case(s) in ''internal'' packages.
** '''Always''' use the following template to create your own test suite and '''change only''' the ''&lt;YOUR ...&gt;'' markers!

   <pre>
   &lt;YOUR COPYRIGHT HEADER&gt;
   package &lt;YOUR PACKAGE DECLARATION&gt;;

   import junit.framework.Test;
   import junit.framework.TestSuite;

   import org.eclipse.rse.tests.framework.DelegatingTestSuiteHolder;

   &lt;YOUR TEST SUITE COMMENT HEADER&gt;
   public class &lt;YOUR TEST SUITE NAME&gt; extends DelegatingTestSuiteHolder {

	/**
	 * Standard Java application main method. Allows to launch the test
	 * suite from outside as part of nightly runs, headless runs or other.
	 * <p><b>Note:</b> Use only <code>junit.textui.TestRunner</code> here as
	 * it is explicitly supposed to output the test output to the shell the
	 * test suite has been launched from.
	 * <p>
	 * @param args The standard Java application command line parameters passed in.
	 */
	public static void main(String[] args) {
		junit.textui.TestRunner.run(suite());
	}

	/**
	 * Combine all test into a suite and returns the test suite instance.
	 * <p>
	 * <b>Note: This method must be always called <i><code>suite</code></i> ! Otherwise
	 * the JUnit plug-in test launcher will fail to detect this class!</b>
	 * <p>
	 * @return The test suite instance.
	 */
	public static Test suite() {
		TestSuite suite = new TestSuite("&lt;YOUR TEST SUITE HUMAN READABLE NAME&gt;"); //$NON-NLS-1$
		// add the single test suites to the overall one here.
		suite.addTestSuite(&lt;YOUR TEST CASE IMPLEMENTATION CLASS NAME&gt;.class);
		
		return suite;
	}
	
	/* (non-Javadoc)
	 * @see org.eclipse.rse.tests.framework.AbstractTestSuiteHolder#getTestSuite()
	 */
	public TestSuite getTestSuite() {
		return (TestSuite)RSEConnectionTestSuite.suite();
	}
   }
   </pre>

=== Which base class should a test case inherit from? ===

That is quite simple. ''Do you write test cases which requires the existance of a valid connection to a remote system?''

* If '''Yes''': Inherit from ''RSEBaseConnectionTestCase''.
* If '''No''': Inherit from ''RSECoreTestCase''.

=== Dynamic single test enablement ===

It has been proved more than useful to introduce the possiblity to enable and disable single tests via configuration. This basically prevents administrator to disable failing tests by commenting out the corresponding test method within a test case. Furthermore, it enables users, having access only to the binary class files, and JUnit test developers to temporarly disable single tests of one or more test cases and/or test suites in order to focus on a specific limited set of tests. To satisfy this desire, a '''certain style''' of writing single test methods '''is required'''.

'''Test method template:'''

   <pre>a)
   ...
   public class MyTestCase extends RSEBaseConnectionTestCase {
      ...
      public void testMyFunctionality() {
         // Call isTestCaseEnabled with the test id and return immediatelly if the
         // call should return false!
         if (!RSETestsPlugin.isTestCaseEnabled("MyTestCase.testMyFunctionality") return; //$NON-NLS-1$
         ...
      }
      ...
   }

b)
   Add the test id to the RSETestsResources.properties file and adjust the default (true == enabled, false == disabled).

   ...
   RSEConnectionTestCase.testConnect=true
   RSEConnectionTestCase.testResolveFilterString=true

   MyTestCase.testMyFunctionality=true
   ...</pre>

Single tests can be enabled/disabled by
** Adjusting the test enabled state within ''RSETestsResources.properties'' file or
** Specifying ''-D&lt;test id&gt;=&lt;true|false&gt;''.

If ''-D&lt;test id&gt;'' is explicitly provided, it overrides all other settings. If neither ''-D&lt;test id&gt;'' provided nor the test id is defined within ''RSETestsResources.properties'', the test is ''enabled by default''.

'''Related methods:'''
   <pre>
   * RSETestsPlugin.isTestCaseEnabled(String)

   See the corresponding javadoc for details on these method.
   </pre>

=== ''RSECoreTestCase'' features in detail ===

==== Test Properties Management ====

The RSE JUnit test framework provides the possibility to set and test for test case wide properties. The framework itself uses the test properties to control global test case behavior like the perspective to switch to before a test case runs or if and which view might be maximized to eliminate other view redraws for performance sensitive test cases. You can choose to manipulate the standard RSE JUnit test framework properties or you may add your own properties controlling specific behaviour of your own test framework base classes. The framework initialize the test properties once at the time the test case is instanciated. Most of the global framework properties are controlling ''setUp()'' and/or ''tearDown()'' behavior, but you may use the test properties management framework for controlling the behavior of your test or framework methods besides these two core JUnit test framework methods.

'''Related methods:'''
   <pre>
   * initializeProperties()
   * setProperty(String, boolean)
   * setProperty(String, String)
   * isProperty(String, boolean)
   * initializeProperties()
   * isProperty(String, String)
   * getProperty(String)

   See the corresponding javadoc for details on these methods.</pre>

'''Related interfaces:'''
   * <code>IRSECoreTestCaseProperties</code> - Defines static constants identifying the RSE JUnit test framework global properties

'''Global properties:'''
* <tt>''PROP_SWITCH_TO_PERSPECTIVE''</tt>
** Default perspective id. This perspective will be activated, if not active anyway, within the tests ''setUp()'' method. Default value is ''org.eclipse.rse.ui.view.SystemPerspective''.
* <tt>''PROP_MAXIMIZE_REMOTE_SYSTEMS_VIEW''</tt>
** Maximize the ''Remote Systems'' view within the tests ''setUp()'' method? Default value is ''false''.
* <tt>''PROP_FORCE_BACKGROUND_EXECUTION''</tt>
** Enforce the execution of the test within a non SWT display (UI) thread? Default value is ''false''.
* <tt>''PROP_PERFORMANCE_TIMING_INCLUDE_SETUP_TEARDOWN''</tt>
** Print the test performance timing information including the time consumed from ''setUp()'' and ''tearDown()''? Default value is ''false''.

==== Views and Perspective Management ====

The RSE JUnit test framework provides convenience methods to find, show and hide views. You may use these methods to manipulate certain view attributes like toggle the zoom state of views. The framework itself uses these methods to switch to the default perspective and maximize the ''Remote Systems'' view, if configured.

'''Related methods:'''
   <pre>
   * findView(String, String)
   * showView(String, String)
   * hideView(String, String)

   See the corresponding javadoc for details on these methods.</pre>

'''Related interfaces:'''
   * <code>IRSEViews</code> - Defines static constants for all RSE related view id's.

==== Test Data Location Management ====

Single tests may require to store and access whatever kind of data file(s) to test certain RSE functionality. The test framework provides access to a centralized test data location. '''Any''' test data accessed from any of the single tests, have to be stored relative to ''''org.eclipse.rse.tests/test.data''''. Create a senseful named subdirectory there, identifying the test suite, test case or single test which is accessing this specific test data. If the test data is ''host operating system'' specific, place the test data in subdirectories named the host operating system specific id (see ''Platform.getOS()'' for details).

'''Related methods:'''
   <pre>
   * getTestDataLocation(String, boolean)

   See the corresponding javadoc for details on these method.</pre>

==== Test Failure Log Collection Management ====

In many cases, if a single test if failing, it is not enough to see the ''assert'' which caused the failure, to analyse and understand ''why'' the tested attribute or object does not have the expected value. Connections and communication to and from remote systems may easely fail by any unpredictable external event causing either the remote system or any of the involved communication tools (probably external processes) to fail. As these tools may log not necessarly to the Eclipse error log, especially if external processes are involved, it is essential to get access and collect all possible existing logs in case of a test failure. The collected logs may explicitly include information dynamically dumped to a temporary file at the time the test log collector delegate is called. The default RSE JUnit test framework test failure log collector is collecting the Eclipse error log and the current set of Java system properties at the time of the failure. Write and register your own test log collector delegate if you have to collect other files and information in case of a test failure. You may register the delegate either globally for collecting the information on all test failures or only for your specific test cases.

''All logs are collected into a ZIP archive named RSEUnittestFailureLogs_&lt;test name&gt;.zip. The ZIP archive is located within the org.eclipse.rse.tests plugin state location (&lt;workspace_loc&gt;/.metadata/.plugins/org.eclipse.rse.tests). It needs to be cleared how we can get access to these log files for the nightly running tests.''

'''Related methods:'''
   <pre>
   * collectTestLogs(Test)
   * RSETestsPlugin.addDelegate(IRSETestLogCollectorDelegate)
   * RSETestsPlugin.removeDelegate(IRSETestLogCollectorDelegate)
   * RSETestsPlugin.getTestLogCollectorDelegates()
   * IRSETestLogCollectorDelegate.getAbsoluteLogFileLocations()
   * IRSETestLogCollectorDelegate.dispose()

   See the corresponding javadoc for details on these method.</pre>

'''Related interfaces:'''
   * <code>IRSETestLogCollectorDelegate</code> - RSE Test log collector delegate public API.

==== Waiting for Results (RSEWaitAndDispatchUtil) ====

As by the multi-threaded nature of Eclipse and Java, it is not guaranteed that all API calls, events and other possible activities are executed synchronously or are finished or even started at the moment the API call returns. A lot of activities, important for test cases to wait for, Eclipse or other API implementors can decide to initiate and execute asynchronously via the very powerful Eclipse Jobs framework. It is ''absolutly essential'' for successfully writing complex (up to very complex) ''functionality'' JUnit test cases to hold the execution of the current thread time out based and/or interrupt condition based ''without preventing the SWT UI events from dispatching''! The ''RSEWaitAndDispatchUtil'' is providing methods and interfaces to deal with these requirements.

'''Related methods:'''
   <pre>
   * RSEWaitAndDispatchUtil.isDispatchThread()
   * RSEWaitAndDispatchUtil.waitAndDispatch(long)
   * RSEWaitAndDispatchUtil.waitAndDispatch(long, IInterruptCondition)
   * IInterruptCondition.isTrue()
   * IInterruptCondition.dispose()

   See the corresponding javadoc for details on these method.</pre>

'''Related interfaces:'''
   * <code>IInterruptCondition</code> - Wait and dispatch interrupt condition public API.

=== ''RSEBaseConnectionTestCase'' features in detail ===

==== The Connection Manager ====

The RSE connection manager is providing and encapsulating all functionality related to finding, creating, removing or manipulating RSE connections and sub systems.

'''Related methods:'''
   <pre>
   * getConnectionManager()
   * IRSEConnectionManager.loadConnectionProperties(IPath, boolean)
   * IRSEConnectionManager.loadConnectionProperties(Properties, boolean)
   * IRSEConnectionManager.removeConnection(String, String)
   * IRSEConnectionManager.findOrCreateConnection(IRSEConnectionProperties)
   * IRSEConnectionManager.getFileSubSystem(IHost, String)
   * IRSEConnectionManager.getShellSubSystem(IHost)
   * IRSEConnectionManager.getTestSubSystem(IHost)
   * IRSEConnectionProperties.getProperty(String)
   * IRSEConnectionProperties.setProperty(String, String)

   See the corresponding javadoc for details on these method.</pre>

'''Related interfaces:'''
   * <code>IRSEConnectionManager</code> - RSE connection manager plugin API.
   * <code>IRSEConnectionProperties</code> - Defines static constants for accessing the single connection properties and related public API.

'''RSE Connection Properties:'''
* ''ATTR_NAME''
** The connection name. ''Required''.
* ''ATTR_PROFILE_NAME''
** The profile name to store the connection to. ''Optional''.
* ''ATTR_SYSTEM_TYPE''
** The system type of the connection. ''Required''.
* ''ATTR_ADDRESS''
** The remote system IP address or DNS name. ''Required''
* ''ATTR_USERID''
** The user id to use for connecting to the remote system. ''Optional''.
* ''ATTR_PASSWORD''
** The password to use for connecting to the remote system. ''Optional''.

''If a connection property is not specified, default values are assumed. The default values are loaded from org.eclipse.rse.tests/test.data/connectionDefault.properties. The load of the default connection properties can be suppressed if necessary.''

==== The Default Local System Connection ====

The RSE JUnit test framework provides convenience access to the default available and always connected "''Local''" connection (system type == "''Local''"). You '''should prefer''' to use this connection for any test requiring a valid connection but does not test any remote system specific functionality.

'''Related methods:'''
   <pre>
   * getLocalSystemConnection()

   See the corresponding javadoc for details on these method.</pre>