Testing Process Part 1 Draft 2 Comments

Jonathan West

Here are my section 5 comments, as requested:

BTW, 5.1 section has a typo, should read the 'JVMPI tests' rather than the 'JVMTI' tests.

5.2 JVMTI Profiler Test Suites:

Like the JVMPI Profiler tests, the JVMTI profiler tests are run like regular JUnit tests, but require manual configuration of a local configuration file. Unlike the JVMPI tests, it is not required to install a test server on the remote machine, or to edit remote configuration files, as these functions are now fully integrated into the local test installation.

5.3 Platform.Communication.Control.Channel.Test automated test:

The control channel tests are run like a regular jUnit test, with test execution being fully automated. The jUnit test will connect to the remote hosts being tested, configure the environment, run the test, and compare the results from the hosts. Presently, z/OS on zSeries and AS400 are not supported. Execution results must still be generated manually.

'5.4 Platform.Communication.Request.Peer.Monitor automated test

While request peer monitoring is still a manual test, it is a tool-assisted manual test, meaning that through automation of installation, configuration, and execution, the total test run time is reduced by about 80% versus a fully-manual run. Execution results must still be generated manually. z/OS on zSeries and AS400 are not supported.

--Jgwest.ca.ibm.com 11:59, 29 November 2007 (EST)

Paul Slauenwhite

• Can we change the organization to (note new sections):

1.0 Preparation

 1.1 Install the TPTP Test Tools
 1.2 Extracting the Test Resources
 1.3 Structure of the Test Resources

2.0 Creating Tests

 2.1 Manual Test Suites
 2.2 JUnit and JUnit Plug-in Test Suites
 2.3 AGR Test Suites

3.0 Executing Tests

 3.1 Manual Test Suites
 3.2 JUnit and JUnit Plug-in Test Suites
 3.3 AGR Test Suites

4.0 Common Tasks

 4.1 Launching Test Suites
 4.2 Viewing Test Logs
 4.3 Report Test Results

5.0 References

• We need to document the BVT process:
  • The root-level test suites for BVTs.
  • The test infrastructure will execute the org.eclipse.hyades.tests/AllBVT<test type>Tests.testsuite test suites.
  • We do not need to check-in the execution results to CVS due to disk space limitations and polluting of our test pass results.
  • Each developer can rerun the automated tests to reproduce a failure.
• We need to add the new structure in CVS, which now to three sets of root level test suites in the TP1, TP2, BVT directories.
• 1.1:
  • "1. Open the TPTP download site (embedded link to http://www.eclipse.org/tptp/home/downloads/)."
  • "2. Select the release of TPTP to be tested and download...". Capitalize Remember in the note. AGR is now an as-is component (not Technology Preview).
• 1.2:
  • i) "for that subproject".
  • 2) "beginning of each test pass, test report generation, etc.".
• 1.3:
  • Remove steps 1 and 2.
  • Move step 3 to the new 4.1 section.
• 4.1:
  • The follow steps are common for launching all types of test suites
    • Create a new test deploymnet:
      • (add steps - see http://help.eclipse.org/help33/topic/org.eclipse.hyades.test.doc.user/tasks/tdeploymentintro.htm)
    • Create a test launch configuration:
      • (add steps - see http://help.eclipse.org/help33/topic/org.eclipse.hyades.test.doc.user/tasks/trunningmanualtestsuite.htm)
      • Choose the results folder associated with the selected test suite and deployment platform

Note: The name and directory structure containing the generated execution history file MUST be the same as it will be in CVS in order for generated reports to work correctly.