TPTP-Test-Tools-20080204

Attendees

• Present:
  • Paul Slauenwhite
  • DuWayne Morris
  • Jerome Bozier
  • Tony Wang

4.4.1

Release Information

• 4.4.1 will ship with the Europa Winter Maintenance Release (target release date: February 29, 2008).

• Detailed Schedule.

• Defect queries.

• Download.

• Test reports.

Where are we at?

• We are currently starting the last week of the 4.4.1 test pass (TP) (target: February 8).

Testing

• Candidate driver: TPTP-4.4.1-200801280906

• Blocker:
  • (Linux) Cannot connect to the Agent Controller: Paul will verify the fix today in the TPTP-4.4.1-200802010100 driver.

4.4.1 TP1

• Tony completed a most of the 4.4.1 testing as part of his orientation. He did an exceptional job completing the testing on-time and without errors. A job well done Tony!

• The outstanding testing includes:
  • Duwayne:
    • /org.eclipse.hyades.test.URLTest.tests/manual/basic_tests/Test.Execution.URLRunner.Remote.testsuite
  • Paul:
    • /org.eclipse.tptp.test.doc.user/manual/Doc/Test.Doc.smoke.testsuite
    • /org.eclipse.hyades.test.ui.logViewer.tests/manual/Test.UI.FrameworkEditors_logViewer_linux.testsuite
    • /org.eclipse.hyades.test.java.tests/manual/regressionTests/Test.Execution.ManualRunner.Windows_XP.testsuite
    • /org.eclipse.hyades.test.java.tests/manual/regressionTests/Test.Execution.JUnitPluginRunner.testsuite
    • /org.eclipse.tptp.platform.report.tests/manual/Reporting/Platform.Reporting_BIRT_Linux.testsuite
  • Tony:
    • /org.eclipse.tptp.test.auto.gui.tests/junit/TestAutoGUIUtil.testsuite
    • /org.eclipse.hyades.test.ui.navigator.tests/gui/TestNavigator_Logical_auto.testsuite
    • /org.eclipse.hyades.test.ui.navigator.tests/gui/TestNavigator_Resource_auto.testsuite
    • /org.eclipse.tptp.platform.report.tests/manual/Reporting/Plaform.UI.Charting_ChartBoundsSVG.testsuite
    • /org.eclipse.tptp.platform.report.tests/manual/Reporting/Plaform.UI.Charting_ChartInternationlizationSVG.testsuite
    • /org.eclipse.tptp.platform.report.tests/manual/Reporting/Plaform.UI.Charting_ChartStaticImage.testsuite
    • /org.eclipse.tptp.platform.report.tests/manual/Reporting/Plaform.UI.Charting_ChartSVG.testsuite
    • /org.eclipse.tptp.platform.report.tests/manual/Reporting/Plaform.UI.Charting_ChartSWT.testsuite
    • /org.eclipse.tptp.platform.report.tests/manual/Reporting/Platform.Reporting_BIRT.testsuite

4.5

Release Information

• 4.5 will ship with the Ganymede Simultaneous Release (target release date: June 30, 2008).

• Detailed Schedule.

• Defect queries.

• Download.

• Test reports.

Where are we at?

• We are starting a 1.5 week I5 TP1 (target: February 13) since the 4.4.1 TP is nearly complete.
• I5 development is closed and requires Project/PMC approval.

Testing

• We want to complete the I5 TP1 by Friday, February 8.
• The candidate driver will be announced by the Release Engineering Team later today.

4.5 I5 TP1

• TP1 testing assignments:
  • Manual:
    • Tony:
      • /org.eclipse.hyades.test.ui.datapool.tests/manual/datapool/Test.UI.DatapoolEditor_editing.testsuite
      • /org.eclipse.hyades.test.ui.datapool.tests/manual/datapool/Test.UI.DatapoolEditor_importExport.testsuite
      • /org.eclipse.hyades.test.ui.logViewer.tests/manual/Test.UI.FrameworkEditors_logViewer_linux.testsuite
      • /org.eclipse.hyades.test.ui.logViewer.tests/manual/Test.UI.FrameworkEditors_logViewer_windows.testsuite
      • /org.eclipse.hyades.test.ui.logViewer.tests/manual/Test.UI.FrameworkEditors_logViewer_extensibility.testsuite
      • /org.eclipse.hyades.execution.harness.tests/manual/defects/Test.Execution.ExecutionHarness.testsuite
    • Paul:
      • /org.eclipse.tptp.test.doc.user/manual/Doc/Test.Doc.smoke.testsuite
      • /org.eclipse.tptp.test.tools.api.tests/manual/Recorder/Test.UI.APIRecorder.Windows_IA32.testsuite
      • /org.eclipse.tptp.platform.report.tests/manual/Reporting/Plaform.UI.Charting_ChartBoundsSVG.testsuite
      • /org.eclipse.tptp.platform.report.tests/manual/Reporting/Plaform.UI.Charting_ChartInternationlizationSVG.testsuite
      • /org.eclipse.tptp.platform.report.tests/manual/Reporting/Plaform.UI.Charting_ChartStaticImage.testsuite
      • /org.eclipse.tptp.platform.report.tests/manual/Reporting/Plaform.UI.Charting_ChartSVG.testsuite
      • /org.eclipse.tptp.platform.report.tests/manual/Reporting/Plaform.UI.Charting_ChartSWT.testsuite
      • /org.eclipse.hyades.test.java.tests/manual/regressionTests/Test.Execution.JUnitPluginRunner.testsuite
    • Duwayne:
      • /org.eclipse.hyades.test.URLTest.tests/manual/basic_tests/Test.UI.URLTest.Windows.testsuite
      • /org.eclipse.hyades.test.URLTest.tests/manual/basic_tests/Test.UI.URLTest.Linux.testsuite
      • /org.eclipse.hyades.test.URLTest.tests/manual/basic_tests/Test.Execution.URLRunner.Remote.testsuite
  • JUnit:
    • Paul:
      • /org.eclipse.tptp.test.testservices.tests/junit/Test.IBM142_TestServices.testsuite
      • /org.eclipse.tptp.test.testservices.tests/junit/Test.IBM15_TestServices.testsuite
      • /org.eclipse.tptp.test.testservices.tests/junit/Test.Sun142_TestServices.testsuite
      • /org.eclipse.tptp.test.testservices.tests/junit/Test.Sun15_TestServices.testsuite
    • Tony:
      • /org.eclipse.hyades.test.ui.logViewer.tests/JUnit/Test.UI.FrameworkEditors_logViewer_UtilitiesTest.testsuite
  • JUnit Plug-in:
    • Paul:
      • /org.eclipse.hyades.test.java.tests/junit/Test.JUnit.CodeSynchronizationTest.testsuite
      • /org.eclipse.hyades.test.java.tests/junit/Test.JUnit.JUnitTestGeneration.testsuite
      • /org.eclipse.hyades.test.java.tests/junit/Test.JUnit.JUnitTestSuiteFacadeTest.testsuite
      • /org.eclipse.hyades.test.core.tests/junit/launch/facade/TestLaunchConfigurationFacadeTest.testsuite
      • /org.eclipse.hyades.test.core.tests/junit/launch/facade/DeploymentLaunchConfigurationFacadeTest.testsuite
      • /org.eclipse.hyades.test.core.tests/junit/launch/facade/ExecutionHistoryLaunchConfigurationFacadeTest.testsuite
      • /org.eclipse.hyades.test.core.tests/junit/launch/extensions/DeploymentsProviderTest.testsuite
      • /org.eclipse.hyades.test.core.tests/junit/launch/extensions/ExecutionHistoryDefaultsProviderTest.testsuite
  • AGR:
    • Paul:
      • /org.eclipse.hyades.test.java.tests/autoui/Test.Execution.IBM142_JUnitRunner.testsuite
      • /org.eclipse.hyades.test.java.tests/autoui/Test.Execution.IBM15_JUnitRunner.testsuite
      • /org.eclipse.hyades.test.java.tests/autoui/Test.Execution.Sun142_JUnitRunner.testsuite
      • /org.eclipse.hyades.test.java.tests/autoui/Test.Execution.Sun15_JUnitRunner.testsuite
    • Tony:
      • /org.eclipse.hyades.test.ui.logViewer.tests/gui/Test.UI.FrameworkEditors_logViewer_linux.testsuite
      • /org.eclipse.hyades.test.ui.logViewer.tests/gui/Test.UI.FrameworkEditors_logViewer_windows.testsuite
      • /org.eclipse.hyades.test.ui.navigator.tests/gui/TestNavigator_Logical_auto.testsuite
      • /org.eclipse.hyades.test.ui.navigator.tests/gui/TestNavigator_Resource_auto.testsuite

Test Automation Initiative

• Paul and the Release Engineering Team are working on consolidating our testing process and automation infrastructure:
  • On Friday, Paul and Joel were able to run JUnit/JUnit Plug-in BVTs on the Reference Platform. The next step is for Joel to integrate these BVTs with the nightly build process and post the Test Reports.
  • For more information, see the WIKI page.
  • Outstanding defect against the Test Project include:
    • Paul:
      • Test log not accurate: Completed in i5.
      • publication task problems: Completed in i5.
      • Remove XMLUnit dependency and convert reporting tests to TPTP tests: Targeted to i5.
      • Content missing in Tabular Report: Targeted to i6.
    • Duwayne:
      • ASF Test Execution Service races with AutoBuildJob -- can lead to unexpected test results: Targeted to i6.
      • ANT test results property requires trailing slash to specify location: Targeted to i6.
      • ANT importExistingProjects throws Exception: Targeted to i6.
      • Running ASF through Ant requires display export: Targeted to i6.

Enhancements

• Queries.
• Schedule.

• Xin Ying Huang:
  • Encrypted datapool variable support: The implementation is complete. Paul and Joe will be finishing the code review this week.

• Duwayne:
  • Test execution should support parallel execution of tests: See the meeting minutes for more details. Duwayne will complete this enhancement, including test cases and product documentation in I5/I6 development. This past week, Duwayne finished code changes for parallel agent start-up and running multiple simultaneous agents but is still seeing an IllegalMonitorStateException exception. Test cases are outstanding. Duwayne will schedule a call for next week to discuss the progress and outstanding items.

• Jerome:
  • Move, Copy, Delete, Rename and Import/Export test assets: Jerome completed the engine work (bugs remain in tee delete use case) and chaired a follow-on call/demo. Test cases are outstanding. For more information, see:
    • Design Review
    • Design Decisions
    • Core Use Cases
    • Detailed Design Document
    • Design Discussion and Demo

• Paul:
  • Generic Recorder Wizard enhancement to filter the list of recorders: Paul has closed with the consuming product on removing the duplicate HTTP recording wizard (see [https://bugs.eclipse.org/bugs/show_bug.cgi?id=137591 defect 137591) and wizard to import HTTP recordings (see [https://bugs.eclipse.org/bugs/show_bug.cgi?id=206078 defect 206078) since both are currently supported by the GRF. Paul will complete the implementation/testing in I6.

Defects

• Queries.
• Schedule.

• Defects requiring a sizing:
  • Duwayne:
    • • publication task problems.
      • Running ASF through Ant requires display export.
  • Jerome:
    • • Copying lots of test results to a newly created folder hangs the system
      • Wrong behaviour when using Working Sets in Test Navigator

• Remaining I5 defects:
  • Paul:
    • Remove XMLUnit dependency and convert reporting tests to TPTP tests.
  • Duwayne:
    • Repeated launch through Automated Services Framework eventually fails: Required by a consuming product.

• The following I5 defects have been deferred to I6:
  • • ASF marshalling return value between service and client adapter does not have support for failures / exceptions
    • RecorderException when HTTP recording with Internet Explorer on Windows XP.
    • Vista-specific error dialog when recording using IAC/IE on XP.
    • Text in Recorder Control should be copyable.
    • Get rid of the Remove option from the Http Requests tab of the Hyades URL Test Suite
    • (Test Automation) Automate manual test suites using the Automated GUI Recorder.
    • (Test Automation) Automate manual test suites using the Automated GUI Recorder.
    • Duplicate wizards for creating a URL recording/test.
    • Copy / Paste should not be available in the logical test navigator
    • (Usability) Hyperlinks in Test Log view are rendered differently for the Test Execution event type.
    • Remove references to internal APIs.
    • Exported datapool with deleted variables is incorrect.
    • Importing large datapool freezes wizard.
    • scroll bars not appearing when needed in test suite editor and other Hyades editors when content doesn't all fit in a page
    • Log Viewer : navigation button on event part of test log are not enabled by default
    • TPTP test pass reports show test type ID instead of meaningful name.
    • Need new icon for testlog files
    • Cannot import HTTP recording file.
    • (Usability) Switch to Test Perspective after test execution.
    • (Usability) Test invocation error message cannot be copied
    • (Usability) Switch to Test Perspective after creating a test asset.
    • Report Icon is incorrect for Test Pass and Time Frame Historic
    • (Usability) Missing/inconsistent titles/descriptions/icons in test report dialogs.

Miscellaneous

• (Scalability) Address TPTP EMF model scalability problems: Although withdrawn as a 4.5 candidate enhancement, the outcome/status of the work done in 4.4 on this enhancement was not updated in the WIKI page. Joe will update the WIKI.

• Test project topics for upcoming AG calls include:
  • ASF and Test Services: Joe
  • Test automation initiative: Alan/Joel
  • EMMA: Paul

Round-table

Paul

• Last Week:
  • Worked on enhancement Generic Recorder Wizard enhancement to filter the list of recorders.
  • Completed defects:
    • (Test Project) (Web) (Matrix) Update the TPTP Matrix and Bugzilla component owners.
    • Ant impoertExistingProjects task does not build projects
    • publication task problems
    • Fix project/compilation errors in the TPTP test bucket.
    • Platform.Communication.Installation.Uninstallation testsuite failed to run
    • Test log not accurate

• This Week:
  • 4.4.1 TP.
  • 4.5 I5 TP.
  • Defect:
    • Remove XMLUnit dependency and convert reporting tests to TPTP tests.
  • Finish reviewing:
    • Encrypted datapool variable support
    • Move, Copy, Delete, Rename and Import/Export test assets

Duwayne

• Last Week:
  • 4.4.1 TP.
  • Working on enhancement Test execution should support parallel execution of tests including investigating sequence of events and setting up a test environment for multiple agents.
  • Defect:
    • Repeated launch through Automated Services Framework eventually fails

• This Week:
  • 4.4.1 TP.
  • 4.5 I5 TP.
  • Defect:
    • Repeated launch through Automated Services Framework eventually fails

Jerome

• Last Week:
• Working on enhancement Move, Copy, Delete, Rename and Import/Export test assets.
  • Completed a stable patch for review.

• This Week:
  • No TPTP contribution.

Tony

• Last Week:
  • 4.4.1 TP.

• This Week:
  • 4.4.1 TP.
  • 4.5 I5 TP.

Action items

Duwayne

• Schedule a call for next week to discuss the progress and outstanding items for enhancement Test execution should support parallel execution of tests.

Notes

News

• The New York Giants are the Super Bowl XLII champions!

• The PMC approved Jerome as the Lead Committer for the following components:
  • Test.UI
  • Test.UI.ConfigEditors
  • Test.UI.DatapoolEditor
  • Test.UI.FrameworkEditors
  • Test.UI.TestPerspective

• Paul created a Test Tools Project Code Snippets page (linked from the Miscellaneous page in the TPTP WIKI) for code snippets that demonstrate specific programmatic tasks commonly requested by Test Tools users on the news group or mailing list.

EclipseCon 2008

• Test and Performance Category.
• Test and Performance Presentations

Reminders

Lead Committers

• Continually review copyright reports and ensure files will missing copyright or incorrect copyright year are corrected.
• Add the Original Estimate (PH) when triaging a new defect.
• Update the Hours Worked (PH) when completing a defect.
• Test their components or review/track testing contributed by other committers/resources.
• Monitor the TPTP news group and answer posts specific to their Test Project components.
• Verify that all API changes (modifications, additions, and removals) do not breech the TPTP API Contract.
• Use the MyFoundation Portal for committer elections.