TPTP-Test-Tools-20080211

Attendees

• Present:
  • Paul Slauenwhite
  • DuWayne Morris
  • Jerome Bozier
  • Tony Wang

4.4.1

Release Information

• 4.4.1 will ship with the Europa Winter Maintenance Release (target release date: February 29, 2008).

• Detailed Schedule.

• Defect queries.

• Download.

• Test reports.

Where are we at?

• We are currently starting one week of fixing any stop-ship defects and regression testing (target: February 15).

Testing

• Candidate driver: TPTP-4.4.1-200802061217

• We have completed the 4.4.1 TP1 (see the test reports).

Defects

• Blockers:
  • (Linux) Cannot connect to the Agent Controller: Paul has verified the fix.
  • (Linux) Cannot start the Agent Controller: Paul has verified the fix.

• Criticals:
  • BIRT Test Reports do not contain any content: Caused by BIRT defect Data source is not used during report generation.

4.5

Release Information

• 4.5 will ship with the Ganymede Simultaneous Release (target release date: June 30, 2008).

• Detailed Schedule.

• Defect queries.

• Download.

• Test reports.

Where are we at?

• We are completing a 1.5 week I5 TP1 (target: February 13).
• I5 development is closed and requires Project/PMC approval.

Testing

• We need to complete the I5 TP1 by Wednesday, February 13.
• Candidate driver: TPTP-4.5.0-200802100400

4.5 I5 TP1

• TP1 testing assignments:
  • Manual:
    • Tony:
      • /org.eclipse.hyades.test.ui.datapool.tests/manual/datapool/Test.UI.DatapoolEditor_editing.testsuite
      • /org.eclipse.hyades.test.ui.datapool.tests/manual/datapool/Test.UI.DatapoolEditor_importExport.testsuite
      • /org.eclipse.hyades.test.ui.logViewer.tests/manual/Test.UI.FrameworkEditors_logViewer_windows.testsuite
      • /org.eclipse.hyades.test.ui.logViewer.tests/manual/Test.UI.FrameworkEditors_logViewer_linux.testsuite
      • /org.eclipse.hyades.test.ui.logViewer.tests/manual/Test.UI.FrameworkEditors_logViewer_extensibility.testsuite
      • /org.eclipse.hyades.execution.harness.tests/manual/defects/Test.Execution.ExecutionHarness.testsuite
    • Paul:
      • /org.eclipse.tptp.test.doc.user/manual/Doc/Test.Doc.smoke.testsuite
      • /org.eclipse.tptp.test.tools.api.tests/manual/Recorder/Test.UI.APIRecorder.Windows_IA32.testsui
      • /org.eclipse.tptp.platform.report.tests/manual/Reporting/Plaform.UI.Charting_ChartBoundsSVG.testsuite
      • /org.eclipse.tptp.platform.report.tests/manual/Reporting/Plaform.UI.Charting_ChartInternationlizationSVG.testsuite
      • /org.eclipse.tptp.platform.report.tests/manual/Reporting/Plaform.UI.Charting_ChartStaticImage.testsuite
      • /org.eclipse.tptp.platform.report.tests/manual/Reporting/Plaform.UI.Charting_ChartSVG.testsuite
      • /org.eclipse.tptp.platform.report.tests/manual/Reporting/Plaform.UI.Charting_ChartSWT.testsuite
      • /org.eclipse.hyades.test.java.tests/manual/regressionTests/Test.Execution.JUnitPluginRunner.testsuite
    • Duwayne:
      • /org.eclipse.hyades.test.URLTest.tests/manual/basic_tests/Test.UI.URLTest.Windows.testsuite
      • /org.eclipse.hyades.test.URLTest.tests/manual/basic_tests/Test.UI.URLTest.Linux.testsuite
      • /org.eclipse.hyades.test.URLTest.tests/manual/basic_tests/Test.Execution.URLRunner.Remote.testsuite
  • JUnit:
    • Tony:
      • /org.eclipse.tptp.test.testservices.tests/junit/Test.IBM142_TestServices.testsuite
      • /org.eclipse.tptp.test.testservices.tests/junit/Test.IBM15_TestServices.testsuite
      • /org.eclipse.tptp.test.testservices.tests/junit/Test.Sun142_TestServices.testsuite
      • /org.eclipse.tptp.test.testservices.tests/junit/Test.Sun15_TestServices.testsuite
      • /org.eclipse.hyades.test.ui.logViewer.tests/JUnit/Test.UI.FrameworkEditors_logViewer_UtilitiesTest.testsuite
  • JUnit Plug-in:
    • Tony:
      • /org.eclipse.hyades.test.java.tests/junit/Test.JUnit.CodeSynchronizationTest.testsuite
      • /org.eclipse.hyades.test.java.tests/junit/Test.JUnit.JUnitTestGeneration.testsuite
      • /org.eclipse.hyades.test.java.tests/junit/Test.JUnit.JUnitTestSuiteFacadeTest.testsuite
      • /org.eclipse.hyades.test.core.tests/junit/launch/facade/TestLaunchConfigurationFacadeTest.testsuite
      • /org.eclipse.hyades.test.core.tests/junit/launch/facade/DeploymentLaunchConfigurationFacadeTest.testsuite
      • /org.eclipse.hyades.test.core.tests/junit/launch/facade/ExecutionHistoryLaunchConfigurationFacadeTest.testsuite
      • /org.eclipse.hyades.test.core.tests/junit/launch/extensions/DeploymentsProviderTest.testsuite
      • /org.eclipse.hyades.test.core.tests/junit/launch/extensions/ExecutionHistoryDefaultsProviderTest.testsuite
  • AGR:
    • Tony:
      • /org.eclipse.hyades.test.java.tests/autoui/Test.Execution.IBM142_JUnitRunner.testsuite
      • /org.eclipse.hyades.test.java.tests/autoui/Test.Execution.IBM15_JUnitRunner.testsuite
      • /org.eclipse.hyades.test.java.tests/autoui/Test.Execution.Sun142_JUnitRunner.testsuite
      • /org.eclipse.hyades.test.java.tests/autoui/Test.Execution.Sun15_JUnitRunner.testsuite
      • /org.eclipse.hyades.test.ui.navigator.tests/gui/TestNavigator_Logical_auto.testsuite
      • /org.eclipse.hyades.test.ui.navigator.tests/gui/TestNavigator_Resource_auto.testsuite

Test Automation Initiative

• Paul and the Release Engineering Team are working on consolidating our testing process and automation infrastructure:
  • Joel is working on integrating these BVTs with the nightly build process and post the Test Reports to the Build Reports page.
  • For more information, see the WIKI page.
  • Outstanding defect against the Test Project include:
    • Paul:
      • Remove XMLUnit dependency and convert reporting tests to TPTP tests: Targeted to i5.
      • Content missing in Tabular Report: Targeted to i6.
    • Duwayne:
      • ASF Test Execution Service races with AutoBuildJob -- can lead to unexpected test results: Targeted to i6.
      • ANT test results property requires trailing slash to specify location: Targeted to i6.
      • ANT importExistingProjects throws Exception: Targeted to i6.
      • Running ASF through Ant requires display export: Targeted to i6.

Enhancements

• Queries.
• Schedule.

• Xin Ying Huang:
  • Encrypted datapool variable support: The implementation is complete. Paul and Joe will be finish the code review by Thursday.

• Duwayne:
  • Test execution should support parallel execution of tests: See the meeting minutes for more details. Duwayne will complete this enhancement, including test cases and product documentation in I5/I6 development. This past week, Duwayne solved the IllegalMonitorStateException exception but now is experiencing a deadlock. Test cases are outstanding. Duwayne will schedule a call for next week to discuss the progress and outstanding items.

• Jerome:
  • Move, Copy, Delete, Rename and Import/Export test assets: Jerome completed the engine work (bugs remain in tee delete use case) and chaired a follow-on call/demo. Test cases are outstanding. For more information, see:
    • Design Review
    • Design Decisions
    • Core Use Cases
    • Detailed Design Document
    • Design Discussion and Demo

• Paul:
  • Generic Recorder Wizard enhancement to filter the list of recorders: Paul has closed with the consuming product on removing the duplicate HTTP recording wizard (see defect 137591) and wizard to import HTTP recordings (see [https://bugs.eclipse.org/bugs/show_bug.cgi?id=206078 defect 206078) since both are currently supported by the GRF. Paul will complete the implementation/testing in I6.

Defects

• Queries.
• Schedule.

• Defects requiring a sizing:
  • Duwayne:
    • Running ASF through Ant requires display export.
    • Confusing exceptions when running automated services.
    • Only the name of the execution result is used from the results property.
  • Jerome:
    • Copying lots of test results to a newly created folder hangs the system
    • Wrong behaviour when using Working Sets in Test Navigator

• Remaining I5 defects:
  • Paul:
    • Remove XMLUnit dependency and convert reporting tests to TPTP tests: Will resolve this week.
    • Update copyright dates and translatability errors.: Requires Release Engineering defect copyright year report shows files changed in other branches.
    • TPTP Test report generation is very slow with a larger input stream.: Caused by BIRT defect Report generation is very slow with a larger input stream.
  • Duwayne:
    • Repeated launch through Automated Services Framework eventually fails: Required by a consuming product. Using Probekit to instrument the code to determine the reason for the hang. Not able to reproduce the symptoms. Duwayne will set the test bed to run for 500,000 runs to try to reproduce the symptoms. Simple fix is a timeout/exception for the invocation of the working thread.

Miscellaneous

• (Scalability) Address TPTP EMF model scalability problems: Although withdrawn as a 4.5 candidate enhancement, the outcome/status of the work done in 4.4 on this enhancement was not updated in the WIKI page. Joe will update the WIKI.

• Test project topics for upcoming AG calls include:
  • ASF and Test Services: Joe
  • Test automation initiative: Alan/Joel
  • EMMA: Paul

Round-table

Paul

• Last Week:
  • 4.4.1 TP.
  • Worked on defect Some TPTP plug-ins are not loading using TPTP-4.5.0-200802061708.

• This Week:
  • 4.5 I5 TP (Monday - Wednesday).
  • EclipseCon 2008 presentation slides.
  • Defect:
    • Remove XMLUnit dependency and convert reporting tests to TPTP tests.
  • Finish reviews:
    • Encrypted datapool variable support
    • Move, Copy, Delete, Rename and Import/Export test assets

Duwayne

• Last Week:
  • Reviewed I6 defects for Todd.
  • 4.4.1 TP.
  • Working on enhancement Test execution should support parallel execution of tests.
  • Defect:
    • Repeated launch through Automated Services Framework eventually fails

• This Week:
  • 4.5 I5 TP (Monday - Wednesday).
  • Defect:
    • Repeated launch through Automated Services Framework eventually fails
  • Working on enhancement Test execution should support parallel execution of tests.

Jerome

• Last Week:
  • No TPTP contribution.

• This Week:
  • No TPTP contribution.

Tony

• Last Week:
  • 4.4.1 TP.
  • Completed defect:
    • (Usability) Test invocation error message cannot be copied

• This Week:
  • 4.5 I5 TP (Monday - Wednesday).
  • Defects:
    • Report Icon is incorrect for Test Pass and Time Frame Historic
    • (Usability) Missing/inconsistent titles/descriptions/icons in test report dialogs.
    • API Recorder's Generate Test Suite dialog does not select a default selection.
    • (Usability) Switch to Test Perspective after test execution.
    • (Usability) Switch to Test Perspective after creating a test asset.

Action items

Duwayne

• Schedule a call for next week to discuss the progress and outstanding items for enhancement Test execution should support parallel execution of tests.

Notes

News

• Next week's call will be canceled for a Canadian holiday. Since February 18 is the start of I6, focus for the first week of development will be on finishing the 4.5 enhancements.

• For future knowledge, the Plug-in Dependencies View with Show State Status and Show Only Unresolved Plug-ins enabled (see toolbar in the view) will display all of the unresolved plug-ins (reference: Some TPTP plug-ins are not loading using TPTP-4.5.0-200802061708).

EclipseCon 2008

• Test and Performance Category.
• Test and Performance Presentations

Reminders

Lead Committers

• Continually review copyright reports and ensure files will missing copyright or incorrect copyright year are corrected.
• Add the Original Estimate (PH) when triaging a new defect.
• Update the Hours Worked (PH) when completing a defect.
• Test their components or review/track testing contributed by other committers/resources.
• Monitor the TPTP news group and answer posts specific to their Test Project components.
• Verify that all API changes (modifications, additions, and removals) do not breech the TPTP API Contract.
• Use the MyFoundation Portal for committer elections.