TPTP-Test-Tools-20071029

Attendees

• Present:
  • Paul Slauenwhite
  • Kendric Wang
  • DuWayne Morris
  • Joseph Toomey
  • Jerome Bozier

• Absent:
  • Jerome Gout

4.4.1

• 4.4.1 will ship with the Europa Winter Maintenance Release (target release date: February 29, 2008).

• Detailed Schedule.

• Defect query.

• Download.

• Test reports.

• All defects will require project/PMC approval.

• All fixes MUST be checked-in to both 4.4.1 (TPTP-4_1_1) and 4.5 (HEAD) streams.

• Outstanding defect(s):
  • 168597[Vista] HTTP Recorder requires work-around on Windows Vista: PMC approved and checked in to both 4.4.1 (TPTP-4_4_1) and 4.5 (HEAD) streams. Duwayne will update the final hours worked.
  • 207781 Mark new ImportExistingProjects class as javadoc provisional per API contract: Will request project/PMC approval this week.

• There will be a smoke test pass in the next few weeks (target date: TBD) aligned with the latest release of the IBM 1.5 JRE.

4.5

• 4.5 will ship with the Ganymede Simultaneous Release (target release date: June 30, 2008).

• Detailed Schedule.

• Defect query.

• Download.

• Test reports.

• The Test Project has only completed three defects in 4.5 i1:
  • 168597
  • 187242
  • 205293

• We are at the 1/3 milestone of TP1 (target date: November 2, 2007).
  • The candidate driver (200710260400) has been posted.
  • Testing responsibilities are based on the new component ownership (see below table).
  • TP1 is a full test pass, with the exception of smoke testing the Technology Previews (Manual Test, API Recorder, and AGR), since it is the first 4.5 test pass on the Eclipse 3.4 base and we need to generate meaningful code coverage statistics.
  • All of the Test Project committers must test with the EMMA-instrumented driver (see the instructions for details).

• Agreement with the current Component Leads has been reached for the following component changes:
  • Notes:
    • Current Component Leads maintain their committer status as developer committers for the component.
    • Paul will send instructions to the current Component Leads for nominating the proposed Component Leads.
    • Paul will remove the unnecessary components this week.
    • TPTP Project Matrix updates will be handled under defect 207968.

Bugzilla Component	Current Component Lead	New Component Lead
Platform.UI.Reporting	Sheldon Lee-Loy	Paul Slauenwhite
Test.Agents	Mark D Dunn	Paul Slauenwhite
Test.Agents.ComptestAgent	Joseph P Toomey	Paul Slauenwhite
Test.Agents.Recorder	Mark D Dunn	Duwayne Morris
Test.Analysis	Joseph P Toomey	[remove]
ASF.Documentation	Joseph P Toomey	Duwayne Morris
Test.Execution	Kent Siefkes	Paul Slauenwhite
Test.Execution.AutoGUIRunner	Liz Dancy	Duwayne Morris
Test.Execution.CommonRunner	Joseph P Toomey	Paul Slauenwhite
Test.Execution.ExecutionHarness	Joseph P Toomey	Paul Slauenwhite
Test.Execution.JUnitRunner	Joseph P Toomey	Paul Slauenwhite
ASF.Runtime	Joseph P Toomey	Duwayne Morris
Test.Model	Joseph P Toomey	Paul Slauenwhite
Test.UI.AutoGUIUI	Liz Dancy	Duwayne Morris
Test.UI.Charting	Sheldon Lee-Loy	[remove]
Test.UI.CodeCoverage	Eugene Chan	[remove]
Test.UI.JUnit	Jerome Gout	Paul Slauenwhite
Test.UI.Reporting	Jerome Gout	Paul Slauenwhite
Test.UI.URLTest	Mark D Dunn	Duwayne Morris

• As discussed at the TPTP F-2-F, we are proposing the following to the PMC to accommodate the decrease in resources for 4.5:
  • Isolating and moving Manual Test to Technology Preview (see defect 207777).
  • Smoke testing the Technology Previews (Manual Test, API Recorder, and AGR) per iteration and maintenance release.
  • Fixing only critical and blocking defects for the Technology Previews (Manual Test, API Recorder, and AGR).
  • Reduced user support (news group and mailing list posts) for the Technology Previews (Manual Test, API Recorder, and AGR).
  • Focus on GA function for TPTP/Test Project presentations (e.g. EclipseCon 2008).

• Enhancements:
  • The following AG-approved enhancements will be requested for PG/PMC approval:
    • 202695 Encrypted datapool variable support: Will be completed by a non-committer from IBM (Xin Ying Huang). Paul and Joe are working with Xin Ying to close on the design.
    • 162605 Test execution should support parallel execution of tests: Duwayne has reviewed the enhancement and Description Document and with limited assistance from Joe, will start the design after i1 TP1 is completed.
    • 166025 Move, Copy, Delete, Rename and Import/Export test assets.: See the meeting minutes for the design review.
    • 145146 Generic Recorder Framework improvements.: Paul has reviewed the enhancement with the consuming product request the feature. They no longer require this feature but do require an extension to disable individual recorders. As such, we will defer this enhancement and open a new enhancement for the specific function requested by the consumer.
  • We will start/continue the design reviews after i1 TP1 is completed. Please start/continue the design work this week.

• Current defects:
  • 107432 (Test) IPv6 support in URL Test HTTP Recorder (required by Platform enhancement 165409): Duwayne has two IPv6 machines (Vista) installed/configured. Testing with JVM configured to use IPv6 addresses, by default. There are no known issues. This is primarily a testing effort.
  • 200194 ASF Test Execution Service races with AutoBuildJob -- can lead to unexpected test results.: Duwayne is investigating.
  • 123650 [Usability] HTTP tests do not automatically generate JUnit Java code like JUnit tests.: Duwayne is investigating.
  • 171083 AGR Recorder does not distinguish between two controls on the same panel if they have the same text and type: Paul is investigating.
  • 187864 Skewed completion rates in BIRT test reports.: Paul is finishing the implementation.
  • 206355 [Usability] Switch to Test Perspective after test execution.: Paul is investigating.
  • 206356 [Usability] Switch to Test Perspective after creating a test asset.: Paul is investigating.
  • The remaining test automation defects from 4.4:
    • Duwayne:
      • 87285: Will be completed by Duwayne early in 4.5.
      • 170654: Will be completed by Duwayne early in 4.5.
    • Paul:
      • 121726: Will be returned since cannot be automated and Manual Test is moving to Technology Preview (smoke testing only).
      • 170655: Will be completed by Paul early in 4.5.
      • 170656: Will be completed by Paul early in 4.5.
      • 170652: Will be completed by Paul early in 4.5.
    • Jerome:
      • 170651: Will be completed by Jerome B. early in 4.5. Work-around required until 171083 is resolved.
      • 170653: Will be completed by Jerome B. early in 4.5. Paul has provided some AGR test suites created from the Test Navigator manual test suites for review. Once reviewed, please check them into CVS (HEAD), remove the manual test suites that they cover from the behavior AllManualTests.testsuite (and AllManualSmokeTests.testsuite), and add the new AGR tests to the behavior of AllGUITests.testsuite (and AllGUISmokeTests.testsuite).

• 144950 [Scalability] Address TPTP EMF model scalability problems: Although withdrawn as a 4.5 candidate enhancement, the outcome/status of the work done in 4.4 on this enhancement was not updated in the WIKI page. Joe will update the WIKI this week.

• Test project topics for upcoming AG calls include:
  • Enhancement 166025 (Move, Copy, Delete, Rename and Import/Export test assets) as it relates to the other EMF models in TPTP.
  • The effort by the Release Engineering Team on consolidating our testing process and automation infrastructure.
  • The effort by the Release Engineering Team on integrating EMMA with our build process.

Roundtable

• Status updates and summary of defects, blocking issues, and architectural issues/concerns that we will focus on completing the upcoming week:
  • Paul:
    • 4.5 i1 TP1.
    • Additional 4.5 AG reviews.
    • Review design for enhancement 202695.
    • Start design of enhancement 145146.
  • Jerome B.:
    • 4.5 i1 TP1.
    • Continue the design of enhancement 166025.
  • Jerome G.:
    • Assisting Jerome B.
  • DuWayne:
    • 4.5 i1 TP1.
    • Start the design of enhancement 162605.
  • Kendric:
    • 4.5 i1 TP1.
    • Test automation.
  • Joe:
    • Review design for enhancement 202695.
    • Preparation for EclipseWorld 2007.
    • News group posts.

Action items

• Paul will send instructions to the current Component Leads for nominating the proposed Component Leads.

Notes

• The RedSox won the World Series!

• Paul and the Release Engineering Team are working on consolidating our testing process and automation infrastructure including:
  • Committers responsible for automating their manual tests.
  • Defining a common process and automation framework
  • Running the automated tests on the automation framework
  • Integrate the automation framework with the build framework (BVT).

• EclipseCon 2008:
  • Reminder to participate in EclipseCon 2008 by proposing a talk or tutorial for the Test and Performance category. This category covers topics related to test and performance tooling that leverage Eclipse. Test and Performance tooling is used throughout the software development and test life cycle, from early unit testing, profiling, and tuning to post-deployment monitoring, tracing, and log analysis. This category is focused on the implementation, extension, and use of test and performance tooling - both open source and commercial - targeting a broad spectrum of systems including embedded, stand-alone, distributed, and enterprise. There is particular interest in exploitation and user experience of the Test and Performance Tools Platform (TPTP) project. Presentation types include:
    • Tutorial: A two hour presentation, with coding and/or a hands on workshop.
    • Long Talk: A one hour presentation that goes into detail and/or demonstration on a topic.
    • Short Talk: A five to ten minute presentation that covers the highlights of a topic.
    • Posters: Posters are presented at the poster reception session.
    • Birds of a Feather (BoF): BoF meetings are informal one-and-a-half-hour gatherings of people who want to talk about a particular topic.
  • If you are interested in proposing a talk or tutorial, please:
    • Review the EclipseCon 2008 Submission Guidelines
    • Submit your talk or tutorial
  • Important dates include:
    • November 19, 2007: Submission deadline.
    • December 10, 2007: Tutorials, Long Talks, and Short Talks selected.
    • January 19, 2008: Submissions open for posters and BoFs.

Reminders

• Lead Committers:
  • Continually review copyright reports and ensure files will missing copyright or incorrect copyright year are corrected.
  • Add the Original Estimate (PH) when triaging a new defect.
  • Update the Hours Worked (PH) when completing a defect.
  • Test their components or review/track testing contributed by other committers/resources.
  • Monitor the TPTP news group and answer posts specific to their Test Project components.
  • Verify that all API changes (modifications, additions, and removals) do not breech the TPTP API Contract.
  • Use the MyFoundation Portal for committer elections. The Portal adds more automation to the elections and makes the individual project elections more transparent to the entire Eclipse community. If you have been experiencing problems or have other issues with the Portal election process, please contact the EMO (emo@eclipse.org).

• All Committers:
  • Monitor the TPTP news group and answer posts on topics other than the Test Project to support our user community.