Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

TPTP-Test-Tools-20071105

Attendees

  • Present:
    • Paul Slauenwhite
    • Jerome Bozier
    • DuWayne Morris
    • Kendric Wang
  • Absent:
    • Joseph Toomey - vacation
    • Jerome Gout - vacation

4.4.1

  • 4.4.1 will ship with the Europa Winter Maintenance Release (target release date: February 29, 2008).
  • All defects will require project/PMC approval.
  • All fixes MUST be checked-in to both 4.4.1 (TPTP-4_1_1) and 4.5 (HEAD) streams.
  • Defect(s):
    • 168597[Vista] HTTP Recorder requires work-around on Windows Vista: Done!
    • 207781 Mark new ImportExistingProjects class as javadoc provisional per API contract: PMC approved and checked in to both 4.4.1 (TPTP-4_4_1) and 4.5 (HEAD) streams.
    • 209263 Deleted datapool row back in datapool upon import: Blocking a consuming product so we will need to provide a patch early next week.
  • There will be a smoke test pass in the next few weeks (target date: TBD) aligned with the latest release of the IBM 1.5 JRE.

4.5

  • 4.5 will ship with the Ganymede Simultaneous Release (target release date: June 30, 2008).
  • The iteration numbers have been changed to reflect the Ganymede milestone numbers. For example, I3 has been changed to I3, which is aligned with Ganymede M3. See the detailed schedule and defect 208587.
  • We are almost nearing the end of TP1 (target date: November 7, 2007).
    • Note the target date for TP1 has been changed to this Wednesday, November 7, 2007 (see the detailed schedule) to ensure we complete TP1 on the candidate driver.
    • A new candidate driver will be posted later today, including regression fixes and without the OpenSSL code (see defect 195644). All remaining test suites need to completed using the new candidate.
    • Testing responsibilities are based on the new component ownership (see below table).
    • TP1 is a full test pass, with the exception of smoke testing the Technology Previews (Manual Test, API Recorder, and AGR), since it is the first 4.5 test pass on the Eclipse 3.4 base and we need to generate meaningful code coverage statistics.
    • All of the Test Project committers must test with the EMMA-instrumented driver (see the instructions for details).
    • Status:
      • Paul: No problems found when testing using the October 26, 2007 candidate build. Will complete by Wednesday.
      • Duwayne: No problems found when testing using the October 26, 2007 candidate build. Initially had problems with the Eclipse 3.4 driver referenced for the October 26, 2007 candidate build. Will complete by Wednesday.
      • Kendric: Defect 165140 found when running org.eclipse.hyades.test.java.tests\junit\CodeSynchronizationTest.testsuite, which will be addressed in I4. Will complete the Test Log Viewer and JUnit Plug-in test suites by Wednesday
      • Jerome B.: Problems found (e.g. opening the Test Perspective and Test Navigator) when using the October 26, 2007 candidate build all-in-one. Will work with Paul to resolve problems. Will not test with the EMMA-instrumented driver since this is the first time testing TPTP. Will complete by Wednesday.
  • When experiencing problems with the TPTP builds and download site (e.g. broken links, incorrect/failing dependencies), please open a defect against the Release Engineering Product.
  • Please vote this week on the following component changes (see the mailing list):
    • Notes:
      • Current Component Leads maintain their committer status as developer committers for the component.
      • Paul will send instructions to the current Component Leads for nominating the proposed Component Leads.
      • Paul will remove the unnecessary components this week.
      • TPTP Project Matrix updates will be handled under defect 207968.
Bugzilla ComponentCurrent Component LeadNew Component Lead
Platform.UI.ReportingSheldon Lee-LoyPaul Slauenwhite
Test.AgentsMark D DunnPaul Slauenwhite
Test.Agents.ComptestAgentJoseph P ToomeyPaul Slauenwhite
Test.Agents.RecorderMark D DunnDuwayne Morris
Test.AnalysisJoseph P Toomey[remove]
ASF.DocumentationJoseph P ToomeyDuwayne Morris
Test.ExecutionKent SiefkesPaul Slauenwhite
Test.Execution.AutoGUIRunnerLiz DancyDuwayne Morris
Test.Execution.CommonRunnerJoseph P ToomeyPaul Slauenwhite
Test.Execution.ExecutionHarnessJoseph P ToomeyPaul Slauenwhite
Test.Execution.JUnitRunnerJoseph P ToomeyPaul Slauenwhite
ASF.RuntimeJoseph P ToomeyDuwayne Morris
Test.ModelJoseph P ToomeyPaul Slauenwhite
Test.UI.AutoGUIUILiz DancyDuwayne Morris
Test.UI.ChartingSheldon Lee-Loy[remove]
Test.UI.CodeCoverageEugene Chan[remove]
Test.UI.JUnitJerome GoutPaul Slauenwhite
Test.UI.ReportingJerome GoutPaul Slauenwhite
Test.UI.URLTestMark D DunnDuwayne Morris
  • As discussed at the TPTP F-2-F, we have proposed the following to the PMC to accommodate the decrease in resources for 4.5:
    • Isolating and moving Manual Test to Technology Preview (see defect 207777).
    • Smoke testing the Technology Previews (Manual Test, API Recorder, and AGR) per iteration and maintenance release.
    • Fixing only critical and blocking defects for the Technology Previews (Manual Test, API Recorder, and AGR).
    • Reduced user support (news group and mailing list posts) for the Technology Previews (Manual Test, API Recorder, and AGR).
    • Focus on GA function for TPTP/Test Project presentations (e.g. EclipseCon 2008).
  • Enhancements:
    • The following AG-approved enhancements will be requested for PG/PMC approval:
      • 202695 Encrypted datapool variable support: Will be completed by a non-committer from IBM (Xin Ying Huang). Paul and Joe are working with Xin Ying to close on the design.
      • 162605 Test execution should support parallel execution of tests: Duwayne has reviewed the enhancement and Description Document and with limited assistance from Joe, will start the design after I3 TP1 is completed.
      • 166025 Move, Copy, Delete, Rename and Import/Export test assets.: See the meeting minutes for the design review.
      • 145146 Generic Recorder Framework improvements.: Paul has reviewed the enhancement with the consuming product request the feature. They no longer require this feature but do require an extension to disable individual recorders. As such, we will defer this enhancement and open a new enhancement for the specific function requested by the consumer.
    • We will start/continue the design reviews after I3 TP1 is completed. Please start/continue the design work this week.
  • Defects:
    • Required functional defects (target: 4.5i4):
      • 187864 Skewed completion rates in BIRT test reports.: Paul is finishing the implementation.
    • Required by consuming product(s) (target: 4.5i4):
      • 107432 (Test) IPv6 support in URL Test HTTP Recorder (required by Platform enhancement 165409): Duwayne has two IPv6 machines (Vista) installed/configured. Testing with JVM configured to use IPv6 addresses, by default. There are no known issues. This is primarily a testing effort.
      • 139589 Allow programmatic context for initially selected project in Generic Recorder Wizard
      • 201412 Repeated launch through Automated Services Framework eventually fails
      • 201150 Text in Recorder Control should be copyable.
      • 145070 Test Log Viewer: Link to deployment is displayed when using temp deploy not named 'default'
      • 180262 variables are listed twice in the "EquivalenceClass" pane of Datapool
      • 205476 copy/paste of a simple folder does not work in Test Navigator view
      • 205477 Copy / Paste should not be available in the logical test navigator
      • 162407 Test assets' editors (locations, datapools etc) allow save with empty internal names
      • 180280 Unmovable rows in Datapool Editor
      • 163150 Usability - adding existing artifacts/locations to deployment should select by name (not filename)
      • 195203 Accessibility-Need shortcuts for all Test Navigator pulldowns
      • 200586 Context sensitive help requires refactoring and new content.
      • 201849 No context help for New Test Deployment - Add Location Association dialog
      • 201892 doc updates for section Creating a test deployment
      • 201844 Time Frame Historical Report view does not have tabs and is missing info
      • 201827 Generate Report: Test Navigator does not refresh to show new report
      • 201828 Report Icon is incorrect for Test Pass and Time Frame Historic
    • Required by test automation initiative (target: 4.5i4):
      • 205093 API Recorder's Generate Test Suite dialog does not select a default selection.
      • 194770 Verdict Summary BIRT chart in Test Log viewer displays 0%.
      • 200194 ASF Test Execution Service races with AutoBuildJob -- can lead to unexpected test results.
      • 171083 AGR Recorder does not distinguish between two controls on the same panel if they have the same text and type
      • 135103 [Test Automation] All external APIs should have JUnit tests
      • 171827 [Test Automation] Merge the various existing/proposed TPTP test execution frameworks.
      • 204849 AGR test suites model disabled test invocations incorrectly.
      • 87285 [Test Automation] [Quality Assurance] Automated Test Execution Harness Tests:
      • 170654 [Test Automation] Automate manual test suites using the Automated GUI Recorder.
      • 170655 [Test Automation] Automate manual test suites using the Automated GUI Recorder.
      • 170656 [Test Automation] Automate manual test suites using the Automated GUI Recorder.
      • 170652 [Test Automation] Automate manual test suites using the Automated GUI Recorder.
      • 170651 [Test Automation] Automate manual test suites using the Automated GUI Recorder.: Work-around required until 171083 is resolved.
      • 170653 [Test Automation] Automate manual test suites using the Automated GUI Recorder.: Paul has provided some AGR test suites created from the Test Navigator manual test suites for review. Once reviewed, please check them into CVS (HEAD), remove the manual test suites that they cover from the behavior AllManualTests.testsuite (and AllManualSmokeTests.testsuite), and add the new AGR tests to the behavior of AllGUITests.testsuite (and AllGUISmokeTests.testsuite).
    • Required to improve usability (target: 4.5i5):
      • 123650 [Usability] HTTP tests do not automatically generate JUnit Java code like JUnit tests.: Duwayne is investigating.
      • 206355 [Usability] Switch to Test Perspective after test execution.: Paul is investigating.
      • 206356 [Usability] Switch to Test Perspective after creating a test asset.: Paul is investigating.
    • Required for translation/TVT (target: 4.5i5):
      • 190453 OutOfMemoryError when generating TPTP test reports.
      • 190558 BIRT test report types generate incorrect verdicts in test suite window.
      • 194752 Confusing label in Test preferences.
      • 164835 Remove unused messages for BIRT test report genreators.
      • 203273 Incorrect german translation of the word "Pass"
      • 200969 Formatting messages make translation difficult.
      • 200725 Simplified Chinese translation when editing a datapool is incorrect
    • Required/strategic/easily fixed (target: TBD):
      • 199782 Irreproducible test results when running AGR test suite from Ant script
      • 189128 Vista-specific error dialog when recording using IAC/IE on XP.
      • 200923 URL test runner does not capture standard out and error.
      • 195683 Datapools are not portable.
      • 194754 Location does not contain port number.
      • 200237 Can not debug launched eclipse process via ASF without rebuilding TPTP jar
      • 202269 Details page extension does not show any properties.
      • 164699 JUnit/Plug-in tests do not allow plugin root as source folder.
      • 200201 Need new icon for testlog files
      • 199783 NullPointerException when requesting publication of AGR test results from Ant script
      • 201402 [Testing Tool] Invalid characters inserted in source folder textbox when using Browse button of "Generate..." menu item
      • 199421HTTP recording does not work with Internet Explorer 7 and Firefox 2.0
  • 144950 [Scalability] Address TPTP EMF model scalability problems: Although withdrawn as a 4.5 candidate enhancement, the outcome/status of the work done in 4.4 on this enhancement was not updated in the WIKI page. Joe will update the WIKI this week.
  • Test project topics for upcoming AG calls include:
    • Enhancement 166025 (Move, Copy, Delete, Rename and Import/Export test assets) as it relates to the other EMF models in TPTP.
    • The effort by the Release Engineering Team on consolidating our testing process and automation infrastructure.
    • The effort by the Release Engineering Team on integrating EMMA with our build process.

Roundtable

  • Status updates and summary of defects, blocking issues, and architectural issues/concerns that we will focus on completing the upcoming week:
    • See above 4.5 i3 TP1 testing status.

Action items

Notes

  • EclipseCon 2008:
    • Reminder to participate in EclipseCon 2008 by proposing a talk or tutorial for the Test and Performance category. This category covers topics related to test and performance tooling that leverage Eclipse. Test and Performance tooling is used throughout the software development and test life cycle, from early unit testing, profiling, and tuning to post-deployment monitoring, tracing, and log analysis. This category is focused on the implementation, extension, and use of test and performance tooling - both open source and commercial - targeting a broad spectrum of systems including embedded, stand-alone, distributed, and enterprise. There is particular interest in exploitation and user experience of the Test and Performance Tools Platform (TPTP) project. Presentation types include:
      • Tutorial: A two hour presentation, with coding and/or a hands on workshop.
      • Long Talk: A one hour presentation that goes into detail and/or demonstration on a topic.
      • Short Talk: A five to ten minute presentation that covers the highlights of a topic.
      • Posters: Posters are presented at the poster reception session.
      • Birds of a Feather (BoF): BoF meetings are informal one-and-a-half-hour gatherings of people who want to talk about a particular topic.
    • If you are interested in proposing a talk or tutorial, please:
    • Important dates include:
      • November 19, 2007: Submission deadline.
      • December 10, 2007: Tutorials, Long Talks, and Short Talks selected.
      • January 19, 2008: Submissions open for posters and BoFs.
  • Paul and the Release Engineering Team are working on consolidating our testing process and automation infrastructure including:
    • Committers responsible for automating their manual tests.
    • Defining a common process and automation framework
    • Running the automated tests on the automation framework
    • Integrate the automation framework with the build framework (BVT).
    • Integrate EMMA with the build and test process for generating code coverage reports for all manual and automated testing.

Reminders

  • Lead Committers:
    • Continually review copyright reports and ensure files will missing copyright or incorrect copyright year are corrected.
    • Add the Original Estimate (PH) when triaging a new defect.
    • Update the Hours Worked (PH) when completing a defect.
    • Test their components or review/track testing contributed by other committers/resources.
    • Monitor the TPTP news group and answer posts specific to their Test Project components.
    • Verify that all API changes (modifications, additions, and removals) do not breech the TPTP API Contract.
    • Use the MyFoundation Portal for committer elections. The Portal adds more automation to the elections and makes the individual project elections more transparent to the entire Eclipse community. If you have been experiencing problems or have other issues with the Portal election process, please contact the EMO (emo@eclipse.org).
  • All Committers:
    • Monitor the TPTP news group and answer posts on topics other than the Test Project to support our user community.

Back to the top