Notice: This Wiki is now read only and edits are no longer possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
TPTP-Test-Automation-20071106
Revision as of 17:08, 6 November 2007 by Unnamed Poltroon (Talk) (New page: == Attendees == *Present: ** Paul ** Joanna ** Kiryl ** Alan ** Joel * Absent: ** Jonathan == Testing Process == ** A very lightweight testing process for TPTP. ** Extends the existing...)
Contents
Attendees
- Present:
- Paul
- Joanna
- Kiryl
- Alan
- Joel
- Absent:
- Jonathan
Testing Process
- A very lightweight testing process for TPTP.
- Extends the existing TPTP Testing Strategy.
- Possibly two documents:
- 1) High steps to use the testing infrastructure intended for first time users. This document will reference the second document.
- 2) Low level detailed discussion on the motivation and design of the infrastructure intended for TPTP adopters or extenders.
Reference Platform
- Target: IBM Java 1.5 (latest SR) and XP/x86.
- Developers are expected to unit test on reference platform before check-in.
- Nightly builds invoke automated test execution on reference platform.
- Goal to use a second reference platform for better test coverage (target: Sun Java 5 and Linux/x86).
- Move to Java 6 once a complete test pass is completed to expose initial defects.
- Update:
- The dedicate test machine for the reference platform has been clean and does not need to be re-imaged.
- IBM JREs for Windows and Linux (see IBM Development Package for Eclipse) are publicly available from developerWorks.
- Once the PoC is complete, Intel will focus on integrating the testing fro the native components (e.g. JVMTI profiler) with the test infrastructure.
Common Test Infrastructure
- ASF integrated with the TPTP builds (invoked via ANT):
- Invoke tests.
- Interrogate results.
- Generate test reports.
- TPTP test types:
- AGR for functional UI testing.
- TPTP JUnit and TPTP JUnit Plug-in test for unit testing.
- TPTP build infrastructure:
- Extract test suites from CVS.
- Provision reference platform (Agent Controller and Eclipse/TPTP).
- Install and configure Agent Controller and Eclipse/TPTP on the reference platform.
- Clean-up reference platform.
- Post test reports (test and code coverage) on TPTP web site.
- Update:
- Joel and Paul discussed the detailed design of this infrastructure last week.
- Joel will attach a diagram of this infrastructure to the WIKI.
- Joel will document this infrastructure for use in the second testing process document, which will be required by TPTP adopters or extenders.
- This infrastructure will not allow testing with the Integrated Agent Controller (IAC) since it cannot coexist with the Agent Controller running on the reference platform.
- Intel will inform IBM that the Intel portion of the build is complete (see defect https://bugs.eclipse.org/bugs/show_bug.cgi?id=200351 200351]).
- Joel and Paul discussed the detailed design of this infrastructure last week.
Deliverables
- Alan:
- No update.
- Waiting for the PoC to be completed.
- Jonathan:
- n/a
- Joel:
- Eclipse/TPTP and the Agent Controller is now deployed and expanded on the target platform.
- Jonathan will assist Joel with configuring and running the Agent Controller.
- A sample ASF script is working, which handles TPTP JUnit test execution, results interrogation and BIRT reporting.
- Since there is defect when deploying TPTP JUnit tests remotely, Joel will continue with using TPTP JUnit tests.
- See the Common Base Event JUnit tests.
- Joel will work on automatically posting the Test Report to
/tptp/test/reports/<release>
.
Delivery dates
- November 14:
- Complete the first testing process document.
- Proof of concept (PoC) for integration of ASF and TPTP builds for PMC approval and integrate with TPTP builds.
- Measure cost savings.
- November 21:
- Identify and resolve bugs.
- November 28:
- Each project automates their manual test suites and converge existing automated test suites over time to leverage the test infrastructure.
- Presentation to TPTP committers.
- Report cost savings to PMC.
Action Items
- Alan: Complete the new test process.
- Jonathan: Assist Joel with configuring and running the Agent Controller.
- Joel: Integrate the ASF with the TPTP builds to check-out a 'Hello World' TPTP JUnit test suite from CVS, run the test suite on an Eclipse instance running on the target platform, and generate a BIRT report on the test execution using existing sample code, possibly reuse some function from the Test Dashboard.
- Joel: Attach a diagram of this infrastructure to the WIKI.
- Joel: Document this infrastructure for use in the second testing process document, which will be required by TPTP adopters or extenders.
- Paul: Determine labor cost (PW) for a) full test pass and b) smoke test pass from project leads.
- Paul: Encourage Project Leads to automate existing manual test suites.
- Paul: Book another meeting for next week (November 12, 2007).
- Paul: Fix defect when deploying TPTP JUnit tests remotely.
References
- ASF
- ASF Tutorial
- TPTP Testing Strategy
- Test Dashboard (see enhancement 112928)
- Test Dashboard Plug-in