Notice: This Wiki is now read only and edits are no longer possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Difference between revisions of "TPTP-Test-Automation-20071129"
(New page: == Attendees == * Present: ** Paul ** Kiryl ** Joanna ** Jonathan ** Joel ** Alan == Deliverables == * Testing Process: ** Only Paul and Jonathan reviewed/commented on the first draft o...) |
|||
Line 12: | Line 12: | ||
* Testing Process: | * Testing Process: | ||
− | ** | + | ** Alan integrated the [http://wiki.eclipse.org/Testing_Process_Part_1_Draft_1_Comments review comments] and Paul posted the second draft of the Testing Process (part 1) document for review. |
− | ** | + | *** Allan will integrate [http://wiki.eclipse.org/Testing_Process_Part_1_Draft_2_Comments review comments] for the second draft of the Testing Process (part 1) document by Monday, December 3. |
− | + | ||
** Alan will add more detailed content to the first draft of the Testing Process (part 2) by Wednesday, November 28. | ** Alan will add more detailed content to the first draft of the Testing Process (part 2) by Wednesday, November 28. | ||
− | ** Please review the [http://wiki.eclipse.org/4.5_Test_Automation_Initiative#Testing_Process Testing Process | + | ** Please review the latest drafts for both [http://wiki.eclipse.org/4.5_Test_Automation_Initiative#Testing_Process Testing Process documents] for Monday, December 3. |
* Common Test Infrastructure: | * Common Test Infrastructure: | ||
** Completed: | ** Completed: | ||
*** CVS check-out of all of the test plug-ins in the <code>/test-results/<project>/*</code> modules. | *** CVS check-out of all of the test plug-ins in the <code>/test-results/<project>/*</code> modules. | ||
+ | *** Configure and run the Agent Controller on the reference platform. | ||
+ | *** Paul created the [http://dev.eclipse.org/viewcvs/index.cgi/test-results/platform/org.eclipse.hyades.tests/BVT/?root=TPTP_Project root-level test suites for BVTs]. The test infrastructure will execute the <code>org.eclipse.hyades.tests/AllBVT<test type>Tests.testsuite</code> test suites. We do not need to check-in the execution results to CVS due to disk space limitations and polluting of our test pass results. Each developer can rerun the automated tests to reproduce a failure. | ||
+ | *** Paul created the [http://dev.eclipse.org/viewcvs/index.cgi/test-results/platform/org.eclipse.hyades.tests/deployment/?root=TPTP_Project BVT deployment] to the reference platform. | ||
+ | *** Automatically generating the Test Report based on the existing test execution results in CVS. | ||
+ | *** Paul modified the BIRT tabular test report to walk an execution result to generate a detailed test report from a root-level execution result. | ||
** Outstanding: | ** Outstanding: | ||
*** Check for Intel build to complete (every x hours) before launching the BVTs. | *** Check for Intel build to complete (every x hours) before launching the BVTs. | ||
− | |||
− | |||
− | |||
− | |||
− | |||
*** Automatically posting the Test Report to <code>/tptp/test/reports/<release></code>. | *** Automatically posting the Test Report to <code>/tptp/test/reports/<release></code>. | ||
− | |||
*** Cleanup of the reference platform. | *** Cleanup of the reference platform. | ||
*** Documenting the test infrastructure for use in the second testing process document, which will be required by TPTP adopters or extenders. | *** Documenting the test infrastructure for use in the second testing process document, which will be required by TPTP adopters or extenders. | ||
Line 35: | Line 33: | ||
** Blocking Issues: | ** Blocking Issues: | ||
# When generating reports, the report has no entries: Paul and Sheldon are investigating. | # When generating reports, the report has no entries: Paul and Sheldon are investigating. | ||
− | # [https://bugs.eclipse.org/bugs/show_bug.cgi?id=208616 Defect 208616]: | + | # [https://bugs.eclipse.org/bugs/show_bug.cgi?id=208616 Defect 208616]: Fixed and verified by Kendric/Joel. |
− | # [https://bugs.eclipse.org/bugs/show_bug.cgi?id=200194 Defect 200194]: | + | # [https://bugs.eclipse.org/bugs/show_bug.cgi?id=200194 Defect 200194]: Targeted to i5. |
− | # [https://bugs.eclipse.org/bugs/show_bug.cgi?id=190453 Defect 190453]: | + | # [https://bugs.eclipse.org/bugs/show_bug.cgi?id=190453 Defect 190453]: Targeted to i6. |
− | # [https://bugs.eclipse.org/bugs/show_bug.cgi?id=210289 Defect 210289]: | + | # [https://bugs.eclipse.org/bugs/show_bug.cgi?id=210289 Defect 210289]: Targeted to i6. |
− | # [https://bugs.eclipse.org/bugs/show_bug.cgi?id=210292 Defect 210292]: | + | # [https://bugs.eclipse.org/bugs/show_bug.cgi?id=210292 Defect 210292]: Targeted to i6. |
− | # [https://bugs.eclipse.org/bugs/show_bug.cgi?id=210955 Defect 210955]: Paul | + | # [https://bugs.eclipse.org/bugs/show_bug.cgi?id=210955 Defect 210955]: DTP has resolved the issue, available in the December 1st build. Paul has request a patch. |
* Metrics: | * Metrics: | ||
− | ** | + | ** The monitoring project lead has replied to a request to estimate the [http://wiki.eclipse.org/4.5_Test_Automation_Initiative#Metrics current testing costs]. |
− | *** | + | *** Joanna and Guru are gathering their estimates. |
− | + | ||
== Action Items == | == Action Items == | ||
* All: | * All: | ||
− | ** Review the [http://wiki.eclipse.org/4.5_Test_Automation_Initiative#Testing_Process Testing Process | + | ** Review the latest drafts for both [http://wiki.eclipse.org/4.5_Test_Automation_Initiative#Testing_Process Testing Process documents] for Monday, December 3. |
* Alan: | * Alan: | ||
− | + | ** Integrate [http://wiki.eclipse.org/Testing_Process_Part_1_Draft_2_Comments review comments] for the second draft of the Testing Process (part 1) document by Monday, December 3. | |
− | + | ** Add more detailed content to the first draft of the Testing Process (part 2) by Monday, December 3. | |
− | ** Add more detailed content to the first draft of the Testing Process (part 2) by | + | |
* Joel: | * Joel: | ||
Line 66: | Line 62: | ||
== Upcoming Meeting == | == Upcoming Meeting == | ||
− | * We will meet next on | + | * We will meet next on Monday, December 3 (9 AM ET). |
== Upcoming Discussion == | == Upcoming Discussion == |
Revision as of 16:05, 29 November 2007
Attendees
- Present:
- Paul
- Kiryl
- Joanna
- Jonathan
- Joel
- Alan
Deliverables
- Testing Process:
- Alan integrated the review comments and Paul posted the second draft of the Testing Process (part 1) document for review.
- Allan will integrate review comments for the second draft of the Testing Process (part 1) document by Monday, December 3.
- Alan will add more detailed content to the first draft of the Testing Process (part 2) by Wednesday, November 28.
- Please review the latest drafts for both Testing Process documents for Monday, December 3.
- Alan integrated the review comments and Paul posted the second draft of the Testing Process (part 1) document for review.
- Common Test Infrastructure:
- Completed:
- CVS check-out of all of the test plug-ins in the
/test-results/<project>/*
modules. - Configure and run the Agent Controller on the reference platform.
- Paul created the root-level test suites for BVTs. The test infrastructure will execute the
org.eclipse.hyades.tests/AllBVT<test type>Tests.testsuite
test suites. We do not need to check-in the execution results to CVS due to disk space limitations and polluting of our test pass results. Each developer can rerun the automated tests to reproduce a failure. - Paul created the BVT deployment to the reference platform.
- Automatically generating the Test Report based on the existing test execution results in CVS.
- Paul modified the BIRT tabular test report to walk an execution result to generate a detailed test report from a root-level execution result.
- CVS check-out of all of the test plug-ins in the
- Outstanding:
- Check for Intel build to complete (every x hours) before launching the BVTs.
- Automatically posting the Test Report to
/tptp/test/reports/<release>
. - Cleanup of the reference platform.
- Documenting the test infrastructure for use in the second testing process document, which will be required by TPTP adopters or extenders.
- Email notifications to component leads for failing test suites.
- Blocking Issues:
- Completed:
- When generating reports, the report has no entries: Paul and Sheldon are investigating.
- Defect 208616: Fixed and verified by Kendric/Joel.
- Defect 200194: Targeted to i5.
- Defect 190453: Targeted to i6.
- Defect 210289: Targeted to i6.
- Defect 210292: Targeted to i6.
- Defect 210955: DTP has resolved the issue, available in the December 1st build. Paul has request a patch.
- Metrics:
- The monitoring project lead has replied to a request to estimate the current testing costs.
- Joanna and Guru are gathering their estimates.
- The monitoring project lead has replied to a request to estimate the current testing costs.
Action Items
- All:
- Review the latest drafts for both Testing Process documents for Monday, December 3.
- Alan:
- Integrate review comments for the second draft of the Testing Process (part 1) document by Monday, December 3.
- Add more detailed content to the first draft of the Testing Process (part 2) by Monday, December 3.
- Joel:
- Outstanding issues for the PoC (integrate the ASF with the TPTP builds to check-out a 'Hello World' TPTP JUnit test suite from CVS, run the test suite on an Eclipse instance running on the target platform, and generate a BIRT report on the test execution using existing sample code).
- Paul:
- Determine labor cost (PW) for a) full test pass and b) smoke test pass from project leads.
- Encourage Project Leads to automate existing manual test suites.
Upcoming Meeting
- We will meet next on Monday, December 3 (9 AM ET).
Upcoming Discussion
- Test framework reuse:
- See defect 171827.
- Existing test frameworks include:
- Test Server for the Agent Controller/Profiler
- Test Server for the JVMTI Profiler.
- Proposed test frameworks include:
- Test Dashboard (see enhancement 112928 and Test Dashboard Plug-in).
- Test automation service (see defect 87285)