Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "4.5 Test Automation Initiative"

(Testing Process)
(Common Test Infrastructure)
Line 68: Line 68:
 
== Common Test Infrastructure ==
 
== Common Test Infrastructure ==
  
* Demonstration [http://www.eclipse.org/tptp/home/documents/process/test/automation/infastructure/viewlet/TestAutomationInitiative_viewlet_swf.html viewlet].
+
* '''Summary''':
  
* Diagram detailing the architecture:
+
* '''Demonstration''':
[[Image:AsfBVT.JPG]]
+
** [http://www.eclipse.org/tptp/home/documents/process/test/automation/infastructure/viewlet/TestAutomationInitiative_viewlet_swf.html Viewlet].
 +
** Diagram detailing the architecture: [[Image:AsfBVT.JPG]]
  
 
== Meeting Minutes ==
 
== Meeting Minutes ==

Revision as of 13:15, 19 November 2007

Members

  • Paul Slauenwhite (IBM)
  • Alan Haggarty (IBM)
  • Joel Cayne (IBM)
  • Jonathan West (IBM)
  • Joanna Kubasta (IBM)
  • Kiryl Kazakevich (Intel)

Goals

  • Consolidation of our testing process and infrastructure.
  • Consolidation of custom test frameworks (Test Dashboard, AC/Profiler test server, variants).
  • Reuse of Test Project framework (ASF, reporting, etc.) and tools (AGR, JUnit, etc.).
  • Automatic test execution and report generation with every build (BVT).
  • Integrate EMMA with the build and test process for generating code coverage reports for all manual and automated testing.

Benefits

  • Specialization of testing skills (e.g. builds).
  • Provide an infrastructure that makes it easy for committers to contribute automated test cases/suites.
  • Manual labor savings by using automated tests, which are automatically executed with every build.
  • Decrease time/cost to resolve defects since uncovered earlier in the code-test-build cycle.
  • Decrease testing overlap by a) localizing test executing and reporting to one group and b) using code coverage statistics.

Delivery dates

  • November 14:
    • First draft of the first testing process document for review.
    • Proof of concept (PoC) for integration of ASF and TPTP builds for PMC approval and integrate with TPTP builds.
    • Status update for the PMC.
    • Measure cost savings.
  • November 21:
    • Review of the first testing process document.
    • First draft of the remaining testing process documents for review
    • Identify and resolve bugs.
  • November 28:
    • Final version of the the first testing process document.
    • Review of the remaining testing process documents for review
    • Status update including cost savings for the PMC.
    • Presentation to TPTP committers.
    • Each project automates their manual test suites and converge existing automated test suites over time to leverage the test infrastructure.

Testing Process

  • Summary:
    • A very lightweight testing process for TPTP.
    • Extends the existing TPTP Testing Strategy. The existing technical content will remain unchanged, aside from review changes.
    • Considered as a instruction manual (e.g. step-by-step) for testing TPTP for each type of testing scenario.
    • Sections include
      • Getting tests.
      • Creating tests.
      • Running tests.
      • Save test results.
      • Reporting results.
      • BVTs.
    • Two (or more) documents:
  1. High steps to use the testing infrastructure intended for first time users. This document will reference the other document(s).
  2. Low level detailed discussion on the motivation and design of the infrastructure intended for TPTP adopters or extenders.
  3. Additional documents detailing the specifics of a testing topics.

Common Test Infrastructure

  • Summary:
  • Demonstration:
    • Viewlet.
    • Diagram detailing the architecture: AsfBVT.JPG

Meeting Minutes

Back to the top