Testing Process Part 1 Draft 1 Comments

• Alan Haggarty:
  • The goal for this document is a set of straightforward instructions for executing TPTP tests and requirements when creating them.
  • It is very much taken from Paul's strategy and represents much more reorganization than original content.
  • The AGR execution part I could not get to work myself (only the first case passes in a suite) and do not understand the steps as written. They are currently a direct copy of the original and I will rewrite although anyone with experience running AGR tests can comment on what should be in a generic, straightforward set of steps for executing our gui test suites.
  • For the execution section I am torn. In this document I tried to list only the steps which are physically required in all cases (ie: comments, project structure and references to allTests) and the items that are general "best practices" such as short cases, reduce redundancy, etc will go in part 2. However I am wondering if we need/should have step by step instructions for creating the tests in this document (new->TPTP JUnit Test-> ....) similar to the step by step for execution?
  • There are a few [red] sections of text which I have my own questions or comments on the current draft.
  • Part 2 will contain the expanded information that was in the original strategy and any new information we want to add. This way Part 1 can remain short, direct and hopefully easy to use to quickly get running tests and contributing to the project.
    • Topics for inclusion in Part 2.
    • Introduction
    • TPTP Test Naming Conventions
    • Test Creation Best Practices
    • Test Execution Best Practices
    • TPTP Test Project Exceptions
    • The Common Test Infrastructure