Jump to: navigation, search

Difference between revisions of "Testing Process Part 1 Draft 1 Comments"

Line 17: Line 17:
 
*'''Jonathan West''':
 
*'''Jonathan West''':
 
** Would it be possible to glue together some of the tests into one single .testsuite file? So for instance, aggregate all of the various Platform.Communication.Request_Peer_Monitoring.(platform).testsuite files into a single Platform.Communication.Request_Peer_Monitoring.testsuite. That is, keep the existing files, but run the aggregate and check in the aggregate .execution file. This would save a significant amount of time on the results generation side, as we would not have to go through the rigmarole of execution generation for each individual test. Is this feasible on the report generation side? -- [[User:Jgwest.ca.ibm.com|Jgwest.ca.ibm.com]] 11:54, 14 November 2007 (EST)
 
** Would it be possible to glue together some of the tests into one single .testsuite file? So for instance, aggregate all of the various Platform.Communication.Request_Peer_Monitoring.(platform).testsuite files into a single Platform.Communication.Request_Peer_Monitoring.testsuite. That is, keep the existing files, but run the aggregate and check in the aggregate .execution file. This would save a significant amount of time on the results generation side, as we would not have to go through the rigmarole of execution generation for each individual test. Is this feasible on the report generation side? -- [[User:Jgwest.ca.ibm.com|Jgwest.ca.ibm.com]] 11:54, 14 November 2007 (EST)
 +
** As per Alan's suggestion that any questions that I answered for myself upon review of the document would be beneficial, here is my comment that I answered for myself: ''IUnder '2.2 Extracting the Test Resources', the second step is, "2. To ensure that the local workspace contains the latest test projects, re-extract at the beginning of each unit of work (for example, test pass, report generation, etc.)." Based on the context of this section the extraction term implies a check-out from CVS. So presumably re-extraction at the beginning of each "unit of work" would be selecting, in Eclipse, the results project -> Team -> Update? Is this correct? Or does it mean a straight wipe of the directory contents, and then a check-out? '' My answer to this was, yes, it could be either.

Revision as of 10:37, 19 November 2007

  • Alan Haggarty:
    • The goal for this document is a set of straightforward instructions for executing TPTP tests and requirements when creating them.
    • It is very much taken from Paul's strategy and represents much more reorganization than original content.
    • The AGR execution part I could not get to work myself (only the first case passes in a suite) and do not understand the steps as written. They are currently a direct copy of the original and I will rewrite although anyone with experience running AGR tests can comment on what should be in a generic, straightforward set of steps for executing our gui test suites.
    • For the execution section I am torn. In this document I tried to list only the steps which are physically required in all cases (ie: comments, project structure and references to allTests) and the items that are general "best practices" such as short cases, reduce redundancy, etc will go in part 2. However I am wondering if we need/should have step by step instructions for creating the tests in this document (new->TPTP JUnit Test-> ....) similar to the step by step for execution?
    • There are a few [red] sections of text which I have my own questions or comments on the current draft.
    • Part 2 will contain the expanded information that was in the original strategy and any new information we want to add. This way Part 1 can remain short, direct and hopefully easy to use to quickly get running tests and contributing to the project.
      • Topics for inclusion in Part 2.
      • Introduction
      • TPTP Test Naming Conventions
      • Test Creation Best Practices
      • Test Execution Best Practices
      • TPTP Test Project Exceptions
      • The Common Test Infrastructure


  • Jonathan West:
    • Would it be possible to glue together some of the tests into one single .testsuite file? So for instance, aggregate all of the various Platform.Communication.Request_Peer_Monitoring.(platform).testsuite files into a single Platform.Communication.Request_Peer_Monitoring.testsuite. That is, keep the existing files, but run the aggregate and check in the aggregate .execution file. This would save a significant amount of time on the results generation side, as we would not have to go through the rigmarole of execution generation for each individual test. Is this feasible on the report generation side? -- Jgwest.ca.ibm.com 11:54, 14 November 2007 (EST)
    • As per Alan's suggestion that any questions that I answered for myself upon review of the document would be beneficial, here is my comment that I answered for myself: IUnder '2.2 Extracting the Test Resources', the second step is, "2. To ensure that the local workspace contains the latest test projects, re-extract at the beginning of each unit of work (for example, test pass, report generation, etc.)." Based on the context of this section the extraction term implies a check-out from CVS. So presumably re-extraction at the beginning of each "unit of work" would be selecting, in Eclipse, the results project -> Team -> Update? Is this correct? Or does it mean a straight wipe of the directory contents, and then a check-out? My answer to this was, yes, it could be either.