Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Difference between revisions of "Testing Process Part 2 Draft 1 Comments"
(→Alan Haggarty) |
(→Alan Haggarty) |
||
| Line 3: | Line 3: | ||
* One comment I have from them though - the implication that all future test pass 1's will be full test passes and test pass 2's will be smoke test passes? I did not realize this was set as policy, nor do I agree with it. In the last test pass it seemed to just be a coping strategy to the fact that we started test pass 1 without a good driver. It doesn't solve that problem. What assurance do we have that the code changes between test pass 1 and test pass 2 will be appropriate for smoke testing only? | * One comment I have from them though - the implication that all future test pass 1's will be full test passes and test pass 2's will be smoke test passes? I did not realize this was set as policy, nor do I agree with it. In the last test pass it seemed to just be a coping strategy to the fact that we started test pass 1 without a good driver. It doesn't solve that problem. What assurance do we have that the code changes between test pass 1 and test pass 2 will be appropriate for smoke testing only? | ||
** '''Response (Paul Slauenwhite):''' Ultimately it is the responsibility of the project lead to determine what is ran for either test pass (e.g. test plan). From a project perspective, we schedule less time for the TP2, so by definition it is a subset of the test cases ran for TP1. For example, if a defect was fixed between TP1 and TP2, the project lead would ensure test cases are ran in TP2 to cover the function in and around the fix. Given the resources, this test automation, and the short iteration cycles, there should be no need to run two full test passes per iteration. Ideally, we should be trying to get TP1 and TP2 down from 3 weeks to 1 - 1.5 weeks. | ** '''Response (Paul Slauenwhite):''' Ultimately it is the responsibility of the project lead to determine what is ran for either test pass (e.g. test plan). From a project perspective, we schedule less time for the TP2, so by definition it is a subset of the test cases ran for TP1. For example, if a defect was fixed between TP1 and TP2, the project lead would ensure test cases are ran in TP2 to cover the function in and around the fix. Given the resources, this test automation, and the short iteration cycles, there should be no need to run two full test passes per iteration. Ideally, we should be trying to get TP1 and TP2 down from 3 weeks to 1 - 1.5 weeks. | ||
| + | |||
| + | == Paul Slauenwhite == | ||
| + | |||
| + | ** Remove 1.0. | ||
| + | ** Move 2.0, 3.0, and 4.0 to part 1 (see [http://wiki.eclipse.org/Testing_Process_Part_1_Draft_2_Comments#Paul_Slauenwhite my comments on part 1]). | ||
| + | ** Why do we need section 5.0? Are there really exceptions now that Jonathan has automated the control channel tests? If they cannot be fully automated, they should be manual test with instructions on how to run the automated bits. | ||
| + | ** Section 6.0 is the really value of this section document, which needs to be completed. | ||
Revision as of 09:38, 3 December 2007
Alan Haggarty
- One comment I have from them though - the implication that all future test pass 1's will be full test passes and test pass 2's will be smoke test passes? I did not realize this was set as policy, nor do I agree with it. In the last test pass it seemed to just be a coping strategy to the fact that we started test pass 1 without a good driver. It doesn't solve that problem. What assurance do we have that the code changes between test pass 1 and test pass 2 will be appropriate for smoke testing only?
- Response (Paul Slauenwhite): Ultimately it is the responsibility of the project lead to determine what is ran for either test pass (e.g. test plan). From a project perspective, we schedule less time for the TP2, so by definition it is a subset of the test cases ran for TP1. For example, if a defect was fixed between TP1 and TP2, the project lead would ensure test cases are ran in TP2 to cover the function in and around the fix. Given the resources, this test automation, and the short iteration cycles, there should be no need to run two full test passes per iteration. Ideally, we should be trying to get TP1 and TP2 down from 3 weeks to 1 - 1.5 weeks.
Paul Slauenwhite
- Remove 1.0.
- Move 2.0, 3.0, and 4.0 to part 1 (see my comments on part 1).
- Why do we need section 5.0? Are there really exceptions now that Jonathan has automated the control channel tests? If they cannot be fully automated, they should be manual test with instructions on how to run the automated bits.
- Section 6.0 is the really value of this section document, which needs to be completed.