Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Difference between revisions of "Talk:COSMOS Design 214576"
| Line 13: | Line 13: | ||
End to End Testing - What is the strategy to automate this testing and provide a more comprehensive coverage ? | End to End Testing - What is the strategy to automate this testing and provide a more comprehensive coverage ? | ||
| + | |||
| + | start --[[User:Martin.simmonds.ca.com|Marty]] 08:32, 16 January 2008 (EST) | ||
| + | |||
| + | Quality Expectations:<br> | ||
| + | Bug Free Implementation<br> | ||
| + | <ol> | ||
| + | <li>The Junits should contain the ER number, they should have a javadoc that shows the bugzilla URL | ||
| + | <li>The Junits should have javadoc added. The CMDBF junits are a good example of this (org.eclipse.cosmos.dc.cmdbf.services.tests). | ||
| + | <li>The Junits should state what is required in order for them to be executed. e.g The management domain should already be started. | ||
| + | <li>It would be nice to say that 100% of the code is covered, but how do you insure that ? | ||
| + | We are using Junit version 3. Will 4 be allowed ? Any reason why we are using 3 and not 4? | ||
| + | <li>I think the Junits should state what they are testing, and not left up to the reader to figure it out. | ||
| + | <li>What about Junits that test error-handling? If my code is supposed to write a log record somewhere if a certain condition is found, | ||
| + | there should be a Junit that forces that condition to occur, and it has to check that the log record is created. So far I have not seen these kinds of tests, but they should be there, and if they are not, then you can clearly state that the junits do not cover 100% of the code. | ||
| + | <li>Does release engineering produce javadoc for junits and put them somewhere ? (It should do) | ||
| + | </ol> | ||
| + | <br><br> | ||
| + | Easy to use deployment package.<br> | ||
| + | As well as having an easy to use deployment package, we should have an easy to execute 'self-test' program that runs out of the box, that determines if the test environment is correctly setup and working. This should be without the user having to start x number of things, execute x number of commands etc. This will save so much time in future. | ||
| + | |||
| + | <br><br> | ||
| + | Unit Level testing | ||
| + | Code review | ||
| + | <ol> | ||
| + | <li>How do I know I wrote the code in the right way ? Could I have done this better? More efficiently? You are never going to know the answers to these kinds of questions without reviewing the code. | ||
| + | <li>Template programs..... If for example we have an Mdr that we would like all Mdrs to based upon, that has good documentation, Junits tests, etc, Lets flag those programs and Junit tests as being templates for how we do things. At least if we don't have the time to code review we can state that the code written was based upon an agreed template. | ||
| + | <br><br></ol> | ||
| + | |||
| + | Operational Efficiency<br> | ||
| + | Quality Characteristics | ||
| + | <ol> | ||
| + | <li>It depends on the use case as to what you expect on the Quality Characteristics. Its not 1 size fits all. | ||
| + | <li>If we are querying a data source currently with the current systems, that has to be a guide as to how long things may take. In general with the added COSMOS layers you expected things to have some degradation. But what degradation is acceptable? If we are saying 15% then someone somewhere has to test the query with current ways of doing it, in order to determine x+15%. That I think is necessary if you want to test degradation, but should not be underestimated on how much time is need to determine X. | ||
| + | <li>If a client retrieves data from more than one source, performance perception could be based on what actually is the slowest DataManager/Mdr. Which brings up an interesting point of whether client queries, are threaded or not. | ||
| + | </ol> | ||
| + | <br><br> | ||
| + | end --[[User:Martin.simmonds.ca.com|Marty]] 08:32, 16 January 2008 (EST) | ||
Revision as of 09:32, 16 January 2008
--Popescu.ca.ibm.com 09:36, 8 January 2008 (EST)
- Assess localization support; validate that the code can run on non-english locales.
- This type of testing will not happen on every iteration, most likely not even on every release. This is a placeholder to keep track of translatability testing once COSMOS offers localization support.
Tmakins.us.ibm.com 16:33, 10 January 2008 (EST)
- We should clarify which activities will be executed for each iteration and which will only be executed for milestones.
- General Question: I assume that specifics on test cases, platforms, metrics, etc. will be documented in a QA plan which will be based on this document?
Paul Stratton. CA. 11-jan-2008
UNIT testing - what framework will be used to run the JUNIT tests - Eclipse ? Will QA perfrom regression testing using JUNITS from previous iterations ? Will manual tests be run within TPTP ? Are there any plans for QA to supplement current UNIT tests where they are considered inadequate ?
End to End Testing - What is the strategy to automate this testing and provide a more comprehensive coverage ?
start --Marty 08:32, 16 January 2008 (EST)
Quality Expectations:
Bug Free Implementation
- The Junits should contain the ER number, they should have a javadoc that shows the bugzilla URL
- The Junits should have javadoc added. The CMDBF junits are a good example of this (org.eclipse.cosmos.dc.cmdbf.services.tests).
- The Junits should state what is required in order for them to be executed. e.g The management domain should already be started.
- It would be nice to say that 100% of the code is covered, but how do you insure that ? We are using Junit version 3. Will 4 be allowed ? Any reason why we are using 3 and not 4?
- I think the Junits should state what they are testing, and not left up to the reader to figure it out.
- What about Junits that test error-handling? If my code is supposed to write a log record somewhere if a certain condition is found, there should be a Junit that forces that condition to occur, and it has to check that the log record is created. So far I have not seen these kinds of tests, but they should be there, and if they are not, then you can clearly state that the junits do not cover 100% of the code.
- Does release engineering produce javadoc for junits and put them somewhere ? (It should do)
Easy to use deployment package.
As well as having an easy to use deployment package, we should have an easy to execute 'self-test' program that runs out of the box, that determines if the test environment is correctly setup and working. This should be without the user having to start x number of things, execute x number of commands etc. This will save so much time in future.
Unit Level testing
Code review
- How do I know I wrote the code in the right way ? Could I have done this better? More efficiently? You are never going to know the answers to these kinds of questions without reviewing the code.
- Template programs..... If for example we have an Mdr that we would like all Mdrs to based upon, that has good documentation, Junits tests, etc, Lets flag those programs and Junit tests as being templates for how we do things. At least if we don't have the time to code review we can state that the code written was based upon an agreed template.
Operational Efficiency
Quality Characteristics
- It depends on the use case as to what you expect on the Quality Characteristics. Its not 1 size fits all.
- If we are querying a data source currently with the current systems, that has to be a guide as to how long things may take. In general with the added COSMOS layers you expected things to have some degradation. But what degradation is acceptable? If we are saying 15% then someone somewhere has to test the query with current ways of doing it, in order to determine x+15%. That I think is necessary if you want to test degradation, but should not be underestimated on how much time is need to determine X.
- If a client retrieves data from more than one source, performance perception could be based on what actually is the slowest DataManager/Mdr. Which brings up an interesting point of whether client queries, are threaded or not.
end --Marty 08:32, 16 January 2008 (EST)