Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Difference between revisions of "COSMOS QA Criteria"
Jim.gyng.com (Talk | contribs) (→'''Is COSMOS 1.0 a consumable entity?''') |
Jim.gyng.com (Talk | contribs) (→'''Is COSMOS 1.0 well-formed software?''') |
||
| Line 77: | Line 77: | ||
|-{{BMTHStyle}} | |-{{BMTHStyle}} | ||
! Acceptance Criteria | ! Acceptance Criteria | ||
| − | ! | + | ! Other Teams |
! QA Role | ! QA Role | ||
|- | |- | ||
| align="left" | Valid use cases | | align="left" | Valid use cases | ||
| − | | | + | | COSMOS team to define use cases |
| Validate ERs against the use cases - manual | | Validate ERs against the use cases - manual | ||
|- | |- | ||
| align="left" | Bug free implementation | | align="left" | Bug free implementation | ||
| − | | | + | | COSMOS team must provide Junits covering 100% of code; Code and Junit walk through to QA must be provided |
| Run the Junits and validate their code / ER coverage – TPTP; Black box functional testing – manual / SOAPUI recorded as TPTP manual tests | | Run the Junits and validate their code / ER coverage – TPTP; Black box functional testing – manual / SOAPUI recorded as TPTP manual tests | ||
|- | |- | ||
| align="left" | Clear API documentation | | align="left" | Clear API documentation | ||
| − | | | + | | COSMOS team must provide API documentation |
| Manual verification of availability of API documentation | | Manual verification of availability of API documentation | ||
|- | |- | ||
| Line 97: | Line 97: | ||
|- | |- | ||
| align="left" | Base platforms support | | align="left" | Base platforms support | ||
| − | | | + | | COSMOS team specified the supported platforms |
| QA certifies product on these platforms | | QA certifies product on these platforms | ||
|- | |- | ||
Revision as of 00:19, 13 January 2008
Contents
COSMOS Quality Expectations
This has been put together to address Bugzilla ER 214576.
Change History
| Name: | Date: | Revised Sections: |
|---|---|---|
| Jimmy Mohsin | 01/3/2008 |
|
| Shivashankari N | 01/09/2008 |
|
| Jimmy Mohsin | 01/12/2008 |
|
Workload Estimation
| Process | Sizing | Names of people doing the work |
|---|---|---|
| Initial specification | 2 | Jimmy Mohsin |
| Review by COSMOS team | 2 | COSMOS Team |
| Review and sign-off | 1 | QA Team |
| TOTAL | 5 |
Terminologies/Acronyms
The terminologies/acronyms below are commonly used throughout this document. The list below defines each term regarding how it is used in this document.
| Term | Definition |
|---|---|
| Quality Expectations | COSMOS (the software) must deliver these expectations |
| Acceptance Criteria | Proof of Quality Expectations being met |
Purpose
We intent to set the quality expectations of COSMOS (the software) and matching acceptance criteria that would serve as a preamble to the COSMOS QA team while executing their work.
The QA team will use these criteria as a specification to define and plan all their testing efforts.
Is COSMOS 1.0 well-formed software?
| Acceptance Criteria | Other Teams | QA Role |
|---|---|---|
| Valid use cases | COSMOS team to define use cases | Validate ERs against the use cases - manual |
| Bug free implementation | COSMOS team must provide Junits covering 100% of code; Code and Junit walk through to QA must be provided | Run the Junits and validate their code / ER coverage – TPTP; Black box functional testing – manual / SOAPUI recorded as TPTP manual tests |
| Clear API documentation | COSMOS team must provide API documentation | Manual verification of availability of API documentation |
| Simple deployment package | RE team to provide an easy install | Validate the ease of use of the package and accompanying install instructions – manual; RE process will not be scrutinized |
| Base platforms support | COSMOS team specified the supported platforms | QA certifies product on these platforms |
| Wiki documentation | Owners take responsibility of the quality of content | QA will not validate wiki content |
Is COSMOS 1.0 a consumable entity?
| Acceptance Criteria | Others | QA Role |
|---|---|---|
| Valid use cases | Product manager to input use cases | Validate ERs against the use cases - manual |
| Successful integration of COSMOS components | Dev must provide helper applications for integration testing with scenarios | Perform integration testing and execute scenarios – manual recorded as manual TPTP tests |
| COSMOS stability during production deployments | Dev must state the minimum system requirements for production run; Dev to recommend parameters (number of data brokers that may be added, volume of data that can be queried) that would be considered for these tests. | Execute performance / scalability /volume/stress/ availability testing with minimum resources recommended – Load (any open source tool) |
| COSMOS support across products / data sources | Dev must state the kinds of MDRs that can be integrated; provide samples | Execute integration tests with these samples – manual recorded as TPTP manual tests |
| User documentation | Doc will write manuals | QA validates the information - manual |
| Samples / skeleton MDR implementation / any collateral | Dev must provide these samples | QA validates the existence and may look forward to Dev while using the skeleton implementations – manual recorded as TPTP manual tests |
| Other platforms | Product manager may recommend | QA will certify the product on these platforms - manual |
| Dependencies on other open source software | Dev to set a process to forward integrate with the newer versions of these dependent software | QA will validate the process - manual |
| Future Enhancement / bug reporting mechanism | Dev to set a process | QA validates the process - manual |
Unit level testing
This section details the quality expectations at the code unit level
- TBD
End To End testing
This section details the quality expectations at the system / integration level
- TBD
Criteria specification
| Acceptance Criteria | Details |
|---|---|
| Availability | There are no Availability quality expectations |
| Capacity | There are no Capacity quality expectations |
| Concurrency | There are no Concurrency quality expectations |
| Data Volumes | There are no Data Volumes quality expectations |
| Performance | The Data Managers and MDRs should not degrade the performance of Data Adapters by more than 15% |
| Scalability | There are no Scalability quality expectations |
| Stability | There are no Stability quality expectations |
| Stress | There are no Stress quality expectations |
Task Breakdown
This section includes the tasks required to complete this enhancement.
- Jimmy Mohsin has generated this page to address bugzilla 214576
- The COSMOS team needs to provide input to this page
- Shivvy, representing the QA team, is supposed to review and sign-off on these criteria
Open Issues/Questions
All reviewer feedback should go in the Talk page for 214576.
- Should there be additional criteria for Milestones?