Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Difference between revisions of "COSMOS QA Criteria"
(→'''Change History''') |
(→'''Change History''') |
||
Line 11: | Line 11: | ||
|<ul><li>Initial version to address bugzilla 214576</li></ul> | |<ul><li>Initial version to address bugzilla 214576</li></ul> | ||
|- | |- | ||
− | | | + | |Shivashankari N |
|01/9/2008 | |01/9/2008 | ||
− | |< | + | |<ul><li>Modified</li></ul> |
|} | |} | ||
Revision as of 03:20, 9 January 2008
Contents
- 1 COSMOS QA Creiteria
- 2 Workload Estimation
- 3 Terminologies/Acronyms
- 4 Purpose
- 5 Is COSMOS 1.0 well-formed software?
- 6 Identify all the end-to-end tests
- 7 Is COSMOS 1.0 a consumable entity?
- 8 Is COSMOS 1.0 an extensible framework?
- 9 Does COSMOS 1.0 offer good features & reference implementations?
- 10 Task Breakdown
- 11 Open Issues/Questions
COSMOS QA Creiteria
Change History
Name: | Date: | Revised Sections: |
---|---|---|
Jimmy Mohsin | 01/3/2008 |
|
Shivashankari N | 01/9/2008 |
|
Workload Estimation
Process | Sizing | Names of people doing the work |
---|---|---|
Initial specification | 2 | Jimmy Mohsin |
Review by COSMOS team | 2 | COSMOS Team |
Review and sign-off | 1 | QA Team |
TOTAL | 5 |
Terminologies/Acronyms
The terminologies/acronyms below are commonly used throughout this document. The list below defines each term regarding how it is used in this document.
Term | Definition |
---|---|
Quality Expectations | COSMOS (the software) must deliver these expectations |
Acceptance Criteria | Proof of Quality Expectations being met |
Purpose
We intent to set the quality expectations of COSMOS (the software) and matching acceptance criteria that would serve as a preamble to the COSMOS QA team while executing their work.
The QA team will use these criteria as a master guide to define and plan all their testing efforts.
Is COSMOS 1.0 well-formed software?
Acceptance Criteria | Others | QA Role |
---|---|---|
Valid use cases | Product manager to input user stories; use cases | Validate ERs against the use cases - manual |
Bug free implementation | Dev must provide Junits covering 100% of code; Code and Junit walk through to QA must be provided | Run the Junits and validate their code / ER coverage – TPTP; Black box functional testing – manual / SOAPUI recorded as TPTP manual tests |
Clear API documentation | Dev must provide API documentation | Manual verification of availability of API documentation |
Identify all the end-to-end tests
All COSMOS 1.0 iterations must pass the following end-to-end tests:
- Scalability
- We need to specifiy details of this capability
- Performance
- We need to specifiy details of this capability
- Concurrency
- We need to specifiy details of this capability
- Availability
- We need to specifiy details of this capability
- Stress testing
- We need to specifiy details of this capability
Is COSMOS 1.0 a consumable entity?
In order to ensure ease of adoption for COSMOS 1.0, the following items need to be addressed:
- Asses install documentation, i.e. is there enough information on the download page to be able to install the required functions ?
- Asses Update Manager capabilities (not there yet…)
- Are the offered packages fully functional, i.e. the content described on the install page matches the downloaded package
- Assess case for COSMOS 1.0 to Executive Management
- Assess package for adopting developers
- Asses package for other types of adopters, if applicable
- Evaluate hardware & software prerequisite definition
- Evaluate upgrade and patch process for COSMOS 1.0
Is COSMOS 1.0 an extensible framework?
In order to ensure that COSMOS 1.0 can be extended as desired by the adopters, the following items need to be addressed:
- Asses user documentation and API doc
- Assess process for extending framework components
- Assess process for extending supported platforms / operating systems
- Review process for reporting bugs and ehnancement requests
Does COSMOS 1.0 offer good features & reference implementations?
- Is it easy to deploy the central infrastructure, i.e. the Broker / Domain / et al?
- How hard is it to plug in an MDR into an existing COSMOS 1.0 framework?
- What Data Managers / MDR types are supported out of the box?
Task Breakdown
The following section includes the tasks required to complete this enhancement.
- Jimmy Mohsin has generated this page to address bugzilla 214576
- The COSMOS team needs to provide input to this page
- Shivvy, representing the QA team, is supposed to review and sign-off on these criteria
Open Issues/Questions
All reviewer feedback should go in the Talk page for 214576.
- Should there be additional criteria for Milestones?