COSMOS QA Criteria

COSMOS QA Creiteria

Change History

Name:	Date:	Revised Sections:
Jimmy Mohsin	01/3/2008	Initial version to address bugzilla 214576
Shivashankari N	01/9/2008	Modified

Workload Estimation

Rough workload estimate in person weeks

Process	Sizing	Names of people doing the work
Initial specification	2	Jimmy Mohsin
Review by COSMOS team	2	COSMOS Team
Review and sign-off	1	QA Team
TOTAL	5

Terminologies/Acronyms

The terminologies/acronyms below are commonly used throughout this document. The list below defines each term regarding how it is used in this document.

Term	Definition
Quality Expectations	COSMOS (the software) must deliver these expectations
Acceptance Criteria	Proof of Quality Expectations being met

Purpose

We intent to set the quality expectations of COSMOS (the software) and matching acceptance criteria that would serve as a preamble to the COSMOS QA team while executing their work.

The QA team will use these criteria as a master guide to define and plan all their testing efforts.

Is COSMOS 1.0 well-formed software?

Quality expectation 1: Is COSMOS well formed ?

Acceptance Criteria	Others	QA Role
Valid use cases	Product manager to input user stories; use cases	Validate ERs against the use cases - manual
Bug free implementation	Dev must provide Junits covering 100% of code; Code and Junit walk through to QA must be provided	Run the Junits and validate their code / ER coverage – TPTP; Black box functional testing – manual / SOAPUI recorded as TPTP manual tests
Clear API documentation	Dev must provide API documentation	Manual verification of availability of API documentation
Simple deployment package	RE team to provide an easy install	Validate the ease of use of the package and accompanying install instructions – manual; RE process will not be scrutinized
Base platforms support	Product manager states the supported platforms	QA certifies product on these platforms
Wiki documentation	Owners take responsibility of the quality of content	QA will not validate wiki content

Identify all the end-to-end tests

All COSMOS 1.0 iterations must pass the following end-to-end tests:

1. Scalability
   1. We need to specifiy details of this capability
2. Performance
   1. We need to specifiy details of this capability
3. Concurrency
   1. We need to specifiy details of this capability
4. Availability
   1. We need to specifiy details of this capability
5. Stress testing
   1. We need to specifiy details of this capability

Is COSMOS 1.0 a consumable entity?

In order to ensure ease of adoption for COSMOS 1.0, the following items need to be addressed:

1. Asses install documentation, i.e. is there enough information on the download page to be able to install the required functions ?
2. Asses Update Manager capabilities (not there yet…)
3. Are the offered packages fully functional, i.e. the content described on the install page matches the downloaded package
4. Assess case for COSMOS 1.0 to Executive Management
5. Assess package for adopting developers
6. Asses package for other types of adopters, if applicable
7. Evaluate hardware & software prerequisite definition
8. Evaluate upgrade and patch process for COSMOS 1.0

Is COSMOS 1.0 an extensible framework?

In order to ensure that COSMOS 1.0 can be extended as desired by the adopters, the following items need to be addressed:

1. Asses user documentation and API doc
2. Assess process for extending framework components
3. Assess process for extending supported platforms / operating systems
4. Review process for reporting bugs and ehnancement requests

Does COSMOS 1.0 offer good features & reference implementations?

1. Is it easy to deploy the central infrastructure, i.e. the Broker / Domain / et al?
2. How hard is it to plug in an MDR into an existing COSMOS 1.0 framework?
3. What Data Managers / MDR types are supported out of the box?

Task Breakdown

The following section includes the tasks required to complete this enhancement.

1. Jimmy Mohsin has generated this page to address bugzilla 214576
2. The COSMOS team needs to provide input to this page
3. Shivvy, representing the QA team, is supposed to review and sign-off on these criteria

Open Issues/Questions

All reviewer feedback should go in the Talk page for 214576.

• Should there be additional criteria for Milestones?

---