Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Difference between revisions of "COSMOS QA i14 Activities"
(→Iteration QA Entrance and Exit Criteria) |
(→Scope Definition) |
||
| Line 28: | Line 28: | ||
''Primary Tasks:'' | ''Primary Tasks:'' | ||
#E2E testing on Windows XP, 2003 and Red Hat Enterprise Linux Server 5.0 (http://wiki.eclipse.org/COSMOS_DEMO_i14) | #E2E testing on Windows XP, 2003 and Red Hat Enterprise Linux Server 5.0 (http://wiki.eclipse.org/COSMOS_DEMO_i14) | ||
| − | |||
#Data Visualization TPTP Manual Tests Execution (DataVisualization.UI.Component.Widget.testsuite / UI.Reports.testsuite) | #Data Visualization TPTP Manual Tests Execution (DataVisualization.UI.Component.Widget.testsuite / UI.Reports.testsuite) | ||
#Execution of i14 supplement e2e tests(negative) from QA. | #Execution of i14 supplement e2e tests(negative) from QA. | ||
#MDR Toolkit and SML Tooling tests | #MDR Toolkit and SML Tooling tests | ||
| − | |||
| − | |||
#Basic operational efficiency tests (limited performance tests) | #Basic operational efficiency tests (limited performance tests) | ||
Revision as of 06:56, 7 November 2008
Contents
COSMOS QA Activities for i14
This has been put together to address bugzilla 243998.
For all these activities, test execution has been tracked over http://wiki.eclipse.org/COSMOS_i14_QA_Phase_Execution
Terminologies/Acronyms
The terminologies/acronyms below are commonly used throughout this document. The list below defines each term regarding how it is used in this document.
| Term | Definition |
|---|---|
| Quality Expectations | Is a statement of some behaviour, characteristic or operational facility that a product must exhibit for it to be deemed ‘fit for purpose’. Quality expectations are normally grouped into four main categories: functional/behavioural, operational efficiency, inter operability factors; and admin/management factors (to control TCO). |
| Acceptance Criteria | This is a quantification of how a quality expectation is to be validated. For functional/behavioral quality expectations this is a simple Boolean test – it either works or it doesn’t. Hence, for most scope docs there is no need to specifically define functional acceptance criteria. However, other types of quality expectations – especially performance related areas – do require specific acceptance criteria because the quantification is normally some form of numeric threshold (with optional margin/tolerance) that states minimum levels of acceptable operational efficiency. |
Scope Definition
The COSMOS quality expectations and the matching acceptance criteria, that would serve as a preamble to the COSMOS QA team while executing their work, were completed via ER 214576.
QA Scope for i14 testing:
Primary Tasks:
- E2E testing on Windows XP, 2003 and Red Hat Enterprise Linux Server 5.0 (http://wiki.eclipse.org/COSMOS_DEMO_i14)
- Data Visualization TPTP Manual Tests Execution (DataVisualization.UI.Component.Widget.testsuite / UI.Reports.testsuite)
- Execution of i14 supplement e2e tests(negative) from QA.
- MDR Toolkit and SML Tooling tests
- Basic operational efficiency tests (limited performance tests)
In scope i14 ERs
Please refer to http://wiki.eclipse.org/COSMOS/COSMOS_iteration_i14_plan (OR)http://wiki.eclipse.org/Cosmos1.0Features#Iteration_14_Enhancements for the list of ERs that QA will need to test as part of the i14 testing phase that runs from 27th Oct thru 14th Nov 2008.
In scope platforms, OS's & configurations
For i14, QA will certify the following platforms:
- Microsoft Windows XP SP2
- Micorsoft Windows 2003 Server SP1
- RedHat Linux Enterprise Server 5.0 (Smoke test/Limited End2End testing only)
For i14, QA will use following softwares:
- Tomcat 5.5.25 & Axis2 1.3
- SUN JDK 1.5.0_15
- Firefox version (2.0.0.11)
- Internet Explorer (6.0.2900.2180.XP SP2)
- Eclipse 3.4
- DOJO 1.1.0
- BIRT 2.3
Iteration QA Entrance and Exit Criteria
The following items need to be in place before the 2 weeks i14 QA phase can commence:
- A stable i14 build that has undergone multiple integration build during the course of the Development phase which ends on Oct 25, 2008
- The wiki page http://wiki.eclipse.org/COSMOS_DEMO_i14 is completed (by Development)
The following items need to be in place before the 2 week i14 QA phase can be declared complete:
- All ERs should have JUnits or a manual TPTP test in place. If JUnits are not applicable to a given ER, alternative verification means need to be specified. To reiterate, QA is not expected to run the JUnits. In i14, this activity will be completed as part of the weekly iteration builds. In future, this will be fully automated once ER 215135 is completed.
- The e2e tests for i14 (http://wiki.eclipse.org/COSMOS_DEMO_i14) are run against the final i14 build
i14 Test Cases
All End2End tests from QA can be found here - http://wiki.eclipse.org/COSMOS_QA_End2End_Tests
Resources & timeframe
i14 QA will be completed by two dedicated CA resources. The i14 QA phase will run from 27th Oct'08 to 14th Nov'08 (3 weeks).
Task Breakdown
This section includes the tasks required to complete this enhancement.
- Srinivas Reddy has generated this page to address bugzilla 243998
- The COSMOS team needs to review the relevant sections for this page.
- Srinivas, representing the QA team, is supposed to complete this activity by Oct 24, 2008. This is prior to the commencement of the QA phase for i14.
Open Issues/Questions
All reviewer feedback should go in the Talk page for 238087 "Define scope and detail the i14 QA activities".