Jump to: navigation, search

COSMOS dev process

Revision as of 12:14, 12 February 2008 by Tmakins.us.ibm.com (Talk | contribs)

COSMOS Development Process

Introduction

This document describes the development process for the COSMOS project. The intent is for this document to complement the Eclipse Development Process.

Development & Testing

Enhancement Requests

  • At any time during the release cycle, enhancement requests can be submitted by creating a Bugzilla entry.


Defect Reporting, Fixing and Verification

Please reference the Eclipse guidelines for the lifecycle of a bugzilla entry available at Bugzilla Use. These are part of the overall Eclipse Development Process. Another good source of information on enhancements/bug is the Bug Reporting FAQ.


Reporting:

  • At any time during the release cycle, defects can be reported by opening a Bugzilla entry against the appropriate component.


Fixing and Verification:

  1. When a developer decides to work on the defect he/she assigns the defect to him/herself, completes the fix and checks it in to CVS
    1. The developer must include the Bugzilla entry number and summary in the CVS comment during the CVS checkin. This step is important because these comments will be automatically extracted by the build process to produce a list of bugs that have been fixed in the next build.
  2. When the fix has been checked in, the developer marks the defect as FIXED
    1. The testing and verification information (e.g. specific test cases) MUST be included in the bugzilla entry as a comment when the developer marks it "Fixed"
    2. When the test cases (JUnit/Manual) are checked in, they MUST have the bugzilla entry number specificed that they test
    3. The test case description MUST include the bugzilla entry that it addresses
  3. After the next build, the user who opened the defect tests to ensure the fix is correct, and marks the defect as VERIFIED
    1. If the user is not available/capable to verify the code, then it is the responsibility of the component owner to validate the code and move the bugzilla entry to "Verify"
  4. Prior to the start of the next iteration, the team lead reviews and closes all verified defects

Milestones

  • Milestone releases, which will be spaced out across the release schedule, contain complete DCUT (code complete, unit tested) increments of functionality (they are a subset of the full functionality of the target release).
  • A set of use cases will be identified and documented on the COSMOS Use Cases wiki page for each milestone release.


Iterations

  • An iteration is a 4-6 week period consisting of requirements, design, code and test activities, the output of which is a stable, integrated and tested release of the system.
  • The number and schedule of iterations will be determined by the project team at the beginning of the release cycle and documented in the Release Plan.


The following steps are required for each iteration:

1. Identify and accept requirements

  • The Community will identify the set of use cases that should be targeted for this iteration based on the milestone goals, and create associated enhancement requests. (Note: Use cases are documented on the COSMOS Use Cases wiki page.)
  • Developers will be assigned to create high-level designs, dependency analysis and sizing estimates for each enhancement request.
  • The Community should review open enhancement requests (and associated designs and sizing estimates), assign a priority based on the release themes and use cases, and assign them to a development resource. Each enhancement request should be associated with a specific use case, as defined in the previous bullet.
  • The priority of enhancements should be marked according to the following definitions:
    • P1 - This is a must-have enhancement OR this enhancement blocks another enhancement that has a priority of P1 or P2.
    • P2 - This is a high priority enhancement, key to satisfying the goals of the iteration.
    • P3 - This is a medium priority enhancement that can be moved to a future iteration with minimal impact to the goals of the current iteration.
    • P4 & P5 - This is a low priority enhancement that will be addressed as time and resources permit.
  • A review of the requirements list for all subprojects will be completed during the COSMOS Community call prior to the start of the iteration for final acceptance.


2. Complete Designs

  • Assigned developer(s) should create designs and post them on the COSMOS wiki for Community review.


3. Complete Coding/Implementation

A. Assigned developer(s) should check code into CVS to be integrated into the build.

  • Developers should reference the Eclipse Development Conventions and Guidelines for consistent development
  • All files should have appropriate copyrights. Developers can periodically use the Eclipse copyright tool to ensure this happens.
  • If a developer modifies a file that contains a copyright from a company other than his/her own employer, the developer should make sure that "and others" is added after the company name.

Example:

/*******************************************************************************
 * Copyright (c) 2008 CA Corporation and others.
 * All rights reserved. This program and the accompanying materials
 * are made available under the terms of the Eclipse Public License v1.0
 * which accompanies this distribution, and is available at
 * http://www.eclipse.org/legal/epl-v10.html
 *
 * Contributors:
 *     CA Corporation - initial API and implementation
 *     IBM Corporation - features xx
 *******************************************************************************/
  • The developer must include the Bugzilla entry number and summary in the CVS comment during the CVS checkin. This step is important because these comments will be automatically extracted by the build process to produce a list of bugs that have been fixed in the next build.

B. When the code has been checked in, the developer marks the enhancement as FIXED

  • The testing and verification information (e.g. specific test cases) MUST be included in the bugzilla entry as a comment when the developer marks the enhancement "Fixed"
  • When the test cases (JUnit/Manual) are checked in, they MUST have the bugzilla entry number specificed that they test
  • The test case description MUST include the bugzilla entry that it addresses

C. After the next build, the user who opened the defect or enhancement tests to ensure the fix is correct, and marks the bugzilla entry as VERIFIED

  • If the user is not available/capable to verify the code, then it is the responsibility of the component owner to validate the code and move the bugzilla entry to "Verify"

D. Prior to the start of the next iteration, the team lead reviews and closes all verified enhancements and defects


4. Complete Test Pass

  • During the iteration test phase, the code base is frozen. Each subproject should run the appropriate test cases, and check in only fixes for problems found during testing.
  • Once a subproject team determines that the test has completed successfully, the team lead should post a message to the dev mailing list indicating that the test pass for the subproject is complete.
  • Once all subprojects have confirmed successful completion of the test pass, the release engineering team will generate a candidate build.
  • Each subproject team is responsible for doing a final test pass against this build.
  • All tests should be in CVS as COSMOS manual and JUnit tests

Iteration Exit Criteria

  • All P1 and P2 enhancements complete
  • No high severity defects and 100% test attempt/pass (95% pass on early iterations)
  • Test results posted; All exceptions reviewed before deferral/discharge

TPTP Test Framework Overview

TPTP Test Framework Overview

Daily Builds

Release Engineering Team

  • The Release Engineering Team will be responsible for seeing to it that the builds and tests run to completion.
  • In the event of problems with the build & test system itself they are expected to resolve them as soon as possible and get the build and test restarted.
  • In the event of problems with the build caused by developer checkins they are expected to follow up with developers to ensure fixes are made quickly, and then get the build and test restarted.
  • Any build failure will be reported as a bugzilla to the build component for tracking of the actual issue as well as trend analysis, etc.

Ownership

  • All committers are responsible for clean builds and tests. Clean daily builds and tests keep the system stable, prevent regressions, and avoid down time. Anytime you check something in, you are responsible for monitoring the builds and tests until they pass. This means that for at least 24 hours after your checkin, you are available, responsive and prepared to help with any issues that crop up.

Daily Builds

  • Every 24 hours at GMT-5 10:00 PM (ET) the state of the head branch of the CVS repository will be labeled with a time stamp
  • The time stamp will be encoded with the release number, date, time, and serial build number: COSMOS-RMm-YYMMDDHHMMSS
  • A module list will be maintained (in CVS) that lists all the modules that are part of the current build, labeling and extracting will be restricted to those modules. This could be a PDE build style map file or .pfd file. This map file will be named COSMOS-RM.m.{map|pfd} and will be labeled with the time stamp at build time as well.

Daily Tests??

  • Where are build and/or test results posted?

Weekly Integration Builds

Weekly Integration Builds

  • Weekly integration builds will be run every Tuesday at 4:00 p.m. ET
  • Code cutoff for integration builds is as follows:
    • All patches should be in bugzilla by Monday at 4:00 p.m. ET
    • All code should be checked into CVS by 3:45 p.m. ET on Tuesday
  • Each subproject team should run JUnits on the candidate build and post results by 1:00 p.m. on Wednesday
  • RE team will post notification to cosmos-dev when the integration build is ready - target availability is 4:00 p.m. on Wednesdays

Milestone Release Builds (MS)

  • Milestone releases, which will be spaced out across the release schedule, contain complete DCUT (code complete, unit tested) increments of functionality (they are a subset of the full functionality of the target release).
  • Milestone releases are number RVM.m.MS1 through RVM.n.MSn (R1.0.MS2 for example).
  • A milestone release is created by relabeling and renaming a GOOD build, updating the downloads page and sending out an email saying the milestone release is available. The contents of the milestone are determined by the project schedule.
  • Users can look at the download page and find latest milestone release.
  • Notification of new milestones should be posted to the web site and COSMOS newsgroup with a high-level list of significant features and fixes

Release Candidate Builds (RC)

  • Near the end of the project when all tests have been run, bugs fixed, and the builds are stable, the release engineering team will promote a release candidate build.
  • Release Candidates are number RVM.m.RC1 through RVM.n.RCn (R0.4.RC7 for example).
  • A release candidate is created by relabeling and renaming a GOOD build, updating the downloads page and sending out an email saying the release candidate is available.

GA Builds (GA)

  • A GA build is created by relabeling and renaming a GOOD release candidate build, updating the downloads page and sending out an email saying the GA release is available.
  • A release candidate build can be declared GA when the following criteria have been satisfied:
    • No high severity defects and 100% test attempt/pass
    • Test results posted; All exceptions reviewed before deferral/discharge
  • Notification of new GA releases should be posted to the web site and COSMOS newsgroup with a high-level list of significant features and fixes


References