Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "Development Resources/HOWTO/Release Reviews"

(Other Results)
(Checklist)
(53 intermediate revisions by 5 users not shown)
Line 1: Line 1:
<div style="float: right; border: 1px dashed black;
+
{{Warning|See  
background-color: #FFFFDD;"><table><tr><td width="150px">See  
+
"[http://www.eclipse.org/projects/dev_process/development_process.php#6_3_3_Release_Review 6.3.3 Release Review]" and "[http://www.eclipse.org/projects/dev_process/development_process.php#6_3_Reviews 6.3 Reviews]" in the Eclipse Development Process.}}
"[http://www.eclipse.org/projects/dev_process/development_process.php#6_3_3_Release_Review 6.3.3 Release Review]" and "[http://www.eclipse.org/projects/dev_process/development_process.php#6_3_Reviews 6.3 Reviews]" in the Eclipse Development Process</td>
+
''The purposes of a Release Review are: to summarize the accomplishments of the release, to verify that the IP Policy has been followed and all approvals have been received, to highlight any remaining quality and/or architectural issues, and to verify that the project is continuing to operate according to the Principles and Purposes of Eclipse.''
</tr></table></div><blockquote><em>The purposes of a Release Review are: to summarize the accomplishments of the release, to verify that the IP Policy has been followed and all approvals have been received, to highlight any remaining quality and/or architectural issues, and to verify that the project is continuing to operate according to the Principles and Purposes of Eclipse.</em></blockquote>
+
  
=What are the Requirements?=
+
=Overview=
  
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px; padding-left:2px; padding-right:2px; font-family:times; ">R</span>The only requirements pertaining to a Release Review is that the review be held successfully and all remedial actions completed before the Project's official release.
+
Please see [[Development Resources/HOWTO/Release Cycle|the release cycle]], which includes a nice (simple) overview of the [[Development Resources/HOWTO/Release Cycle#Release_Review|release review process]].
  
 +
[[Image:ReleaseReview.png|center]]
 +
 +
Helpful Documentation from the [http://www.eclipse.org/projects/dev_process/development_process.php Eclipse Development Process]
 +
* [http://www.eclipse.org/projects/dev_process/development_process.php#6_3_3_Release_Review About Release Reviews]
 +
* [http://www.eclipse.org/projects/dev_process/release-review.php Guidelines for Release Reviews]
 +
 +
==Reviews are Not Required for Service Releases==
 +
Note that a release review is '''not''' required for a service--or "bug fix only"--release. If your release contains no new features, APIs, or significant changes in functionality, you may release it without engaging in a community review. You should still create the release record in the [[Project Management Infrastructure/Release Metadata|PMI]] to notify your community. Service releases are indicated by an increment in the third segment of the version number (e.g. 2.3.'''2''').
 +
 +
Note also that projects are not required to submit IP logs for review by the IP Team for service releases. Projects are required to ensure that they have the appropriate coverage for all intellectual property (e.g. CQs and contribution records) at all times.
 +
 +
==Intellectual Property==
 +
Before you can consider a Release Review, all of the relevant CQs must be approved by the Eclipse Legal team. We cannot schedule a Review before the Legal team has completed their work. If you are waiting for CQs, please review where your CQs are, and when they are scheduled to be reviewed, in the [https://dev.eclipse.org/ipzilla/buglist.cgi?cmdtype=dorem&remaction=run&namedcmd=IP%20Team%20Work%20Queue&sharer_id=854 IP team work queue].
 +
 +
After you have verified that all relevant CQs are approved, the following items must be completed '''at least one week before the scheduled review date''':
 +
# PMC Approval ''(Can occur in parallel with, prior to, and after, the IP clearance)'';
 +
# [[Development Resources/IP Log | IP Log]] Approval '''This is essential, and no release review can proceed without it!'''; and
 +
# Review document.
 +
 +
===IP Log Approval===
 +
Because IP Log approval is essential and can take varying amounts of time (depending on the work queue of the IP Team), projects '''must''' have an approved IP Log before their review date is confirmed. After you have submitted the IP Log to Eclipse Legal for approval, the review is added to the schedule with the notation ''Waiting for IP Log.'' After EMO is notified that the IP Log is approved, that notation is replaced with the date of the review.
 +
 +
Please use the automated IP Log Tool to update and and submit your IP Log. You can use the [[Development Resources/Automatic IP Log|IP Log Generator]] for this (the [[Development Resources/Automatic IP Log#Submitting the IP Log to Eclipse Legal|Submitting the IP Log to Eclipse Legal]] section explains how to submit the IP Log). The URL for using the tool for your project uses your '''project ID''', in this format: ''<nowiki>http://www.eclipse.org/projects/ip_log.php?projectid=</nowiki>'''project ID'''.
 +
 +
Before submitting your IP Log, we recommend that you review the contents of your project's download directory to ensure that you have the necessary CQs in place. The [https://www.eclipse.org/projects/tools/downloads.php Project Downloads Review] tool can help you in with this review.
 +
 +
While you are waiting for IP Log approval, we suggest that you obtain PMC approval for the review and begin work on your documentation.
 +
 +
== PMC Approval ==
 +
Please forward an email showing that you have the approval of your PMC for the review and corresponding review documentation. The easiest way to do this is to request approval on your PMC mailing list (which requires subscription) and forward the response to EMO.
 +
 +
Note that you do not have to wait for the IP Log to be approved by the IP Team. You can seek PMC review/approval of your review documentation while the IP Log separate from the IP log approval process.
 +
 +
== Project Plan ==
 +
Your [[Development Resources/Project Plan|project plan]] must be current and available for public consumption.
 +
 +
== Review Documentation ==
 +
 +
The content requirements for review documentation are detailed below. The preferred means of providing review documentation is to enter it directly into the record associated with your release in the [[Project Management Infrastructure/Release Metadata|project metadata]].
 +
 +
=== Alternative Formats===
 +
 +
In the past, a presentation file (PPT or ODP) rendered in PDF from was the preferred format for review documentation (we used to host actual review calls for which the presentation format was appropriate). Presentation files are still acceptable (and are quite common). Other open formats, including HTML, a wiki page, or a document file rendered in PDF format, are also acceptable.
 +
 +
Please submit your document via email to the [mailto:emo@eclipse.org EMO]. Include the review document as either a link or an attachment.
 +
 +
Many people underestimate the time and effort needed to create the document, so be sure to allow enough time for this task. The "official" due date for the document is one week before the scheduled start of the review period. However, the document need to be reviewed by the EMO before posting. If you wait until the due date to submit the first draft and the EMO requests changes, then you'll probably miss your deadline and your review will be postponed.
 +
 +
=Checklist=
 +
This looks scarier than it is. Be brave.
 +
 +
*Project website up-to-date:
 +
** Project incubation status (if applicable) correctly noted
 +
** Doing the right sorts of things to facilitate [[Community Development for Eclipse Projects|community development]]
 +
** Project has a [[Architecture Council/Contributor Guide Recommendation|Contribution Guide]]
 +
* [[Project Management Infrastructure/Project Metadata|Project Metadata is up-to-date]]
 +
** [[Project Management Infrastructure/Release Metadata|Release metadata]] is correct and up-to-date
 +
** Includes well-written project description (see [http://www.bigsoft.co.uk/blog/index.php/2010/10/18/the-whats-and-whys-to-creating-project-descriptions The Whats and Whys to creating project descriptions]).
 +
*Project plan is up-to-date and in the [[Development Resources/Project Plan|standard format]]
 +
*Code and builds are in order:
 +
**All project code available in eclipse.org VCS
 +
**All project source code repositories include a [[Architecture Council/Contributor Guide Recommendation#Source_Code_Repositories|CONTRIBUTING]] file
 +
**[[Naming Conventions]] followed:
 +
*** Bundle and package names e.g. <code>org.eclipse.<subproject>.<component>[.*]</code>
 +
**[[Version Numbering]] rules followed
 +
** Bundle manifests contains a "Bundle-Vendor" entry set to "Eclipse <project name>"
 +
** Every bundles contains an about.html file as required by [http://www.eclipse.org/legal/guidetolegaldoc.php A Guide to the Legal Documentation for Eclipse-Based Content].
 +
** Features names and descriptions are captured, are spelled correctly, use proper grammar, and contain content that is actually useful for the intended audience
 +
**(If in incubation phase) [[Development Resources/HOWTO/Conforming Incubation Branding|Incubation branding]] on distributed artefacts
 +
**Well-defined retention policy for distributed artefacts (e.g. p2 repositories)
 +
*Review security-related bugs that may affect your project and prepare mitigation strategy
 +
** [https://bugs.eclipse.org/bugs/buglist.cgi?keywords=security;query_format=advanced;keywords_type=allwords Disclosed security issues]
 +
** [https://bugs.eclipse.org/bugs/buglist.cgi?query_format=advanced;field0-0-0=bug_group;bug_status=RESOLVED;type0-0-0=equals;value0-0-0=Security_Advisories Undisclosed security issues] (issues not yet disclosed to the general public)
 +
* Downloads/Products
 +
** At least one milestone build available
 +
** Builds hosted on download.eclipse.org
 +
** Archived builds hosted on archive.eclipse.org
 +
** Every download includes the necessary legal documentation (including a copy of the project licenses) as required by [http://www.eclipse.org/legal/guidetolegaldoc.php A Guide to the Legal Documentation for Eclipse-Based Content].
 +
** If applicable, project downloads are signed using forge-specific signing (as provided by the Eclipse Foundation)
 +
*Intellectual Property (IP) Logs in order
 +
** Approved [[Development Resources/Contribution Questionnaire|Contribution Questionnaire]] (CQ) for project's initial contribution;
 +
** Approved CQs for all third-party libraries used directly by project code (regardless of whether or not the library is directly distributed by the project)
 +
**: Use the [https://www.eclipse.org/projects/tools/downloads.php Project Downloads Review] tool to confirm that the necessary CQs are in place;
 +
* Distribution channels cleaned up (e.g. project [[IT Infrastructure_Doc#Downloads|downloads]])
 +
** Have older releases been moved to the archive server?
 +
** Have older nightly and integration builds been removed?
 +
* IP Logs submitted for review by the Eclipse IP Team at least one week in advance of the review period
 +
** Review [http://eclipse.org/projects/tools/ip_contribution_review.php contributions] for inclusion in your IP Log
 +
** Are there any [https://www.eclipse.org/projects/tools/downloads.php third-party libraries in your downloads] that need a CQ?
 +
*PMC approval obtained (generally obtained through project mailing list)
 +
*Review Documentation (described below in greater detail)
 +
** Correct [http://wiki.eclipse.org/Development_Resources/HOWTO/Review_Information_for_Project_Leads#Copyright_Notice copyright notice] and [http://wiki.eclipse.org/Development_Resources/HOWTO/Review_Information_for_Project_Leads#EPL_Notice EPL notice];
 +
** URLs for the project plan in the format [Development Resources/Project Plan standard format], and the IP Log
 +
** Logistic information: Dates, Communication Channel (e.g. project mailing list)
 +
*PMC-approved review Documentation submitted to EMO in advance of the start of the review period (one week preferred)
 +
If you have any questions, please ask [mailto:emo@eclipse.org EMO].
 +
 +
Note that the lead time required to process IP Logs and review documentation may change based on current load. We typically ask, for example, that IP Logs be submitted a full month in advance for projects participating in the annual simultaneous release.
 +
 +
=What are the Requirements?=
 
The Release Review is conducted before each major release to verify that the key goals of the release have been accomplished and to verify that all intellectual property rights issues have been resolved. For Eclipse projects with typical six week milestones, a plausible time for a release review is one to two weeks into the final six week period, i.e., four or five weeks before the release is scheduled to go live.
 
The Release Review is conducted before each major release to verify that the key goals of the release have been accomplished and to verify that all intellectual property rights issues have been resolved. For Eclipse projects with typical six week milestones, a plausible time for a release review is one to two weeks into the final six week period, i.e., four or five weeks before the release is scheduled to go live.
  
 
=What Does a Release Review Cover?=
 
=What Does a Release Review Cover?=
The Release Review is the final review of the project, process, release and most importantly, the IP issues, against the characteristics that help ensure [http://www.eclipse.org/projects/dev_process/eclipse-quality.php Eclipse Quality].
+
The Release Review is the final review of the project, process, release and most importantly, the IP issues, against the characteristics that help ensure [[Eclipse Quality]].
  
A Release Review is a fairly comprehensive process. We anticipate that   gathering the material for the review and preparing the docuware is a     non-trivial effort, but we believe that the introspection offered by this      exercise is useful for the project and that the results are very useful     for the community.
+
A Release Review is a fairly comprehensive process. We anticipate that gathering the material for the review and preparing the documentation is a non-trivial effort, but we believe that the introspection offered by this      exercise is useful for the project and that the results are very useful for the community.
  
The items to be covered in the Review include the following. If the docuware      is a [http://wiki.eclipse.org/images/4/4a/ReleaseReview20081211.zip slide deck], it will probably average one or two slides       on each of these issues. (Note the use of the     word &quot;summarize&quot; throughout this agenda. Judgment must be used     to determine how much to include in each of these bullet items.)
+
The items to be covered in the Review include the following. If the document is a slide deck (there's a [http://wiki.eclipse.org/images/4/4a/ReleaseReview20081211.zip template] available), it will probably average one or two slides on each of these issues. A text document (provided in an open format such as HTML) should have one or two paragraphs, or a handful of bullets on each issue. Note the use of the word &quot;summarize&quot; throughout this agenda; judgment must be used to determine how much to include in each of these bullet items.
  
 
==Features==
 
==Features==
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px;
+
Summarize the major features of this release as well as any other features that have generated significant discussion amongst the community during the development cycle. Compare the features against the Roadmap to understand the project's conformance or divergence.
padding-left:2px; padding-right:2px; font-family:times; ">R</span> Summarize the major features of this release as well as any other features that have generated significant discussion amongst the community during the development cycle. Compare the features against the Roadmap to understand the project's conformance or divergence.
+
  
<i><b><font color="#0080C0">Reason:</font></b> </i>The community will use this release and the ecosystem will build products on top of this release, and both need to know what features were included or excluded.
+
'''Reason:''' The community will use this release and the ecosystem will build products on top of this release, and both need to know what features were included or excluded.
  
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#00CC99; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px; padding-left:2px; padding-right:2px; font-family:times; ">G</span>References to existing [[Architecture Council/New and Noteworthy|New &amp; Noteworthy]] documentation is a useful addition to this summary. (Reference [http://www.eclipse.org/projects/dev_process/development_process.php#2_2_Quality_Culture])
+
References to existing [[Architecture Council/New and Noteworthy|New &amp; Noteworthy]] documentation is a useful addition to this summary.
  
 
==Non-Code Aspects==
 
==Non-Code Aspects==
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px;
+
Summarize the state of the non-code aspects of the release including: user documentation, localization/externalization, examples, tutorials, articles, and so on. Have the existing artifacts been          updated? Are there new artifacts? Have the obsolete ones been retired or at least marked as pertaining only to older material?
padding-left:2px; padding-right:2px; font-family:times; ">R</span> Summarize the state of the non-code aspects         of the release including: user documentation, localization/externalization, examples,         tutorials, articles, and so on. Have the existing artifacts been          updated? Are there new artifacts? Have the obsolete ones been retired         or at least marked as pertaining only to older material?
+
  
<i><b><font color="#0080C0">Reason:</font></b></i> The non-code aspects are essential for the             wide-spread adoption of the release.              (Reference [http://www.eclipse.org/projects/dev_proceess/development_process.php#2_4_Eclipse_Ecosystem])
+
'''Reason:''' The non-code aspects are essential for the wide-spread adoption of the release (Reference [http://www.eclipse.org/projects/dev_process/development_process.php#2_2_Eclipse_Ecosystem Eclipse Ecosystem]).
  
 
==APIs==
 
==APIs==
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px;
+
Certify that the APIs in this release are [[Eclipse Quality]]. The project lead will ''personally'' certify that the requirements for quality have been met and/or discuss any deficiences.
padding-left:2px; padding-right:2px; font-family:times; ">R</span> Certify that the APIs in this release are <i>[http://www.eclipse.org/projects/dev_process/eclipse-quality.php Eclipse Quality]</i>. <span style="background-color: #FFFFCC">The project lead will         personally certify that the requirements for quality have been met       and/or discuss any deficiences</span>.
+
  
<i><b><font color="#0080C0">Reason:</font></b></i> Eclipse members build commercial tools on top of             the extensible frameworks and thus the quality of the APIs is              extremely important.         (Reference [http://www.eclipse.org/projects/dev_process/development_process.php#2_2_Quality_Culture])
+
'''Reason:''' Eclipse members build commercial tools on top of the extensible frameworks and thus the quality of the APIs isextremely important.
  
 
==Architectural Issues==
 
==Architectural Issues==
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px;
+
Summarize the architectural quality of the release. Discuss the ''intrinsic nature of being extensible''         embodied by this project. Discuss issues such as unresolved overlap with other projects, unpaid &quot;merge debt&quot; from incorporating various components, and so on.
padding-left:2px; padding-right:2px; font-family:times; ">R</span> Summarize the architectural quality of         the release. Discuss the <i>intrinsic nature of being extensible</i>         embodied by this project. Discuss issues such as unresolved overlap with other         projects, unpaid &quot;merge debt&quot; from incorporating various         components, and so on.
+
 
 +
'''Reason:''' Eclipse members build commercial tools on top of the extensible frameworks and thus the quality of the              architecture is important.
 +
 
 +
==Security Issues==
 +
Prior to your release, check Bugzilla for the list of [https://bugs.eclipse.org/bugs/buglist.cgi?keywords=security;query_format=advanced;keywords_type=allwords resolved vulnerabilities] and [https://bugs.eclipse.org/bugs/buglist.cgi?query_format=advanced;field0-0-0=bug_group;type0-0-0=equals;value0-0-0=Security_Advisories currently committer-private vulnerabilities] (there may be overlap between the two) to determine if any affect your project's code. Also check Bugzilla for bugs that may not have been marked for security (e.g. perform a simple search for words like "vulnerability" and "security" in the bug summary.
  
<i>Reason: </i>Eclipse members build commercial tools              on top of the extensible frameworks and thus the quality of the              architecture is important.          (Reference [http://www.eclipse.org/projects/dev_process/development_process.php#2_2_Quality_Culture])
+
In this section of the document, describe the work that you've done to remediate any issues that may arise as a result of these vulnerabilities.
  
 
==Tool Usability==
 
==Tool Usability==
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px; padding-left:2px; padding-right:2px; font-family:times; ">R</span> Summarize the usability of the tools.         Usability in this sense is about using the tools to solve development          problems, not the more academic sense of UI evaluation/testing.
+
Summarize the usability of the tools. Usability in this sense is about using the tools to solve development          problems, not the more academic sense of UI evaluation/testing.
  
<i><b><font color="#0080C0">Reason:</font></b></i> Without usable             tools, the project will not attract the user community necessary             to enable the ecosystem.&nbsp;          (Reference [http://www.eclipse.org/projects/dev_process/development_process.php#2_5_Three_Communities])
+
'''Reason:''' Without usable tools, the project will not attract the user community necessary to enable the ecosystem (Reference [http://www.eclipse.org/projects/dev_process/development_process_2010.php#2_3_Three_Communities Three Communities]).
  
 
==End-of-Life==
 
==End-of-Life==
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px; padding-left:2px; padding-right:2px; font-family:times; ">R</span> Summarize the features (APIs and any significant         user features)         from previous releases that are being end-of-life'd in this release.         End of life includes both deprecation and actual removal.
+
Summarize the features (APIs and any significant user features) from previous releases that are being end-of-life'd in this release. End of life includes both deprecation and actual removal.
  
<i><b><font color="#0080C0">Reason:</font></b></i> The community builds products that rely on             features and so they need to know when these features are              changing.          (Reference [http://www.eclipse.org/projects/dev_process/development_process.php#2_5_Three_Communities])
+
'''Reason:''' The community builds products that rely on features and so they need to know when these features are              changing (Reference [http://www.eclipse.org/projects/dev_process/development_process_2010.php#2_3_Three_Communities Three Communities]).
  
 
==Bugzilla==
 
==Bugzilla==
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px;
+
Summarize the bugzilla situation. How many bug records (defects and enhancements) have been opened/closed/deferred/new, etc? How many P1, P2, ..., bug records are outstanding?  
padding-left:2px; padding-right:2px; font-family:times; ">R</span> Summarize the bugzilla situation. How many bug         records (defects and enhancements) have been         opened/closed/deferred/new, etc? How many P1, P2, ..., bug records are         outstanding?  
+
  
<i><b><font color="#0080C0">Reason:</font></b></i> Summaries of the bugzilla records offer a glimpse             into the project productivity. They also offer an estimate of the             outstanding risk. And the summary is used to alert the community             to known issues.              (Reference [http://www.eclipse.org/projects/dev_process/development_process.php#2_4_Eclipse_Ecosystem])
+
'''Reason:''' Summaries of the bugzilla records offer a glimpse into the project productivity. They also offer an estimate of the outstanding risk. And the summary is used to alert the community to known issues           (Reference [http://www.eclipse.org/projects/dev_process/development_process.php#2_2_Eclipse_Ecosystem]).
  
 
==Standards==
 
==Standards==
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px;
+
Summarize the standards compliance of this release. If the features are based on defined, pending, or ad-hoc standards, what is the state of those standards and what is the state of the support for those standards in this release.
padding-left:2px; padding-right:2px; font-family:times; ">R</span> Summarize the standards compliance of this release.         If the features are based on defined, pending, or ad-hoc standards,         what is the state of those standards and what is the state of the         support for those standards in this release.
+
  
<i><b><font color="#0080C0">Reason: </font></b></i>Eclipse is about building frameworks and tools             based on standards, so we need to make sure that we are conforming             to the appropriate standards.              (Reference [http://www.eclipse.org/projects/dev_process/development_process.php#2_4_Eclipse_Ecosystem])
+
'''Reason:''' Eclipse is about building frameworks and tools based on standards, so we need to make sure that we are conforming to the appropriate standards (Reference [http://www.eclipse.org/projects/dev_process/development_process.php#2_2_Eclipse_Ecosystem]).
  
 
==UI Usability==
 
==UI Usability==
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px;
+
Summarize the user interface usability and the conformance to the [[User_Interface_Guidelines]]. Include section 508 compliance, language pack conformance (does the code support multiple languages), etc. Explain any deviations from the user interface guidelines and standards.
padding-left:2px; padding-right:2px; font-family:times; ">R</span> Summarize the user interface usability and the conformance to the [[User_Interface_Guidelines]]. Include section 508 compliance, language pack conformance (does the code         support multiple languages), etc. Explain any deviations from the         user interface guidelines and standards.
+
  
<i><b><font color="#0080C0">Reason:</font></b></i>  The user             community is larger than just mouse-wielding, English-speaking,             computer jockeys. We need to support that larger community.          (References [http://www.eclipse.org/projects/dev_process/development_process.php#2_2_Quality_Culture],[[User_Interface_Guidelines]])
+
'''Reason:''' The user community is larger than just mouse-wielding, English-speaking, computer jockeys. We need to support that larger community (Reference [[User_Interface_Guidelines]]).
  
 
==Schedule==
 
==Schedule==
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px;
+
Discuss the initial schedule and any changes to the schedule over the course of the release, i.e., what the project team achieved. Discuss whether milestones were met or slipped.
padding-left:2px; padding-right:2px; font-family:times; ">R</span> Discuss the initial schedule and any changes to the         schedule over the course of the release, i.e., what the project team         achieved. Discuss whether milestones were met or slipped.
+
  
<i><b><font color="#0080C0">Reason:</font></b></i> The community relies on consistent schedules from             Eclipse so that projects and products can plan for the correct             dependencies.         (Reference [http://www.eclipse.org/projects/dev_process/development_process.php#2_3_Collective_Reputation])
+
'''Reason:''' The community relies on consistent schedules from Eclipse so that projects and products can plan for the correct dependencies.
  
 
==Communities==
 
==Communities==
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px;
+
Summarize the project's development of its three communities. Consider the interactions on bugzilla, the          mailing lists, the newsgroups, public conference calls, blogs, public-relations activities, code camps, conference tutorials, coordinating with other Eclipse projects and other open source projects (Apache, ObjectWeb,          etc), ...
padding-left:2px; padding-right:2px; font-family:times; ">R</span> Summarize the project's development of its three         communities. Consider the interactions on bugzilla, the          mailing lists, the newsgroups, public conference calls, blogs, PR          activities, code camps, conference tutorials, coordinating with other         Eclipse projects and other open source projects (Apache, ObjectWeb,          etc), ...
+
  
<i><b><font color="#0080C0">Reason:</font></b></i> It is important for Eclipse projects to build a             community around the project, not just deliver code for a project.             This review item is about the success of building a community.          (Reference [http://www.eclipse.org/projects/dev_process/development_process.php#2_5_Three_Communities])
+
'''Reason:''' It is important for Eclipse projects to build a community around the project, not just deliver code for a project. This review item is about the success of building a community (Reference [http://www.eclipse.org/projects/dev_process/development_process.php#2_3_Three_Communities Three Communities]).
  
 
==IP Issues==
 
==IP Issues==
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px;
+
As per the [http://www.eclipse.org/org/documents/Eclipse_IP_Policy.pdf Eclipse IP Policy], these steps must be done:
padding-left:2px; padding-right:2px; font-family:times; ">R</span> As per the [http://www.eclipse.org/org/documents/Eclipse_IP_Policy.pdf Eclipse IP Policy], these steps must be done:
+
  
 
# The project leadership verifies that:
 
# The project leadership verifies that:
Line 102: Line 196:
 
## the About files which contain any non-standard terms                    (e.g., a reference to a license other than the EPL, etc)
 
## the About files which contain any non-standard terms                    (e.g., a reference to a license other than the EPL, etc)
 
# The EMO will validate for (a) and (b) that Contribution Questionnaires              have been properly submitted and EMO approvals have been              completed.
 
# The EMO will validate for (a) and (b) that Contribution Questionnaires              have been properly submitted and EMO approvals have been              completed.
# A frozen copy of the reviewed-and-approved-by-Eclipse-legal Project              Log is part of the Release Review documentation. It              can be included in the docuware or as a separate document. (Reference [http://www.eclipse.org/org/documents/Eclipse_IP_Policy.pdf])
+
# A frozen copy of the reviewed-and-approved-by-Eclipse-legal Project              Log is part of the Release Review documentation. It              can be included in the documentation or as a separate document. (Reference [http://www.eclipse.org/org/documents/Eclipse_IP_Policy.pdf])
  
 
The provider name for contributed features/bundles should always start with "Eclipse ..." followed by project name. It is up to each PMC to decide how best to describe their project in a meaningful, balanced fashion (either as one top level one or as multiple pieces). Avoid special symbols (e.g. ':' or "-") unless they are part of the project name. Avoid acronyms (unless in common use in software industry, such as XML, or PHP). Capitalization should follow "headline-sytyle capitalization": The only words that are not capitalized are articles (except as the first word), coordinating conjunctions (for example, and, but, and or), prepositions (except as the first or last word), and the to in an infinitive.) For discussion, see [https://bugs.eclipse.org/bugs/show_bug.cgi?id=252813 Bug 252813].
 
The provider name for contributed features/bundles should always start with "Eclipse ..." followed by project name. It is up to each PMC to decide how best to describe their project in a meaningful, balanced fashion (either as one top level one or as multiple pieces). Avoid special symbols (e.g. ':' or "-") unless they are part of the project name. Avoid acronyms (unless in common use in software industry, such as XML, or PHP). Capitalization should follow "headline-sytyle capitalization": The only words that are not capitalized are articles (except as the first word), coordinating conjunctions (for example, and, but, and or), prepositions (except as the first or last word), and the to in an infinitive.) For discussion, see [https://bugs.eclipse.org/bugs/show_bug.cgi?id=252813 Bug 252813].
  
 
==IP Issues Speak-Up-Now==
 
==IP Issues Speak-Up-Now==
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px; padding-left:2px; padding-right:2px; font-family:times; ">R</span> The EMO explicitly asks during the Release          Review if any Member would like to assert that this release infringes          their IP rights. If so, the EMO and the project will follow the          Eclipse IP Policy in discussions with that Member.  
+
The EMO explicitly asks during the Release          Review if any Member would like to assert that this release infringes          their IP rights. If so, the EMO and the project will follow the          Eclipse IP Policy in discussions with that Member.  
  
 
<i><b><font color="#0080C0">Reason:</font></b></i> One of the important benefits that the Eclipse              Foundation provides for its members is the consistent application              of the [http://www.eclipse.org/org/documents/Eclipse_IP_Policy.pdf Eclipse IP Policy] which helps ensure (but does not guarantee) that the              framework and tools are useable in commercial products.
 
<i><b><font color="#0080C0">Reason:</font></b></i> One of the important benefits that the Eclipse              Foundation provides for its members is the consistent application              of the [http://www.eclipse.org/org/documents/Eclipse_IP_Policy.pdf Eclipse IP Policy] which helps ensure (but does not guarantee) that the              framework and tools are useable in commercial products.
Line 114: Line 208:
  
 
==Project Plan==
 
==Project Plan==
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px; padding-left:2px; padding-right:2px; font-family:times; ">R</span>  Per Board resolution of December 11, 2008, to pass a Continuation Review, Graduation Review or Release Review a project must have a current project plan, in the format specified by the EMO, available to the community.
+
Per Board resolution of December 11, 2008, to pass a Continuation Review, Graduation Review or Release Review a project must have a current project plan, in the format specified by the EMO, available to the community.
  
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#00CC99; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px; padding-left:2px; padding-right:2px; font-family:times; ">G</span>If there is a Project Plan (full or even a draft)          for the next release, the final issue to cover in the Release Review          is the unveiling of the new plan.
+
If there is a Project Plan (full or even a draft)          for the next release, the final issue to cover in the Release Review          is the unveiling of the new plan.
  
=Preparing the Docuware=
+
=Preparing the Documentation=
[[Development_Resources/FAQs/About_Docuware|About Docuware]]
+
[[Development_Resources/FAQs/About_Docuware|About Documentation]]
  
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px; padding-left:2px; padding-right:2px; font-family:times; ">R</span><br>The docuware must cover the issues from (2) above.       
+
The documentation must cover the issues from (2) above.       
  
Example docuware from previous Release Reviews can be found in the [http://www.eclipse.org/projects/previous-release-reviews.php archives]. A [http://wiki.eclipse.org/images/4/4a/ReleaseReview20081211.zip template] (Open Office Presentation format) is also available.
+
Example documentation from previous Release Reviews can be found in the [http://www.eclipse.org/projects/previous-release-reviews.php archives]. A [http://wiki.eclipse.org/images/4/4a/ReleaseReview20081211.zip template] (Open Office Presentation format) is also available.
  
 
=The Review Process=
 
=The Review Process=
Line 140: Line 234:
 
*          Fail - there are significant problems with the release and/or the          project.  
 
*          Fail - there are significant problems with the release and/or the          project.  
  
''This page is moderated by the EMO''
+
<hr/>
 +
This page is moderated by the EMO.
 +
[[Category:Eclipse Development Process]]
 
[[Category:Development_Resources]]
 
[[Category:Development_Resources]]
 
[[Category:How to Contribute]]
 
[[Category:How to Contribute]]

Revision as of 17:28, 26 January 2016

Warning2.png
See "6.3.3 Release Review" and "6.3 Reviews" in the Eclipse Development Process.

The purposes of a Release Review are: to summarize the accomplishments of the release, to verify that the IP Policy has been followed and all approvals have been received, to highlight any remaining quality and/or architectural issues, and to verify that the project is continuing to operate according to the Principles and Purposes of Eclipse.

Overview

Please see the release cycle, which includes a nice (simple) overview of the release review process.

ReleaseReview.png

Helpful Documentation from the Eclipse Development Process

Reviews are Not Required for Service Releases

Note that a release review is not required for a service--or "bug fix only"--release. If your release contains no new features, APIs, or significant changes in functionality, you may release it without engaging in a community review. You should still create the release record in the PMI to notify your community. Service releases are indicated by an increment in the third segment of the version number (e.g. 2.3.2).

Note also that projects are not required to submit IP logs for review by the IP Team for service releases. Projects are required to ensure that they have the appropriate coverage for all intellectual property (e.g. CQs and contribution records) at all times.

Intellectual Property

Before you can consider a Release Review, all of the relevant CQs must be approved by the Eclipse Legal team. We cannot schedule a Review before the Legal team has completed their work. If you are waiting for CQs, please review where your CQs are, and when they are scheduled to be reviewed, in the IP team work queue.

After you have verified that all relevant CQs are approved, the following items must be completed at least one week before the scheduled review date:

  1. PMC Approval (Can occur in parallel with, prior to, and after, the IP clearance);
  2. IP Log Approval This is essential, and no release review can proceed without it!; and
  3. Review document.

IP Log Approval

Because IP Log approval is essential and can take varying amounts of time (depending on the work queue of the IP Team), projects must have an approved IP Log before their review date is confirmed. After you have submitted the IP Log to Eclipse Legal for approval, the review is added to the schedule with the notation Waiting for IP Log. After EMO is notified that the IP Log is approved, that notation is replaced with the date of the review.

Please use the automated IP Log Tool to update and and submit your IP Log. You can use the IP Log Generator for this (the Submitting the IP Log to Eclipse Legal section explains how to submit the IP Log). The URL for using the tool for your project uses your project ID, in this format: http://www.eclipse.org/projects/ip_log.php?projectid=project ID.

Before submitting your IP Log, we recommend that you review the contents of your project's download directory to ensure that you have the necessary CQs in place. The Project Downloads Review tool can help you in with this review.

While you are waiting for IP Log approval, we suggest that you obtain PMC approval for the review and begin work on your documentation.

PMC Approval

Please forward an email showing that you have the approval of your PMC for the review and corresponding review documentation. The easiest way to do this is to request approval on your PMC mailing list (which requires subscription) and forward the response to EMO.

Note that you do not have to wait for the IP Log to be approved by the IP Team. You can seek PMC review/approval of your review documentation while the IP Log separate from the IP log approval process.

Project Plan

Your project plan must be current and available for public consumption.

Review Documentation

The content requirements for review documentation are detailed below. The preferred means of providing review documentation is to enter it directly into the record associated with your release in the project metadata.

Alternative Formats

In the past, a presentation file (PPT or ODP) rendered in PDF from was the preferred format for review documentation (we used to host actual review calls for which the presentation format was appropriate). Presentation files are still acceptable (and are quite common). Other open formats, including HTML, a wiki page, or a document file rendered in PDF format, are also acceptable.

Please submit your document via email to the EMO. Include the review document as either a link or an attachment.

Many people underestimate the time and effort needed to create the document, so be sure to allow enough time for this task. The "official" due date for the document is one week before the scheduled start of the review period. However, the document need to be reviewed by the EMO before posting. If you wait until the due date to submit the first draft and the EMO requests changes, then you'll probably miss your deadline and your review will be postponed.

Checklist

This looks scarier than it is. Be brave.

  • Project website up-to-date:
  • Project Metadata is up-to-date
  • Project plan is up-to-date and in the standard format
  • Code and builds are in order:
    • All project code available in eclipse.org VCS
    • All project source code repositories include a CONTRIBUTING file
    • Naming Conventions followed:
      • Bundle and package names e.g. org.eclipse.<subproject>.<component>[.*]
    • Version Numbering rules followed
    • Bundle manifests contains a "Bundle-Vendor" entry set to "Eclipse <project name>"
    • Every bundles contains an about.html file as required by A Guide to the Legal Documentation for Eclipse-Based Content.
    • Features names and descriptions are captured, are spelled correctly, use proper grammar, and contain content that is actually useful for the intended audience
    • (If in incubation phase) Incubation branding on distributed artefacts
    • Well-defined retention policy for distributed artefacts (e.g. p2 repositories)
  • Review security-related bugs that may affect your project and prepare mitigation strategy
  • Downloads/Products
    • At least one milestone build available
    • Builds hosted on download.eclipse.org
    • Archived builds hosted on archive.eclipse.org
    • Every download includes the necessary legal documentation (including a copy of the project licenses) as required by A Guide to the Legal Documentation for Eclipse-Based Content.
    • If applicable, project downloads are signed using forge-specific signing (as provided by the Eclipse Foundation)
  • Intellectual Property (IP) Logs in order
    • Approved Contribution Questionnaire (CQ) for project's initial contribution;
    • Approved CQs for all third-party libraries used directly by project code (regardless of whether or not the library is directly distributed by the project)
      Use the Project Downloads Review tool to confirm that the necessary CQs are in place;
  • Distribution channels cleaned up (e.g. project downloads)
    • Have older releases been moved to the archive server?
    • Have older nightly and integration builds been removed?
  • IP Logs submitted for review by the Eclipse IP Team at least one week in advance of the review period
  • PMC approval obtained (generally obtained through project mailing list)
  • Review Documentation (described below in greater detail)
    • Correct copyright notice and EPL notice;
    • URLs for the project plan in the format [Development Resources/Project Plan standard format], and the IP Log
    • Logistic information: Dates, Communication Channel (e.g. project mailing list)
  • PMC-approved review Documentation submitted to EMO in advance of the start of the review period (one week preferred)

If you have any questions, please ask EMO.

Note that the lead time required to process IP Logs and review documentation may change based on current load. We typically ask, for example, that IP Logs be submitted a full month in advance for projects participating in the annual simultaneous release.

What are the Requirements?

The Release Review is conducted before each major release to verify that the key goals of the release have been accomplished and to verify that all intellectual property rights issues have been resolved. For Eclipse projects with typical six week milestones, a plausible time for a release review is one to two weeks into the final six week period, i.e., four or five weeks before the release is scheduled to go live.

What Does a Release Review Cover?

The Release Review is the final review of the project, process, release and most importantly, the IP issues, against the characteristics that help ensure Eclipse Quality.

A Release Review is a fairly comprehensive process. We anticipate that gathering the material for the review and preparing the documentation is a non-trivial effort, but we believe that the introspection offered by this exercise is useful for the project and that the results are very useful for the community.

The items to be covered in the Review include the following. If the document is a slide deck (there's a template available), it will probably average one or two slides on each of these issues. A text document (provided in an open format such as HTML) should have one or two paragraphs, or a handful of bullets on each issue. Note the use of the word "summarize" throughout this agenda; judgment must be used to determine how much to include in each of these bullet items.

Features

Summarize the major features of this release as well as any other features that have generated significant discussion amongst the community during the development cycle. Compare the features against the Roadmap to understand the project's conformance or divergence.

Reason: The community will use this release and the ecosystem will build products on top of this release, and both need to know what features were included or excluded.

References to existing New & Noteworthy documentation is a useful addition to this summary.

Non-Code Aspects

Summarize the state of the non-code aspects of the release including: user documentation, localization/externalization, examples, tutorials, articles, and so on. Have the existing artifacts been updated? Are there new artifacts? Have the obsolete ones been retired or at least marked as pertaining only to older material?

Reason: The non-code aspects are essential for the wide-spread adoption of the release (Reference Eclipse Ecosystem).

APIs

Certify that the APIs in this release are Eclipse Quality. The project lead will personally certify that the requirements for quality have been met and/or discuss any deficiences.

Reason: Eclipse members build commercial tools on top of the extensible frameworks and thus the quality of the APIs isextremely important.

Architectural Issues

Summarize the architectural quality of the release. Discuss the intrinsic nature of being extensible embodied by this project. Discuss issues such as unresolved overlap with other projects, unpaid "merge debt" from incorporating various components, and so on.

Reason: Eclipse members build commercial tools on top of the extensible frameworks and thus the quality of the architecture is important.

Security Issues

Prior to your release, check Bugzilla for the list of resolved vulnerabilities and currently committer-private vulnerabilities (there may be overlap between the two) to determine if any affect your project's code. Also check Bugzilla for bugs that may not have been marked for security (e.g. perform a simple search for words like "vulnerability" and "security" in the bug summary.

In this section of the document, describe the work that you've done to remediate any issues that may arise as a result of these vulnerabilities.

Tool Usability

Summarize the usability of the tools. Usability in this sense is about using the tools to solve development problems, not the more academic sense of UI evaluation/testing.

Reason: Without usable tools, the project will not attract the user community necessary to enable the ecosystem (Reference Three Communities).

End-of-Life

Summarize the features (APIs and any significant user features) from previous releases that are being end-of-life'd in this release. End of life includes both deprecation and actual removal.

Reason: The community builds products that rely on features and so they need to know when these features are changing (Reference Three Communities).

Bugzilla

Summarize the bugzilla situation. How many bug records (defects and enhancements) have been opened/closed/deferred/new, etc? How many P1, P2, ..., bug records are outstanding?

Reason: Summaries of the bugzilla records offer a glimpse into the project productivity. They also offer an estimate of the outstanding risk. And the summary is used to alert the community to known issues (Reference [1]).

Standards

Summarize the standards compliance of this release. If the features are based on defined, pending, or ad-hoc standards, what is the state of those standards and what is the state of the support for those standards in this release.

Reason: Eclipse is about building frameworks and tools based on standards, so we need to make sure that we are conforming to the appropriate standards (Reference [2]).

UI Usability

Summarize the user interface usability and the conformance to the User_Interface_Guidelines. Include section 508 compliance, language pack conformance (does the code support multiple languages), etc. Explain any deviations from the user interface guidelines and standards.

Reason: The user community is larger than just mouse-wielding, English-speaking, computer jockeys. We need to support that larger community (Reference User_Interface_Guidelines).

Schedule

Discuss the initial schedule and any changes to the schedule over the course of the release, i.e., what the project team achieved. Discuss whether milestones were met or slipped.

Reason: The community relies on consistent schedules from Eclipse so that projects and products can plan for the correct dependencies.

Communities

Summarize the project's development of its three communities. Consider the interactions on bugzilla, the mailing lists, the newsgroups, public conference calls, blogs, public-relations activities, code camps, conference tutorials, coordinating with other Eclipse projects and other open source projects (Apache, ObjectWeb, etc), ...

Reason: It is important for Eclipse projects to build a community around the project, not just deliver code for a project. This review item is about the success of building a community (Reference Three Communities).

IP Issues

As per the Eclipse IP Policy, these steps must be done:

  1. The project leadership verifies that:
    • ... that the about files and use licenses are in place as per the Guidelines to Legal Documentation
    • ... all contributions (code, documentation, images, etc) has been committed by individuals who are either Members of the Foundation, or have signed the appropriate Committer Agreement. In either case, these are individuals who have signed, and are abiding by, the Eclipse IP Policy.
    • ... that all significant contributions have been reviewed by the Foundation's legal staff. Include references to the IPZilla numbers of all clearances.
    • ... that all non-Committer code contributions, including third-party libraries, have been documented in the release and reviewed by the Foundation's legal staff. Include references to the IPZilla numbers of all clearances.
    • ... that all Contribution Questionnaires have been completed
    • ... the "copyright" field of each feature is set to the copyright owner (the Eclipse Foundation is rarely the copyright owner).
    • ... that any third-party logos or trademarks included in the distribution (icons, help file logos, etc) have been licensed under the EPL.
    • ... that any fonts or similar third-party images included in the distribution (e.g. in PDF or EPS files) have been licensed under the EPL.
  2. The PMC provides a Project Log that enumerates:
    1. every piece of third party software including information on the license
    2. every major contribution
    3. the name of every contributor including non-committer contributions via bug fixes with bug #'s
    4. the About files which contain any non-standard terms (e.g., a reference to a license other than the EPL, etc)
  3. The EMO will validate for (a) and (b) that Contribution Questionnaires have been properly submitted and EMO approvals have been completed.
  4. A frozen copy of the reviewed-and-approved-by-Eclipse-legal Project Log is part of the Release Review documentation. It can be included in the documentation or as a separate document. (Reference [3])

The provider name for contributed features/bundles should always start with "Eclipse ..." followed by project name. It is up to each PMC to decide how best to describe their project in a meaningful, balanced fashion (either as one top level one or as multiple pieces). Avoid special symbols (e.g. ':' or "-") unless they are part of the project name. Avoid acronyms (unless in common use in software industry, such as XML, or PHP). Capitalization should follow "headline-sytyle capitalization": The only words that are not capitalized are articles (except as the first word), coordinating conjunctions (for example, and, but, and or), prepositions (except as the first or last word), and the to in an infinitive.) For discussion, see Bug 252813.

IP Issues Speak-Up-Now

The EMO explicitly asks during the Release Review if any Member would like to assert that this release infringes their IP rights. If so, the EMO and the project will follow the Eclipse IP Policy in discussions with that Member.

Reason: One of the important benefits that the Eclipse Foundation provides for its members is the consistent application of the Eclipse IP Policy which helps ensure (but does not guarantee) that the framework and tools are useable in commercial products.

(Reference [4])

Project Plan

Per Board resolution of December 11, 2008, to pass a Continuation Review, Graduation Review or Release Review a project must have a current project plan, in the format specified by the EMO, available to the community.

If there is a Project Plan (full or even a draft) for the next release, the final issue to cover in the Release Review is the unveiling of the new plan.

Preparing the Documentation

About Documentation

The documentation must cover the issues from (2) above.

Example documentation from previous Release Reviews can be found in the archives. A template (Open Office Presentation format) is also available.

The Review Process

Once the review content has been created, the operational side of a Release Review is described here: Development_Resources/HOWTO/Review_Process

After a Successful Review

After the successful advisory vote and the approval of the Release Review by the EMO:

  1. the EMO will send an email to the project development list affirming the positive result
  2. then, when the final release files are ready, the project may post them to the download server and let the powerful replication system distribute the bits throughout the known universe

Other Results

Other possible results of the Release Review include:

  • Postpone - issues have arisen that must be solved before the release proceeds
  • Pass w/ Notes - the release has some issues that need to be addressed, but they can be addressed with release notes and/or documentation
  • Pass w/ Issues - the release has some issues, but they can be addressed in a future release as agreed to in the review
  • Fail - there are significant problems with the release and/or the project.

This page is moderated by the EMO.

Copyright © Eclipse Foundation, Inc. All Rights Reserved.