Difference between revisions of "Development Resources/HOWTO/Release Reviews"

From Eclipsepedia

Jump to: navigation, search
(Added a link to a template slide deck document.)
Line 4: Line 4:
 
</tr></table></div><blockquote><em>The purposes of a Release Review are: to summarize the accomplishments of the release, to verify that the IP Policy has been followed and all approvals have been received, to highlight any remaining quality and/or architectural issues, and to verify that the project is continuing to operate according to the Princples and Purposes of Eclipse.</em></blockquote>
 
</tr></table></div><blockquote><em>The purposes of a Release Review are: to summarize the accomplishments of the release, to verify that the IP Policy has been followed and all approvals have been received, to highlight any remaining quality and/or architectural issues, and to verify that the project is continuing to operate according to the Princples and Purposes of Eclipse.</em></blockquote>
  
===(1) What are the Requirements?===
+
=What are the Requirements?=
  
<h2>(1) What are the Requirements?</h2>
 
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px; padding-left:2px; padding-right:2px; font-family:times; ">R</span>The only requirements pertaining to a Release Review is that the review be held and be successfully and all remedial actions completed before the Project's official release.
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px; padding-left:2px; padding-right:2px; font-family:times; ">R</span>The only requirements pertaining to a Release Review is that the review be held and be successfully and all remedial actions completed before the Project's official release.
  
 
The Release Review is conducted before each major release to verify that the key goals of the release have been accomplished and to verify that all intellectual property rights issues have been resolved. For Eclipse projects with typical six week milestones, a plausible time for a release review is one to two weeks into the final six week period, i.e., four or five weeks before the release is scheduled to go live.
 
The Release Review is conducted before each major release to verify that the key goals of the release have been accomplished and to verify that all intellectual property rights issues have been resolved. For Eclipse projects with typical six week milestones, a plausible time for a release review is one to two weeks into the final six week period, i.e., four or five weeks before the release is scheduled to go live.
  
===(2) What Does a Release Review Cover?===
+
=What Does a Release Review Cover?=
 
The Release Review is the final review of the project, process, release and most importantly, the IP issues, against the characteristics that help ensure [http://www.eclipse.org/projects/dev_process/eclipse-quality.php Eclipse Quality].
 
The Release Review is the final review of the project, process, release and most importantly, the IP issues, against the characteristics that help ensure [http://www.eclipse.org/projects/dev_process/eclipse-quality.php Eclipse Quality].
  
Line 18: Line 17:
 
The items to be covered in the Review include the following. If the docuware      is a [http://wiki.eclipse.org/images/4/4a/ReleaseReview20081211.zip slide deck], it will probably average one or two slides      on each of these issues. (Note the use of the      word &quot;summarize&quot; throughout this agenda. Judgment must be used      to determine how much to include in each of these bullet items.)
 
The items to be covered in the Review include the following. If the docuware      is a [http://wiki.eclipse.org/images/4/4a/ReleaseReview20081211.zip slide deck], it will probably average one or two slides      on each of these issues. (Note the use of the      word &quot;summarize&quot; throughout this agenda. Judgment must be used      to determine how much to include in each of these bullet items.)
  
====(2.1) Features====
+
==Features==
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px;  
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px;  
 
padding-left:2px; padding-right:2px; font-family:times; ">R</span> Summarize the major features of this release as well          as any other features that have generated significant discussion          amongst the community during the development cycle. Compare the          features against the Roadmap to understand the project's conformance          or divergence.
 
padding-left:2px; padding-right:2px; font-family:times; ">R</span> Summarize the major features of this release as well          as any other features that have generated significant discussion          amongst the community during the development cycle. Compare the          features against the Roadmap to understand the project's conformance          or divergence.
Line 26: Line 25:
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#00CC99; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px; padding-left:2px; padding-right:2px; font-family:times; ">G</span>References to existing New &amp; Noteworthy          documentation is a useful addition to this summary.          (Reference [http://www.eclipse.org/projects/dev_process/development_process.php#2_2_Quality_Culture])
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#00CC99; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px; padding-left:2px; padding-right:2px; font-family:times; ">G</span>References to existing New &amp; Noteworthy          documentation is a useful addition to this summary.          (Reference [http://www.eclipse.org/projects/dev_process/development_process.php#2_2_Quality_Culture])
  
====(2.2) Non-Code Aspects====
+
==Non-Code Aspects==
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px;  
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px;  
 
padding-left:2px; padding-right:2px; font-family:times; ">R</span> Summarize the state of the non-code aspects          of the release including: user documentation, localization/externalization, examples,          tutorials, articles, and so on. Have the existing artifacts been          updated? Are there new artifacts? Have the obsolete ones been retired          or at least marked as pertaining only to older material?
 
padding-left:2px; padding-right:2px; font-family:times; ">R</span> Summarize the state of the non-code aspects          of the release including: user documentation, localization/externalization, examples,          tutorials, articles, and so on. Have the existing artifacts been          updated? Are there new artifacts? Have the obsolete ones been retired          or at least marked as pertaining only to older material?
Line 32: Line 31:
 
<i><b><font color="#0080C0">Reason:</font></b></i> The non-code aspects are essential for the              wide-spread adoption of the release.              (Reference [http://www.eclipse.org/projects/dev_proceess/development_process.php#2_4_Eclipse_Ecosystem])
 
<i><b><font color="#0080C0">Reason:</font></b></i> The non-code aspects are essential for the              wide-spread adoption of the release.              (Reference [http://www.eclipse.org/projects/dev_proceess/development_process.php#2_4_Eclipse_Ecosystem])
  
====(2.3) APIs====
+
==APIs==
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px;  
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px;  
 
padding-left:2px; padding-right:2px; font-family:times; ">R</span> Certify that the APIs in this release are <i>[http://www.eclipse.org/projects/dev_process/eclipse-quality.php Eclipse Quality]</i>. <span style="background-color: #FFFFCC">The project lead will          personally certify that the requirements for quality have been met        and/or discuss any deficiences</span>.
 
padding-left:2px; padding-right:2px; font-family:times; ">R</span> Certify that the APIs in this release are <i>[http://www.eclipse.org/projects/dev_process/eclipse-quality.php Eclipse Quality]</i>. <span style="background-color: #FFFFCC">The project lead will          personally certify that the requirements for quality have been met        and/or discuss any deficiences</span>.
Line 38: Line 37:
 
<i><b><font color="#0080C0">Reason:</font></b></i> Eclipse members build commercial tools on top of              the extensible frameworks and thus the quality of the APIs is              extremely important.          (Reference [http://www.eclipse.org/projects/dev_process/development_process.php#2_2_Quality_Culture])
 
<i><b><font color="#0080C0">Reason:</font></b></i> Eclipse members build commercial tools on top of              the extensible frameworks and thus the quality of the APIs is              extremely important.          (Reference [http://www.eclipse.org/projects/dev_process/development_process.php#2_2_Quality_Culture])
  
====(2.4) Architectural Issues====
+
==Architectural Issues==
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px;  
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px;  
 
padding-left:2px; padding-right:2px; font-family:times; ">R</span> Summarize the architectural quality of          the release. Discuss the <i>intrinsic nature of being extensible</i>          embodied by this project. Discuss issues such as unresolved overlap with other          projects, unpaid &quot;merge debt&quot; from incorporating various          components, and so on.
 
padding-left:2px; padding-right:2px; font-family:times; ">R</span> Summarize the architectural quality of          the release. Discuss the <i>intrinsic nature of being extensible</i>          embodied by this project. Discuss issues such as unresolved overlap with other          projects, unpaid &quot;merge debt&quot; from incorporating various          components, and so on.
Line 44: Line 43:
 
<i>Reason: </i>Eclipse members build commercial tools              on top of the extensible frameworks and thus the quality of the              architecture is important.          (Reference [http://www.eclipse.org/projects/dev_process/development_process.php#2_2_Quality_Culture])
 
<i>Reason: </i>Eclipse members build commercial tools              on top of the extensible frameworks and thus the quality of the              architecture is important.          (Reference [http://www.eclipse.org/projects/dev_process/development_process.php#2_2_Quality_Culture])
  
====(2.5) Tool Usability====
+
==Tool Usability==
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px; padding-left:2px; padding-right:2px; font-family:times; ">R</span> Summarize the usability of the tools.          Usability in this sense is about using the tools to solve development          problems, not the more academic sense of UI evaluation/testing.
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px; padding-left:2px; padding-right:2px; font-family:times; ">R</span> Summarize the usability of the tools.          Usability in this sense is about using the tools to solve development          problems, not the more academic sense of UI evaluation/testing.
  
 
<i><b><font color="#0080C0">Reason:</font></b></i>  Without usable              tools, the project will not attract the user community necessary              to enable the ecosystem.&nbsp;          (Reference [http://www.eclipse.org/projects/dev_process/development_process.php#2_5_Three_Communities])
 
<i><b><font color="#0080C0">Reason:</font></b></i>  Without usable              tools, the project will not attract the user community necessary              to enable the ecosystem.&nbsp;          (Reference [http://www.eclipse.org/projects/dev_process/development_process.php#2_5_Three_Communities])
  
====(2.6) End-of-Life====
+
==End-of-Life==
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px; padding-left:2px; padding-right:2px; font-family:times; ">R</span> Summarize the features (APIs and any significant          user features)          from previous releases that are being end-of-life'd in this release.          End of life includes both deprecation and actual removal.
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px; padding-left:2px; padding-right:2px; font-family:times; ">R</span> Summarize the features (APIs and any significant          user features)          from previous releases that are being end-of-life'd in this release.          End of life includes both deprecation and actual removal.
  
 
<i><b><font color="#0080C0">Reason:</font></b></i> The community builds products that rely on              features and so they need to know when these features are              changing.          (Reference [http://www.eclipse.org/projects/dev_process/development_process.php#2_5_Three_Communities])
 
<i><b><font color="#0080C0">Reason:</font></b></i> The community builds products that rely on              features and so they need to know when these features are              changing.          (Reference [http://www.eclipse.org/projects/dev_process/development_process.php#2_5_Three_Communities])
  
====(2.7) Bugzilla====
+
==Bugzilla==
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px;  
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px;  
 
padding-left:2px; padding-right:2px; font-family:times; ">R</span> Summarize the bugzilla situation. How many bug          records (defects and enhancements) have been          opened/closed/deferred/new, etc? How many P1, P2, ..., bug records are          outstanding?  
 
padding-left:2px; padding-right:2px; font-family:times; ">R</span> Summarize the bugzilla situation. How many bug          records (defects and enhancements) have been          opened/closed/deferred/new, etc? How many P1, P2, ..., bug records are          outstanding?  
Line 60: Line 59:
 
<i><b><font color="#0080C0">Reason:</font></b></i> Summaries of the bugzilla records offer a glimpse              into the project productivity. They also offer an estimate of the              outstanding risk. And the summary is used to alert the community              to known issues.              (Reference [http://www.eclipse.org/projects/dev_process/development_process.php#2_4_Eclipse_Ecosystem])
 
<i><b><font color="#0080C0">Reason:</font></b></i> Summaries of the bugzilla records offer a glimpse              into the project productivity. They also offer an estimate of the              outstanding risk. And the summary is used to alert the community              to known issues.              (Reference [http://www.eclipse.org/projects/dev_process/development_process.php#2_4_Eclipse_Ecosystem])
  
====(2.7) Standards====
+
==Standards==
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px;  
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px;  
 
padding-left:2px; padding-right:2px; font-family:times; ">R</span> Summarize the standards compliance of this release.          If the features are based on defined, pending, or ad-hoc standards,          what is the state of those standards and what is the state of the          support for those standards in this release.
 
padding-left:2px; padding-right:2px; font-family:times; ">R</span> Summarize the standards compliance of this release.          If the features are based on defined, pending, or ad-hoc standards,          what is the state of those standards and what is the state of the          support for those standards in this release.
Line 66: Line 65:
 
<i><b><font color="#0080C0">Reason: </font></b></i>Eclipse is about building frameworks and tools              based on standards, so we need to make sure that we are conforming              to the appropriate standards.              (Reference [http://www.eclipse.org/projects/dev_process/development_process.php#2_4_Eclipse_Ecosystem])
 
<i><b><font color="#0080C0">Reason: </font></b></i>Eclipse is about building frameworks and tools              based on standards, so we need to make sure that we are conforming              to the appropriate standards.              (Reference [http://www.eclipse.org/projects/dev_process/development_process.php#2_4_Eclipse_Ecosystem])
  
====(2.8) UI Usability====
+
==UI Usability==
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px;  
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px;  
 
padding-left:2px; padding-right:2px; font-family:times; ">R</span> Summarize the user interface usability and the conformance to the [[User_Interface_Guidelines]]. Include section 508 compliance, language pack conformance (does the code          support multiple languages), etc. Explain any deviations from the          user interface guidelines and standards.
 
padding-left:2px; padding-right:2px; font-family:times; ">R</span> Summarize the user interface usability and the conformance to the [[User_Interface_Guidelines]]. Include section 508 compliance, language pack conformance (does the code          support multiple languages), etc. Explain any deviations from the          user interface guidelines and standards.
Line 72: Line 71:
 
<i><b><font color="#0080C0">Reason:</font></b></i>  The user              community is larger than just mouse-wielding, English-speaking,              computer jockeys. We need to support that larger community.          (References [http://www.eclipse.org/projects/dev_process/development_process.php#2_2_Quality_Culture],[[User_Interface_Guidelines]])
 
<i><b><font color="#0080C0">Reason:</font></b></i>  The user              community is larger than just mouse-wielding, English-speaking,              computer jockeys. We need to support that larger community.          (References [http://www.eclipse.org/projects/dev_process/development_process.php#2_2_Quality_Culture],[[User_Interface_Guidelines]])
  
====(2.9) Schedule====
+
==Schedule==
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px;  
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px;  
 
padding-left:2px; padding-right:2px; font-family:times; ">R</span> Discuss the initial schedule and any changes to the          schedule over the course of the release, i.e., what the project team          achieved. Discuss whether milestones were met or slipped.
 
padding-left:2px; padding-right:2px; font-family:times; ">R</span> Discuss the initial schedule and any changes to the          schedule over the course of the release, i.e., what the project team          achieved. Discuss whether milestones were met or slipped.
Line 78: Line 77:
 
<i><b><font color="#0080C0">Reason:</font></b></i> The community relies on consistent schedules from              Eclipse so that projects and products can plan for the correct              dependencies.          (Reference [http://www.eclipse.org/projects/dev_process/development_process.php#2_3_Collective_Reputation])
 
<i><b><font color="#0080C0">Reason:</font></b></i> The community relies on consistent schedules from              Eclipse so that projects and products can plan for the correct              dependencies.          (Reference [http://www.eclipse.org/projects/dev_process/development_process.php#2_3_Collective_Reputation])
  
====(2.10) Communities====
+
==Communities==
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px;  
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px;  
 
padding-left:2px; padding-right:2px; font-family:times; ">R</span> Summarize the project's development of its three          communities. Consider the interactions on bugzilla, the          mailing lists, the newsgroups, public conference calls, blogs, PR          activities, code camps, conference tutorials, coordinating with other          Eclipse projects and other open source projects (Apache, ObjectWeb,          etc), ...
 
padding-left:2px; padding-right:2px; font-family:times; ">R</span> Summarize the project's development of its three          communities. Consider the interactions on bugzilla, the          mailing lists, the newsgroups, public conference calls, blogs, PR          activities, code camps, conference tutorials, coordinating with other          Eclipse projects and other open source projects (Apache, ObjectWeb,          etc), ...
Line 84: Line 83:
 
<i><b><font color="#0080C0">Reason:</font></b></i> It is important for Eclipse projects to build a              community around the project, not just deliver code for a project.              This review item is about the success of building a community.          (Reference [http://www.eclipse.org/projects/dev_process/development_process.php#2_5_Three_Communities])
 
<i><b><font color="#0080C0">Reason:</font></b></i> It is important for Eclipse projects to build a              community around the project, not just deliver code for a project.              This review item is about the success of building a community.          (Reference [http://www.eclipse.org/projects/dev_process/development_process.php#2_5_Three_Communities])
  
====(2.11) IP Issues====
+
==IP Issues==
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px;  
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px;  
 
padding-left:2px; padding-right:2px; font-family:times; ">R</span> As per the [http://www.eclipse.org/org/documents/Eclipse_IP_Policy.pdf Eclipse IP Policy], these steps must be done:
 
padding-left:2px; padding-right:2px; font-family:times; ">R</span> As per the [http://www.eclipse.org/org/documents/Eclipse_IP_Policy.pdf Eclipse IP Policy], these steps must be done:
Line 107: Line 106:
  
  
<h3>(2.12) IP Issues Speak-Up-Now</h3>
+
==IP Issues Speak-Up-Now==
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px; padding-left:2px; padding-right:2px; font-family:times; ">R</span> The EMO explicitly asks during the Release          Review if any Member would like to assert that this release infringes          their IP rights. If so, the EMO and the project will follow the          Eclipse IP Policy in discussions with that Member.  
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px; padding-left:2px; padding-right:2px; font-family:times; ">R</span> The EMO explicitly asks during the Release          Review if any Member would like to assert that this release infringes          their IP rights. If so, the EMO and the project will follow the          Eclipse IP Policy in discussions with that Member.  
  
Line 114: Line 113:
 
(Reference [http://www.eclipse.org/org/documents/Eclipse_IP_Policy.pdf])
 
(Reference [http://www.eclipse.org/org/documents/Eclipse_IP_Policy.pdf])
  
====(2.13) Project Plan====
+
==Project Plan==
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px; padding-left:2px; padding-right:2px; font-family:times; ">R</span>  Per Board resolution of December 11, 2008, to pass a Continuation Review, Graduation Review or Release Review a project must have a current project plan, in the format specified by the EMO, available to the community.
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#FF9999; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px; padding-left:2px; padding-right:2px; font-family:times; ">R</span>  Per Board resolution of December 11, 2008, to pass a Continuation Review, Graduation Review or Release Review a project must have a current project plan, in the format specified by the EMO, available to the community.
  
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#00CC99; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px; padding-left:2px; padding-right:2px; font-family:times; ">G</span>If there is a Project Plan (full or even a draft)          for the next release, the final issue to cover in the Release Review          is the unveiling of the new plan.
 
<span style="margin-right:6px; margin-top:5px; float:left; color:ivory; background:#00CC99; border:1px solid #444; font-size:30px; line-height:25px; padding-top:2px; padding-left:2px; padding-right:2px; font-family:times; ">G</span>If there is a Project Plan (full or even a draft)          for the next release, the final issue to cover in the Release Review          is the unveiling of the new plan.
  
===(3) Preparing the Docuware===
+
=Preparing the Docuware=
 
[[Development_Resources/FAQs/About_Docuware|About Docuware]]
 
[[Development_Resources/FAQs/About_Docuware|About Docuware]]
  
Line 126: Line 125:
 
Example docuware from previous Release Reviews can be found in the [http://www.eclipse.org/projects/previous-release-reviews.php archives]. A [http://wiki.eclipse.org/images/4/4a/ReleaseReview20081211.zip template] (Open Office Presentation format) is also available.
 
Example docuware from previous Release Reviews can be found in the [http://www.eclipse.org/projects/previous-release-reviews.php archives]. A [http://wiki.eclipse.org/images/4/4a/ReleaseReview20081211.zip template] (Open Office Presentation format) is also available.
  
===(4) The Review Process===
+
=The Review Process=
 
Once the review content has been created, the operational side of a Release Review is described here: [[Development_Resources/HOWTO/Review_Process]]
 
Once the review content has been created, the operational side of a Release Review is described here: [[Development_Resources/HOWTO/Review_Process]]
  
===(5) After a Successful Review===
+
=After a Successful Review=
 
After the successful advisory vote and the approval of the Release Review by the EMO:
 
After the successful advisory vote and the approval of the Release Review by the EMO:
 
# the EMO will send an email to the project development list affirming the positive result
 
# the EMO will send an email to the project development list affirming the positive result
 
# then, when the final release files are ready, the project may post them to the download server and let the powerful replication system distribute the bits throughout the known universe
 
# then, when the final release files are ready, the project may post them to the download server and let the powerful replication system distribute the bits throughout the known universe
  
====(5.1) Other Results====
+
==Other Results==
 
Other possible results of the Release Review include:
 
Other possible results of the Release Review include:
 
*          Postpone -&nbsp;issues have arisen that must be solved before the          release proceeds  
 
*          Postpone -&nbsp;issues have arisen that must be solved before the          release proceeds  
Line 141: Line 140:
 
*          Fail - there are significant problems with the release and/or the          project.  
 
*          Fail - there are significant problems with the release and/or the          project.  
  
''This page is moderated by Anne Jacko and Bjorn Freeman-Benson (Eclipse Foundation)''
+
''This page is moderated by Anne Jacko and Wayne Beaton (Eclipse Foundation)''
 
[[Category:Development_Resources]]
 
[[Category:Development_Resources]]
 
[[Category:How to Contribute]]
 
[[Category:How to Contribute]]

Revision as of 21:03, 12 May 2009

See "6.3.3 Release Review" and "6.3 Reviews" in the Eclipse Development Process
The purposes of a Release Review are: to summarize the accomplishments of the release, to verify that the IP Policy has been followed and all approvals have been received, to highlight any remaining quality and/or architectural issues, and to verify that the project is continuing to operate according to the Princples and Purposes of Eclipse.

Contents

What are the Requirements?

RThe only requirements pertaining to a Release Review is that the review be held and be successfully and all remedial actions completed before the Project's official release.

The Release Review is conducted before each major release to verify that the key goals of the release have been accomplished and to verify that all intellectual property rights issues have been resolved. For Eclipse projects with typical six week milestones, a plausible time for a release review is one to two weeks into the final six week period, i.e., four or five weeks before the release is scheduled to go live.

What Does a Release Review Cover?

The Release Review is the final review of the project, process, release and most importantly, the IP issues, against the characteristics that help ensure Eclipse Quality.

A Release Review is a fairly comprehensive process. We anticipate that gathering the material for the review and preparing the docuware is a non-trivial effort, but we believe that the introspection offered by this exercise is useful for the project and that the results are very useful for the community.

The items to be covered in the Review include the following. If the docuware is a slide deck, it will probably average one or two slides on each of these issues. (Note the use of the word "summarize" throughout this agenda. Judgment must be used to determine how much to include in each of these bullet items.)

Features

R Summarize the major features of this release as well as any other features that have generated significant discussion amongst the community during the development cycle. Compare the features against the Roadmap to understand the project's conformance or divergence.

Reason: The community will use this release and the ecosystem will build products on top of this release, and both need to know what features were included or excluded.

GReferences to existing New & Noteworthy documentation is a useful addition to this summary. (Reference [1])

Non-Code Aspects

R Summarize the state of the non-code aspects of the release including: user documentation, localization/externalization, examples, tutorials, articles, and so on. Have the existing artifacts been updated? Are there new artifacts? Have the obsolete ones been retired or at least marked as pertaining only to older material?

Reason: The non-code aspects are essential for the wide-spread adoption of the release. (Reference [2])

APIs

R Certify that the APIs in this release are Eclipse Quality. The project lead will personally certify that the requirements for quality have been met and/or discuss any deficiences.

Reason: Eclipse members build commercial tools on top of the extensible frameworks and thus the quality of the APIs is extremely important. (Reference [3])

Architectural Issues

R Summarize the architectural quality of the release. Discuss the intrinsic nature of being extensible embodied by this project. Discuss issues such as unresolved overlap with other projects, unpaid "merge debt" from incorporating various components, and so on.

Reason: Eclipse members build commercial tools on top of the extensible frameworks and thus the quality of the architecture is important. (Reference [4])

Tool Usability

R Summarize the usability of the tools. Usability in this sense is about using the tools to solve development problems, not the more academic sense of UI evaluation/testing.

Reason: Without usable tools, the project will not attract the user community necessary to enable the ecosystem.  (Reference [5])

End-of-Life

R Summarize the features (APIs and any significant user features) from previous releases that are being end-of-life'd in this release. End of life includes both deprecation and actual removal.

Reason: The community builds products that rely on features and so they need to know when these features are changing. (Reference [6])

Bugzilla

R Summarize the bugzilla situation. How many bug records (defects and enhancements) have been opened/closed/deferred/new, etc? How many P1, P2, ..., bug records are outstanding?

Reason: Summaries of the bugzilla records offer a glimpse into the project productivity. They also offer an estimate of the outstanding risk. And the summary is used to alert the community to known issues. (Reference [7])

Standards

R Summarize the standards compliance of this release. If the features are based on defined, pending, or ad-hoc standards, what is the state of those standards and what is the state of the support for those standards in this release.

Reason: Eclipse is about building frameworks and tools based on standards, so we need to make sure that we are conforming to the appropriate standards. (Reference [8])

UI Usability

R Summarize the user interface usability and the conformance to the User_Interface_Guidelines. Include section 508 compliance, language pack conformance (does the code support multiple languages), etc. Explain any deviations from the user interface guidelines and standards.

Reason: The user community is larger than just mouse-wielding, English-speaking, computer jockeys. We need to support that larger community. (References [9],User_Interface_Guidelines)

Schedule

R Discuss the initial schedule and any changes to the schedule over the course of the release, i.e., what the project team achieved. Discuss whether milestones were met or slipped.

Reason: The community relies on consistent schedules from Eclipse so that projects and products can plan for the correct dependencies. (Reference [10])

Communities

R Summarize the project's development of its three communities. Consider the interactions on bugzilla, the mailing lists, the newsgroups, public conference calls, blogs, PR activities, code camps, conference tutorials, coordinating with other Eclipse projects and other open source projects (Apache, ObjectWeb, etc), ...

Reason: It is important for Eclipse projects to build a community around the project, not just deliver code for a project. This review item is about the success of building a community. (Reference [11])

IP Issues

R As per the Eclipse IP Policy, these steps must be done:

  1. The project leadership verifies that:
    • ... that the about files and use licenses are in place as per the Guidelines to Legal Documentation
    • ... all contributions (code, documentation, images, etc) has been committed by individuals who are either Members of the Foundation, or have signed the appropriate Committer Agreement. In either case, these are individuals who have signed, and are abiding by, the Eclipse IP Policy.
    • ... that all significant contributions have been reviewed by the Foundation's legal staff. Include references to the IPZilla numbers of all clearances.
    • ... that all non-Committer code contributions, including third-party libraries, have been documented in the release and reviewed by the Foundation's legal staff. Include references to the IPZilla numbers of all clearances.
    • ... that all Contribution Questionnaires have been completed
    • ... the "provider" field of each feature is set to "Eclipse.org"
    • ... the "copyright" field of each feature is set to the copyright owner (the Eclipse Foundation is rarely the copyright owner).
    • ... that any third-party logos or trademarks included in the distribution (icons, help file logos, etc) have been licensed under the EPL.
    • ... that any fonts or similar third-party images included in the distribution (e.g. in PDF or EPS files) have been licensed under the EPL.
  2. The PMC provides a Project Log that enumerates:
    1. every piece of third party software including information on the license
    2. every major contribution
    3. the name of every contributor including non-committer contributions via bug fixes with bug #'s
    4. the About files which contain any non-standard terms (e.g., a reference to a license other than the EPL, etc)
  3. The EMO will validate for (a) and (b) that Contribution Questionnaires have been properly submitted and EMO approvals have been completed.
  4. A frozen copy of the reviewed-and-approved-by-Eclipse-legal Project Log is part of the Release Review documentation. It can be included in the docuware or as a separate document. (Reference [12])


IP Issues Speak-Up-Now

R The EMO explicitly asks during the Release Review if any Member would like to assert that this release infringes their IP rights. If so, the EMO and the project will follow the Eclipse IP Policy in discussions with that Member.

Reason: One of the important benefits that the Eclipse Foundation provides for its members is the consistent application of the Eclipse IP Policy which helps ensure (but does not guarantee) that the framework and tools are useable in commercial products.

(Reference [13])

Project Plan

R Per Board resolution of December 11, 2008, to pass a Continuation Review, Graduation Review or Release Review a project must have a current project plan, in the format specified by the EMO, available to the community.

GIf there is a Project Plan (full or even a draft) for the next release, the final issue to cover in the Release Review is the unveiling of the new plan.

Preparing the Docuware

About Docuware

R
The docuware must cover the issues from (2) above.

Example docuware from previous Release Reviews can be found in the archives. A template (Open Office Presentation format) is also available.

The Review Process

Once the review content has been created, the operational side of a Release Review is described here: Development_Resources/HOWTO/Review_Process

After a Successful Review

After the successful advisory vote and the approval of the Release Review by the EMO:

  1. the EMO will send an email to the project development list affirming the positive result
  2. then, when the final release files are ready, the project may post them to the download server and let the powerful replication system distribute the bits throughout the known universe

Other Results

Other possible results of the Release Review include:

  • Postpone - issues have arisen that must be solved before the release proceeds
  • Pass w/ Notes - the release has some issues that need to be addressed, but they can be addressed with release notes and/or documentation
  • Pass w/ Issues - the release has some issues, but they can be addressed in a future release as agreed to in the review
  • Fail - there are significant problems with the release and/or the project.

This page is moderated by Anne Jacko and Wayne Beaton (Eclipse Foundation)