Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "SimRel/Simultaneous Release Requirements"

(Re-use and share common third party code)
(Project Metrics)
Line 199: Line 199:
 
=== Project Metrics  ===
 
=== Project Metrics  ===
  
Projects should provide some summary metrics, such as number of bundles, number of committers, lines of code, number of bugs opened and fixed. This is so some statements can be made and tracked year-to-year about the size of the simultaneous release.  
+
Projects should provide some summary metrics, such as number of bundles, number of committers, lines of code, number of bugs opened and fixed. This is so some statements can be made and tracked year-to-year about the size of the simultaneous release.
 +
 
 +
=== Platforms, OSs, and Java Levels ===
 +
 
 +
[added 10/18/2011]
 +
 
 +
We do not have any "requirement" of what platforms or Operating Systems projects must support, mostly because its never been an issue, but simply encourage all projects to well document what platforms they support, especially if they have native (non-Java) code and especially if it is
 +
different than the [http://www.eclipse.org/projects/project-plan.php?projectid=eclipse#target_environments set of platforms supported by the Eclipse Platform itself].
 +
 
 +
Similarly, we have never found a reason to "require" support of a particular minimum (or, maximum) Java level to use (such as Java 5, Java 6, Java 7). But, the question often gets asked, so we simply clarify here projects should document it well (which is, mostly done by stating the BREE level bundle's manifest). But otherwise a) use a level that makes life easy for committers, but b) does not negatively impact adopters (such as some "low level" server code might still need to run on 1.4 JREs), and c) does not "force" others on the release train to use a level higher than they would prefer or be able to accommodate (some language constructs, such as enum, can essentially become part of your API, and force others to move up, but others things, such as syntax of for loops, do not effect your API or downstream consumers since should still run and be API compatible with lower levels).
 +
 
 +
It would be anticipated that this year, for Juno, many Eclipse users would still be using Java 6, a few on Java 5, but projects should test extensively on Java 7 since its use will likely grow and become common place by the time Juno ships, especially over Juno's life time. In other words, in general, write your code to the lowest level possible, but test on the highest level possible.
 +
 
 +
Be sure to raise any "platform level" questions or issues to the cross-project list.
  
 
=== Specify, in Plans, support for auxiliary, or previous Eclipse Platforms  ===
 
=== Specify, in Plans, support for auxiliary, or previous Eclipse Platforms  ===

Revision as of 13:30, 18 October 2011

The Eclipse Simultaneous Release Requirements

Updated October 3, 2011

Authored and maintained by the Eclipse Planning Council

Contact: David Williams

This document defines the rules and criteria for participating in the yearly Simultaneous Release. There are more criteria than when releasing at other times. There are more requirements partially because there are more projects releasing at once, so the workload needs to streamlined and made uniform. But also, the extra criteria are included by mutual agreement between projects (via their representatives to Planning Council) so that as a whole, the release will be of better quality, maintainability, and improved consumability.

The spirit of this document is not be so much as a "contract" of what has to be done to release, but instead a statement of what minimally is necessary to make the Yearly Release good, if not great! While each Project does their individual things to make the Release great, this document describes how we, as a group, do that by our voluntary agreement to participate and provide the minimum requirements. We are always open to better documentation and more meaningful agreements, so please feel to make suggestions on how to make our yearly release better from year to year (preferably through your Planning Council representative). Changes may be made to this document throughout the development cycle for clarity or to improve reference links, but nothing new will be added after M4 (that is, things that would effect workload).

To allow for some flexibility, exceptions to these requirements are allowed, but to provide balance and foster good communication, any exceptions to the criteria or deadlines must follow the Planning Council Exception Process.

The requirements are divided into three categories:

  1. The minimum requirements to be released as part of the "yearly release" ... that is, to able to "claim" you were part of the yearly release . These are the normal release requirements, but done earlier than usual.
  2. The "must do" requirements to be part of the common software repository and, consequently, the minimum requirements to be part of an EPP package.
  3. Requirements to demonstrate good Eclipse Citizenship, following "the Eclipse Way". These are requirements you do not have to do, but they are recommended, encouraged, and the thing that you do "have" to do is to document if and how you do them.

Normal release requirements ... but early

The requirements and conditions stated in this section are the basic minimum required for a project to claim they are part of the yearly Simultaneous Release.

State intent early

To join a Simultaneous Release, Projects must have stated their intent to do so by M4, at the latest. The "statement of intent" is done formally by marking the "Simultaneous Release Flag" in the project's Portal meta-data. For a normal release, a month's notice is typical, but we ask for more advanced planning for the yearly release.

Formal plans, early

All projects must have their project plan in the Eclipse Foundation standard XML Format (a normal Eclipse requirement). Committing to be in the Simultaneous Release means you commit to having these plans early: by M4 at the latest. Naturally, plans will change as development continues, and we encourage teams to update them periodically, such as every milestone, to reflect reality and progress, but an initial version is required by at least M4 and the final version, due by the release in June, should be a clear statement of what was planned, what was achieved, and what was deffered.

IP Documentation

Projects must have their IP Logs approved (a normal Eclipse requirement) and will follow the Eclipse Legal deadlines to do so. The IP log deadline is typically RC2.

Projects are encouraged to created drafts of the Projects IP Logs even earlier. The development process requires the IP Log to always be accurate, but experience shows there often are not, until examined for a release, and there are typically many issues to resolve, or fixed, before the final submission to the IP staff. For example, sometimes a CQ might have the wrong flag, and cause it to not show up in the Auto IP Log, or perhaps a common, already approved 3rd party jar was was used, but no CQ for that specific project entered. The purpose of having these early drafts is so that projects get familiar with what is required, and do not allow work to build up at the end, since that could cause a "bottleneck" at a critical time and jeopardize having the ability to resolve all issues in time to be released. Also, some adopters will want to look at early drafts to see what 3rd party requirements are associated with the code they are planning to adopt.

A good guideline is to have a draft of your IP Log by M5, and plan for it to be complete for M7.

Note that being in the Simultaneous Release will give your IP CQ requests some higher priority in getting evaluated, in order to make the date. The higher priority treatment is only for the 5 months or so before the release (after the deadline for CQs). The reason being, of course, is that the rest of the year the IP staff must also get work done for maintenance releases and projects not on the release train. During that part of the year (roughly July to February every year) all CQs are prioritized in a uniform way. But, there is a deadline for CQs for the yearly release, usually in February, near M5, to have all required CQs submitted, on file (but not necessarily approved by then).

Release Review

The release review archival materials must be complete by the date specified by the EMO, which is usually staged in earlier than for a usual release. (Typically around RC2.)

A Project's PMC must approve the projects request for review (a normal Eclipse requirement). It is recommended, to help organize and streamline the yearly Simultaneous release, that a PMC provide their approval in writing, in the form of a short summary of their projects that are requesting review and summary of the PMC's discussion or method of approving them. (This is meant to be very brief, such as 1/2 page). The short summary can be documented in a mailing list, PMC Meeting notes, or even a wiki document.

More is required to be in common repository

The requirements in this section must be met for a project to be on the common, central repository (e.g. /releases/juno) for end users to discover easily and therefore are the minimum requirements to be included in EPP Packages. The criteria in this section are designed to make sure projects work relatively well, and work well together. This is especially required for adopters who may be using these projects in complicated, interwoven ways so each piece of the puzzle must fit together well and be dependable and be maintainable, as well as being on time and IP clean.

Integrate Early and Often

First-time participants are expected to be in an aggregation build by M4, at the latest. Then, once in, always in. This firstly and mostly means by agreeing to be in the yearly release, in June, you will also participate in the two Service Releases planned. But, even more than that, it is assumed that once you are in one Simultaneous Release, you will continue to be, so the following year, it is assumed you will be in M1 ... that is, you should not wait until M4 every year, even though that is the deadline for first-timers. Put another way, being part of the Simultaneous Release is not a "one time" activity, covering only the release part of the development cycle. Instead it is a commitment to stay "simultaneous" on an on-going basis. Once in, if a project decides to not be part of future simultaneous releases, they need to communicate that widely, and as early as possible, since could effect adopters or downstream projects.

Communication

At least one person from each project in a Simultaneous Release must subscribe to cross-project mailing list, since that is the primary communication channel for issues related to the Simultaneous Release. Also, at least one person from each project must subscribe to cross-project bugzilla inbox, as that is the primary bugzilla components for bugs that are truly cross-project, or bugs which are not known to be in one particular component.

Your representative to the Planning Council, either from PMC or Strategic Member, must attend Planning Council meetings and represent you there. Presumably, of course, after meeting or communicating with you and the other projects they represent, so they can fairly bring forward concerns and vote on issue that effect all projects, if required. Put another way, by committing to be in the Simultaneous Release, you agree to abide by all the Planning Council decisions and rules, so be sure your representative understands your project and your situation.

Build-team members (or their designated alternates) from each project may be asked to provide direct communication channels: at least email (if not phone, IM, IRC) and at least one build team member must be "on call" during the integration periods.

API

Projects should leverage only published APIs of dependencies. All deviations must be documented in bugzillas. These bugzillas may be of the type that a dependent project should provide a required API, or of the type that a consuming project must move to some API that already exists. Note that technically there is no obligation for consumed projects to provide API that is requested ... that depends on many things ... but the main goal of requiring these bugzilla entries is to provide some documentation and measure of the amount of risk associated with non-API use.

Message Bundles

Projects must use Eclipse message bundles unless there are technical reasons not to, since these are known to be more memory efficient that some other forms of handling UI-releated strings.

Version Numbering (tested)

Projects must use 4-part version numbers.

OSGi bundle format

All plug-ins (bundles) must use the true bundle form. That is, provide a manifest.mf file, and not rely on the plugin.xml file being 'translated' into a manifest.mf file at initial startup. With that, empty plugin.xml files in the presence of a manifest.mf file should not be included in a bundle. (For some old history, see bug 130598.)

Execution Environment (tested)

All plug-ins (that contain Java code) must correctly specify their Bundle Required Execution Environment (BREE).

Signing (tested)

Projects must use signed plugins using the Eclipse certificate.

Jarred Bundles

Projects must use jarred plug-ins (with unpack=false) unless there are technical reasons not to do so. Also, nested jars should be avoided if possible since it creates problems for projects that has dependencies to such plug-ins. The OSGi runtime is fine with it but the PDE environment is not able to handle classpaths that contain nested jars. Exceptions to these principles should be documented by the project, so we can learn the reasons and extent of unjarred bundles.

Re-use and share common third party code (partially tested)

Any third-party plug-ins that are common between projects must be consumed via Orbit. The Simultaneous Release must not have duplicate third-party libraries (note that this only applies to versions of the libraries; thus if project A requires foo.jar 1.6 and project B uses foo.jar 1.7, that's normally ok, different service versions a little less ok, such as 1.7.1 vs 1.7.2 (those should be explained, if required), and a qualifier-only difference is definitely not ok).

Note: the "partially tested", for this case, means there is a report of "Non Unique Versions used in repository" which can catch issues of not using common bundles. See current report for an example.

Optimization (tested)

Projects must optimize their p2 repositories to reduce bandwidth utilization and provide a better install and update experience for users.

Provide p2 repository

Projects must provide their own project p2 repository for their own project and updates. In addition, they must provide their archives and metadata in a specified format and method to allow at least parts of their repository to be aggregated and mirrored to a common repository. The current process may be modified throughout the year, if improvements can be made. Clarification on 03/31/2010: Note that a project's repositories must contain original (conditioned) jars, and pack.gz files (where original jar means the jar produced by the build, but which has been conditioned for pack200). Clarification on 11/08/2010: feature "includes" must be strict, that is "include" an exact version of that other feature. This is required so installs and builds can be repeatable independent of the exact day of the install or the exact repos enabled. This is the way things are, and have been for years, and this statement is just making it explicit. While there may, in the future, be new mechanisms that allow some "line up collection" to be specified, it will be something new, not the feature "includes" element.

Support Translations

All strings must be externalized, and Projects must participate in Babel, meaning it is registered and available for string translation, etc. Projects must freeze the UI sufficiently early to allow the Babel project time to translate strings so there can be simultaneous release of translated versions. The UI should be frozen by M6 (a "freeze" all major changes and additions are done by M6, and changes after that are done in a controlled, well documented fashion, so Babel translators can more easily "keep up" with late changes).

Excel in NL support

The Project must use ICU4J, where appropriate, to excel in NL support. (The latest ICU4J bundles will be in Orbit).

Branding

Each major project (as determined by participating PMCs) must have an 'About' dialog icon with hover text that displays the provider name. Every plug-in and feature must specify a descriptive provider-name (for features), or Bundle-Vendor header (for plug-ins), as determined by the project's PMC (e.g. "Eclipse Modeling Project" rather than "Eclipse.org"). Also, Projects must contribute to the welcome page when appropriate.

Do No Harm

Projects must work together in any combination of any install. Put another way, this means that users can install any subset of the projects participating in Simultaneous Release, and each of the installed projects will work as well as if it had been loaded independently. If such a problem is identified, the affected projects must track down and fix the problem.

License text consistency (tested)

Use standard forms of license documents so it is displayed in the most usable, and concise way during install and update. It is a normal requirement to use a standard Eclipse Foundation "about" template, but where those templates are edited by each project, care must be taken to be sure they are edited in similar ways. That is, substantial differences are fine, if required, but we need to avoid minor differences based on case, dates, and formatting. Note that the Eclipse Foundation's license or user agreement files may change from year to year (such as, see bug 316152 but ideally in Indigo and future releases, it will be easier to point to a "symbolic" representation of the license, so it will be accurate with less manual updates from each project (see bug 306818).

Support Primary Eclipse Platform to be in EPP Package.

For Juno, that means EPP packages (and the features and bundles that go into them) must be built on and tested with Eclipse 4.2.

Required for good adoption (but, optional)

Projects should exhibit good Eclipse Citizenship, to Release and participate in Common Discovery Site and EPP Packages. These are often "best practices" that some projects have found helpful at Eclipse. These criteria often speak to the quality of the Project, as an Eclipse Project, as opposed to their code or architecture. They are a bit more subjective than some of the other criteria, and the relevancy to any particular project may not be as universal, so there is no set number of items to satisfy. But, it is required that each project document their level of compliance to each item. Especially good Eclipse Citizens will get a gold star, and especially bad ones might get a frowny face.

Engage Community

The Project should actively engage their community to get feedback on milestone builds, and document how they do that. One way to do this is to have a New & Noteworthy for each milestone. New and Noteworthy documents should be something readable and usable not just a static list of all the bugs. Corollary: individual new & noteworthy should be linked in to the collective New & Noteworthy.

Usability

Should follow the User Interface Guidelines. The UI Checklist is a good place to start. Also, when possible, it is good to have a UI walkthrough.

Performance

Project should have measurable performance criteria that are regularly tested against. Projects should devote at least one milestone to performance and scalability improvements.

Test Localization

The project should use the Babel Pseudo Translation Test to verify their translatability. See bug 217339 [Need better reference link.]

Capabilities

Each project should provide basic capability/activity definitions to allow for their UI contributions to be hidden. These can be provided in a separate plugins and feature to facilitate inclusion and reuse by consumers, or ... as most projects do ... simply document some examples, so adopters can create their own, or reuse via copy/paste. Ideally, projects should also provide triggers to facilitate progressive discovery of functionality (but, not many do, other than the Platform). As with other "good citizen" items, projects are free to document "we don't do this" ... but, then at least it is known and better communicated.

Enable Use with All Languages

Should design and test for enabling all languages including bidi, unicode characters, etc. This is different than "translating" the software. For example, while using an English version of Eclipse Web Tools Platform, someone should be able to create a Chinese language web application. [Need "how to" reference link.]

Builds

Projects must have a mature, stable build process: documented, scripted, repeatable, and executable by others.

Ramp Down Planned and Defined

Projects must have a written ramp down policy by M6, at the latest, and provide link. The plan should describe when the project plans to be feature complete, have API frozen, and similar. See Platform 3.5 Endgame plan as a guideline. See also Indigo Final Daze.)

Accessibility

Projects should design and test for accessibility compliance, following established guidelines and Eclipse fundamental techniques to achieve accessibility. Projects must document their accessibility work and compliance. Ideally this would be by using a publicly available checklists, such as

but, given the advice of the Accessibility Cross Project Team, for this year's Simultaneous Release, projects can document their work or compliance as a negative, such as "we did not do any accessibility work or testing and do not know the degree of our compliance". But its important to document, so adopters know. If possible, and appropriate, accessibility testing tools can be leveraged such as NVDA. The main accessibility article at Eclipse Corner has been made current (thanks goes to Todd Creasey).

[TODO: These entry seems out of date. I recall we used to have some "example templates" that projects could fill out regarding specific areas of compliance/testing ... I need to find those and update this entry].

Unit Tests

Projects must have some unit tests that can verify at least basic functionality of a build or distribution. The steps to build and run the tests must be documented and executable by others.

API Policy

Defined and Documented. Typically would include how 'APIs' are distinguished from non-API and 'provisional' API, if any. It is recommended that non-API be marked with x-internal in the bundles manifest. Also, should include what the commitment is to API, how long maintained after deprecated, etc. As one example, see WTP API Policy.

Retention Policy

Projects should define and document their retention policy. This should include both zip distributions and repositories. For examples, see WTP Retention Policy and Eclipse Project Update Sites.

Project Metrics

Projects should provide some summary metrics, such as number of bundles, number of committers, lines of code, number of bugs opened and fixed. This is so some statements can be made and tracked year-to-year about the size of the simultaneous release.

Platforms, OSs, and Java Levels

[added 10/18/2011]

We do not have any "requirement" of what platforms or Operating Systems projects must support, mostly because its never been an issue, but simply encourage all projects to well document what platforms they support, especially if they have native (non-Java) code and especially if it is different than the set of platforms supported by the Eclipse Platform itself.

Similarly, we have never found a reason to "require" support of a particular minimum (or, maximum) Java level to use (such as Java 5, Java 6, Java 7). But, the question often gets asked, so we simply clarify here projects should document it well (which is, mostly done by stating the BREE level bundle's manifest). But otherwise a) use a level that makes life easy for committers, but b) does not negatively impact adopters (such as some "low level" server code might still need to run on 1.4 JREs), and c) does not "force" others on the release train to use a level higher than they would prefer or be able to accommodate (some language constructs, such as enum, can essentially become part of your API, and force others to move up, but others things, such as syntax of for loops, do not effect your API or downstream consumers since should still run and be API compatible with lower levels).

It would be anticipated that this year, for Juno, many Eclipse users would still be using Java 6, a few on Java 5, but projects should test extensively on Java 7 since its use will likely grow and become common place by the time Juno ships, especially over Juno's life time. In other words, in general, write your code to the lowest level possible, but test on the highest level possible.

Be sure to raise any "platform level" questions or issues to the cross-project list.

Specify, in Plans, support for auxiliary, or previous Eclipse Platforms

For Juno, this means that each project needs to have a section (or theme) in their plans (and, to be clear, that is the standard format plan), on how they intend to support Eclipse 3.8. We will have 4.2 as primary (hence the one used for EPP Packages) but ask participating projects to have a clear plan item titled, exactly, "Support for Eclipse 3.8 workbench" where possible descriptions might be similar to:

  • Not at all. No support for 3.8 based apps.
  • We will accept bugs against 3.8 based apps, but do not test or compile against it.
  • We will compile against and somewhat test 3.8, though 4.2 is primary.
  • We will support 3.8 as well as 4.2, but the exact functionality may differ.
  • We will support 3.8 and 4.2 equally, and the functionality will be the same.

Additional Information

Planning Council Exception Process

Exceptions for any rule or schedule can be made if there are good enough reasons to. This same exception process will be followed for things like "requests to respin" a build after a deadline. The process to get any exception must be open and well documented and follow these steps:

First, the Project's PMC must approve the request for an exception and it is the PMC (not the Project) that makes the request to the Planning Council. The Planning Council member that represents the PMC would bring the issue forward to the Planning Council.

Second, the exception requires at least 3 positive votes from active Planning Council members and no negative votes. When time is a factor (e.g. requests for rebuilds) the deadline for voicing a negative vote is basically by the time 3 votes are documented. But when time is not a factor, such as when requesting an exception to one of the criteria, then a period of one week will pass before being final, to allow times for concerns or negative votes to be voiced even after 3 positive votes. If there are not enough positive votes within one week, then the request for exception will be considered 'failed'. Note that "3" was chosen under the assumption that the Planning Council member representing the PMC would vote for it (since that PMC must approve it initially) so that means 2 others must also vote for it, for 3 total.

Depending on the timing, the issue and votes will be documented in either the Planning Council Meeting minutes, or on the Planning Council mailing list. If possible, some automation may be added to the release reporting tool to aid this documentation.

Testing of Common Repository

Some of the items marked "(tested)" means that there is some degree of automatic checking and reporting done on the common repository. The reports, for example, see the Juno Reports do not test "project by project" but as a whole, give some idea of where there are violations or problems. Ideally, the "common repo" tests should be simply a final sanity check, and projects can include their own automatic testing during their own build processes. See reporting FAQs for more information.

Back to the top