This page is to list plans, questions, policies, use-cases, and recommendations for how to approach the 3.7 versus 4.1 platform in the Indigo release.
Note that this is a "rough notes" page, to serve as a straw-man statement, to assist the Planning Council in flushing out these issues. Many statements here are "speculation" and may change significantly as more is learned. This page will be updated as questions are answered, new issues listed, and possibilities turns into plans.
As most readers know, in June 2011 there will be two versions of the Platform released: 3.7 and 4.1. 3.7 is intended to be the most stable, production-proven version, which some adopters (many?) will need to use to "ship product" on, especially if they use non-API from the Platform, which may not be supported in 4.1. 4.1, though, is seen as the future of Eclipse, becoming the primary (only) version release stream in years to come. While it is anticipated to be production-ready by June 2011, it is not production ready as of this writing, so there is some risk to "putting all our eggs" in this one basket.
This puts us in a "transition" phase for the Simultaneous Release and this page is to list issues (and eventually plans) on how to accomplish this transition. Please feel free to add issues or questions (via your Planning Council representative).
Only one stream
Of course, one option is just to pick one stream, and put all the focus there: either 3.7, implying 4.1 is still the "experimental" stream (which would not be good for 4.1), or focus on 4.1, but this might lose support from many adopters that still need a solid release on 3.7. It it thought some form or "both streams in the release" is required.
Completely split streams
One option is to simply have two completely different "streams", different repositories, different EPP packages, even different release "names". This simplifies some things, but a) should not be required (as far as is known) and b) is a prohibitive amount of extra work for contributing projects (essentially doubling workload). Also, this option might also make a negative impression about the perceived compatibility of the two platforms -- they really should be nearly "the same thing" from the compatibility point of view of running code.
One, compatible stream
Since 4.1 will have a compatibility layer ... so that code written to 3.x API will continue to run on 4.1 Platforms .... it is feasible that the Indigo release can consist of one set of bundles, that runs on both 3.7 and 4.1. This partially depends on a) how good the compatibility layer is, and b) how many non-api are used by contributing projects. This is likely the option that will take least amount of work from contributing projects, so will be the focus of the rest of these notes. The success of this option will depend on getting some quick answers on if it is feasible, or not, so will will be requesting that participating projects "report back" on their ability to participate not only in a "Simultaneous Release" but also a "Simultaneous Transition".
Question: If one were to do an API scan to check compatibility, what is (are) the key bundles from which to not use non-API? For example, someone may use non-API from, say, JDT ... and while that's not good, seems it'd be good to know where to put the focus?
Mostly one, compatible stream
It was pointed out in 8/18 meeting that some people could have a 4.1 only function or bundle and should be fine, exist in repo, etc. Could even, in theory, say they do not support 3.7 ... especially for incubating or leaf components. But we'll be consistent with EPP packages and say those are 3.7 only (and must be ... to participate there).
Binary compatibility between 3.7 and 4.1
Note that in this context, "4.1 Platform" means "4.1 Platform plus the compatibility layer" (if someone had a "pure" 4.1 application, they would not need the compatibility layer ... but we are assuming for now nearly everyone will need or use the compatibility layer).
It is assumed that the two platform's are binary compatible, so that if code is compiled for 3.7, is can be ran on 4.1.
Recommended requirement: Code for delivery is compiled against 3.7 platform. Naturally, it is good practice to also compile against 4.1 so that the compiler can catch compile-time compatibility errors, but for consistency and easy of maintenance, we'd need consistency on compilation for the code that is delivered for release.
In theory, should be able to compile against in 4.1 and produce identical binaries as compiling against 3.7 ... but we'll recommend 3.7 anyway, for consistency, and repeatability, etc. In addition to "should compile against 4.1 to catch errors early" we will specify "should run API scan tools, especially looking for "migration report" against org.eclipse.ui.workbench
Implication: (or assumption?) Implicit in this and other assumptions is that no one makes use of new 4.1 specific platform features ... except the platform itself. Is this a safe assumption?
Leaf and non-EPP components should be fine to use 4.1 ... more discussion needed if "in betweens" used (optional) 4.1 function (such as JDT). For example, would that double work that WTP had to do? More likely to break WTP?).
Question: What are recommendations for pre-req version ranges? Does anyone have to "span" major releases? Such as specify [3.4, 5.0) (or which bundles?) or is this handled by compatibility layer? Do new, 4.x bundles have new names? Or same names, new version?
The new versioning is 'major change' only in feature level. If someone really did want to restrict bundles to 3.7 only they maybe could specify an upper range of [3.6, 3.100) ... but usually best to restrict by specifying a feature level constraint.
One common repository
It is assumed there can be just one Indigo repository that has both platform's, plus all the compatible bundles. This seems pretty important for at least the "common" repository. Note we currently "point to" the Platform's repository via a composite repo file ... so, one question is if the Platform itself will be one repository (recommended) or two?
It is important that the "root features" used for EPP packages not have dual stream versions in one repo ... or else people will be offered to upgrade and get something pretty different
Need to writeup advice for others on using care with "root features" ... such as "Platform" or "RCP" should not be a root feature, if you do not want an update to 4.1 offered
Platform team will investigate if they can work with one repo, do their builds from one repo, etc. Then they'll discuss and ask on cross project list if other teams (still) need dual repos/zips for their own build? Or if they can make do with the Platform's one repo?
It would greatly assist user adoption, and committer testing and development, if they could use one workspace or at least their same of projects with each platform.
No way one workspace was going to get any buy in ... but, we might get 'cross version tolerance' at "project level" and this would (still) aid people using/trying both Platforms. DW to writeup proposed statement similar to platform's read-me on the topic.
Plan A. One workspace
It is assumed (required) that users can use 4.1 platform or the 3.7 platform and all their data, metadata, and preferences would be compatible between the two. This would be "round trip" for data or preferences that existed in both platforms. For new features/data that exist in only 4.1, then 3.7 based versions should ignore or gracefully handle things that it did not recognize. This scenario should be tested (to some degree) by all projects, and community, and bugs in this area given priority.
Plan B. One set of projects, two workspaces
If having literally one workspace is not technically feasible, then we should provide instructions for how to 'share projects' between workspaces using linked resources. This still implies project metadata is completely compatible with both Platforms, but the workspace-wide metadata itself would not need to be.
Plan C. Improve "sharing" functions
If neither of the above are possible, we might want to consider improving or providing some new "import/export" functions that would make it easier to move back and forth from one IDE to another, using the same projects or preferences.
This issue is complicated by use-cases needed by adopters, in addition to the "normal" one of what we provide to the general Eclipse user. Some adopters of course have their own, standalone products, which would not be effected by EPP packaging. But, sometimes, some adopters have a "product" that simply installs into existing installations of Eclipse, so they need some simple, fool proof way of telling their users what to install before installing their product. In addition, even those with "stand alone products", often support the scenario where users already have Eclipse or an EPP package installed, and then install the product into that existing version. We need to assess and test these various use cases.
In general, it was felt the focus for EPP packages would be 3.7 only based.
Plan A: one package for both platforms
Ideally, it is desirable that both 3.7 and 4.1 Platform plugins could be present and the one used would be a function of a start-up parameter or "product" definition, so a particular EPP package could have 2 product definitions, and users could control which platform was used for their start up. (The default one would be chosen by EPP project or package maintainer).
This was seen as technically possible, perhaps. But would need a fair amount of investigation, a lot of testing, perhaps a few bugs/features fixed (e.g. should be able to have two products installed, and two profiles, .... but ... two p2 configuration areas?). Plus, would change the install "layout" in ways that people might not like, might have to react to. Plus, might not be as "user friendly" as anticipated ... for example, if someone was running on "3.7 mode" and updated some feature or applied a patch ... they'd have to go into "4.1 mode" and update the same feature again for it to "take" in both. Plus, there's no particular practical application for this outside of EPP packages. That is, we don't anticipate any adopter would ever want to run or ship product in this way.
Plan B: one package for one platform
Let's say the "package owner" (with input from community and committers) would decide which Platform to use in their EPP package.
Plan C: two package for each platform
This is one possibility, but would essentially double diskspace (and impact the mirroring system) and may add to some confusion about "which to download", instead of providing a simple "get going quick" web page.
Plan D: two packages for each platform, but mirror only one
This was mentioned as a possibility ... to produce both, but one would be mostly for testing and early adopters and not mirrored ... available from a separate download page, so avoid confusion. Its a possibility ... but ... not too much interest.
Plan E: One package, but with easy upgrade to 4.1
This had most interest. One solution is simply to have a set of EPP package features available (in a separate repo) that were just like the others, except pre-req'd 4.x platform. Has the nice aspect of minimizing duplicated diskspace. Would have to test if can be "undone" ... should be possible ... but should be tested. Markus agreed to investigate.
Participating projects would be expected to test on both platforms (projects may emphasize one more than the other ... but expected to test on both to some extent).
No splitting steams for compatibly
Is is assumed (and would be required?) that no one splits streams to have a 3.7 based version and a 4.1 based version simply due to compatibility issues. For example, if it was discovered some coded used some non-API in the 3.x based versions, then the "fix" is not to have a new stream based on 4.1 using some other non-API ... but instead for participating projects to fix their code to use only API and/or the Platform to provide a new API that worked in both streams.
The case of splitting streams to make use of new API that is only in 4.1 is a slightly different case. In theory, that should be ok ... but, intuitively would seem to complicate things in a multiplicative fashion, not simply additive. This will have to be reassessed if we learn participating projects do have 4.1 specific function they want to take advantage of?
The dates chosen below are based on our regularly scheduled Planning Council meetings ... first Wednesday of the month.
- September 1: have a draft-plan addressing this issue, from Planning Council, with PC member agreement on direction
- October 6: update plan based on feedback from projects. Projects should state if they agree or disagree with direction. Hopefully some "empirical" validation that it will work. (For example, if a project found they used non-compatible internals (non-APIs) in thousands of lines of code, that might mean they could not commit to supporting a compatibility based transition plan.
- November 3: update plan based on experience gained.
- December 1: finalize plan, to be part of 2011 road map for Board Review.
M3: basics in place for common repo to have both Platform milestones. EPP packages begin implementing "dual platform" configurations (say, at least for a few packages, for wider testing, before committing to all packages?)
M4: dual platforms in place in repository and EPP packages. All participating projects test on both platforms (to some degree).
Naming, marketing and messaging
The planning council has nothing to do with "marketing" Eclipse ... but the following statement may help clarify our plan.
It is assumed it suffices to have one release name, Indigo. When there is a need to distinguish "Indigo with 4.1 Platform" from "Indigo with 3.7 Platform", it is recommended to use a name variants, such as "Indigo Major" and "Indigo Minor", or perhaps Indigo4 versus plain 'ol Indigo?. For this transition period, we want a naming or labeling scheme that emphasizes they are "nearly the same thing". While it is true a key component is different, nearly all the code is the same and (hopefully) they will be practically interchangeable. There might be some differences that some adopters and users might care about ... such as one might have 'capabilities' defined and the other might not ... but the casual user (hopefully) should be able to use either, and their choice may depend simply on which "look and feel" they prefer. We don't want to literally use "3.7" or "4.1" in the name, for the same reasons we wanted a "word" to name the release to begin with. Also, we don't want to use completely separate names, such as "Indigo" vs. "Janus" since that implies "completely different" things, instead of the "nearly the same thing" we want to encourage.
Some other (good) terminology I've heard recently would be to refer to 3.7 as the "primary platform" and 4.1 as the "secondary platform". This is similar to how we refer to OSes and VMs ... primary are used/tested, secondary are "should run" cases, but have not really been tested, or tested well.
Can we require that bugs against a project (say, webtools) running on 4.1 must be accepted as valid bugs? (Or, could participating projects say "oh, we don't support that platform and only accept bugs when running on 3.7"? (Note, we never dictate what has to be fixed, what priorities are ... this is just about if it is a valid bug).
I suspect we can require that projects document their support of 4.1: with several options; a) not supported at all, b) tested as secondary platform, c) will investigate bugs on 4.1, if problem can not be reproduced on 3.7 ... or similar. What would adopters/projects find useful?
Improve pre-req startup error messages
Some have expressed concern that, with having two platform's to choose from, there may be "adopter problems" if a commercial product, say, only supports one but not the other, Lets say they support only 3.7 platform, but someone tries to run/install into a 4.1 Platform. In the worse case, this may "appear" to run for a while, and problems only show up later, at which point it would be complicated to "move back" to a previous level. In the best case, there would be a bunch of vague OSGi low level messages about certain bundles not being available, or not installable, etc. You know, those messages that no casual user can understand. Is it possible to improve the errors (or scenario) to that end-users get a better message? Maybe two possibilities; 1) message says "this requires 3.7 platform, install that first"? or 2) "installing this will force your workbench to run (only) the 3.7 platform, do you want to continue"? Or similar? Naturally, here's where terminology is important ... users may think "what's 3.7 ... I just installed 'Indigo version' so what now"? Suggestions? Recommendations on how to improve the handling of this scenario?