Callisto Coordinated Update Sites
Purpose of this document
This page is to document processes and procedures for providing improved coordination of update sites provided by the many Eclipse Projects. It's focus is on the 10 or so projects as part of the Callisto simultaneous release but some of the information would be helpful to any Eclipse adopter.
Note: as of this initial writing, this document should be taken as a 'proposal', and as its reviewed and discussed and improved by others, it will become a 'plan'. And then, eventually, once the plan seems to be working, it will become a true "process and procedures" document.
This document was started because of a cross-project agreement reached at the December, 2005, EMO Architecture Council Meeting. There, through their representatives to the Architecture Council, the projects of the Callisto Release agreed to improve the cross-project update experience. The agreement was that I (David Williams) would coordinate the effort ... but the projects still had to do the work!
This document is in no way to "take over" any of the Eclipse base project Update Team's function, proposals, or responsibilities.
It is also not to add to their responsibilities. The goal for Callisto release is to use and "push the limits" of the current capabilities and technologies of the Update Manager. Of course, bugs and feature requests may result, and that is fine, but I wanted to be clear this proposal is not for anything fundamentally new ... it is just to document what's already possible, and make sure it is coordinated and carried out with "best practices".
There are 3 primary use cases that cross-project, coordinated update sites provide:
1. Allow end-users to install some minimum "platform" and from that be able to use Update Manager to install all of the Callisto release, just by going to just one update site and selecting just one thing.
2. Allow committers and developers to install an "SDK" version of Callisto, to be used while developing their own plugins.
3. Allow adopters to provide their own update sites, and "point to" appropriate sites to pick up prerequisite features. While there is nothing in this Callisto effort to support this directly, Callisto should serve as a good example of "best practices" for other projects to follow.
Objectives and Constraints
Besides promoting the use cases given above, there are other objectives and constraints to meet:
1. The distribution of Eclipse projects must fit in to its current "mirror system" to allow for distributed bandwidth.
2. There should be something of a "central site" that should make it easy to "get started" with "all" of Callisto. But, in the Eclipse spirit of allowing projects to "do their own thing" each project should also have their own discovery (and update) sites as well. This allows the central "get started" site to be one-size fits all, but still allow projects to offer even more, special features such as tools, examples, etc., that may not be part of their "main' deliverables available via Callisto.
3. Some sites (corporations) can establish an "corporate" update site, which may have its own policies about what updates when, etc. The process and procedures here should do nothing to interfere with that capability.
Each project will still have its own disk space and area on Eclipse, and will still be responsible for providing jarred features and plugins there, as they would for their own update site.
The features (and archive tags, if needed) will be specified using the new "find the closest mirror" luxury support provided by the Eclipse.org download.php script.
Issue: We (any volunteers?) should begin to collect approximate sizes and make rough estimates of required bandwidth for various scenarios, to make sure our mirroring system will suffice ... or we should panic now!?
The Get Started Download
There will be one ZIP provided (by the Eclipse Project), that corresponds to the minimum amount of Eclipse that enables the update manager to work. This is, literally, the Eclipse Platform ... NOT including JDT and PDE. This feature will have the Callisto site as its implanted discovery site. It will still have 'eclipse' as its update site as it comes from the Eclipse Project (the update site being where "fixes" can be obtained automatically by update manager). All other features, from all other projects, will have their own project's discovery and update sites implanted in their features, not the Callisto site). The few features available from the central Callisto site will have discovery sites for all 10 Callisto projects.
Issue: In this day and age, I'm surprised we do not offer "updates" (or "get started") in the form of a WebStart object. I think we do not only due to people constraints ... so, if there is participation from the community (including the community of Callisto projects :) then we might be able to improve the user's experience so they'd never have to see another zip file again?
Central Callisto Sites
For the Callisto release, there will be one central "discovery" site, such as
This one site will have only two features, I propose. Each an "aggregate feature" that only includes other features (no actual code jars itself). One of the features will be the minimum "runtimes" that enables all the functionality of all 10 Callisto projects. The second would be the SDK version of all 10 projects (would include 'source', basic programming examples, etc.).
Issue: I expect this "all or nothing" aspect of this Callisto site proposal to be the most controversial. We will certainly evaluate as we go, but suspect the major driving force will be community feedback. (That is, will Java developers complain if they automatically get C++ related plugins, for example). We (the Eclipse Foundation), do, however, want to make it easier to use all of the Eclipse projects, such as BIRT and TPTP, whose value is often not well understood, if not installed automatically, and made part of a seamless Eclipse experience). Some projects, such as "EMF" and "GEF" are, I believe, exclusively enabling technologies and should never be "visible" to an end user where they have to say they "want" to download EMF or GEF support. The idea is that some developer who wants something very specific, such as GEF and the XSD portion of EMF, will really have no trouble putting the appropriate sites in their update manager and getting exactly what they need, where as more casual users should not even see a hint of the complication of deciding exactly which of the enabling technologies they need. Besides community feedback, we also need to 'get some numbers' and see really how much difference in size there would be.
Issue: We will have to ensure that all licenses are appropriate displayed by update manager, while using this only-a-few-aggregate features strategy. If not, we'll somehow have to come up with one big huge one that covers all aggregated features (yuck).
For Callisto milestones and release candidates, there will be a separate site
All projects must participate in these milestone and release updates. This requires that all projects begin to use qualified plugin versions.
Weekly Integration Update Sites
For those projects wishing to participate, there will be a site for integration builds:
Working Staging Area
Lastly, there will be a working directory of
that is used exclusively by central "Callisto developers" in the days or weeks ahead of the other update "releases" to test that the central features and site.xml is ready before they are copied over to the "public" sites.
Web and User Interface Consistency
All update sites will have an "index" file, appropriate for viewing with a web browser ... and we need to work on giving this a consistent look.
All update sites will provide a "site.xml" file, with a consistent level of abstraction for separately installable features. (Remember, not literally every feature provided by a project has to be "visible" to the update site user ... many can be "included" as being required by other features, and installed automatically when other features selected).
Planned Tests and Trial runs
During February, initial test versions will be created to download "all of Callisto" to see how it does, from several parts of the world. Its not expected these will be "usable" versions of Callisto.
By the end of the M5 period, there will be a usable version of Callisto available via update manager. (This is the version that can (will) be pointed to during EclipseCon, as an example of what's coming).
After M5, there will be updates to the "milestones" site for each Release Candidate leading up to the release.
Note: after M5, projects that elect to use the weekly I-build site can do so. (This part will need some investigation, to be sure this is no great extra work for anyone ... but the idea is that for developers that are building on top of other features, it's very handy to be able to update pre-reqs in a frequent, easy way).
This document and this "coordinated effort" is in no way meant to substitute for the projects themselves understanding update manger and update sites and actively contributing the necessary resources to ensure the correct end result.