Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "Callisto Coordinated Update Sites"

(updated for consistent "project" placement and noted lack of zip file)
 
(38 intermediate revisions by 7 users not shown)
Line 1: Line 1:
 +
{{Callisto Deprecation Notice}}
 +
<p>
 
== Purpose of this document ==
 
== Purpose of this document ==
  
Line 25: Line 27:
 
1. Allow end-users to install some minimum "platform" and from that be able to use Update Manager to install all of the Callisto release, just by going to just one update site and selecting just one thing.
 
1. Allow end-users to install some minimum "platform" and from that be able to use Update Manager to install all of the Callisto release, just by going to just one update site and selecting just one thing.
  
2. Allow committers and developers to install an "SDK" version of Callisto, to be used while developing their own plugins.
+
<del>2. Allow committers and developers to install an "SDK" version of Callisto, to be used while developing their own plugins.</del>
  
3. Allow adopters to provide their own update sites, and "point to" appropriate sites to pick up prerequisite features. While there is nothing in this Callisto effort to support this directly, Callisto should serve as a good
+
<del>3. Allow adopters to provide their own update sites, and "point to" appropriate sites to pick up prerequisite features. While there is nothing in this Callisto effort to support this directly, Callisto should serve as a good
example of "best practices" for other projects to follow.
+
example of "best practices" for other projects to follow.</del>
  
 
== Objectives and Constraints ==
 
== Objectives and Constraints ==
Line 36: Line 38:
  
 
=== Eclipse.org Infrastructure constraints ===
 
=== Eclipse.org Infrastructure constraints ===
 +
 +
The distribution of Eclipse projects must fit in to the current disk space and "mirror system".
  
 
* Disk space.  
 
* Disk space.  
Line 51: Line 55:
  
  
1. The distribution of Eclipse projects must fit in to its current "mirror system" to allow for distributed bandwidth.
+
=== Coordination verus Unification ===
  
2. There should be something of a "central site" that should make it easy to "get started" with "all" of Callisto. But, in the Eclipse spirit of allowing projects to "do their own thing" each project should also have their
+
For ease-of-use, there should be something of a "central site" that should make it easy to "get started" with "all" of Callisto. But, in the Eclipse spirit of allowing projects to "do their own thing" each project should also have their
 
own
 
own
discovery (and update) sites as well. This allows the central "get started" site to be one-size fits all, but still allow projects to offer even more, special features such as tools, examples, etc., that may not be part of
+
discovery and update sites as well. This allows the central "get started" site to be one-size fits all, but still allow projects to offer even more, special features such as tools, examples, etc., that may not be part of
their "main' deliverables available via Callisto.
+
their "main' deliverables available via Callisto, not to mention have their own schedule for fixes and point releases.  
 +
 
 +
=== Update Policies ===
  
3. Some sites (corporations) can establish an "corporate" update site, which may have its own policies about what updates when, etc. The process and procedures here should do nothing to interfere with that
+
Some sites (corporations) can establish an "corporate" update site, which may have its own policies about what updates when, etc. The process and procedures here should do nothing to interfere with that
 
capability.
 
capability.
  
Line 83: Line 89:
 
</cite>
 
</cite>
  
=== The Get Started Download ===
+
=== The Get-Started Platform Download ===
  
 
There will be one ZIP provided (by the Eclipse Project), that corresponds to the minimum amount of Eclipse that enables the update manager to work. This is, literally, the Eclipse Platform ... NOT including JDT and
 
There will be one ZIP provided (by the Eclipse Project), that corresponds to the minimum amount of Eclipse that enables the update manager to work. This is, literally, the Eclipse Platform ... NOT including JDT and
Line 91: Line 97:
  
 
<cite>
 
<cite>
<b>Issue: </b>In this day and age, I'm surprised we do not offer "updates" (or "get started") in the form of a WebStart object. I think we do not only due to people constraints ... so, if there is participation from
+
<del><b>Issue: </b>In this day and age, I'm surprised we do not offer "updates" (or "get started") in the form of a WebStart object. I think we do not only due to people constraints ... so, if there is participation from
 
the community (including the community of Callisto projects :) then we might be able to improve the user's experience so they'd never have to see another zip file again?
 
the community (including the community of Callisto projects :) then we might be able to improve the user's experience so they'd never have to see another zip file again?
</cite>
+
</cite></del>
  
=== Central Callisto Sites ===
+
<b>Note: </b>We should be explicit that the Callisto Update Site, contrary to the name, must begin with a 3.2 based Platform Feature ... there is no capability planned to update from a 3.1 based environment. Likewise, there are no plans to update from an 3.2 M4 environment to a 3.2 M5 environment. We will plan to test updating from an M5 environment to an M6 environment.
  
==== Release Site ====
+
=== The User's Experience ===  
For the Callisto release, there will be one central "discovery" site, such as
+
http://update.eclipse.org/callisto/updates/
+
  
Note: the URL above, its assumed, will be aliased to the normal download/update area, so that 'callisto' will
+
==== Minimum Runtimes versus full SDK's ====
be a sibling to all the other Eclipse projects on Edlipse downloads.
+
  
Also, we should be explicit, there is no plans to provide a big-huge-zip that contains all the Callisto projects ... its expected to be provided entirely via update manager.  
+
Its been decided that for this Callisto Release, there will not be a  
 +
coordinated discovery site for the "SDK" contributions that projects  
 +
provide. This is partially due to having the focus on simultaneous release itself,
 +
not necessarily what each user audience might need. Plus, as it turns out,
 +
the meaning and design of what's in "runtime" and what's in "SDK" is a little different from
 +
project to project, and will take a longer effort to coordinate an approach that's meaningful to all.  
  
This one site will have only two features, I propose. Each an "aggregate feature" that only
+
But, this actually turns out to be a benefit, of sorts, rather than a limitation.  
includes other features (no actual code jars itself). One of the features will be the minimum "runtimes" that enables all the functionality of all 10 Callisto projects. The second would be the SDK version of all 10
+
The expected workflow is that Callisto users can install minimum "runtimes" to get the required or interesting functionality, and as part of those features installing, they each provide their own implanted update or discovery site URL. Hence, it should be easier for a user to install lots of functionality, but then get only the specific SDK portions they need. This will reduce users on-disk footprint, and reduce overall bandwidth on networks.  
projects (would include 'source', basic programming examples, etc.).
+
  
<cite>
+
However, during development of Callisto, this will make it hard to actually test or exercise this workflow, since the implanted URL's usually refer to official release sites, not the milestone or interim release sites. So, if anyone is interested in testing or using this workflow during Callisto milestones, the following are the milestone or interim sites that projects are currently using.  
<b>Issue: </b> I expect this "all or nothing" aspect of this Callisto site proposal to be the most controversial. We will certainly evaluate as we go, but suspect the major driving force will be community feedback. (That is, will
+
Java developers complain if they automatically get C++ related plugins, for example). We (the Eclipse Foundation), do, however, want to make it easier to use all of the Eclipse projects, such as BIRT and TPTP, whose value is often not well
+
understood, if not installed automatically, and made part of a seamless Eclipse experience). Some projects, such as "EMF" and "GEF" are, I believe, exclusively enabling technologies and should never be "visible" to
+
an end user where they have to say they "want" to download EMF or GEF support. The idea is that some developer who wants something very specific, such as GEF and the XSD portion of EMF, will really have no trouble putting the appropriate sites in their update manager and getting exactly what they need, where as more casual users should not even see a hint of the complication of deciding exactly which of the enabling technologies they need. Besides community feedback, we also need to 'get some numbers' and see really how much difference in size there would be.
+
</cite>
+
  
<cite>
 
<b>Issue: </b> We will have to ensure that all licenses are appropriate displayed by update manager, while using this only-a-few-aggregate features strategy. If not, we'll somehow have to come up with one big
 
huge one that covers all aggregated features (yuck).
 
</cite>
 
  
==== Milestone Site ====
+
http://download.eclipse.org/birt/update-site/site.xml
 +
http://cdt.eclipse.org/callisto/site.xml
 +
http://download.eclipse.org/datatools/updates/site.xml
 +
http://download.eclipse.org/tools/emf/updates/site-interim.xml
 +
http://download.eclipse.org/eclipse/updates/3.2milestones/site.xml
 +
http://download.eclipse.org/tools/gef/update-site/milestones/site.xml
 +
http://download.eclipse.org/technology/gmf/update-site/callisto/site.xml
 +
http://download.eclipse.org/tptp/updates/4.2milestones/site.xml
 +
http://download.eclipse.org/tools/ve/updates/1.0/development/site.xml
 +
http://download.eclipse.org/webtools/milestones/site.xml
 +
 
 +
 
 +
 
 +
==== The View from Update Manager Dialog ====
 +
 
 +
The current plan is, for this Callisto Release, to simply list Features "by Project" [[Image:ExampleDialog.gif]].
 +
 
 +
=== Central Callisto Sites ===
 +
 
 +
==== Release Site ====
 +
For the Callisto releases and milestones, there will be one central "discovery" site, such as
 +
http://download.eclipse.org/callisto/releases
 +
 
 +
Note: Its desriable to avoid any reliance on "aliases" as it makes mirroring more difficult to manage.
 +
 
 +
<del>Also, we should be explicit, there is no plans to provide a big-huge-zip that contains all the Callisto projects ... its expected to be provided entirely via update manager.</del> There has been some recent talk of providing "big zips" made easy to get by the use of a form on a web page, and then the user would get, in one download, a zipped up version of their selections. If this is done, would be handled by the Eclipse Foundation (not this cross-project group).
  
For Callisto milestones and release candidates, there will be a separate site
 
http://update.eclipse.org/updates/callisto/milestones
 
All projects must participate in these milestone and release updates. This requires that all projects begin to use qualified plugin versions.
 
  
  
Line 132: Line 152:
  
 
For those projects wishing to participate, there will be a site for integration builds:
 
For those projects wishing to participate, there will be a site for integration builds:
  http://update.eclipse.org/updates/callisto/weeklies
+
  http://download.eclipse.org/callisto/interim
 
+
  
 
==== Working Staging Area ====
 
==== Working Staging Area ====
  
 
Lastly, there will be a working directory of
 
Lastly, there will be a working directory of
  http://update.eclipse.org/updates/callisto/staging
+
  http://download.eclipse.org/callisto/staging
 
that is used exclusively by central "Callisto developers" in the days or weeks ahead of the other update "releases" to test that the central features and site.xml is ready before they are copied over to the "public" sites.
 
that is used exclusively by central "Callisto developers" in the days or weeks ahead of the other update "releases" to test that the central features and site.xml is ready before they are copied over to the "public" sites.
 
  
 
== User Interface Consistency ==
 
== User Interface Consistency ==
Line 165: Line 183:
 
There's currently great inconsistency here. I guess as long as its permanent, and long term, there's no harm. But looks sloppy.
 
There's currently great inconsistency here. I guess as long as its permanent, and long term, there's no harm. But looks sloppy.
  
For example, GEF is in Eclipse, no update site of its own.
+
For example, GEF is in Eclipse, and has no update site of its own. [[https://bugs.eclipse.org/bugs/show_bug.cgi?id=128348 128348]]
  
 
Some sites have a version number in their URL, such as tools/ve/updates/1.0/ and some do not, such as
 
Some sites have a version number in their URL, such as tools/ve/updates/1.0/ and some do not, such as
Line 187: Line 205:
  
 
This document and this "coordinated effort" is in no way meant to substitute for the projects themselves understanding update manger and update sites and actively
 
This document and this "coordinated effort" is in no way meant to substitute for the projects themselves understanding update manger and update sites and actively
contributing the necessary [[callisto_update_people | people]] to ensure the
+
contributing [[callisto_update_people | the necessary people]] to ensure the
 
correct end result.
 
correct end result.
  
The following are some requirements placed on the projects participating in Callisto (most should not be anything "new", but some will be, and feasibility and need will have to be evaluated as we go).  
+
The following are some requirements placed on the projects participating in Callisto (most should not be anything "new", but some will be, and feasibility and need will have to be evaluated as we go).
  
* provide project update sites, with jars, urls, site.xml files, etc.  
+
* provide [[project update sites]], with jars, urls, site.xml files, etc. pretty much as has been done before ... but, might be a good time to change past decisions (for example, GEF does not have its own update site, for ... historical reasons, I guess).  The Callisto site.xml file will "point to" these project-level update sites, so, basically, each project must have a working update site before Callisto can have a working update site. See [[project update sites]] for more details.
  
* provide list of minimum features for inclusion in "runtime" version of Callisto (and, please, minimum for "end users" to take advantage of your functionality). It would include 'end user' documentation, but not 'isv documentation nor JavaDoc.  
+
* provide to "[[callisto build]]" a list of minimum features for inclusion in "runtime" version of Callisto (and, please, minimum for "end users" to take advantage of your functionality). It would include 'end user' documentation, but not 'isv documentation nor JavaDoc.
  
* provide list of features for inclusion in "SDK" version of Callisto (should be no duplicates with first list, the SDK features should "require" runtime versions, so that someone may install runtime first, then install SDK later, without re-downloading or installing the runtime versions). These SDK features dos not have to be ALL your projects features, but should, at least, include 'source' and 'isv documentation'.  
+
<del>* provide to [[User:David williams|David williams]] list of features for inclusion in "SDK" version of Callisto (should be no duplicates with first list, the SDK features should "require" runtime versions, so that someone may install runtime first, then install SDK later, without re-downloading or installing the runtime versions). These SDK features dos not have to be ALL your projects features, but should, at least, include 'source' and 'isv documentation'.</del>
  
* provide accurate "feature size" so the update time and progress can be accurate.  
+
* provide accurate "feature size" so the update time and progress can be accurate. This is just good practice, but since I've noticed many don't sizes in their updates, I thought I would call it out here, since this will be more important in Callisto.
  
* (maybe) provide signed jars, so users get some assurance they are installing verifiable Callisto features (this is probably more important using the "nearest mirror" techniques, since users may not know exactly where the jar is coming from.  
+
* (maybe) provide signed jars, so users get some assurance they are installing verifiable Callisto features. This is more important now, using the "nearest mirror" techniques, since users may not know exactly where the jar is coming from. [I just say "maybe" because I do not think any of us know the technical steps or requirements for this.]
 +
 
 +
** <b>ISSUE: </b> If we have signed jars, it would be "nice" to have a "signed by Eclipse Foundation" corporate certificate, which I mention here, because I think to get such a certificate is a little time consuming (and not free :)
 +
 
 +
== Update Manager and Site Optimization ==
 +
The [[Update Site Optimization]] page outlines steps for making update sites more scalable.
  
 
== Help Wanted ==  
 
== Help Wanted ==  
Line 212: Line 235:
 
* specialists to help test and tweak update performance would likely not be refused.  
 
* specialists to help test and tweak update performance would likely not be refused.  
  
* etc.  
+
* etc.
  
 +
== Bugzilla Queries ==
  
 +
The following are some handy bugzilla queries related to update manager.
 +
 +
[https://bugs.eclipse.org/bugs/buglist.cgi?query_format=advanced&short_desc_type=allwordssubstr&short_desc=&classification=Eclipse&product=Platform&component=Update&long_desc_type=allwordssubstr&long_desc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&status_whiteboard_type=allwordssubstr&status_whiteboard=&keywords_type=allwords&keywords=&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&emailtype1=substring&email1=&emailtype2=substring&email2=&bugidtype=include&bug_id=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&field0-0-0=noop&type0-0-0=noop&value0-0-0=&order=bugs.bug_severity,bugs.target_milestone%2Cbugs.target_milestone%2Cbugs.bug_severity%2Cmap_components.name%2Cbugs.bug_severity%2Cbugs.target_milestone%2Cbugs.bug_severity%2Cmap_components.name%2Cbugs.bug_severity%2Cmap_reporter.login_name%2Cmap_reporter.login_name%2Cmap_components.name%2Cbugs.target_milestone%2Cbugs.bug_severity%2Cbugs.target_milestone%2Cbugs.resolution%2Cmap_components.name%2Cmap_assigned_to.login_name%2Cbugs.bug_status%2Cbugs.target_milestone%2Cbugs.target_milestone%2Cbugs.target_milestone%2Cbugs.bug_severity%2Cbugs.bug_status%2Cbugs.target_milestone%2Cbugs.priority%2Cbugs.bug_id&query_based_on=  All Opened]
 +
 +
 +
[https://bugs.eclipse.org/bugs/buglist.cgi?query_format=advanced&short_desc_type=allwordssubstr&short_desc=&classification=Eclipse&product=Platform&component=Update&long_desc_type=allwordssubstr&long_desc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&status_whiteboard_type=allwordssubstr&status_whiteboard=&keywords_type=allwords&keywords=&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&emailreporter1=1&emailtype1=substring&email1=david_williams&emailtype2=substring&email2=&bugidtype=include&bug_id=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&cmdtype=doit&order=Reuse+same+sort+as+last+time&field0-0-0=noop&type0-0-0=noop&value0-0-0=  All Opened by David Williams :) since starting to play with Callisto Updates.]
 +
 +
== Tips and Tricks ==
 +
 +
In addition to the written works in the reference section, I often some across little tidbits in bug reports, newsgroups, etc., and hope we can collect some of them on our [[Callisto Build and Update Tips and Tricks]] page. (So I can find them in the future :)
  
 
== Related Links for Further Enjoyable Reading and Reference ==
 
== Related Links for Further Enjoyable Reading and Reference ==
Line 222: Line 256:
 
[http://www.eclipse.org/articles/Article-Update/keeping-up-to-date.html How To Keep Up To Date]
 
[http://www.eclipse.org/articles/Article-Update/keeping-up-to-date.html How To Keep Up To Date]
  
[http://www.eclipse.org/eclipse/platform-core/documents/plugin-versioning.html Plugin Versioning Proposal]
+
[http://www.eclipse.org/equinox/documents/plugin-versioning.html Plugin Versioning Proposal]
  
 
[http://www.eclipse.org/eclipse/platform-core/documents/update.html Update Site Proposal]
 
[http://www.eclipse.org/eclipse/platform-core/documents/update.html Update Site Proposal]
 +
 +
[http://www.eclipse.org/eclipse/platform-core/documents/3.1/run_from_jars.html Jarring Plugins]
 +
 +
[[Development_Conventions_and_Guidelines]]
 +
 +
[[JAR_Signing]]
 +
 +
[[Update Site Optimization]]
 +
 +
[[Category:Callisto]] [[Category:Coordinated]]

Latest revision as of 22:53, 26 August 2007

Note: This page discusses topics related to the 2006 Callisto Simultaneous Release. Most of the Callisto pages are deprecated. You are urged to find current information on pages related to the 2007 effort, currently named the Europa Simultaneous Release.

Purpose of this document

This page is to document processes and procedures for providing improved coordination of update sites provided by the many Eclipse Projects. It's focus is on the 10 or so projects as part of the Callisto simultaneous release but some of the information would be helpful to any Eclipse adopter.

Note: as of this initial writing, this document should be taken as a 'proposal', and as its reviewed and discussed and improved by others, it will become a 'plan'. And then, eventually, once the plan seems to be working, it will become a true "process and procedures" document.

This document was started because of a cross-project agreement reached at the December, 2005, EMO Architecture Council Meeting. There, through their representatives to the Architecture Council, the projects of the Callisto Release agreed to improve the cross-project update experience. The agreement was that I (David Williams) would coordinate the effort ... but the projects still had to do the work!

This document is in no way to "take over" any of the Eclipse base project Update Team's function, proposals, or responsibilities.

It is also not to add to their responsibilities. The goal for Callisto release is to use and "push the limits" of the current capabilities and technologies of the Update Manager. Of course, bugs and feature requests may result, and that is fine, but I wanted to be clear this proposal is not for anything fundamentally new ... it is just to document what's already possible, and make sure it is coordinated and carried out with "best practices" in mind.

If I understand Wiki's at all committers can use the "talk" feature to provide feedback. Others may open a bugzilla on the Cross Project bugzilla component.

Use cases

There are 3 primary use cases that cross-project, coordinated update sites provide:

1. Allow end-users to install some minimum "platform" and from that be able to use Update Manager to install all of the Callisto release, just by going to just one update site and selecting just one thing.

2. Allow committers and developers to install an "SDK" version of Callisto, to be used while developing their own plugins.

3. Allow adopters to provide their own update sites, and "point to" appropriate sites to pick up prerequisite features. While there is nothing in this Callisto effort to support this directly, Callisto should serve as a good example of "best practices" for other projects to follow.

Objectives and Constraints

Besides promoting the use cases given above, there are other objectives and constraints to meet:


Eclipse.org Infrastructure constraints

The distribution of Eclipse projects must fit in to the current disk space and "mirror system".

  • Disk space.
    • JAR files should not be duplicated to avoid wasting bandwidth and disk space on our mirrors.
    • The site.xml file can be configured to include JARs from various projects and still work with mirrors. See Distributed Bandwidth, below.
  • Mirroring.
    • Callisto must make the best use of our mirror sites.
    • Our mirror sites should have at least 24-hours of exclusive access to the Callisto files before releasing them to the public. This is done by waiting for Webmaster approval before sending files to download.eclipse.org.
    • Webmaster will need to advise mirrors ahead of time, so they can anticipate the rush.
  • Bandwidth.
    • Co-ordinate with the Webmaster to "open the floodgates" on the bandwidth ahead of time.
    • R3.1 required additional bandwidth for a foundation-hosted, separate mirror site. We will need this again.


Coordination verus Unification

For ease-of-use, there should be something of a "central site" that should make it easy to "get started" with "all" of Callisto. But, in the Eclipse spirit of allowing projects to "do their own thing" each project should also have their own discovery and update sites as well. This allows the central "get started" site to be one-size fits all, but still allow projects to offer even more, special features such as tools, examples, etc., that may not be part of their "main' deliverables available via Callisto, not to mention have their own schedule for fixes and point releases.

Update Policies

Some sites (corporations) can establish an "corporate" update site, which may have its own policies about what updates when, etc. The process and procedures here should do nothing to interfere with that capability.

Fundamentals

Distributed Storage

Each project will still have its own disk space and area on Eclipse, and will still be responsible for providing jarred features and plugins there, as they would for their own update site.

Distributed Bandwidth

The features (and archive tags, if needed) will be specified using the new "find the closest mirror" luxury support provided by the Eclipse.org download.php script.

Issue: We (any volunteers?) should begin to collect approximate sizes and make rough estimates of required bandwidth for various scenarios, to make sure our mirroring system will suffice ... or we should panic now!?

Here is WebMaster's plan to make this happen:

  • Go mirror shopping to get new mirror sites - especially in US and Asia
  • Substantially increase bandwidth a few days ahead of release
  • Allow mirrors 24 hours to sync up
  • Provision a separate, dedicated high-speed mirror within those 24 hours
  • Boom

The Get-Started Platform Download

There will be one ZIP provided (by the Eclipse Project), that corresponds to the minimum amount of Eclipse that enables the update manager to work. This is, literally, the Eclipse Platform ... NOT including JDT and PDE. This feature will have the Callisto site as its implanted discovery site. It will still have 'eclipse' as its update site as it comes from the Eclipse Project (the update site being where "fixes" can be obtained automatically by update manager). All other features, from all other projects, will have their own project's discovery and update sites implanted in their features, not the Callisto site). The few features available from the central Callisto site will have discovery sites for all 10 Callisto projects.

Issue: In this day and age, I'm surprised we do not offer "updates" (or "get started") in the form of a WebStart object. I think we do not only due to people constraints ... so, if there is participation from the community (including the community of Callisto projects :) then we might be able to improve the user's experience so they'd never have to see another zip file again? </cite>

Note: We should be explicit that the Callisto Update Site, contrary to the name, must begin with a 3.2 based Platform Feature ... there is no capability planned to update from a 3.1 based environment. Likewise, there are no plans to update from an 3.2 M4 environment to a 3.2 M5 environment. We will plan to test updating from an M5 environment to an M6 environment.

The User's Experience

Minimum Runtimes versus full SDK's

Its been decided that for this Callisto Release, there will not be a coordinated discovery site for the "SDK" contributions that projects provide. This is partially due to having the focus on simultaneous release itself, not necessarily what each user audience might need. Plus, as it turns out, the meaning and design of what's in "runtime" and what's in "SDK" is a little different from project to project, and will take a longer effort to coordinate an approach that's meaningful to all.

But, this actually turns out to be a benefit, of sorts, rather than a limitation. The expected workflow is that Callisto users can install minimum "runtimes" to get the required or interesting functionality, and as part of those features installing, they each provide their own implanted update or discovery site URL. Hence, it should be easier for a user to install lots of functionality, but then get only the specific SDK portions they need. This will reduce users on-disk footprint, and reduce overall bandwidth on networks.

However, during development of Callisto, this will make it hard to actually test or exercise this workflow, since the implanted URL's usually refer to official release sites, not the milestone or interim release sites. So, if anyone is interested in testing or using this workflow during Callisto milestones, the following are the milestone or interim sites that projects are currently using.


http://download.eclipse.org/birt/update-site/site.xml
http://cdt.eclipse.org/callisto/site.xml
http://download.eclipse.org/datatools/updates/site.xml
http://download.eclipse.org/tools/emf/updates/site-interim.xml
http://download.eclipse.org/eclipse/updates/3.2milestones/site.xml
http://download.eclipse.org/tools/gef/update-site/milestones/site.xml
http://download.eclipse.org/technology/gmf/update-site/callisto/site.xml
http://download.eclipse.org/tptp/updates/4.2milestones/site.xml
http://download.eclipse.org/tools/ve/updates/1.0/development/site.xml
http://download.eclipse.org/webtools/milestones/site.xml


The View from Update Manager Dialog

The current plan is, for this Callisto Release, to simply list Features "by Project" ExampleDialog.gif.

Central Callisto Sites

Release Site

For the Callisto releases and milestones, there will be one central "discovery" site, such as

http://download.eclipse.org/callisto/releases

Note: Its desriable to avoid any reliance on "aliases" as it makes mirroring more difficult to manage.

Also, we should be explicit, there is no plans to provide a big-huge-zip that contains all the Callisto projects ... its expected to be provided entirely via update manager. There has been some recent talk of providing "big zips" made easy to get by the use of a form on a web page, and then the user would get, in one download, a zipped up version of their selections. If this is done, would be handled by the Eclipse Foundation (not this cross-project group).


Weekly Integration Update Sites

For those projects wishing to participate, there will be a site for integration builds:

http://download.eclipse.org/callisto/interim

Working Staging Area

Lastly, there will be a working directory of

http://download.eclipse.org/callisto/staging

that is used exclusively by central "Callisto developers" in the days or weeks ahead of the other update "releases" to test that the central features and site.xml is ready before they are copied over to the "public" sites.

User Interface Consistency

Web site consistency

All update sites will have an "index" file, appropriate for viewing with a web browser ... and we need to work on giving this a consistent look. (currently, some projects have none, some have highly detailed slowly performing ones with jars available for download for some reason I do not understand, but some :) have some really nice looking ones.

Update Manager View Consistency

All update sites will provide a "site.xml" file, with a consistent level of abstraction for separately installable features. (Remember, not literally every feature provided by a project has to be "visible" to the update site user ... many can be "included" as being required by other features, and installed automatically when other features selected).

These will be consistent and follow the following UI guidelines:

Project Name spelled out, followed by initial in parenthesis, if desired. Such as
Graphical Editing Framework (GEF)
Eclipse Modeling Framework (EMF)
Version numbers only at feature level, not project level categories. Such as
Draw2d 3.2 (not GEF 3.2).

Predictable, consistent, long term URL's for update sites

There's currently great inconsistency here. I guess as long as its permanent, and long term, there's no harm. But looks sloppy.

For example, GEF is in Eclipse, and has no update site of its own. [128348]

Some sites have a version number in their URL, such as tools/ve/updates/1.0/ and some do not, such as tools/emf/updates/. The latter is correct, and important with the new qualified versioning conventions.

So, I suggest, this is the time to get consistent, but will need some care and thought to be sure it doesn't "break" anything.

Planned Tests and Trial runs

During February, initial test versions will be created to download "all of Callisto" to see how it does, from several parts of the world. Its not expected these will be "usable" versions of Callisto.

By the end of the M5 period (3/3), there will be a usable version of Callisto available via update manager. (This is the version that can (will) be pointed to during EclipseCon, as an example of what's coming).

After M5, there will be "updates" (what's the right word for updates to updates?) to the milestones site for each Release Candidate leading up to the release.

Note: after M5, projects may elect to use the weekly I-build site. But this part of this proposal is sort of experimental, and will need some investigation, to be sure this is no great extra work for anyone ... but the idea is that for developers that are building on top of other features, it's very handy to be able to update pre-reqs in a frequent, easy way. That is, in other words, these "integration sites" are not literally an end-user requirement, so will get a little less attention.

Project Responsibilities

This document and this "coordinated effort" is in no way meant to substitute for the projects themselves understanding update manger and update sites and actively contributing the necessary people to ensure the correct end result.

The following are some requirements placed on the projects participating in Callisto (most should not be anything "new", but some will be, and feasibility and need will have to be evaluated as we go).

  • provide project update sites, with jars, urls, site.xml files, etc. pretty much as has been done before ... but, might be a good time to change past decisions (for example, GEF does not have its own update site, for ... historical reasons, I guess). The Callisto site.xml file will "point to" these project-level update sites, so, basically, each project must have a working update site before Callisto can have a working update site. See project update sites for more details.
  • provide to "callisto build" a list of minimum features for inclusion in "runtime" version of Callisto (and, please, minimum for "end users" to take advantage of your functionality). It would include 'end user' documentation, but not 'isv documentation nor JavaDoc.

* provide to David williams list of features for inclusion in "SDK" version of Callisto (should be no duplicates with first list, the SDK features should "require" runtime versions, so that someone may install runtime first, then install SDK later, without re-downloading or installing the runtime versions). These SDK features dos not have to be ALL your projects features, but should, at least, include 'source' and 'isv documentation'.

  • provide accurate "feature size" so the update time and progress can be accurate. This is just good practice, but since I've noticed many don't sizes in their updates, I thought I would call it out here, since this will be more important in Callisto.
  • (maybe) provide signed jars, so users get some assurance they are installing verifiable Callisto features. This is more important now, using the "nearest mirror" techniques, since users may not know exactly where the jar is coming from. [I just say "maybe" because I do not think any of us know the technical steps or requirements for this.]
    • ISSUE: If we have signed jars, it would be "nice" to have a "signed by Eclipse Foundation" corporate certificate, which I mention here, because I think to get such a certificate is a little time consuming (and not free :)

Update Manager and Site Optimization

The Update Site Optimization page outlines steps for making update sites more scalable.

Help Wanted

Besides the main deliverables for Callisto, the participating projects are asked to provide some assistance in testing the Callisto update site and, if possible, there are many tools that could be contributed to help. For example,

  • a "peek" tool which doesn't actually update anything, but which could, say, get the date and size headers for all the jars that would be requested by update manager, just for a warm reassuring feeling that all is (still) available as expected in a quick and painless way.
  • specialist to help with security, signed jars would likely be helpful.
  • specialists to help test and tweak update performance would likely not be refused.
  • etc.

Bugzilla Queries

The following are some handy bugzilla queries related to update manager.

All Opened


All Opened by David Williams :) since starting to play with Callisto Updates.

Tips and Tricks

In addition to the written works in the reference section, I often some across little tidbits in bug reports, newsgroups, etc., and hope we can collect some of them on our Callisto Build and Update Tips and Tricks page. (So I can find them in the future :)

Related Links for Further Enjoyable Reading and Reference

Basics of Update Sites

How To Keep Up To Date

Plugin Versioning Proposal

Update Site Proposal

Jarring Plugins

Development_Conventions_and_Guidelines

JAR_Signing

Update Site Optimization

Back to the top