View source for Talk:Callisto Coordinated Update Sites

My comments (Kim Moir platform-releng) :-)

(1)There is a problem with the url 
http://update.eclipse.org/updates/callisto/

Currently, the platform team stores all its updates in this directory 

http://update.eclipse.org/updates

and subdirectories such as these

http://update.eclipse.org/updates/3.0
http://update.eclipse.org/updates/3.1

So I don't know that this is the best url for Callisto

http://update.eclipse.org/updates/callisto/

given that 
<li>we already have platform content in that directory</li>
<li>we can't change the Apache alias for platform because this has been hardcoded in our feature.xmls</li>
<li>the unix permissions on the directory mean that the platform team would need to retain ownership in order to continue update the content in the existing update sites.</li>

[dmw: agreed, we should have 'callisto' first in the URL, and keep it "parallel" other Eclipse projects]

<span style="color:DarkGreen">[denis: I can't host this on update.eclipse.org/callisto. How about download.eclipse.org/callisto?]</span>

[nickb] How about: http://update.eclipse.org/updates/3.2/callisto.xml (in the same folder as site.xml) ?
[dmw] I think we're leaning toward http://download.eclipse.org/callisto, but wanted to comment on the comment ... I think we should avoid having version numbers in (any) directory ... isn't the long term goal to have a constant update site for all time, and just keep putting new plugins and features in there, as needed, so to speak? ... and, hopefully, there may start to be more and more that don't actually change, say from 3.2 to 3.3?]

[kmoir]Until the code to resolve this bug is implemented, we can't ensure that we can just reuse the same update site from release to release.  https://bugs.eclipse.org/bugs/show_bug.cgi?id=123162

(2)Providing Callisto as a single download comprising all 10 projects provides better exposure to the newer projects. By the same token, it also forces people to download stuff that they may not necessarily want or ever use, thus significantly increasing the strain on the distribution infrastructure. It is also rather overwhelming for a new eclipse user.  My point is, perhaps there is a better way to educate users and provide choice while simultaneously exposing the full breadth of functionality available with Callisto.

[dmw: yep, this is the question to be answered empiracally and via community feedback ... but if we get one packaging working ... a few others should be easy --- though, I'd never want to get to the point of a Callisto user having more than, say, 4 or 5 choices ... those users that want more fine grained control than that will have to use project update URLs]

[nickb] I agree that making people download EVERYTHING is both overkill and will overtax the infrastructure. However, as David points out, "enabling" projects like EMF and GEF are required for the more newb-friendly projects, so could we have a set of screenshots just showing people how to use UM and the various requirements they need to check off in order to get the projects they want? I like what WTP has done for their [http://download.eclipse.org/webtools/updates/ UM site], for example. 

By the way, I'm working on a tool to track download stats for EMF (and later, other projects who want to implement it), which combines

* a PHP script which runs SQL queries at the eclipse.org download stats database,
* a set of 3 cronjobs (actually, one shell script w/ three different options) for nightly/weekly/monthly querying using that interface, and
* a [http://emf.torolab.ibm.com/emf/downloads/downloads.php web UI] to display those results for trending and overall results (eg., combining monthly data across a year or weekly data across a quarter; or comparing 4 consecutive weeks to see if downloads increase or decrease over time)

The purpose for us is:

* to track how many hits we get in a day/week/month (survey says: [http://emf.torolab.ibm.com/emf/downloads/downloads.php?type=File&groups%5B%5D=groupSmall&groups%5B%5D=groupType&range=ym&rangeLimit=-1 over 2 million EMF jar downloads and 80,000 zip downloads in Dec & Jan])
* to track how those hits change overtime
* to determine which files are more popular than others (ie., should the Callisto UM site include XSD? survey says: [http://emf.torolab.ibm.com/emf/downloads/downloads.php?type=File&groups%5B%5D=groupSmall&thresh=1000&groups%5B%5D=groupProject&groups%5B%5D=groupType&range=ym "*xsd*"&&!"*emf*" to "*emf*" =~ 20:1 (zips)], which means EMF-only and EMF SDK (incl. XSD) zip downloads are 20x more downloaded than XSD-only zips)
* to compare UM installs vs. zip downloads, to see which is more prominently used (survey says: [http://emf.torolab.ibm.com/emf/downloads/downloads.php?type=File&groups%5B%5D=groupSmall&groups%5B%5D=groupType&range=ym&rangeLimit=-1 jars to zips =~ 19:1])

If you feel these tools/stats would be of value to other projects, I'd be happy to volunteer the code (and time to help set it up) for any other Callisto projects you'd like to track. See /cvsroot/org.eclipse/www/emf/downloads/

== Sizes involved with Calisto ==

Nick, thanks. I'm writing this mostly to experiment with "talking" to "talk" :)

Not sure how it will be displayed.

First, your tool does sound very interesting ... I personally won't have time to investigate it much for a while, but I'd think we'd want these sorts of stats for all Eclipse projects!

Since several have voiced concerned over size of Callisto, I guess we should start to estimate its size.

Here's my off hand estimates of "runtime" sizes of several components (in MegaBytes).
These are mostly guesses at this point (but a few I'm "updated" recently), but if enough people provide estimate, we'll get a little better idea of what tradeoffs are.

[<b>Callisto meisters:</b> feel free to update your portions of table to be accurate ... using current milestones, or previous release milestones as estimates. Any rough estimate will do.]

[nickb] You can get numbers for your most recent builds like this:

* ssh <i>username</i>@dev.eclipse.org
* cd /home/data/httpd/download.eclipse.org/tools/emf/downloads/drops/2.2.0/I200602020000
* find . -name "*.zip" -exec du -sh {} \;

[/nickb]

If we literally included each (sub) project as a choice, that would be 15 choices ... which I think is too many for what I've thought Callisto was for. Perhaps someone could propose some concrete "groupings" that would satisfy majority of users (though, I'm pessimistic if we could reach much agreement?)

If we go that route, though, I don't think we would need to do anything, except maybe to get the platform plugin to provide all 10 "discovery sites"? 

Also, maybe our <b>friendly webmaster</b> could comment on if these "increases in size" are of the size to be of concern .... such as is 75 Megs, say, that much different than 150? Is 150 that much different that 300? (I know "twice as big" sounds like a lot ... but .. its not like 10 times as much). And with our improved "nearest mirror" support, maybe it wouldn't matter?).

<span style="color:DarkGreen">[denis: also consider that the bigger it is, the longer it will take to reach all our mirrors. The good news is that Mike okay'd an extra 200Mb (!!) of bandwidth for (but not limited to) Callisto, in addition to and separate of the 100Mb for eclipse.org, so we'll be in good shape for actual distribution.]</span>

<table border="1" align="center">


                <tr valign="top">
                        <th>Project</th>
                        <th>Runtime Size</th>
                        <th>SDK Size</th>
                </tr>


        <tr valign="top">
                <td>Business Intelligence and Reporting Tools (BIRT)</td>
                <td>3</td>
                <td>?</td>
        </tr>
        <tr valign="top">
                <td>C/C++ IDE (CDT)</td>
                <td>10</td>
                <td>?</td>
        </tr>
        <tr valign="top">
                <td>Data Tools Platform (DTP)</td>
                <td>6</td>
                <td>?</td>
        </tr>
        <tr valign="top">
                <td>Eclipse Modeling Framework (EMF, SDO, XSD)</td>
                <td>EMF/SDO: 3.4, XSD: 1</td>
                <td>EMF/SDO/XSD: 23</td>
        </tr>
        <tr valign="top">
                <td>Eclipse Project (Platform only)</td>
                <td>15</td>
                <td>?</td>
        </tr>
        <tr valign="top">
                <td>Eclipse Project (PDE, JDT)</td>
                <td>60</td>
                <td>?</td>
        </tr>

        <tr valign="top">
                <td>Graphical Editing Framework (GEF)</td>
                <td>3</td>
                <td>?</td>
        </tr>
        <tr valign="top">
                <td>Graphical Modeling Framework (GMF)</td>
                <td>6</td>
                <td>?</td>
        </tr>
        <tr valign="top">
                <td>Test and Performance Tools Platform (TPTP)</td>
                <td>10</td>
                <td>?</td>
        </tr>
        <tr valign="top">
                <td>Visual Editor (VE, JEM)</td>
                <td>5</td>
                <td>?</td>
        </tr>
        <tr valign="top">
                <td>Web Tools Platform (WTP, WST, JST)</td>
                <td>25</td>
                <td>45</td>
        </tr>
                <tr valign="top">
                        <th>Totals</th>
                        <th>150</th>
                        <th>300</th>
                </tr>

</table>

== Page talks ==

I believe the little "+" symbol should be used for these talks.
--[[User:Droy|Denis Roy]] 16:50, 6 February 2006 (EST)

== Concurrent Builds &amp; Releases ==

[https://bugs.eclipse.org/bugs/show_bug.cgi?id=116912 Release Train Cascade / RSS Notification & Response]

[nickb] There's been a fair amount of discussion about getting the Callisto projects to align in terms of their releases (before the subject of UM was broached). Should this be a page by itself, rather than a footnote here, by all means please do. The link above shows what's been done so far in terms of building a build cascade automation scheme, along with some comments from the community. Please post comments about this plan into that bug so that I can integrate your requests into it, in order to make this solution more portable across all Callisto projects.