Jump to: navigation, search

Difference between revisions of "Eclipse Build Available RSS Feeds"

(updates to sample feeds and links to latest code)
(Actual Feeds)
 
(29 intermediate revisions by 2 users not shown)
Line 3: Line 3:
 
[http://wiki.eclipse.org/index.php?title=Eclipse_Build_Available_RSS_Feeds&action=history Recent changes to this document]
 
[http://wiki.eclipse.org/index.php?title=Eclipse_Build_Available_RSS_Feeds&action=history Recent changes to this document]
  
== Sample Hypothetical Feeds ==
+
==Getting Started With RSS Feeds==
  
<ul>
+
To get started using this code, I'd suggest extracting everything from CVS and having a look at the examples provided:
<li>Below are a few sample feeds which will hopefully be implemented within the Eclipse 3.2M6 timeframe. As this spec evolves this document will too, including revised URLs where applicable.</li>
+
  
<li>Everything in the <tt style="color:DarkGreen">&lt;summary/&gt;</tt> field is optional. A schema that backs these examples will be available soon, along with an Ant task to generate these during a build (after tests have run).</li>
+
cvs -d :pserver:anonymous@dev.eclipse.org:/cvsroot/eclipse -q checkout -r '''releng_test''' \
 +
-d org.eclipse.build.tools org.eclipse.releng.basebuilder/plugins/org.eclipse.build.tools
  
<li>The ant task (for the moment, until I think of something else) will take the following parameters:</li>
+
For details on extracting the code using Eclipse, see '''''[[Eclipse Build Available RSS Feeds Getting Started]]'''''.
  
<ul>
 
<li>feed URL to publish</li>
 
<li>user@dev.eclipse.org to use for scp'ing the file</li>
 
<li>overwrite existing feed (one build per feed) or insert new content (many builds per feed)</li>
 
</ul>
 
</ul>
 
  
=== Sample Eclipse Platform Feed ===
+
Make sure you get the '''releng_test''' branch, not '''HEAD''', if you want the latest implementation.
  
<pre>
 
<?xml version="1.0" encoding="UTF-8"?>
 
<?xml-stylesheet href="http://www.blogger.com/styles/atom.css" type="text/css"?>
 
<feed xmlns="http://www.w3.org/2005/Atom">
 
  <title>Eclipse Builds</title>
 
  <link href="http://download.eclipse.org/downloads/builds-eclipse.xml" rel="self" type="application/atom+xml"/>
 
  <updated>2006-05-29T16:31:27Z</updated>
 
  <author>
 
    <name>Eclipse Build Team</name>
 
  </author>
 
  <id>http://download.eclipse.org/downloads/builds-eclipse.xml</id>
 
  <entry>
 
    <title>[announce] Eclipse 3.2RC5 (3.2.0.S-3.2RC5-200605191206) is available</title>
 
    <link href="http://download.eclipse.org/downloads/drops/S-3.2RC5-200605191206"/>
 
    <id>http://download.eclipse.org/downloads/drops/S-3.2RC5-200605191206</id>
 
    <updated>2006-05-29T16:31:27Z</updated>
 
    <summary>
 
      <build xmlns="http://www.eclipse.org/2006/BuildFeed" callisto="COMPLETE" href="http://download.eclipse.org/downloads/drops/S-3.2RC5-200605191206" jars="SIGNED" type="S">
 
        <update>http://download.eclipse.org/updates/3.2milestones</update>
 
        <downloads>http://download.eclipse.org/eclipse/downloads/</downloads>
 
        <releasenotes>http://download.eclipse.org/eclipse/downloads/drops/S-3.2RC5-200605191206/buildNotes.php
 
          </releasenotes>
 
        <releases>
 
          <release os="win32" type="SDK" ws="win">eclipse-SDK-3.2RC5-win32.zip</release>
 
          <release os="linux" type="SDK" ws="gtk">eclipse-SDK-3.2RC5-linux-gtk.tar.gz</release>
 
        </releases>
 
        <tests>
 
          <test href="http://download.eclipse.org/downloads/drops/S-3.2RC5-200605191206/testResults.php" type="junit">
 
            <result os="win32" ws="win">PASS</result>
 
            <result os="linux" ws="gtk">PASS</result>
 
            <result os="linux" ws="gtk-x86_64">PASS</result>
 
            <result os="linux" ws="gtk-ppc">PASS</result>
 
            <result os="linux" ws="motif">PASS</result>
 
            <result os="solaris8" ws="gtk">PASS</result>
 
            <result os="solaris8" ws="motif">PASS</result>
 
            <result os="AIX" ws="motif">PASS</result>
 
            <result os="HP-UX" ws="motif">PASS</result>
 
            <result os="macosx" ws="carbon">PASS</result>
 
          </test>
 
          <test href="http://download.eclipse.org/downloads/drops/S-3.2RC5-200605191206/performance/performance.php" type="performance">
 
            <result os="RHEL4-3GHz-2.5GB" ws="gtk">PENDING</result>
 
            <result os="win32-3GHz-2GB" ws="winxp">AVAILABLE</result>
 
            <result os="RHEL3-3GHz-2GB" ws="gtk">AVAILABLE</result>
 
            <result os="win32-2GHz-512MB" ws="winxp">AVAILABLE</result>
 
          </test>
 
        </tests>
 
      </build>
 
    </summary>
 
  </entry>
 
</feed>
 
</pre>
 
  
=== Sample EMF Feed ===
+
In there, you'll find '''[http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.releng.basebuilder/plugins/org.eclipse.build.tools/scripts_rss/?only_with_tag=releng_test scripts_rss/]''' which includes ant script examples for:
 +
* publishing a feed ('''[http://dev.eclipse.org/viewcvs/index.cgi/*checkout*/org.eclipse.releng.basebuilder/plugins/org.eclipse.build.tools/scripts_rss/feedPublish.xml?only_with_tag=releng_test feedPublish.xml]''')
 +
* manipulating a feed, eg., updating a feed with a new entry, changing attributes on an existing entry, or just querying a feed for some value ('''[http://dev.eclipse.org/viewcvs/index.cgi/*checkout*/org.eclipse.releng.basebuilder/plugins/org.eclipse.build.tools/scripts_rss/feedManipulation.xml?only_with_tag=releng_test feedManipulation.xml]''')
 +
* watching a feed and responding to changes ('''[http://dev.eclipse.org/viewcvs/index.cgi/*checkout*/org.eclipse.releng.basebuilder/plugins/org.eclipse.build.tools/scripts_rss/feedWatch.xml?only_with_tag=releng_test feedWatch.xml]''')
  
<pre>
 
<?xml version="1.0" encoding="UTF-8"?>
 
<?xml-stylesheet href="http://www.blogger.com/styles/atom.css" type="text/css"?>
 
<feed xmlns="http://www.w3.org/2005/Atom">
 
  <title>EMF Builds</title>
 
  <link href="http://www.eclipse.org/emf/news/builds.xml" rel="self" type="application/atom+xml"/>
 
  <updated>2006-05-25T21:25:43Z</updated>
 
  <author>
 
    <name>EMF Build Team</name>
 
  </author>
 
  <id>http://www.eclipse.org/emf/news/builds.xml</id>
 
  <entry>
 
    <title>[announce] EMF 2.2.0RC7 (S200607081234) is available</title>
 
    <link href="http://download.eclipse.org/tools/emf/scripts/downloads-viewer.php?s=2.2.0/S200607081234"/>
 
    <id>http://download.eclipse.org/tools/emf/scripts/downloads-viewer.php?s=2.2.0/S200607081234</id>
 
    <updated>2006-05-25T21:25:43Z</updated>
 
    <summary>
 
      <build xmlns="http://www.eclipse.org/2006/BuildFeed" callisto="COMPLETE" href="http://download.eclipse.org/tools/emf/scripts/downloads-viewer.php?s=2.2.0/S200607081234" jars="SIGNED" type="SC">
 
        <update>http://download.eclipse.org/tools/emf/updates/</update>
 
        <downloads>http://download.eclipse.org/tools/emf/scripts/downloads.php</downloads>
 
        <releasenotes>http://www.eclipse.org/emf/news/release-notes.php?version=2.2.0
 
          </releasenotes>
 
        <releases>
 
          <release os="ALL" type="SDK" ws="ALL">emf-sdo-xsd-SDK-2.2.0RC7.zip</release>
 
        </releases>
 
        <tests>
 
          <test href="http://download.eclipse.org/tools/emf/downloads/drops/2.2.0/S200607081234/testResults.php" type="junit">
 
            <result os="ALL" ws="ALL">PASS</result>
 
          </test>
 
          <test href="http://download.eclipse.org/tools/emf/downloads/drops/2.2.0/S200607081234/testResults.php" type="performance"/>
 
          <test href="http://download.eclipse.org/tools/emf/downloads/drops/2.2.0/S200607081234/testResults.php" type="api"/>
 
        </tests>
 
        <dependencies>
 
          <dependency>http://download.eclipse.org/downloads/drops/S-3.2M2-200509231000/eclipse-SDK-3.2M2-linux-gtk.tar.gz</dependency>
 
        </dependencies>
 
      </build>
 
    </summary>
 
  </entry>
 
</feed>
 
</pre>
 
  
=== Sample UML2 Feed ===
+
These all use '''[http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.releng.basebuilder/plugins/org.eclipse.build.tools/?only_with_tag=releng_test feedTools.jar]''', with sources in '''[http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.releng.basebuilder/plugins/org.eclipse.build.tools/src_rss?only_with_tag=releng_test src_rss/]'''. The properties files used by these scripts are in '''[http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.releng.basebuilder/plugins/org.eclipse.build.tools/properties/?only_with_tag=releng_test properties/]'''. You'll see three for feedPublish.*.properties (publishing 3 different feeds) and two for feedWatch.*.properties (two projects watching upstream feeds). You'll also find a README in there which provides more details.
  
<pre>
 
<?xml version="1.0" encoding="UTF-8"?>
 
<?xml-stylesheet href="http://www.blogger.com/styles/atom.css" type="text/css"?>
 
  
<feed xmlns="http://www.w3.org/2005/Atom">
+
You can also look at the sample data in the '''[http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.releng.basebuilder/plugins/org.eclipse.build.tools/data/?only_with_tag=releng_test data/]''' folder, which provides three feeds (Eclipse, EMF, UML2).
  <title>UML2 Builds</title>
+
 
  <link href="http://www.eclipse.org/uml2/news/builds.xml" rel="self" type="application/atom+xml"/>
+
 
  <updated>2006-05-25T21:26:19Z</updated>
+
The feed schema is in the '''[http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.releng.basebuilder/plugins/org.eclipse.build.tools/schema/?only_with_tag=releng_test schema/]''' folder. The Ecore model and genmodel created from the schema and used for the feed validator are in '''[http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.releng.basebuilder/plugins/org.eclipse.build.tools/model/?only_with_tag=releng_test model/]''', and the feed validator code is best accessed from the '''[http://dev.eclipse.org/viewcvs/index.cgi/*checkout*/org.eclipse.releng.basebuilder/plugins/org.eclipse.build.tools/scripts_rss/feedValidator.xml?only_with_tag=releng_test feedValidator.xml]''' script in '''[http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.releng.basebuilder/plugins/org.eclipse.build.tools/scripts_rss/?only_with_tag=releng_test scripts_rss/]''' and the '''[http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.releng.basebuilder/plugins/org.eclipse.build.tools/?only_with_tag=releng_test feedValidator.jar]''' in the root.
  <author>
+
 
    <name>UML2 Build Team</name>
+
'''Note:''' The code cited above is the ''example/reference implementation''. For an '''''live working implementation''''', see the [[#Working_Examples | examples below]].
  </author>
+
 
  <id>http://www.eclipse.org/uml2/news/builds.xml</id>
+
== Sample Feeds ==
  <entry>
+
 
    <title>[announce] UML2 2.2.0RC3 (2.0.0.S200607082345) is available</title>
+
<ul>
    <link href="http://download.eclipse.org/tools/uml2/scripts/downloads-viewer.php?s=2.0.0/S200607082345"/>
+
<li>Samples feed files can be seen in CVS here: [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.releng.basebuilder/plugins/org.eclipse.build.tools/data/?only_with_tag=releng_test /cvsroot/eclipse/org.eclipse.releng.basebuilder/plugins/org.eclipse.build.tools/data/]
    <id>http://download.eclipse.org/tools/uml2/scripts/downloads-viewer.php?s=2.0.0/S200607082345</id>
+
 
    <updated>2006-05-25T21:26:19Z</updated>
+
<li>Most of the contents of the <tt style="color:DarkGreen">&lt;build/&gt;</tt> node are optional. The schema for these examples is located [[Eclipse_Build_Available_RSS_Schema|here]]. For tasks used to create/modify/view or respond to feeds, see [[#Latest_Code]].</li>
    <summary>
+
</ul>
      <build xmlns="http://www.eclipse.org/2006/BuildFeed" callisto="COMPLETE" href="http://download.eclipse.org/tools/uml2/scripts/downloads-viewer.php?s=2.0.0/S200607082345" jars="SIGNED" type="SC">
+
        <update>http://download.eclipse.org/tools/uml2/updates/</update>
+
        <downloads>http://download.eclipse.org/tools/uml2/scripts/downloads.php</downloads>
+
        <releasenotes>http://www.eclipse.org/uml2/news/release-notes.php?version=2.0.0
+
          </releasenotes>
+
        <releases>
+
          <release os="ALL" type="SDK" ws="ALL">uml2-SDK-2.2.0RC3.zip</release>
+
        </releases>
+
        <tests>
+
          <test href="http://download.eclipse.org/tools/uml2/downloads/drops/2.0.0/S200607082345/testResults.php" type="junit">
+
            <result os="ALL" ws="ALL">PASS</result>
+
          </test>
+
        </tests>
+
        <dependencies>
+
          <dependency>http://download.eclipse.org/downloads/drops/S-3.2M2-200509231000/eclipse-SDK-3.2M6-linux-gtk.tar.gz</dependency>
+
          <dependency>http://download.eclipse.org/tools/emf/downloads/drops/2.2.0/S200607081234/emf-sdo-xsd-SDK-2.2.0M6.tar.gz</dependency>
+
        </dependencies>
+
      </build>
+
    </summary>
+
  </entry>
+
</feed>
+
</pre>
+
  
 
== Support For Eclipse.org Jar Signing ==
 
== Support For Eclipse.org Jar Signing ==
  
So that we can identify which projects' builds are signed, I've implemented an additional node, <tt style="color:DarkGreen">&lt;build/&gt;</tt>, which can take an attribute for <tt style="color:DarkGreen">jars</tt>, thus:
+
So that we can identify which projects' builds are signed, the <tt style="color:DarkGreen">&lt;build/&gt;</tt> node can take an attribute for <tt style="color:DarkGreen">jars</tt>, thus:
  
<pre>
+
<feed xmlns="http://www.w3.org/2005/Atom">
<feed xmlns="http://www.w3.org/2005/Atom">
+
  ...
...
+
  <summary>
    <summary>
+
    <build  
      <build jars="UNSIGNED">http://download.eclipse.org/tools/emf/scripts/downloads-viewer.php?s=2.2.0/S200607081234</build>
+
      xmlns="http://www.eclipse.org/2006/BuildFeed"
...
+
      callisto="COMPLETE"
    </summary>
+
      href="http://download.eclipse.org/downloads/drops/S-3.2RC5-200605191206"
</feed>
+
      '''jars="''UNSIGNED''"'''
</pre>
+
      type="S">
 +
    ...
 +
  </summary>
 +
</feed>
  
 
I'm thinking of 5 status codes, assuming the current jar signing process continues as proposed:
 
I'm thinking of 5 status codes, assuming the current jar signing process continues as proposed:
Line 181: Line 65:
 
*      SIGNED        - signed & bundled on download page and on UM site
 
*      SIGNED        - signed & bundled on download page and on UM site
  
== Support For Build Type ==
+
== Support For Build Type, Branch & ID ==
  
So that we can identify what type of build a given entry is, I propose these values for a <tt style="color:DarkGreen">type</tt> attribute:
+
So that we can identify what type of build a given entry is, these values can be specified for a <tt style="color:DarkGreen">type</tt> attribute:
  
  <build jars="UNSIGNED" callisto="COMPLETE" type="SC">
+
  <build  
   http://download.eclipse.org/tools/emf/scripts/downloads-viewer.php?s=2.2.0/S200607081234
+
  xmlns="http://www.eclipse.org/2006/BuildFeed"
</build>
+
   href="http://download.eclipse.org/downloads/drops/S-3.2RC5-200605191206"
 +
  cvsbranch="HEAD"
 +
  branch="3.2.0"
 +
  datetime="200605191206"
 +
  '''type="''S''">
  
 
:*N : Nightly
 
:*N : Nightly
Line 195: Line 83:
 
:*R : Release
 
:*R : Release
  
:*MC : Maintenance-Callisto
+
If the property buildType is not specified, the ant task will attempt to guess the build type based on the buildID, by looking for, like this:
:*SC : Stable-Callisto
+
:*RC : Release-Callisto
+
  
If the property buildType is not specified, the ant task will attempt to guess the build type based on the buildID, like this:
+
this.buildType = buildID.replaceAll("[^NIMSR]", NS); //$NON-NLS-1$
 +
if (this.buildType.length()>1)
 +
{
 +
  this.buildType=this.buildType.substring(0, 1);
 +
}
  
  public void setBuildType(String buildType)
+
In addition, there are attributes available for <tt style="color:DarkGreen">branch</tt>, <tt style="color:DarkGreen">cvsbranch</tt> and <tt style="color:DarkGreen">datetime</tt>. Branch is a 3-number branch id, like 3.0.0 or 2.2.1. CVS branch is a string like "HEAD" or "R3_2_maintenance", and is optional. Datetime is a 12-digit number defining YYYYMMDDhhmm, the date and time associated with the build.
  {
+
    if (!isNullString(buildType))
+
    {
+
      this.buildType = buildType;
+
    }
+
    else
+
    {
+
      this.buildType = buildID.replaceAll("[^NIMSR]", NS); //$NON-NLS-1$
+
      if (this.buildType.length()>1)
+
      {
+
        this.buildType=this.buildType.substring(0, 1);
+
      }
+
    }
+
  }
+
  
== Support For Callisto Status ==
+
== Support For Coordinated Releases ==
  
So that we can identify which projects' have been posted to callisto, there's an attribute on the <tt style="color:DarkGreen">&lt;build/&gt;</tt> node, thus:
+
So that we can identify which projects' have been posted to Callisto/Europa/Whatnot, there's an element in the <tt style="color:DarkGreen">&lt;build/&gt;</tt> node, thus:
  
<pre>
+
<build
<feed xmlns="http://www.w3.org/2005/Atom">
+
  xmlns="http://www.eclipse.org/2006/BuildFeed"  
...
+
  href="http://download.eclipse.org/downloads/drops/S-3.2RC5-200605191206"
    <summary>
+
  type="S">
      <build jars="UNSIGNED" callisto="COMPLETE">http://download.eclipse.org/tools/emf/scripts/downloads-viewer.php?s=2.2.0/S200607081234</build>
+
  <coordinated status="COMPLETE">More information here</coordinated>
...
+
    </summary>
+
</feed>
+
</pre>
+
  
Status codes here could be:
+
Status codes are limited to:
  
 
{| cellpadding="4"
 
{| cellpadding="4"
Line 242: Line 114:
 
| '''BUILDCOMPLETE'''
 
| '''BUILDCOMPLETE'''
 
| Have you finished your RC1 bits?
 
| Have you finished your RC1 bits?
|- valign="top"
 
| '''2006-05-02T20:50:00Z'''
 
| When do you expect to finish them?
 
|- valign="top"
 
| '''TPTP'''
 
| If you're waiting for another project, which one(s)?
 
 
|- valign="top"
 
|- valign="top"
 
| '''UMSITEREADY'''
 
| '''UMSITEREADY'''
Line 259: Line 125:
 
|}
 
|}
  
= Latest Code =
+
Additional information, if applicable, can be placed in the text field of the node, which might include:
  
* [[Eclipse_Build_Available_RSS_Schema|Latest RSS Feed's XML Schema]]
+
{| cellpadding="4"
 +
| '''<u>Note</u>'''
 +
| '''<u>Purpose/Description</u>'''
 +
|- valign="top"
 +
| '''2006-05-02T20:50:00Z'''
 +
| When do you expect to finish them?
 +
|- valign="top"
 +
| '''TPTP'''
 +
| If you're waiting for another project, which one(s)?
 +
|}
  
::''Note that the latest code will shortly be available in [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.releng.basebuilder/plugins/org.eclipse.build.tools/ org.eclipse.releng.basebuilder/plugins/org.eclipse.build.tools]. The links below are about a week behind (they use the previous version of the feed's schema, not the [[Eclipse_Build_Available_RSS_Schema|absolute latest]]).''
+
= Where To Publish =
 +
 
 +
In order to ensure that your feed data is as current as your promoted bits, it's recommended that you publish your feed to the same place you publish your zips/jars. For the MDT example, this is [http://download.eclipse.org/modeling/mdt/feeds/builds-uml2.xml http://download.eclipse.org/modeling/mdt/feeds/builds-*.xml], and the CVS archived versions of the files are in [http://dev.eclipse.org/viewcvs/index.cgi/www/modeling/mdt/feeds/?root=Eclipse_Website /cvsroot/org.eclipse/www/modeling/mdt/feeds/].
 +
 
 +
It is recommended that you publish one feed per component that people might want to watch uniquely. For MDT, this means [http://www.eclipse.org/modeling/mdt/feeds/ 5 feeds] (at time of writing).
 +
 
 +
 
 +
== Actual Feeds ==
 +
 
 +
These are actual feeds, currently being published. If you publish a feed that's not listed here, [http://wiki.eclipse.org/index.php?title=Eclipse_Build_Available_RSS_Feeds&action=edit please add it].
 +
 
 +
A sample layout for linking to your feed from your downloads page is here: [http://www.eclipse.org/modeling/mdt/downloads/index.php?project=uml2 UML2 Downloads].
 +
 
 +
A sample icon to use is here: [http://www.eclipse.org/modeling/images/rss-atom10.gif Atom 1.0 icon].
 +
 
 +
* http://download.eclipse.org/eclipse/downloads/builds-eclipse-3.3.xml (Eclipse 3.3.x)
 +
* http://www.eclipse.org/modeling/emf/feeds/ (List of EMF feeds)
 +
* http://www.eclipse.org/modeling/mdt/feeds/ (List of MDT feeds)
 +
* http://www.eclipse.org/modeling/m2t/feeds/ (List of M2T feeds)
 +
* http://www.eclipse.org/emft/feeds/ (List of EMFT feeds; components will be removed as they move to MDT, M2T or EMF)
 +
 
 +
= Working Examples =
 +
 
 +
These are working examples implemented and in use, based on the samples provided in [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.releng.basebuilder/plugins/org.eclipse.build.tools/?hideattic=0&only_with_tag=releng_test org.eclipse.build.tools].
 +
 
 +
In this scenario, EMF publishes a scheduled build every Thursday at 00:00, promotes the build to eclipse.org at 01:00, and then UML2, knowing there's an EMF build every week, starts watching the EMF feed, checking hourly at 02:00, 03:00 and 04:00, in order to pick up the latest driver to do its weekly build. As our build server is at IBM, this delay is necessary to ensure that code committed to eclipse.org has time to propagate across the nodes and to the IBM mirrors. Usually the mirrors replicate between 02:00 and 03:00, causing the UML2 build to start at 03:00.
 +
 
 +
This is only one use case. For more responsive feed watching, cron-based checks could be done 24h a day, multiple times an hour, and more than once a week.
 +
 
 +
The most recent version of the scripts documented here is in [http://dev.eclipse.org/viewcvs/index.cgi/releng-common/tools/scripts/?root=Modeling_Project /cvsroot/modeling/releng-common/tools/scripts/].
 +
 
 +
== MDT & EMFT Components ==
 +
 
 +
Currently, MDT & EMFT components only publish RSS, they do not listen to upstream RSS feed(s). Setup is fairly straightforward.
 +
 
 +
1. <code>[http://dev.eclipse.org/viewcvs/index.cgi/releng-common/tools/scripts/promoteToEclipse.sh?root=Modeling_Project&view=markup promoteToEclipse.sh]</code> (see line 349), is [http://dev.eclipse.org/viewcvs/index.cgi/releng-common/tools/scripts/promoteToEclipse.properties?root=Modeling_Project&view=markup passed the property RSS=1] (or executed using the <code>-rssonly</code> flag).
 +
 
 +
2. The promote script calls out to <code>[http://dev.eclipse.org/viewcvs/index.cgi/releng-common/tools/scripts/feedPublish.xml?root=Modeling_Project&view=markup feedPublish.xml]</code> using <code>[http://dev.eclipse.org/viewcvs/index.cgi/releng-common/tools/scripts/feedPublish.mdt.properties?root=Modeling_Project&view=markup feedPublish.mdt.properties]</code>.
 +
 
 +
== EMF ==
 +
 
 +
Publishing an EMF build automatically is done in two steps, both fired by crontab:
 +
 
 +
# M build every Thu (4); runs as web user, www-data
 +
0 0 * * 4 sudo -u www-data /home/www-data/emf-build/scripts/start_cron.sh -proj emf -buildType M \
 +
  -tagBuild true -branchCVS HEAD -noperf -runJDK14Tests -runJDK50Tests -runOldTests > \
 +
  $HOME/cron_logs/start_cron.sh.emf.M.txt 2>&1
 +
 
 +
* [http://dev.eclipse.org/viewcvs/indextools.cgi/*checkout*/emf-home/emf-build/scripts/start_cron.sh start_cron.sh] (cron wrapper for start.sh)
 +
* [http://dev.eclipse.org/viewcvs/indextools.cgi/*checkout*/emf-home/emf-build/scripts/start.sh start.sh] (start a build)
 +
 
 +
# promote M build + update feed (among other things)
 +
0 1 * * 4 /home/www-data/emf-build/scripts/promoteToEclipse_cron.sh -proj emf -branch 2.2.1 -type M \
 +
  -user nickb -userIES nboldt -announce > $HOME/cron_logs/promoteToEclipse_cron.sh.emf.M.txt 2>&1
 +
 
 +
* [http://dev.eclipse.org/viewcvs/indextools.cgi/*checkout*/emf-home/emf-build/scripts/promoteToEclipse_cron.sh promoteToEclipse_cron.sh] (cron wrapper to publish a build)
 +
* [http://dev.eclipse.org/viewcvs/indextools.cgi/*checkout*/emf-home/emf-build/scripts/promoteToEclipse.sh promoteToEclipse.sh] (publish a build)
 +
* [http://dev.eclipse.org/viewcvs/indextools.cgi/*checkout*/emf-home/emf-build/scripts/promoteToEclipse.emf.properties promoteToEclipse.emf.properties] (properties)
 +
 
 +
One of the steps in promoting a build is to check out the latest copy of the feed from CVS, update it, commit it back to CVS, and scp it to eclipse.org. This is done by having <code>promoteToEclipse.sh</code> fire <code>feedPublish.xml</code>:
 +
 
 +
* [http://dev.eclipse.org/viewcvs/indextools.cgi/*checkout*/emf-home/emf-build/scripts/feedPublish.xml feedPublish.xml] (publish a feed)
 +
* [http://dev.eclipse.org/viewcvs/indextools.cgi/*checkout*/emf-home/emf-build/scripts/feedPublish.emf.properties feedPublish.emf.properties] (properties)
 +
 
 +
== UML2 ==
 +
 
 +
In this scenario, UML2 listens from 02:00 to 04:00 by running <code>feedWatch.sh</code>, but has chosen to manually verify and promote builds (also using <code>promoteToEclipse.sh</code>):
 +
 
 +
# respond to emf feed change; runs as web user, www-data
 +
0 2,3,4 * * 4  cd /home/www-data/emf-build/scripts/ && sudo -u www-data \
 +
  /home/www-data/emf-build/scripts/feedWatch.sh -proj uml2 -branchCVS R2_0_maintenance > \
 +
  $HOME/cron_logs/feedWatch.sh.uml2.txt 2>&1
 +
 +
* [http://dev.eclipse.org/viewcvs/indextools.cgi/*checkout*/emf-home/emf-build/scripts/feedWatch.sh feedWatch.sh] (check cached copy of a feed for specific changes on remote host)
 +
* [http://dev.eclipse.org/viewcvs/indextools.cgi/*checkout*/emf-home/emf-build/scripts/feedWatch.uml2.xml feedWatch.uml2.xml] (run UML2-specific check for upstream EMF feed changes)
 +
* [http://dev.eclipse.org/viewcvs/indextools.cgi/*checkout*/emf-home/emf-build/scripts/feedWatch.uml2.properties feedWatch.uml2.properties] (properties)
 +
 
 +
If a change is seen in the feed, run the '''respondToFeedChanged''' target in <code>feedWatch.xml</code>, which has been customized to update dependencies then run a build:
 +
 
 +
<target name="spawnBuild">
 +
  <echo message="Spawn new build:"/>
 +
  <echo message="/home/www-data/emf-build/scripts/start_cron.sh -proj uml2 ${buildType} ${buildAlias} -branchCVS ${branchCVS} ${runOptions} -URL '${feedWatchAction.TheValue.2}/${feedWatchAction.TheValue.1}' -URL '${feedWatchAction.TheValue.3}'"/>
 +
  <exec spawn="true" executable="bash" resolveexecutable="true">
 +
    <arg line="/home/www-data/emf-build/scripts/start_cron.sh -proj uml2 ${buildType} ${buildAlias} -branchCVS ${branchCVS} ${runOptions} -URL '${feedWatchAction.TheValue.2}/${feedWatchAction.TheValue.1}' -URL '${feedWatchAction.TheValue.3}'"/>
 +
  </exec>
 +
</target>
 +
 
 +
= Latest Code =
 +
 
 +
The latest code - shell and ant scripts, ant task code, XML Schema for the feed, and sample data - can be found in CVS, currently in the '''releng_test''' branch. This includes:
  
 
* Shell & Ant tasks to create, update, query & publish feeds (CVS, SCP)
 
* Shell & Ant tasks to create, update, query & publish feeds (CVS, SCP)
 
* Shell & Ant tasks to respond to feed changes (eg., sending an email, firing another ant task, launching a shell process with <exec>)
 
* Shell & Ant tasks to respond to feed changes (eg., sending an email, firing another ant task, launching a shell process with <exec>)
:* [https://bugs.eclipse.org/bugs/show_bug.cgi?id=116912#c31 https://bugs.eclipse.org/bugs/show_bug.cgi?id=116912#c31]
+
**[http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.releng.basebuilder/plugins/org.eclipse.build.tools/?hideattic=0&only_with_tag=releng_test /cvsroot/eclipse/org.eclipse.releng.basebuilder/plugins/org.eclipse.build.tools]
  
* Ant task (and shell wrapper) to validate a feed against the latest schema
+
* Ant task to validate a feed file against an XML schema (requires EMF). Note that the samples and schema are NOT the latest, but the tool WILL work with newer xml and xsd files.
:* [https://bugs.eclipse.org/bugs/show_bug.cgi?id=116912#c32 https://bugs.eclipse.org/bugs/show_bug.cgi?id=116912#c32]
+
** https://bugs.eclipse.org/bugs/attachment.cgi?id=43015&action=view
 +
 
 +
* The latest XML Schema is here:
 +
** [[Eclipse_Build_Available_RSS_Schema|Eclipse_Build_Available_RSS_Schema]]
 +
 
 +
= Discussion / Bugzilla =
 +
 
 +
* Here is a list of the bugzillas associated with RSS:
 +
** https://bugs.eclipse.org/bugs/show_bug.cgi?id=164390 (RSS for eclipse.org releases)
 +
** https://bugs.eclipse.org/bugs/show_bug.cgi?id=141152 (proposed feed enhancements)
 +
** https://bugs.eclipse.org/bugs/show_bug.cgi?id=130150 ("latest" links for project downloads)
 +
** <strike>https://bugs.eclipse.org/bugs/show_bug.cgi?id=124172</strike> (build stack automation)
 +
** <strike>https://bugs.eclipse.org/bugs/show_bug.cgi?id=116912</strike> (RSS release train)
 +
** <strike>https://bugs.eclipse.org/bugs/show_bug.cgi?id=85485</strike> (Platform RSS feeds)
 +
** [https://bugs.eclipse.org/bugs/buglist.cgi?query_format=specific&order=relevance+desc&bug_status=__open__&product=&content=rss more bugs...] (query bugzilla for "rss")
 +
** [https://bugs.eclipse.org/bugs/enter_bug.cgi?product=Platform&component=Releng&assigned_to=codeslave@ca.ibm.com&qa_contact=platform-releng-inbox@eclipse.org&bug_severity=enhancement new bug] (open a new bug in Platform > Releng and assign it to [[User:Nickb | Nick Boldt]])
  
 
= Additional Information =
 
= Additional Information =
  
 
* For more on the Atom 1.0 spec, see [http://www.ietf.org/rfc/rfc4287.txt http://www.ietf.org/rfc/rfc4287.txt].
 
* For more on the Atom 1.0 spec, see [http://www.ietf.org/rfc/rfc4287.txt http://www.ietf.org/rfc/rfc4287.txt].
 +
 +
[[Category:Releng]]
 +
[[Category:RSS]]
 +
[[Category:Modeling]]

Latest revision as of 13:24, 3 May 2007

Eclipse Build Available RSS Feeds

Recent changes to this document

Getting Started With RSS Feeds

To get started using this code, I'd suggest extracting everything from CVS and having a look at the examples provided:

cvs -d :pserver:anonymous@dev.eclipse.org:/cvsroot/eclipse -q checkout -r releng_test \
-d org.eclipse.build.tools org.eclipse.releng.basebuilder/plugins/org.eclipse.build.tools

For details on extracting the code using Eclipse, see Eclipse Build Available RSS Feeds Getting Started.


Make sure you get the releng_test branch, not HEAD, if you want the latest implementation.


In there, you'll find scripts_rss/ which includes ant script examples for:

  • publishing a feed (feedPublish.xml)
  • manipulating a feed, eg., updating a feed with a new entry, changing attributes on an existing entry, or just querying a feed for some value (feedManipulation.xml)
  • watching a feed and responding to changes (feedWatch.xml)


These all use feedTools.jar, with sources in src_rss/. The properties files used by these scripts are in properties/. You'll see three for feedPublish.*.properties (publishing 3 different feeds) and two for feedWatch.*.properties (two projects watching upstream feeds). You'll also find a README in there which provides more details.


You can also look at the sample data in the data/ folder, which provides three feeds (Eclipse, EMF, UML2).


The feed schema is in the schema/ folder. The Ecore model and genmodel created from the schema and used for the feed validator are in model/, and the feed validator code is best accessed from the feedValidator.xml script in scripts_rss/ and the feedValidator.jar in the root.

Note: The code cited above is the example/reference implementation. For an live working implementation, see the examples below.

Sample Feeds

Support For Eclipse.org Jar Signing

So that we can identify which projects' builds are signed, the <build/> node can take an attribute for jars, thus:

<feed xmlns="http://www.w3.org/2005/Atom">
  ...
  <summary>
    <build 
      xmlns="http://www.eclipse.org/2006/BuildFeed" 
      callisto="COMPLETE" 
      href="http://download.eclipse.org/downloads/drops/S-3.2RC5-200605191206" 
      jars="UNSIGNED" 
      type="S">
    ...
  </summary>
</feed>

I'm thinking of 5 status codes, assuming the current jar signing process continues as proposed:

  • NONE (or "") - no status available or not participating
  • UNSIGNED - no jar signage available or done yet
  • SIGNREADY - jars promoted to eclipse.org, ready for signing
  • BUILDREADY - signed on eclipse.org, ready to be collected and bundled as zips and copied to UM site
  • SIGNED - signed & bundled on download page and on UM site

Support For Build Type, Branch & ID

So that we can identify what type of build a given entry is, these values can be specified for a type attribute:

<build 
  xmlns="http://www.eclipse.org/2006/BuildFeed"  
  href="http://download.eclipse.org/downloads/drops/S-3.2RC5-200605191206"
  cvsbranch="HEAD"
  branch="3.2.0"
  datetime="200605191206"
  type="S">
  • N : Nightly
  • I : Integration
  • M : Maintenance
  • S : Stable (Milestone or Release Candidate)
  • R : Release

If the property buildType is not specified, the ant task will attempt to guess the build type based on the buildID, by looking for, like this:

this.buildType = buildID.replaceAll("[^NIMSR]", NS); //$NON-NLS-1$
if (this.buildType.length()>1) 
{
  this.buildType=this.buildType.substring(0, 1);
}

In addition, there are attributes available for branch, cvsbranch and datetime. Branch is a 3-number branch id, like 3.0.0 or 2.2.1. CVS branch is a string like "HEAD" or "R3_2_maintenance", and is optional. Datetime is a 12-digit number defining YYYYMMDDhhmm, the date and time associated with the build.

Support For Coordinated Releases

So that we can identify which projects' have been posted to Callisto/Europa/Whatnot, there's an element in the <build/> node, thus:

<build 
  xmlns="http://www.eclipse.org/2006/BuildFeed" 
  href="http://download.eclipse.org/downloads/drops/S-3.2RC5-200605191206" 
  type="S">
  <coordinated status="COMPLETE">More information here</coordinated>

Status codes are limited to:

Code Purpose/Description
NONE (or "") not part of Callisto or unknown status
BUILDCOMPLETE Have you finished your RC1 bits?
UMSITEREADY Have you placed those bits in your update site?
CALLISTOSITEREADY Have you updated the features.xml file in the Callisto CVS directory?
COMPLETE Are you ready for RC1 to be declared?

Additional information, if applicable, can be placed in the text field of the node, which might include:

Note Purpose/Description
2006-05-02T20:50:00Z When do you expect to finish them?
TPTP If you're waiting for another project, which one(s)?

Where To Publish

In order to ensure that your feed data is as current as your promoted bits, it's recommended that you publish your feed to the same place you publish your zips/jars. For the MDT example, this is http://download.eclipse.org/modeling/mdt/feeds/builds-*.xml, and the CVS archived versions of the files are in /cvsroot/org.eclipse/www/modeling/mdt/feeds/.

It is recommended that you publish one feed per component that people might want to watch uniquely. For MDT, this means 5 feeds (at time of writing).


Actual Feeds

These are actual feeds, currently being published. If you publish a feed that's not listed here, please add it.

A sample layout for linking to your feed from your downloads page is here: UML2 Downloads.

A sample icon to use is here: Atom 1.0 icon.

Working Examples

These are working examples implemented and in use, based on the samples provided in org.eclipse.build.tools.

In this scenario, EMF publishes a scheduled build every Thursday at 00:00, promotes the build to eclipse.org at 01:00, and then UML2, knowing there's an EMF build every week, starts watching the EMF feed, checking hourly at 02:00, 03:00 and 04:00, in order to pick up the latest driver to do its weekly build. As our build server is at IBM, this delay is necessary to ensure that code committed to eclipse.org has time to propagate across the nodes and to the IBM mirrors. Usually the mirrors replicate between 02:00 and 03:00, causing the UML2 build to start at 03:00.

This is only one use case. For more responsive feed watching, cron-based checks could be done 24h a day, multiple times an hour, and more than once a week.

The most recent version of the scripts documented here is in /cvsroot/modeling/releng-common/tools/scripts/.

MDT & EMFT Components

Currently, MDT & EMFT components only publish RSS, they do not listen to upstream RSS feed(s). Setup is fairly straightforward.

1. promoteToEclipse.sh (see line 349), is passed the property RSS=1 (or executed using the -rssonly flag).

2. The promote script calls out to feedPublish.xml using feedPublish.mdt.properties.

EMF

Publishing an EMF build automatically is done in two steps, both fired by crontab:

# M build every Thu (4); runs as web user, www-data
0 0 * * 4 sudo -u www-data /home/www-data/emf-build/scripts/start_cron.sh -proj emf -buildType M \
 -tagBuild true -branchCVS HEAD -noperf -runJDK14Tests -runJDK50Tests -runOldTests > \
 $HOME/cron_logs/start_cron.sh.emf.M.txt 2>&1
# promote M build + update feed (among other things)
0 1 * * 4 /home/www-data/emf-build/scripts/promoteToEclipse_cron.sh -proj emf -branch 2.2.1 -type M \
  -user nickb -userIES nboldt -announce > $HOME/cron_logs/promoteToEclipse_cron.sh.emf.M.txt 2>&1

One of the steps in promoting a build is to check out the latest copy of the feed from CVS, update it, commit it back to CVS, and scp it to eclipse.org. This is done by having promoteToEclipse.sh fire feedPublish.xml:

UML2

In this scenario, UML2 listens from 02:00 to 04:00 by running feedWatch.sh, but has chosen to manually verify and promote builds (also using promoteToEclipse.sh):

# respond to emf feed change; runs as web user, www-data
0 2,3,4 * * 4   cd /home/www-data/emf-build/scripts/ && sudo -u www-data \
 /home/www-data/emf-build/scripts/feedWatch.sh -proj uml2 -branchCVS R2_0_maintenance > \
 $HOME/cron_logs/feedWatch.sh.uml2.txt 2>&1

If a change is seen in the feed, run the respondToFeedChanged target in feedWatch.xml, which has been customized to update dependencies then run a build:

<target name="spawnBuild">
  <echo message="Spawn new build:"/>
  <echo message="/home/www-data/emf-build/scripts/start_cron.sh -proj uml2 ${buildType} ${buildAlias} -branchCVS ${branchCVS} ${runOptions} -URL '${feedWatchAction.TheValue.2}/${feedWatchAction.TheValue.1}' -URL '${feedWatchAction.TheValue.3}'"/>
  <exec spawn="true" executable="bash" resolveexecutable="true">
    <arg line="/home/www-data/emf-build/scripts/start_cron.sh -proj uml2 ${buildType} ${buildAlias} -branchCVS ${branchCVS} ${runOptions} -URL '${feedWatchAction.TheValue.2}/${feedWatchAction.TheValue.1}' -URL '${feedWatchAction.TheValue.3}'"/>
  </exec>
</target>

Latest Code

The latest code - shell and ant scripts, ant task code, XML Schema for the feed, and sample data - can be found in CVS, currently in the releng_test branch. This includes:

Discussion / Bugzilla

Additional Information