View source for Modeling Project Releng/Releasing

==Release Process==

=== Schedule ===
All components are requested to follow this release schedule:

* Tuesday/Wednesday night: 
** Your code in CVS must be "compile error free". Feel free to run N builds that afternoon to verify this.
* Wednesday/Thursday morning: 
** One Integration (I) build from your main branch (HEAD), and/or 
** One Maintenance (M) build from your maintenance branch (eg., R1_0_maintenance).

If your component is participating in a [[:Category:Coordinated | Coordinated]] release such as [[Ganymede]] (June 2008), please be aware of the release schedule for [[Ganymede#Milestones_and_Release_Candidates | Milestones and Release Candidates]], and provide Stable (S) builds in alignment with those dates.

===Rampdown to Release===

After the M7/RC0 builds are published, it's usually time to review the copyrights in your sources to ensure they're accurate. You may also be asked at some point to provide all of the about.* files to the Eclipse Management Organization (EMO) legal department so they can review them prior to the release.

You will also be required to provide a slide deck to your PMC, as he will be representing your component at your [http://www.eclipse.org/projects/dev_process/release-review.php Release Review]. For samples of past release review slides, see
* [http://www.eclipse.org/projects/previous-release-reviews.php Previous Release Reviews]
:* [http://www.eclipse.org/projects/slides/EMF-QVT_Europa_Release_Review.pdf EMF QTV Europa Review]
:* [http://www.eclipse.org/projects/slides/EMF_Europa_Review.pdf EMF (Core) Europa Review]

==== Collecting Abouts ====

* First, sources are collected [[CVS_Source_From_Mapfile|using this script]] and the map file listed on your downloads page for your most recent milestone build. (If you don't have access to your own linux box, you can run this on emft.eclipse.org or build.eclipse.org.)
* Next, check out Jeff McAffer's [http://dev.eclipse.org/mhonarc/lists/cross-project-issues-dev/msg02409.html org.eclipse.orbit.releng.aboutextractor tool] from CVS and use that to process your sources into a new directory containing only the Abouts.

* Finally, that new directory can be zipped and sent to emo@eclipse.org and/or Janet.Campbell@eclipse.org, as required.

==== Maintaining Your IP Log ====

As part of being an Eclipse project, [http://www.eclipse.org/projects/dev_process/ip-process-in-cartoons.php everyone is responsible for keeping the IP clean]. This includes:

* Contributors: by attaching patches to bugs
* Committers: by committing patches and filing CQs for large contributions, and using the 'contributed' keyword on bugs with contributions which have been committed to your code base
* Project/Component Leads: by enforcing the IP rules for your committers/contributors, updating IP Logs in the event they ever get out of date, and filing CQs on behalf of soon-to-be committers
* PMCs: all of the above, plus liasing with the Foundation in terms of [http://www.eclipse.org/projects/dev_process/release-review.php Release & Graduation Reviews] -- collecting Abouts, Slide Decks, and IP Logs.


Currently, there are [http://www.eclipse.org/modeling/emft/eclipse-project-ip-log.php three ways to log your project or component's IP].

<table border="0"><tr valign="top"><th>Method</th>
<th>Provider</th><th>Example</th><th>How to Update</th></tr>
<tr bgcolor="#EEEEFF"><th>
Dynamic</th><td>Eclipse Foundation</td><td>[http://www.eclipse.org/projects/ip_log.php?projectid=modeling.emft ip_log.php]</td><td>
Contribute to this log by modifying bugs in Bugzilla and by opening CQs in IPZilla.</td></tr>
<tr bgcolor="#FFFFEE"><th>
Half & Half</th><td>Modeling Project</td><td>[http://www.eclipse.org/modeling/emft/project-info/ipquery.php ipquery.php]
</td><td>
Contribute to this log by modifying bugs in Bugzilla and by opening CQs in IPZilla; you can also add static information by editing your [http://dev.eclipse.org/viewcvs/index.cgi/www/modeling/emf/project-info/ipquery.php?root=Eclipse_Website&content-type=text%2Fplain&view=co ipquery.php] file.</td></tr>
<tr bgcolor="#EEEEFF"><th>
Static</th><td>project/component committers</td><td>[http://www.eclipse.org/modeling/emft/compare/eclipse-project-ip-log.csv eclipse-project-ip-log.csv]
</td><td>
Contribute to this log by editing your [http://dev.eclipse.org/viewcvs/index.cgi/www/modeling/emft/compare/eclipse-project-ip-log.csv?root=Eclipse_Website&view=co&content-type=text%2Fplain eclipse-project-ip-log.csv] file.</td></tr>
</table>

For more on how to keep your IP Log up to date, see '''''[[Modeling Project Releng/Releasing/IP Log|Modeling Project Releng - Releasing - IP Log]]'''''.

=== Release Reviews / x.y.0 Final Builds ===

* <b style="color:darkred">DO NOT PUBLISH</b> an R build until you've been through a release review. 
* <b style="color:darkred">DO NOT PUBLISH</b> any build using a vanity name ending in ".0" without a release review. 
 
* <b style="color:darkgreen">DO PUBLISH</b> as many I or S builds as you want -- ideally, you should be publishing one build a week, but only if there are changes. 

You can have unlimited weekly Integration (I) builds, then about once every 6 weeks and at the end of your development cycle, you should publish a Stable (S) build. These are usually vanity-named with "0.8.0M2" or "0.8.0RC" to describe the Milestone or Release Candidate status and number.

At the end of your development cycle, you should do an RC build at least one week before your final planned release date. When you pass your [http://www.eclipse.org/projects/dev_process/release-review.php Release Review], you can use the <code>/home/www-data/build/modeling/scripts/renameBuild.sh</code> script to rename your "0.8.0RC" Stable (S) build to a "0.8.0" Release (R) build, and repromote it under the new name. This will ensure the correct metadata is properly updated (RSS feed, release notes).
 
Once your x.y.0 build is reviewed, passed, and released, you can continue in HEAD working on x.y.1 (or branch immediately). Publish as many (M)aintenance builds as you want, then again, as you get close to the end, a Stable 0.8.1RC can be done. Maintenance branch releases need not be reviewed prior to release. 
 
At some point you'll want to branch HEAD (eg., 0.9.0) and R0_8_maintenance (eg., 0.8.1 or 0.8.2). When you start doing 0.9.0 builds, they will be I builds, interspersed with S milestone builds and RC build(s) towards the end of the cycle. Finally, your last RC will be reviewed and you'll once again rename and repromote. 

It is '''''HIGHLY ADVISABLE''''' to not branch midway through a release, in order to preserve accurate release notes.

Please also be aware that your build server is '''SHARED''' -- clean up your old builds (especially any -noclean builds) when they are obsoleted by newer ones in order to save space for others.

===Renaming a Build / Releasing Your GM Build===

* When releasing your final build, you may decide to rename an RC build if nothing bad has happened since that build was released. For example, you might plan to release your final build on Sept. 28, so you'd do an RC1 build two weeks before. If any stop-ship bugs were found, you'd spin an RC2 build the week after, on the 21st. Then, if no more bugs were found, you'd be able to promote that build as your GM (Golden Master) on Sept. 28 - however, you'd have to rename the zips, md5 files, dependencies listed in build.cfg, etc.

* The good news is that there's an easy way to do this - [http://dev.eclipse.org/viewcvs/indextools.cgi/*checkout*/emf-home/emf-build/scripts/renameBuild.sh?content-type=text/plain renameBuild.sh]. This script contains detailed instructions and examples of usage; just run it from the commandline w/o options to see. Additionally, here's an example showing a rename of JET from 0.7.1RC2 to 0.7.1:

 sudo -u www-data ./renameBuild.sh -proj jet -verbose -branch 0.7.1 \
 -buildID S200609210820=R200609210820 -buildAlias 0.7.1RC2=0.7.1 \
 eclipse-SDK-M20060919-1045-linux-gtk.tar.gz=eclipse-SDK-3.2.1-linux-gtk.tar.gz \
 M20060919-1045=R-3.2.1-200609210945 \
 S200609210005=R200609210005 \
 2.2.1RC2=2.2.1 \
 fullmoon.torolab.ibm.com=download.eclipse.org \
 emf.torolab.ibm.com=download.eclipse.org

* When you're done renaming, you can simply promote the new build as you did the previous week (<tt style="color:DarkGreen">-buildID S200609210820</tt>), using the new ID (<tt style="color:DarkGreen">-buildID R200609210820</tt>), and you'll have your GM build published. '''*HOWEVER**''', if you are participating in a coordinated release, you must adhere to the process rules for that release. For example: [[Ganymede/Final_Daze/Modeling_Project|Ganymede - Final Daze - Modeling Project]], [[Ganymede/Final_Daze|Ganymede - Final Daze]].

* After releasing the final build of a given version, you need to remember to upgrade the version of the plugins you change. In other words, if a plugin is not changed then its version remains the same. After upgrading the version of a plugin, you need to ensure that the version of the appropriate features and branding plugins are also upgraded, and that the feature is referring to the new versions of the plugins. Also, if you change a plugin with source code, you may need to upgrade the appropriate documentation plugin (and documentation feature and branding plugin) if the build is adding Javadoc to it.

== Build Naming Conventions ==

Because Modeling consists of many projects with an ever-expanding set of components, it's important that we all adhere to consistent naming rules for each [http://www.eclipse.org/modeling/downloads/build-types.php build type]. It's also important to be consistent within your component or your release notes will be sloppy or incomplete. Build Aliases must conform to one of these patterns in order to be automatically added to the release notes database: 

<blockquote><table><tr><td>{{codeblock|/[IM]\d{12}/}}</td><td>  or  </td><td>{{codeblock|/\d+\.\d+\.\d+.*/}}</td></tr></table></blockquote>

When building, use the Build Alias field to name your S and R [http://www.eclipse.org/modeling/downloads/build-types.php type builds]. 

: [[Image:Uml2tools-build-page.png]]

The rules are very simple.

* If it's a ''Milestone'' (build type '''S'''), call it "x.y.z''M''n", eg., '''0.7.0M6'''
* If it's an ''RC'' (build type '''S'''), call it "x.y.z''RC''n", eg., '''0.8.0RC1''' or '''0.8.2RC'''
* If it's a ''Release'' (build type '''R'''), call it "x.y.z", eg., '''0.9.0''' or '''2.0.2'''
* All other builds ('''I''', '''M''') use no alias and are named using the default pattern of <code>"/[IM]\d{12}/"</code>, eg., '''I200802070245'''

==Promoting A Build==

===The First Time===

* First, ensure you've set up all your [[#SSH_Key_Setup | SSH keys]], including [[#Connecting_the_web_user_to_your_account|the one for the web user]].

* Next, ensure your [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.emf/org.eclipse.emf.ecoretools.releng/promoteToEclipse.ecoretools.properties?root=Modeling_Project&view=markup promoteToEclipse.*.properties] file is up to date. Once your file is correct in CVS, you can run this (sudo'd as the web user, www-data or apache) to check out the latest version of that file from CVS:

{{codeblock|/home/www-data/build/modeling/scripts/getPromotePropertiesFromCVS.sh emf.ecoretools}}

* Contact [[User:Nickb]], and ask that he:
:* add your component to [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.releng.basebuilder/plugins/org.eclipse.build.tools/scripts_cvs/searchcvs/relupdate.php relupdate.php],
:* add a "0.0.0" release to releases table for your component, and
:* check out your project and releng sources for use with [http://build.eclipse.org/modeling/build/updateSearchCVS.php searchcvs]

* Load up [http://emft.eclipse.org/modeling/emft/ecoretools/build/promo.php promo.php]. You can only promote I, M, S, and R builds this way. If your branch is not listed, make sure your [http://dev.eclipse.org/viewcvs/index.cgi/www/modeling/emft/ecoretools/build/_common.php?revision=1.1&root=Eclipse_Website&view=markup _common.php] file is up to date. See also [[Modeling_Project_Releng/Website_Maintenance | Website Maintenance]]. 

* Review the log file generated by the promote process. Log files are stored in <code>/home/www-data/promo_logs/</code>. If you encounter any errors in the log, review [[Modeling Project Releng]] or contact [[User:Nickb]].

* Should a step fail, you can re-run the whole promote (but you may get duplicate entries in your [http://dev.eclipse.org/viewcvs/index.cgi/www/modeling/emft/feeds/?root=Eclipse_Website RSS feed] and [http://dev.eclipse.org/viewcvs/index.cgi/www/modeling/emft/updates/?root=Eclipse_Website update site]). To avoid this, you can remove the new content before re-running the build, or you can run only the relevant parts that failed via the commandline promote script interface. For example, to re-run the javadoc generation step, use:

{{codeblock|cd /home/www-data/build/modeling/scripts/;./promoteToEclipse.sh -sub ecoretools -docsonly -Q -buildID I200712031234 ...}}

* To see all the commandline options available to you, run <code>./promoteToEclipse.sh</code> without options for the inline help.

* '''PLEASE NOTE!''' Once your promote is done, you will not see it on eclipse.org for at least 20-30 minutes as the various eclipse.org cluster nodes replicate. Why?
:* ''download.eclipse.org is served via nfs on all of the nodes from the nfsslave. The master and slave replicate every 4-6 hours depending on system loads.''
::* So, unfortunately, the only recourse is patience. 

* When your build does appear at eclipse.org, [http://www.eclipse.org/modeling/emft/downloads/ download] the zip and test it out to ensure it works for you. Then try and install via the [http://www.eclipse.org/modeling/emft/updates/ update site].

* When you're satisfied, update the [http://dev.eclipse.org/viewcvs/index.cgi/www/modeling/emft/_projectCommon.php?root=Eclipse_Website&view=markup _projectCommon.php] file for your project (EMFT, MDT, etc.) to remove your project from the <code>$nodownloads</code> array. Wait 30-60 seconds and then refresh your project's homepage, downloads page, release notes, etc. to ensure your component is listed correctly. See also [[Modeling_Project_Releng/Website_Maintenance#Site_Navigation | Website Navigation]].

* If you plan to contribute to a coordinated release such as [[Ganymede]], make sure you've added your component to the [[Ganymede#Projects | list]], and that you have write access to <code>/cvsroot/callisto</code>. If you don't, [https://bugs.eclipse.org/bugs/enter_bug.cgi?product=community&component=CVS open a bug] assigned to webmaster@eclipse.org and cc:'d to your PMC [[User:Nickb]]. You should ask:
:* "Please add user ''yourusername'' to group '''callisto-dev''' for Ganymede .sc file contributions."

====Troubleshooting====

* If you don't set up your [[#SSH_Key_Setup | SSH keys]], including [[#Connecting_the_web_user_to_your_account|the one for the web user]] correctly, you may see this in your promo log:

 Permission denied, please try again.
 Permission denied, please try again.
 Permission denied (publickey,gssapi-with-mic,password).

* If you see '''Permission denied''' errors in the promote log, you may not have sufficient permission to create files on the download server.

 mkdir: cannot create directory `/home/data/httpd/download.eclipse.org/modeling/m2t/mtl': Permission denied

:* [https://bugs.eclipse.org/bugs/enter_bug.cgi?product=community&component=CVS Open a bug] assigned to webmaster@eclipse.org and cc:'d to your PMC and [[User:Nickb]], asking "Please add user ''yourusername'' to group '''groupname''' for publishing buildname builds."
:* ''Groupname will be in the form '''m2tadmin''' (old style) or '''modeling.emf.website''' (new style).''

===Scheduled Releases (Promoting Automatically)===

See [[Modeling_Project_Releng/Releasing/Scheduled_Releases|Scheduled Releases]].

===Update Manager Sites &amp; Build Types===

If you go to http://www.eclipse.org/modeling/emft/updates/, you'll see a note about two different UM sites. These two sites are there to split final, release builds from interim builds, so that the two types of users can benefit from the two types of releases.

A release build (R) is anything that's done as a final GM (aka ''Golden Master'') release, eg., 0.7.0, 0.7.1, 1.0.0, 1.0.1.
Everything else is considered "interim" - Nightly, Integration, Maintenance, Stable (N, I, M, S).

So, for example, when you publish an I build (eg., 0.7.5 I200610050000), it gets added to [http://download.eclipse.org/modeling/emft/updates/site-interim.xml site-interim.xml]. When you do your final build for that stream, (eg., 0.7.5 R200612020000), it will be added to [http://download.eclipse.org/modeling/emft/updates/site.xml site.xml].

Note also that the URL to give to Update Manager (or to put in your features) is NOT http://www.eclipse.org/modeling/emft/updates/, but download.eclipse.org. www.eclipse.org stores website content only, and provides a location for extracting, updating, and committing changes to the site.xml and site-interim.xml files, in order to provide CVS versioning &amp; history. The actual site is [http://download.eclipse.org/modeling/emft/updates/ http://download.eclipse.org/modeling/emft/updates/site*.xml], which is also benefitted by the fact that it's mirrored (unlike www.eclipse.org).

* For more on build types, see http://www.eclipse.org/modeling/downloads/build-types.php. 

====Generating Update Manager Site Jars====

* For details on generating Update site XML, jars, digest, and metadata, see: [[Modeling_Project_Releng/Releasing/Update Site Generation|Update Site Generation]].

* For troubleshooting, see [[Modeling_Project_Releng/Releasing/Update_Site_Generation/Troubleshooting|Troubleshooting]].

====Contributing To Ganymede Update Site====

<table><tr valign="top"><td>

* The [[Ganymede/Build]] collects your *.sc files and turns them into features for the combined Ganymede update site.

* To contribute to Ganymede, you must complete these steps:

:* Open a bug to get added to group callisto-dev, eg., {{bug|212325}}. Bug should be assigned to '''webmaster@eclipse.org''' and cc'd to '''bjorn.freeman-benson@eclipse.org''' for PMC-like approval.
:* Use your [http://emft.eclipse.org/modeling/emft/compare/build/promo.php promo.php] build page (snapshot at right) to ensure your contribution (/cvsroot/callisto/org.eclipse.ganymede.sitecontributions/*.sc) is generated and committed to CVS 
::* promoteToEclipse.sh, called from the promote page, will generate the XML contents of your .sc file. That's it!

* To control what features are contributed, tweak your promoteToEclipse.*.properties file. For example:

[http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.emf/org.eclipse.emf.releng/scripts/promoteToEclipse.emf.properties?root=Modeling_Project&view=markup For EMF/SDO/XSD]:

 # pattern to use when searching for features to include in coordsite; defaults to ".*eclipse/features/org.eclipse..*(${subprojectName}|${subprojectName}.sdk)_.*\/$" 
 coordsiteFeaturePattern=".*eclipse/features/org.eclipse..*(emf|emf.sdk|xsd|xsd.sdk|emf.ecore.sdo|emf.ecore.sdo.sdk|emf.common|emf.common.ui|emf.ecore|emf.ecore.edit|emf.edit|emf.edit.ui|xsd.edit)_.*\/$"
 # pattern to use when setting feature to include in coordsite's main feature ('Models and Model Development'); all other matching features will go in 'Enabling Features'
 coordsiteMainFeaturePattern="org.eclipse.(emf|emf.sdk|xsd|xsd.sdk|emf.ecore.sdo|emf.ecore.sdo.sdk)$"

Thus, EMF contributes 6 "main" features:
* emf, emf.sdk, xsd, xsd.sdk, emf.ecore.sdo, emf.ecore.sdo.sdk
And 7 "enabling" features:
* emf.common, emf.common.ui, emf.ecore, emf.ecore.edit, emf.edit, emf.edit.ui, xsd.edit

[http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.mdt/org.eclipse.ocl.releng/promoteToEclipse.ocl.properties?root=Modeling_Project&view=markup For OCL]:

 # pattern to use when searching for features to include in coordsite; defaults to ".*eclipse/features/org.eclipse..*(${subprojectName}|${subprojectName}.sdk)_.*\/$" 
 coordsiteFeaturePattern=".*eclipse/features/org.eclipse.(emf.ocl|ocl.all|ocl.all.sdk|ocl|ocl.uml)_.*\/$"
 coordsiteMainFeaturePattern="org.eclipse.ocl.(all|all.sdk)"

To create aggregate features, such as [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.emf/org.eclipse.emf.transaction.releng/promoteToEclipse.transaction.properties?root=Modeling_Project&view=markup EMF Transaction] does to bundle up [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.emf/org.eclipse.emf.query.releng/promoteToEclipse.query.properties?root=Modeling_Project&view=markup EMF Query], [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.emf/org.eclipse.emf.validation.releng/promoteToEclipse.validation.properties?root=Modeling_Project&view=markup EMF Validation], and EMF Transaction into a single "emfqtv.all.sdk" feature:

 # pattern to use when searching for features to include in coordsite; defaults to ".*eclipse/features/org.eclipse..*(${subprojectName}|${subprojectName}.sdk)_.*\/$" 
 #omit SDK because this is only to create a temp file which will be combined w/ rest of QTV components
 coordsiteFeaturePattern=".*eclipse/features/org.eclipse.emf.(emfqtv.all|emfqtv.all.sdk|query|query.ocl|validation|validation.ocl|transaction|workspace)_.*\/$"
 coordsiteMainFeaturePattern="org.eclipse.emf.emfqtv.(all|all.sdk)"

Then, in the [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.emf/org.eclipse.emf.validation.releng/promoteToEclipse.validation.properties?root=Modeling_Project&view=markup EMF Validation] (and [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.emf/org.eclipse.emf.query.releng/promoteToEclipse.query.properties?root=Modeling_Project&view=markup EMF Query]) properties files:

 # pattern to use when searching for features to include in coordsite; defaults to ".*eclipse/features/org.eclipse..*(${subprojectName}|${subprojectName}.sdk)_.*\/$" 
 #omit SDK because this is only to create a temp file which will be combined w/ rest of QTV components
 coordsiteFeaturePattern=".*eclipse/features/org.eclipse.emf.(validation|validation.ocl)_.*\/$"
 coordsiteMainFeaturePattern="NONE"
</td><td>[[Image:EMFT-Build-Page2.png|thumb|right|Build Promotion Page]]</td>
</tr></table>

=====Troubleshooting=====

See [[Modeling_Project_Releng/Releasing/Update_Site_Generation/Troubleshooting#How_do_I_fix_this.3F|Troubleshooting]].

===Generating Release Notes===

Generating release notes is easy - [http://dev.eclipse.org/viewcvs/indextech.cgi/*checkout*/org.eclipse.emft/releng/common/scripts/promoteToEclipse.sh?rev=HEAD&content-type=text/vnd.viewcvs-markup <tt style="color: DarkGreen">promoteToEclipse.sh</tt>] does it for you! However, there are a few simple steps to follow to ensure the generated content is accurate &amp; non-duplicative. Here's the general process for closing a bug and using Bugzilla &amp; CVS to provide the information required to generate release notes.

* Working in Eclipse, fix your code. Test is as much as possible before committing it to CVS.
* Commit your change(s) to CVS, using the bug's number in square brackets to begin your commit comment, eg:

 [127509] javadocs now available

* If you use Mylyn you can use this template, with a space at the end:

 [{$task.id}] 

* Note that you can include multiple bugs (each number on its own line) and comments are not required. The information that will appear in the release notes comes from the bug's <tt style="color: DarkGreen">Summary</tt> field, not your CVS comments.
* Browse to the bug's webpage, and assign it the status <tt style="color: DarkGreen">Assigned</tt>. Your bugzilla comment could be something like:

 Committed to CVS

* Tip: to save time, you can create &amp; save a named Bugzilla query for [https://bugs.eclipse.org/bugs/colchange.cgi?rememberedquery=product%3DEMFT%26bug_status%3DASSIGNED%26order%3Dbugs.bug_id%26query_format%3Dadvanced&amp;column_changeddate=on&amp;column_bug_severity=on&amp;column_priority=on&amp;column_rep_platform=on&amp;column_bug_status=on&amp;column_product=on&amp;column_component=on&amp;column_version=on&amp;column_target_milestone=on&amp;column_short_short_desc=on&amp;splitheader=0 <tt style="color: DarkGreen">All Assigned</tt>], in order to quickly [https://bugs.eclipse.org/bugs/colchange.cgi?rememberedquery=product%3DEMFT%26bug_status%3DASSIGNED%26order%3Dbugs.bug_id%26query_format%3Dadvanced%26tweak%3D=1&amp;column_changeddate=on&amp;column_bug_severity=on&amp;column_priority=on&amp;column_rep_platform=on&amp;column_bug_status=on&amp;column_product=on&amp;column_component=on&amp;column_version=on&amp;column_target_milestone=on&amp;column_short_short_desc=on&amp;splitheader=0 <tt style="color: DarkGreen">Change Several Bugs At Once</tt>].

* Produce a build, and promote it. Unless you use the <tt style="color: DarkGreen">-norss</tt> flag:
** Your [http://dev.eclipse.org/viewcvs/index.cgi/www/emft/feeds/?root=Eclipse_Website RSS feed] will be updated, 
** The Modeling database will see that new entry,
** A corresponding entry will be added to the Releases table, and
** A refresh of the [http://www.eclipse.org/modeling/emft/news/ release notes] will now show that new entry.

====Troubleshooting====

=====My new build isn't listed on the website!=====

Mirror replication takes time. Please try again in 30 mins, or re-read your promote log to verify everything went OK.

=====My new build isn't listed in the Release Notes!=====

First, check your RSS feed file to ensure it's been updated. If it hasn't, check your promote log for failures in updating the feed. If it has, please be patient and wait for the next database update. Your RSS feed file controls the list of releases that are imported into the Releases table of the Search CVS database, and thus appear on the Release Notes page.

There are three rules to consider when building, naming, and publishing maintenance builds:

# If you've branched, you need to tell the build system (and thus the RSS feed generator). Maintenance builds from HEAD are ok, but may confuse the database if you try to branch during a release rather than immediately afterwards. Branching right after your 1.0 release is recommended.
# Maintenance builds on, say, 1.0.0 MUST BE VERSIONED to 1.0.1, 1.0.2: if 1.0.0 has already been released, everything since must be > 1.0.
# Maintenance builds should NOT use an alias of "1.0.0"; they should simply default to MyyyymmddHHMM. You can only have ONE unique "1.0.0" in the database.

=====There are errors in my release notes! How do I edit?=====

If you want to remove or edit values in your RSS feed and flush old/incorrect values from the database, please edit the file /cvsroot/org.eclipse/www/modeling/''projectName''/feeds/builds-''componentName''.xml (eg., ''projectName'' = emft; ''componentName'' = teneo). When done, send an email to [[User:nickb]] requesting a database flush for your component's releases.

=====Bugs are missing from the Release Notes!=====

Database updates can take a while. If it's been 24hrs and you '''*still*''' are missing data, contact [[User:nickb]].

=====RSS Update Errors When Promoting=====

{| border="1" align="center" cellpadding="3" cellspacing="1"
! Problem !! Solution 
|-valign="top" 
|
 addEntry:
  [AddEntry] [Fatal Error] builds-teneo.xml:13:2: 
    The content of elements must consist of 
    well-formed character data or markup.
  [AddEntry] org.xml.sax.SAXParseException: 
    The content of elements must consist of 
    well-formed character data or markup.

 || Check the file on the build server in [http://emft.eclipse.org/emft/feeds/ /var/www/html/emft/feeds/]. Is it corrupt? Is it full of invalid XML due to a CVS update/merge? If so, so the following:
 cd /var/www/html/emft/feeds/; \
 rm -f builds-<tt style="color: DarkGreen">projectName</tt>.xml; \
 cvs up -Pd; \
 chmod 664 builds-<tt style="color: DarkGreen">projectName</tt>.xml; \
 chgrp www builds-<tt style="color: DarkGreen">projectName</tt>.xml;
Then rerun your promote using the <tt style="color: DarkGreen">-rssonly</tt> flag to redo that single step. After that, you can force an update of the database - including loading any new CVS commits - with the <tt style="color: DarkGreen">-searchcvsonly</tt> flag. If you don't need to refresh commits, just the Releases table, you can run this (on emft.eclipse.org only):
 cd /shared/modeling/searchcvs; \
 php relupdate.php -noheat
|-valign="top" 
|
 Permission denied

 || Check the file on the build server in [http://emft.eclipse.org/emft/feeds/ /var/www/html/emft/feeds/]. Is it writable by you or one of the groups of which you're a member? Try this:
 cd /var/www/html/emft/feeds/; \
 ls -la; groups; \
 chmod 664 *; chgrp www *;
Then rerun your promote using the <tt style="color: DarkGreen">-rssonly</tt> flag to redo that single step. After that, you can force an update of the database - including loading any new CVS commits - with the <tt style="color: DarkGreen">-searchcvsonly</tt> flag. If you don't need to refresh commits, just the Releases table, you can run this (on emft.eclipse.org only):
 cd /shared/modeling/searchcvs; \
 php relupdate.php -noheat
|}

=== Automated 'Fix available in ...' Bugzilla Comments ===

As of 2008-01-21, you can use [http://emft.eclipse.org/modeling/emft/compare/build/promo.php promo.php] or the <tt style="color: DarkGreen">-bugzonly</tt> flag to automatically move your RESOLVED:FIXED bugs to VERIFIED:FIXED, and place a comment stating in which build & branch the bug was fixed. See also {{bugstrike|206558}}, {{bugstrike|191571}}.

If you prefer, you can enable this step to happen automatically with every promoted build using these parameters in your promoteToEclipse.*.properties file

{{codeblock|<nowiki># do search cvs / release notes database update - 1 = do, 0 = skip
searchCVS=1
 
# how long to block before giving up (in seconds, eg., 1200 for 20 mins wait)
searchCVSTimeout=1200
 
#automatically move bugzillas from RESOLVED:FIXED to VERIFIED:FIXED - 1 = do, 0 = skip
bugz=1
 
# check for the existence of the new build in the release notes database 
# before trying to close bugs - 1 = do, 0 = skip
checkReleaseExists=1</nowiki>}}

====Setup / Troubleshooting====

The first time you use this option, or if your bugzilla cookie expires, you will receive an error like this:

{{codeblock|<nowiki>[promote] ERROR: login cannot be read from ~/.ssh/UpdateBugStateTask.properties.</nowiki>}}

If your cookie expires (and one day, it will), you'll see this:

{{codeblock|<nowiki>[UpdateBugState] Could not use your login/logincookie credentials to log in.
[UpdateBugState] Bugzilla replied: "I need a legitimate login and password to continue."</nowiki>}}

To regenerate credentials / cookie information, log in to the build server and run:

{{codeblock|<nowiki>cd /home/www-data/build/modeling/scripts; ./bugzillaLogin.sh</nowiki>}}

If you pipe the command's syserr directly to your UpdateBugStateTask.properties file, you won't have to copy & paste (as per the directions in the script).

{{codeblock|<nowiki>cd /home/www-data/build/modeling/scripts; 
./bugzillaLogin.sh 2> ~/.ssh/UpdateBugStateTask.properties</nowiki>}}

===Newsgroup Announcement Setup===

Setup for posting automatically to the newsgroup to announce new builds is easy.

: 1. Copy <tt style="color: DarkGreen">newsgroup-post.txt</tt> into your <tt style="color: DarkGreen">~/.ssh</tt> folder.

<tt style="color: #990000; background-color: #FFFFCC;">''build_id''@''build_server'':~ $</tt> <tt style="color: DarkGreen">mkdir -p ~/.ssh; cp /home/nickb/newsgroup-post/newsgroup-post.txt ~/.ssh</tt>

You are now set up to post automatically to the newsgroup while promoting a build. If you want to post as a specific person, rather than the [http://dev.eclipse.org/viewcvs/indextech.cgi/*checkout*/org.eclipse.emft/releng/common/scripts/promoteToEclipse.properties default name &amp; email], there are two additional steps.

: 2. Edit <tt style="color: DarkGreen">org.eclipse.emft/releng/''[subproject]''/promoteToEclipse.''[subproject]''.properties</tt> <span style="font-size: 9pt"><[http://dev.eclipse.org/viewcvs/indextech.cgi/*checkout*/org.eclipse.emft/releng/jet/promoteToEclipse.jet.properties example]></span>, placing the correct name and email address for the person whose name will appear on newsgroup posts. This step - and the next one - is '''ONLY''' required if the name &amp; email are not the same as the default value recorded [http://dev.eclipse.org/viewcvs/indextech.cgi/*checkout*/org.eclipse.emft/releng/common/scripts/promoteToEclipse.properties here].

 newsgroupPublisherEmail="John Q. Committer <john_q_emft_committer_guy@eclipse.org>"

: 3. Copy <tt style="color: DarkGreen">org.eclipse.emft/releng/''[subproject]''/promoteToEclipse.''[subproject]''.properties</tt> <span style="font-size: 9pt"><[http://dev.eclipse.org/viewcvs/indextech.cgi/*checkout*/org.eclipse.emft/releng/jet/promoteToEclipse.jet.properties example]></span> into <tt style="color: DarkGreen">/home/www-data/build/emft/scripts</tt>.

To announce a build that's already been built, you can use the <tt style="color: DarkGreen">-announceonly</tt> flag to skip all other steps and ONLY post to the newsgroup. Otherwise, adding <tt style="color: DarkGreen">-announce</tt> when promoting will add the extra step of posting a quick note to the newsgroup.

====Newsgroup Post Threading====

Should you want to thread your announcements so that they can be hidden or categorized, here's how you can do so:

# create the parent thread into which subsequent posts will go (just post a message to the newsgroup), then 
# get the post's <tt style="color:DarkGreen">Message-ID</tt> (using Thunderbird, just turn on all headers to see this), and
# update the appropriate entry in your <tt style="color:DarkGreen">/home/www-data/build/modeling/scripts/promoteToEclipse.*.properties</tt> file on your build server (emf.torolab or emft.eclipse).

Looking at promoteToEclipse.jet.properties as an example:

 # publisher of newsgroup announcements
 newsgroupPublisherEmail="John Q. Committer <john_q_emft_committer_guy@eclipse.org>"
 newsgroup=eclipse.technology.emft
 # to thread posts under an existing post (or posts) enter the slash-escaped Message-ID for which this is a reply,
 # eg: <e0i2gc\$an5\$1@utils.eclipse.org>
 <tt style="color:DarkGreen">newsgroupThreadReferences</tt>=""

If you put the <tt style="color:DarkGreen">Message-ID</tt> in the <tt style="color:DarkGreen">newsgroupThreadReferences</tt> property, announcements about builds will be nested under that existing message.

Note that to edit this file you must sudo to the web user:

* On emf.toro: <tt style="color:DarkGreen">sudo -u '''www-data''' vi promoteToEclipse.jet.properties</tt>
* On emft.eclipse: <tt style="color:DarkGreen">sudo -u '''apache''' vi promoteToEclipse.cdo.properties</tt>

If you'd like to store a copy of your changes in CVS, please contact your releng manager so that your changes can be committed into your releng module for safekeeping/versioning.

===SSH Key Setup===

====Getting Access====
You must have ssh access to your build server, download1.eclipse.org, and dev.eclipse.org. If you do not have access, please contact the administrators of these servers to gain access. You need a '''full login shell''' (eg., bash), not just a CVS login shell (cvssh). If you cannot ssh into dev.eclipse.org or download1.eclipse.org, please open a bug asking for a full login shell, eg., [https://bugs.eclipse.org/bugs/show_bug.cgi?id=206146 bug 206146].

To get access to your build server, you must forward your static IP address to the [[User:Nickb | server's administrator]]. If you do not have a static IP, you can set up your router to broadcast your IP to a domain name registration service like [http://www.dyndns.org dyndns.org] and provide your domain name for access to the build server.

Once you have ssh access, you need to set up '''limited''' ssh key access between your build server and the eclipse.org download and cvs servers. This will allow you to run the promote script without being prompted for a password periodically.

====Connecting in====

To grant yourself ssh access from your build server to eclipse.org, ssh to the build server normally.

Under windows, you can use [http://www.cygwin.com/ cygwin] or [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY]. Under Linux, open a shell and type `ssh`.

====Generating private and public SSH keys====

If you don't already have an ssh key pair created, you will need to generate one.

: <tt style="color: #990000; background-color: #FFFFCC;">''build_id''@''build_server'':~ $</tt> <tt style="color: DarkGreen">mkdir -p ~/.ssh; chmod 700 ~/.ssh; cd ~/.ssh; ssh-keygen -b 2048 -t rsa</tt>

When prompted for a passphrase, hit enter to skip. Having a passphrase, while more secure than having no passphrase, prevents automated login.

Save your public key in the default place: <tt style="color: DarkGreen">~/.ssh/id_rsa.pub</tt>.

Then, you have to create the authorized_keys file and append to it your public key :
: <tt style="color: #990000; background-color: #FFFFCC;">''build_id''@''build_server'':~/.ssh $</tt> <tt style="color: DarkGreen">touch ~/.ssh/authorized_keys; </tt>
: <tt style="color: #990000; background-color: #FFFFCC;">''build_id''@''build_server'':~/.ssh $</tt> <tt style="color: DarkGreen">cat ~/.ssh/id_rsa.pub ~/.ssh/authorized_keys;</tt>

====Connecting out: authorizing remote hosts to let you in====

To copy your local public key to the remote machine, you will need to scp (secure copy) the file. From the build server, copy the file to the remote server, then ssh to that server and combine the new key with the any existing keys in the authorized_keys file. In most cases, build_id == eclipse_id, eg., jlescot == jlescot:

Note that if you have not created yet the .ssh folder on the dev.eclipse.org server, you first need to achieve it using the following instructions :
: <tt style="color: #990000; background-color: #FFFFCC;">''build_id''@''build_server'':~/.ssh $</tt> <tt style="color: DarkGreen">ssh ''eclipse_id''@dev.eclipse.org</tt>
:: <tt style="color: #990000; background-color: #FFFFCC;">''eclipse_id''@dev.eclipse.org:~ $</tt> <tt style="color: DarkGreen">mkdir -p ~/.ssh; chmod 700 ~/.ssh;</tt>
:: <tt style="color: #990000; background-color: #FFFFCC;">''eclipse_id''@dev.eclipse.org:~/.ssh $</tt> <tt style="color: DarkGreen">exit</tt>
Then you can execute the following scp command :

: <tt style="color: #990000; background-color: #FFFFCC;">''build_id''@''build_server'':~/.ssh $</tt> <tt style="color: DarkGreen">scp id_rsa.pub ''eclipse_id''@dev.eclipse.org:~/.ssh/''eclipse_id''.temp</tt>

: <tt style="color: #990000; background-color: #FFFFCC;">''build_id''@''build_server'':~/.ssh $</tt> <tt style="color: DarkGreen">ssh ''eclipse_id''@dev.eclipse.org</tt>
::: ''(you will still be prompted for a password when connecting)''

:: <tt style="color: #990000; background-color: #FFFFCC;">''eclipse_id''@dev.eclipse.org:~ $</tt> <tt style="color: DarkGreen">cd ~/.ssh; </tt>
:: <tt style="color: #990000; background-color: #FFFFCC;">''eclipse_id''@dev.eclipse.org:~/.ssh $</tt> <tt style="color: DarkGreen">touch ~/.ssh/authorized_keys ~/.ssh/known_hosts; </tt>
:: <tt style="color: #990000; background-color: #FFFFCC;">''eclipse_id''@dev.eclipse.org:~/.ssh $</tt> <tt style="color: DarkGreen">cp ~/.ssh/authorized_keys ~/.ssh/authorized_keys_old; # backup</tt>
:: <tt style="color: #990000; background-color: #FFFFCC;">''eclipse_id''@dev.eclipse.org:~/.ssh $</tt> <tt style="color: DarkGreen">cat ~/.ssh/''eclipse_id''.temp ~/.ssh/authorized_keys_old > ~/.ssh/authorized_keys; # prepend new key into old file; save as new file</tt>
:: <tt style="color: #990000; background-color: #FFFFCC;">''eclipse_id''@dev.eclipse.org:~/.ssh $</tt> <tt style="color: DarkGreen">rm -fr ~/.ssh/''eclipse_id''.temp ~/.ssh/authorized_keys_old;</tt>
::  
:: <tt style="color: #990000; background-color: #FFFFCC;">''eclipse_id''@dev.eclipse.org:~/.ssh $</tt> <tt style="color: DarkGreen">chmod 644 authorized_keys id_rsa.pub known_hosts; chmod 600 id_rsa; # fix perms</tt>
:: 
:: <tt style="color: #990000; background-color: #FFFFCC;">''eclipse_id''@dev.eclipse.org:~/.ssh $</tt> <tt style="color: DarkGreen">exit</tt>

: <tt style="color: #990000; background-color: #FFFFCC;">''build_id''@''build_server'':~/.ssh $</tt> <tt style="color: DarkGreen">ssh ''eclipse_id''@dev.eclipse.org</tt>
::: ''(this time you should get in using your key instead of needing a password)''

Once you have <tt style="color: DarkGreen">~/.ssh/authorized_keys</tt> and <tt style="color: DarkGreen">~/.ssh/known_hosts</tt> updated with your public key and server fingerprint(s) (respectively), you should be able to connect to these systems without being prompted, and the promote script should be good to go unassisted.

====Connecting out: creating host keys ====

SSH to your build server (if not already there). Then, one by one, ssh from there to these target systems, accepting whatever yes/no questions are posed about host keys or IP addresses. Once in, <code>exit</code> back to your build server. In most cases, ''build_id'' == ''eclipse_id'', eg., jlescot == jlescot:

* <tt style="color: DarkGreen">''build_id''@''FQDN''</tt>, <tt style="color: DarkGreen">''build_id''@localhost</tt>, 
* <tt style="color: DarkGreen">''eclipse_id''@download1.eclipse.org</tt>, <tt style="color: DarkGreen">''eclipse_id''@dev.eclipse.org</tt>

For example:

<table><tr><td>
 ssh jlescot@emft.eclipse.org
 exit
 ssh jlescot@localhost
 exit
 ssh jlescot@download1.eclipse.org ''[publishing build folder, zips, and javadoc]''
 exit
 ssh jlescot@dev.eclipse.org       ''[CVS access]''
 exit
 ssh jlescot@build.eclipse.org     ''[jar signing and update site generation]''
 exit
</td></tr></table>

All of those should work without being prompted for a password. If you are prompted, you need to set an ssh key in your <tt style="color: DarkGreen">~/.ssh/authorized_keys</tt> file for each user &amp; hostname pair to which you want to connect. The first time you try each of these connections, you should be be prompted to add the host's fingerprint to your <tt style="color: DarkGreen">~/.ssh/known_hosts</tt> file. This is also required.

====Connecting the web user to your account====

In order to run builds from the web site on the build server, the web user (apache or www-data) must be able to ssh to your account on the build server. Put its public key in your authorized_keys file to make this work.

====Additional====

If you require further assistance setting up key access, try here: http://cfm.gs.washington.edu/security/ssh/client-pkauth/.

==Promotion FAQ==

Many things can go wrong when promoting a build (CVS or SCP errors, artifacts not created, java failures). Most are a result of network or permission problems, and can be fixed by trying again or correcting the permission problem. 

===RSS Feeds===

=====The log file for my promoted build shows a FileNotFoundException (Permission denied), NullPointerException, or SAXParseException when publishing my RSS feed!=====

Either the permissions on the file do not allow you to edit it...

 Buildfile: feedPublish.xml
 publishFeed:
 addEntry:
   [AddEntry] java.io.FileNotFoundException: /var/www/html/modeling/emft/feeds/builds-net4j.xml (Permission denied)

... or your file has been corrupted by a CVS merge/update, and contains "<<<<<<<" and "=======" which makes it an invalid XML file.

 addEntry:
   [AddEntry] [Fatal Error] builds-gef.xml:7:2: The content of elements must consist of well-formed character data or markup.
   [AddEntry] org.xml.sax.SAXParseException: The content of elements must consist of well-formed character data or markup.
   [AddEntry]     at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:264)

To fix this, delete the file on filesystem, refresh from cvs & redo the promote. See also {{bugstrike|254645}}
 
Here's how: 
 
 $ cd /var/www/html/modeling/mdt/feeds/ 
 $ less builds-uml2.xml  
   ''(note the <<<<<<<<<<<<< and =============== markers that the cvs server added) ''
 $ rm -fr builds-uml2.xml 
 $ cvs up builds-uml2.xml 
   ''cvs update: warning: `builds-uml2.xml' was lost ''
   ''U builds-uml2.xml ''
 $ chmod g+w builds-uml2.xml
 $ chgrp www builds-uml2.xml

Then rerun the promote to fix the missing feed entry (required for release notes)...  

 ./promoteToEclipse.sh -sub uml2 -Q -buildID ''somebuildID'' -rssonly -cvsbranch ''somebranchvalue'' ...  

If you decide to repromote completely using the web UI, make sure you've removed this build's site*.xml contribution because a full promote will add it again, slowing down the operation of the update site and presenting users with confusing duplicate entries.

Finally, since the RSS is now updated, you can [http://build.eclipse.org/modeling/build/updateSearchCVS.php force the database to be refreshed] and have the new release appear in your Release Notes.

===Update Site===

Please check your promote logs for errors or transient glitches, like CVS disconnects or SCP problems. Sometimes one problem can manifest as several problems, as in the case of this EMFT Validation build's UM jar creation, which threw four warnings that something had gone wrong, all starting from a transient CVS checkout problem (bandwidth overload or connection reset).

 [promote] [9] [13:53:18] Create &amp; promote [[Update_Manager_Site_Generation|Update Manager]] jars:
 [promote] Running buildUpdate.sh:

 [umj-co] [1] Checking out org.eclipse.releng.basebuilder from dev 
   using HEAD
 cvs checkout: failed to open /home/vramaswamy/.cvspass for reading: 
   No such file or directory
 '''cvs [checkout aborted]: recv() from server dev.eclipse.org: 
   Connection reset by peer'''

 '''The java class is not found: org/eclipse/core/launcher/Main
 features/*.jar: No such file or directory
 plugins/*.jar: No such file or directory'''

 [umj] [7d] [13:54:47] Comparing local and remote folders MD5 sums 
   to ensure SCP completeness...

 [compare] Compare md5sums...
 '''[compare] MD5s do not match! Compare failed.'''

 [umj] [13:54:52] '''ERROR! Script exiting from compareFolders.sh'''
 [promote] [9] [13:54:52] Create &amp; promote [[Update_Manager_Site_Generation|Update Manager]] jars done.

In this case, the problems were:

* CVS disconnected while checking out required builder code
* Build failed looking for Eclipse Main, so no build occurred
* Copying files from temp to /var/www/technology/emft/updates/ failed as they were not created
* MD5 check failed as nothing was copied to download1.eclipse.org

In this case, the solution was to:

# Verify that nothing had been produced on either the build server or download1.eclipse.org.
# Confirm that site*.xml had not been changed in CVS, and rollback any changes for this build's ID (if any).
# Re-run the promote using the <tt style="color: DarkGreen">-jarsonly</tt> flag in order to bypass all the steps that completed successfully and ONLY do the UM jar creation step.

[[Category:Modeling]] [[Category:Releng]]