Jump to: navigation, search

Difference between revisions of "Modeling Project Releng/Maintenance"

(Manual Delete From Production Server (download.eclipse.org))
m (Manual Delete From Production Server (download.eclipse.org))
Line 84: Line 84:
 
* ask [[User:Nickb]] to update the Search CVS database to remove the build from the releases table, eg.:
 
* ask [[User:Nickb]] to update the Search CVS database to remove the build from the releases table, eg.:
  
  delete from releases where project='org.eclipse.emf' and component='org.eclipse.net4j' \
+
delete from releases where project='org.eclipse.emf' and component='org.eclipse.net4j' \
  and vanityname='M200809091234' limit 1;
+
and vanityname='M200809091234' limit 1;
  
 
* You may want to rollback the update site, too. Run this script to back up the latest addition to the update site:
 
* You may want to rollback the update site, too. Run this script to back up the latest addition to the update site:

Revision as of 09:24, 3 October 2008

Maintenance Processes

By following a few simple rules, you can ensure that you never run out of memory or hard disc space when building, and that your neighbours on the server are similarly unaffected.

  • Release builds should be kept at all times.
  • Stable builds should be kept until they are considered stale, eg., M1 need not be kept once M4 or M5 is available. This is subject to your discretion or customer needs.
  • Interim builds should be purged after every Milestone build so that you never have more than a few weeks of Interim builds listed.
  • Nightly builds should be scrubbed every week or so in order to save space and keep your build server's downloads page clean.


Removing CVS Tags

To remove CVS tags, you can run the following commands (from any machine that can connect to dev.eclipse.org via ssh):

cvs -d [eclipse_id]@dev.eclipse.org:/cvsroot/technology -q rtag -d [build_tag] org.eclipse.emft/[subproject];
cvs -d [eclipse_id]@dev.eclipse.org:/cvsroot/technology -q rtag -d [build_tag] org.eclipse.emft/releng/[subproject];

where:

  • eclipse_id is your cvs username for dev.eclipse.org
  • build_timestamp is "build_" plus the timestamp of the build for which you want to remove its tag from CVS. For example, for a milestone build 1.0.0M6a (S200604131352) (RC1, use build_200604131352.

Note that in order to remove tags from within org.eclipse.emft/releng you must be a member of the emft-releng group on dev.eclipse.org, or have similar write access via ACL.


Cleaning Failed Builds

Failed builds can occupy up to 2G of space each, which can chew up the disk pretty fast. To purge the temporary files left over from a failed build, you can either delete them by hand, or run the /home/www-data/build/modeling/scripts/cleanBuildFolder.sh script at a folder. For ease of use, you can symlink it in your build folder:

cd /home/www-data/build/modeling/emft/teneo/downloads/drops/0.8.0
sudo -u [web_user] ln -s /home/www-data/build/modeling/scripts/cleanBuildFolder.sh clean
sudo -u [web_user] ./clean ./S2008021*

or, just delete the entire build folder:

cd /home/www-data/build/modeling/emft/teneo/downloads/drops/0.8.0
sudo -u [web_user] rm -fr S200802021234

where:

  • web_user is the id on your build server which runs the Apache webserver processes, eg., apache or www-data

Removing Old Builds

The simplest way to remove old builds is simply to delete them, both from the build server and from the public download1.eclipse.org server. Doing so will also (eventually) cause them to be removed from the eclipse.org mirrors.

Manual Delete From Build Server

To delete an old build, ssh to your build server, then run, for example:

cd /home/www-data/build/emft/[subproject]/downloads/drops/1.0.0;
sudo -u [web_user] rm -fr N200604152353;

where:

  • web_user is the id on your build server which runs the Apache webserver processes, eg., www-data or apache

Automated Delete From Build Server

There is also a nightly crontab script that runs to trash, then purge old builds:

#Min    Hr      Mday    Month   Wday    Cmd
01 01 * * * sudo -u apache /home/www-data/build/modeling/scripts/cleanupBuilds.sh -a > $HOME/cron_logs/cleanupBuilds.sh.txt 2>&1

The rules, as defined in the script, are:

  1. After 14 days, purge from /home/www-data/OLD/
  2. After 2 days, purge old [N] builds
  3. After 21 days, move old [M] builds into OLD/
  4. After 14 days, move old [I] builds into OLD/
  5. After 42 days, move old [S] builds into OLD/
  6. After 42 days, purge old dependency zips in /home/www-data/build/downloads/

Manual Delete From Production Server (download.eclipse.org)

When deleting from dev.eclipse.org (or download1.eclipse.org), you do not need to sudo to the web user:

cd /home/data/httpd/download.eclipse.org/technology/emft/[subproject]/downloads/drops/1.0.0;
rm -fr N200604152353;

If you are removing an accidentally promoted build, you will also need to:

  • update your RSS feed to remove that <entry>
  • ask User:Nickb to update the Search CVS database to remove the build from the releases table, eg.:
delete from releases where project='org.eclipse.emf' and component='org.eclipse.net4j' \
and vanityname='M200809091234' limit 1;
  • You may want to rollback the update site, too. Run this script to back up the latest addition to the update site:
/home/www-data/build/modeling/scripts/buildUpdateSiteRollback.sh

Archiving Builds on Production Server (download.eclipse.org)

Nightly builds can just be destroyed, but consumers of your component may have longer-term dependencies on Integration builds. After a couple of releases, maybe even milestone builds want to be cleared away. This can all be accomplished safely by archiving a build. A build that is in the archive doesn't take up space on the main Eclipse download server or its mirrors, but is still accessible in the event that it is needed via the download.php?file=/path/to/zipfile.zip URL.

Simply move your build from from the main download area to the archive area:

cd /home/data/httpd/download.eclipse.org/technology/emft/[subproject]/downloads/drops/[release]/[buildID]
mkdir -p /home/data/httpd/archive.eclipse.org/technology/emft/[subproject]/downloads/drops/[release]
mv [buildID] /home/data/httpd/archive.eclipse.org/technology/emft/[subproject]/downloads/drops/[release]/

and you're done!

For old releases (once 0.8.2 is out, you might want to archive 0.8.0 and 0.8.1), you can copy the build over to archive.eclipe.org as above, then update your downloads page's extras-*.php file to explicitly list these builds in the $oldrels array. See examples below.

$oldrels = array(
  // release version => timestamp
  "0.7.1" => "200610021452",
  "0.7.0" => "200609251359",
);
$oldrels = array(
  "JET 0.7.2" => array("2007/02/08 11:15",
    "http://archive.eclipse.org/modeling/m2t/jet/downloads/drops/0.7.2/R200702081115/"),
  // ...
  "JET Editor 0.8.0" => array("2007/03/03 00:36",
    "http://archive.eclipse.org/modeling/m2t/jet/downloads/drops/0.8.0/I200703030036/")
);

Removing Update Manager Jars

Update Manager jars should be purged from build and production servers with the same frequencies as above.

Remove Jar Files

To get a list of jars (ordered by modification time) that are older than a certain age (eg., 14 days), run:

 cd /home/data/httpd/download.eclipse.org/technology/emft/updates;
 find . -maxdepth 2 -type f -name "*eodm*" -mtime +14 \
   -exec date -r {} +%s\ {} \; | sort -r | sed -e "s/[0-9]\+\ \.\///g";

Then, if you are satisfied with that list, delete them like this:

 for f in \
   `find . -maxdepth 2 -type f -name "*eodm*" -mtime +14 \
     -exec date -r {} +%s\ {} \; | sort -r | sed -e "s/[0-9]\+\ \.\///g"`; do \
       echo "Delete: "$f; rm -fr $f; \
 done

Remove Entries From site*.xml

In addition to deleting jars, you must also remove the entries from site-interim.xml. This file must be edited in CVS then checked out to the local filesystem to cause the removed builds to disappear.

If you do not update the file in CVS, but only edit it locally, the next UM jar build will recreate the removed entries since it extracts the latest version of that file from CVS before adding to it. To update the file in CVS and push it to eclipse.org, ssh to your build server or to any machine that can ssh to dev.eclipse.org, then:

cd /tmp;
cvs -d [eclipse_id]@dev.eclipse.org:/cvsroot/org.eclipse \
  -q co -d updates www/emft/updates;
cd /tmp/updates;
# edit file: use dd to delete lines, :wq to save & quit
vi site-interim.xml; 
cvs -q ci -m "remove old builds"; 
scp site-interim.xml \
  [eclipse_id]@dev.eclipse.org:/home/data/httpd/download.eclipse.org/technology/emft/updates
cd /tmp; rm -fr /tmp/updates; 

Of course, you can also edit the file in Eclipse or another editor, commit those changes, then check out the latest files and scp them to download.eclipse.org as shown above.


Build.php Dependency URL Cleanup

To remove old dependency URLs shown on your build.php page, edit the following file on your build server:

 sudo -u apache vi /home/www-data/build/requests/dependencies.urls.txt # on emft.eclipse
   - or -
 sudo -u www-data vi /home/www-data/build/requests/dependencies.urls.txt # on emf.torolab

Note that the most recent entries are listed at the bottom of the file, so delete from the top down (starting on line 3). If you delete too many, you can always re-add them using the web UI of build.php.

Removing or Replacing a Bad Build

Yes, it happens. You need to trash a milestone build and replace it with an M5a version. We've been there. Here's the steps you need to follow to make a bad build vanish forever using a hypothetical example.

0. Start a new build. This takes the most time and all the remaining steps can be done in parallel.

1. SSH to download1.eclipse.org and delete the bad build's folder.

cd /home/data/httpd/download.eclipse.org/modeling/mdt/uml2/downloads/drops/2.1.0
rm -fr I200704091234

2. Delete the build from the build server too.

3. Check out your project's RSS feed from CVS, remove the bad entry, and commit your changes. When you promote the replacement build, this file will be promoted to the correct place on download1.eclipse.org. The RSS feed controls what releases are loaded in the Search CVS / Release Notes database, so you must remove the bad entry.

cd /tmp;
cvs -d username@dev.eclipse.org:/cvsroot/org.eclipse -q co -d feeds \
  www/modeling/mdt/feeds
cd feeds;
vi builds-uml2.xml
cvs ci -m "remove bad build" builds-uml2.xml

4. If you're not replacing your bad build w/ a fixed one, you'll have to manually upload the RSS feed file; otherwise, your next build will perform this step for you:

 scp builds-uml2.xml \
  username@download1.eclipse.org:/home/data/httpd/download.eclipse.org/modeling/mdt/feeds/

5. Delete the entry from the releases table of the Search CVS database so it will vanish from the Release Notes. Use this webpage. If you don't see your build listed, then it hasn't been loaded yet, and you can safely skip this step.

6. Remove the build from the appropriate site-interim.xml and/or site-interim-europa.xml file(s).

cd /tmp;
cvs -d username@dev.eclipse.org:/cvsroot/org.eclipse -q co -d updates \
  www/modeling/mdt/updates
cd updates;
vi site-interim*.xml
cvs ci -m "remove bad build" site-interim*.xml

7. Promote your new build, sit back, and call it a day.