EMFT Build Server Setup

You will need to be root for most of these tasks.

Set up web content

Copy content from an existing server setup. Most of the content exists in CVS as well (/cvsroot/technology/org.eclipse.emft/releng/common):

FROM (emf.torolab.ibm.com)		TO (emft.eclipse.org)

/home/www-data/build/emft	-->	/home/www-data/build/emft
/var/www/emft	-->	/var/www/html/emft
/var/www/technology/emft	-->	/var/www/html/technology/emft

Fix permissions & ownership

cd /var/www/html; find . -type f -exec chmod 664 {} \;
cd /var/www/html; find . -type d -exec chmod 775 {} \;
cd /var/www/html; find . -exec chown apache:www {} \;

cd /home/www-data/build; find . -type f -exec chmod 664 {} \;
cd /home/www-data/build; find . -type d -exec chmod 775 {} \;
cd /home/www-data/build; find . -exec chown apache:www {} \;
cd /home/www-data/build/scripts; find . -type f -name "*.sh" -exec chmod 755 {} \;

Apache 2 w/ PHP 5 was already installed. Only a few additional programs needed to be installed as well.

Install via `yum` using yum-xen.conf file

Get a list of the available updates

yum -c http://people.redhat.com/~katzj/yum-xen.conf list 2>&1 | tee /tmp/yum-list.txt

Install required groups of installs - Java Development & Development Tools

yum -c http://people.redhat.com/~katzj/yum-xen.conf groupinstall "Java Development"
yum -c http://people.redhat.com/~katzj/yum-xen.conf groupinstall "Development Tools"

Install gtk- and x11-related packages (in order to do Eclipse UI JUnit tests)

yum -c http://people.redhat.com/~katzj/yum-xen.conf install gtk2.i386

=============================================================================
 Package                 Arch       Version          Repository        Size
=============================================================================
Installing:
 gtk2                    i386       2.6.10-2.fc4.4   updates-released  4.8 M
Installing for dependencies:
 atk                     i386       1.9.1-1          base              178 k
 pango                   i386       1.8.1-2          base              272 k

yum -c http://people.redhat.com/~katzj/yum-xen.conf install xorg-x11.i386 xorg-x11-libs.i386 xorg-x11-xdm.i386

=============================================================================
 Package                 Arch       Version          Repository        Size
=============================================================================
Installing:
 xorg-x11                i386       6.8.2-37.FC4.49.2  updates-released   14 M
 xorg-x11-xdm            i386       6.8.2-37.FC4.49.2  updates-released  170 k
Updating:
 xorg-x11-libs           i386       6.8.2-37.FC4.49.2  updates-released  2.5 M
Installing for dependencies:
 chkfontpath             i386       1.10.0-4         base               13 k
 fonts-xorg-base         noarch     6.8.2-1          base              7.3 M
 ttmkfdir                i386       3.0.9-16.1       updates-released   49 k
 xinitrc                 noarch     4.0.18.1-1       updates-released   28 k
 xorg-x11-font-utils     i386       6.8.2-37.FC4.49.2  updates-released  124 k
 xorg-x11-xauth          i386       6.8.2-37.FC4.49.2  updates-released  102 k
 xorg-x11-xfs            i386       6.8.2-37.FC4.49.2  updates-released  141 k
 xterm                   i386       208-4.FC4        updates-released  191 k

Install & symlink

IBM JDK 1.4.2sr4-1 (symlink to /opt/ibm-java2-1.4)
Sun JDK 1.5.0_06 (symlink to /opt/sun-java2-5.0)
Apache Ant 1.6.5 (symlink to /opt/apache-ant-1.6)

Fix web user (apache)

Edit /etc/group. Add www group:

/etc/group

www:x:501:user,apache

Edit /etc/passwd. Change user's home directory and shell:

/etc/passwd

apache:!!:48:48:Apache:/home/www-data:/bin/bash

Edit /etc/sudoers to let you run commands and switch to the web user w/o needing a password. Add the following lines:

/etc/sudoers

%www  ALL = (apache) NOPASSWD: ALL, (root) /usr/bin/su apache

Switch to the web user. You should NOT be prompted for a password.

sudo -u apache bash

Create an ssh key, WITH NO PASSPHRASE. Store in ~/.ssh/id_rsa and ~/.ssh/id_rsa.pub:

ssh-keygen -b 2048 -t rsa

Copy contents of ~/.ssh/id_rsa.pub into ~/.ssh/authorized_keys file for user on dev.eclipse.org who

will be running builds. This is so that the web user can commit changes to cvs (tagging, updating map files) for I, M, S & R builds.

Test by ssh'ing to dev.eclipse.org, where _username_ should be replaced with your actual username:

ssh _username_@dev.eclipse.org

Run newgrp www so that when new files are created, they will use the group id www instead of apache:

newgrp www

Set umask 022 so that files will be created with group write perms 664 (see #/etc/bashrc):

umask 022

Set the remote shell connection method for CVS to be ssh instead of the default rsh (see #/etc/bashrc):

export CVS_RSH=/usr/bin/ssh

Set an ANT_HOME and JAVA_HOME, and add ant to the PATH (see #/etc/bashrc):

export ANT_HOME=/opt/apache-ant-1.6
export JAVA_HOME=/opt/sun-java2-5.0
export PATH=${PATH}:${ANT_HOME}/bin

Switch to the root user.

Append the following into /etc/bashrc, where _username_ should be replaced with your actual dev.eclipse.org username:

/etc/bashrc

umask 022
export ANT_HOME=/opt/apache-ant-1.6
export JAVA_HOME=/opt/sun-java2-5.0
export PATH=${PATH}:${ANT_HOME}/bin
export CVSROOT=:ext:_username_@dev.eclipse.org:/cvsroot/technology
export CVS_RSH=/usr/bin/ssh

Add the following to .bashrc and .bash_profile files:

~/.bashrc

if [ -f /etc/bashrc ]; then
  . /etc/bashrc
fi

if [ "$PS1" ]; then
  # enable color support of ls and also add handy aliases
  eval `dircolors -b`
  alias ls='ls --color=auto'
  alias ll='ls -l --color=auto'

  # set a fancy prompt
  # 1;30 - grey, 1;31 - red, 1;32 - green, 1;33 - yellow, 1;34 - blue, etc.
  PS1="\[\033[<b>1;30</b>m\]\u@\h:\w\\[\033[0;39m\] \$ "
  export PS1=$PS1"\[\e]30;\u@\H:\w\a\]"
  PATH=$PATH:/opt/ibm-java2-1.4/bin:/opt/apache-ant-1.6/bin
  export PATH
fi

source ~/.alias
cat ~/.alias

~/.bash_profile

if [ -f ~/.bashrc ]; then
  . ~/.bashrc
fi

PATH=$PATH:$HOME/bin
export PATH
unset USERNAME

Secure build script

Since the webserver is public but builds should only be run by authorized users, we must secure access to the build.php script.

Edit Apache config file to allow .htaccess rule changes to take affect. Change None to All:

/etc/httpd/conf/httpd.conf

# AllowOverride controls what directives may be placed in .htaccess files.
# It can be "All", "None", or any combination of the keywords:
#   Options FileInfo AuthConfig Limit
#
  AllowOverride All

Create password file, where _username_ should be replaced with an actual username and /path/to/password/file should be replaced with an actual path:

httpasswd -c /path/to/password/file _username_

You can add additional users to the password file like this:

httpasswd /path/to/password/file _username2_

Create .htaccess file, replacing /path/to/password/file with the same path used in the previous step:

/var/www/html/emft/.htaccess

AuthType Basic
AuthName "EMFT Build Server @ Eclipse.org"
AuthUserFile /path/to/password/file
Require valid-user

Restart apache, eg.:

/usr/sbin/httpd -k restart
 - or -
apache2ctl restart

For more on htaccess, see http://httpd.apache.org/docs/2.0/howto/auth.html.

Run a build

Using the username and password set up in the previous step, go here: http://emft.eclipse.org/emft/build.php.

Check the build log while it's running or after it completes. If you see any messages such as permission denied or the following, something is amiss.

Debugging tips

execvp: Permission denied

Shell scripts must contain unix line endings. Run dos2unix to make sure, if copying them from a non-unix filesystem.
Shell scripts /home/www-data/build/emft/scripts/*.sh must be executable. See #Fix permissions & ownership.

--

Directories, eg. /home/www-data/build/emft/jet/downloads/drops/1.0.0 must be writable by the web user

(or group www). See #Fix permissions & ownership.

--

eclipse.org: Connection refused
cvs [checkout aborted]: end of file from server (consult above messages if any)

CVS connection refusals are the result of not being able to automatically ssh as the web user to _username_@dev.eclipse.org. This

could be a problem with ssh keys (see above - #Fix web user (apache)) or the method CVS uses to connect (rsh instead of ssh - see #/etc/bashrc).

--

For any compilation-related problems, missing file problems, or other issues not touched upon in this document, see EMFT_Procedures.

Display build logs, details & test results

Builds - including unpublished Nightly builds - are listed on the downloads page here: http://emft.eclipse.org/technology/emft/downloads/.

Published builds are located here: http://downloads.eclipse.org/technology/emft/downloads/.

Add additional users

As root, run the following script:

#!/bin/bash

if [ $# -lt 1 ]; then
  echo "Usage: ./adduser <username>"
else
  last=`tail -2 /etc/group | head -1`; last=${last##*:x:}; last=${last%%:*}; (( last++ )); #echo $last
  newgrp=$1":x:"$last":"$1;

  echo -n "Adding: "
  echo $1"::"$last":"$last"::/home/"$1":/bin/bash"
  echo $1"::"$last":"$last"::/home/"$1":/bin/bash" >> /etc/passwd

  wwwgrp=`tail -1 /etc/group`","$1; #echo $wwwgrp;
  wc=`wc -l /etc/group`; wc=${wc%% *}; (( wc-- )); wc="-"$wc; #echo $wc;
  head $wc /etc/group > /etc/group.new

  echo -n "Adding: "
  echo $newgrp
  echo $newgrp >> /etc/group.new

  echo -n "Adding: "
  echo $wwwgrp
  echo $wwwgrp >> /etc/group.new

  mv /etc/group.new /etc/group

  echo $1 | passwd --stdin $1
  echo "Creating home dir: /home/$1"
  cp -r /home/user /home/$1
  echo "Setting ownership: $1:$1"
  chown -R $1:$1 /home/$1
fi

--Nick Boldt 21:48, 14 March 2006 (EST)

Breadcrumbs

EMFT Build Server Setup

Contents

EMFT Build Server Setup

Set up web content

Fix permissions & ownership

Install via yum using yum-xen.conf file

Install & symlink

Fix web user (apache)

/etc/group

/etc/passwd

/etc/sudoers

/etc/bashrc

~/.bashrc

~/.bash_profile

Secure build script

/etc/httpd/conf/httpd.conf

/var/www/html/emft/.htaccess

Run a build

Debugging tips

Display build logs, details & test results

Add additional users

Install via `yum` using yum-xen.conf file