Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "EMF Build Server Setup"

m (Install via apt-get)
(Verify X Server: add note about Fedora)
 
(43 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 +
This install log is for an Ubuntu server. For a Xen/Fedora server, see [[EMFT Build Server Setup]].
 +
 
= EMF Build Server Setup =
 
= EMF Build Server Setup =
  
Line 12: Line 14:
  
 
== Install via apt-get ==
 
== Install via apt-get ==
 +
 +
apt-get update; apt-get upgrade
  
 
  apt-get install apache2 perl5 curl cvs sun-java6-jdk
 
  apt-get install apache2 perl5 curl cvs sun-java6-jdk
Line 17: Line 21:
 
  apt-get install zip unzip tar bzip2 sudo screen sendmail
 
  apt-get install zip unzip tar bzip2 sudo screen sendmail
 
  apt-get install openssh-server ccrypt tofrodos
 
  apt-get install openssh-server ccrypt tofrodos
  apt-get install libcvsservice0 cvs tkcvs
+
  apt-get install libcvsservice0 cvs tkcvs bc
 
  apt-get install vim vim-common vim-scripts vim-gtk
 
  apt-get install vim vim-common vim-scripts vim-gtk
 
  apt-get install libgtk2.0-common libgtk2.0-0 libgtk2.0-bin
 
  apt-get install libgtk2.0-common libgtk2.0-0 libgtk2.0-bin
 +
 
  apt-get install xserver-xorg-core xvfb xserver-xorg-driver-vesa xrgb \
 
  apt-get install xserver-xorg-core xvfb xserver-xorg-driver-vesa xrgb \
 
   xserver-xorg xserver-xorg-input-mouse xfs xkill xlsfonts xlsclients \
 
   xserver-xorg xserver-xorg-input-mouse xfs xkill xlsfonts xlsclients \
 
   xinit xinitrc xkeyboard-config kbd xserver-xorg-input-kbd xkbutils \
 
   xinit xinitrc xkeyboard-config kbd xserver-xorg-input-kbd xkbutils \
   xset xsetroot
+
   xset xsetroot libxp6 libxpm4 libxp-java xnest menu twm metacity \
 +
  xkbutils xxkb xkbset xkbsel libxkbsel-dev x11-apps
  
 
== Install & symlink ==
 
== Install & symlink ==
Line 40: Line 46:
 
   - or -  
 
   - or -  
 
  cd /etc/alternative; rm -f editor; ln -s /usr/bin/vim editor
 
  cd /etc/alternative; rm -f editor; ln -s /usr/bin/vim editor
 +
 +
== Disable system bell / PC speaker ==
 +
 +
Annoyed by beeps when you backspace or tab in a console? [http://ubuntuforums.org/showthread.php?t=315217 Disable the system bell] by removing the pcspkr module.
 +
 +
{{codeblock|modprobe -r pcspkr}}
 +
 +
Or, to kill it forever, and add this to <code>/etc/modprobe.d/blacklist</code> so the pc speaker will be disabled on your next reboot:
 +
 +
{{codeblock|blacklist pcspkr}}
  
 
== Recreate ssh keys ==
 
== Recreate ssh keys ==
Line 89: Line 105:
  
 
  addgroup www
 
  addgroup www
 +
:* <i>On Fedora, run <tt>groupadd www</tt> instead</i>
  
 
* Edit <tt style="color:DarkGreen">/etc/group</tt>. Add <tt style="color:DarkGreen">www</tt> group:
 
* Edit <tt style="color:DarkGreen">/etc/group</tt>. Add <tt style="color:DarkGreen">www</tt> group:
Line 94: Line 111:
 
==== /etc/group ====
 
==== /etc/group ====
 
<pre>www:x:1008:www-data,user1,user2,user3,...</pre>
 
<pre>www:x:1008:www-data,user1,user2,user3,...</pre>
 +
:* <i>On Fedora, edit /etc/gshadow to contain the same line</i>
  
 
* Edit <tt style="color:DarkGreen">/etc/passwd</tt>. Change users' home directories and shell:
 
* Edit <tt style="color:DarkGreen">/etc/passwd</tt>. Change users' home directories and shell:
Line 155: Line 173:
  
 
umask 022
 
umask 022
 +
export CVS_RSH=/usr/bin/ssh
 +
 +
export ANT_HOME=/opt/apache-ant-1.6
 +
export JAVA_HOME=/opt/sun-java2-5.0
 +
export PATH=${PATH}:${ANT_HOME}/bin
 +
 
export GREP_OPTIONS='--color=auto'
 
export GREP_OPTIONS='--color=auto'
 
export GTK2_RC_FILES=$HOME/.gtkrc-2.0
 
export GTK2_RC_FILES=$HOME/.gtkrc-2.0
Line 199: Line 223:
  
 
* For consistency w/ other servers, edit <tt style="color:DarkGreen">/etc/apache2/sites-enabled/000-default</tt> and set Apache's default <tt style="color:DarkGreen">DocumentRoot</tt> to <tt style="color:DarkGreen">/var/www/html</tt> instead of <tt style="color:DarkRed">/var/www</tt>.
 
* For consistency w/ other servers, edit <tt style="color:DarkGreen">/etc/apache2/sites-enabled/000-default</tt> and set Apache's default <tt style="color:DarkGreen">DocumentRoot</tt> to <tt style="color:DarkGreen">/var/www/html</tt> instead of <tt style="color:DarkRed">/var/www</tt>.
 +
 +
:* <i>Not required on Fedora (config file found in /var/httpd/conf/httpd.conf).</i>
 +
 +
* This is particularly important if you plan to run more than one http server on your build box. See [[EMF Build Server Setup/Lighttpd]].
  
 
== Set up web content ==
 
== Set up web content ==
Line 204: Line 232:
 
* Copy content from an existing server partition or else extract from CVS:
 
* Copy content from an existing server partition or else extract from CVS:
  
 +
mkdir -p /home/www-data/build; cd /home/www-data/build;
 +
cvs -d :pserver:anonymous@dev.eclipse.org:/cvsroot/modeling -q co -d config [http://dev.eclipse.org/viewcvs/index.cgi/releng-common/server-config/?root=Modeling_Project releng-common/server-config]
 
  mkdir -p /home/www-data/build/modeling; cd /home/www-data/build/modeling;
 
  mkdir -p /home/www-data/build/modeling; cd /home/www-data/build/modeling;
 
  cvs -d :pserver:anonymous@dev.eclipse.org:/cvsroot/modeling -q co -d scripts [http://dev.eclipse.org/viewcvs/index.cgi/releng-common/tools/scripts/?root=Modeling_Project releng-common/tools/scripts]
 
  cvs -d :pserver:anonymous@dev.eclipse.org:/cvsroot/modeling -q co -d scripts [http://dev.eclipse.org/viewcvs/index.cgi/releng-common/tools/scripts/?root=Modeling_Project releng-common/tools/scripts]
Line 209: Line 239:
 
  cvs -d :pserver:anonymous@dev.eclipse.org:/cvsroot/org.eclipse -q co -d modeling [http://dev.eclipse.org/viewcvs/index.cgi/www/modeling/?root=Eclipse_Website www/modeling]
 
  cvs -d :pserver:anonymous@dev.eclipse.org:/cvsroot/org.eclipse -q co -d modeling [http://dev.eclipse.org/viewcvs/index.cgi/www/modeling/?root=Eclipse_Website www/modeling]
 
  cvs -d :pserver:anonymous@dev.eclipse.org:/cvsroot/org.eclipse -q co -d eclipse.org-common [http://dev.eclipse.org/viewcvs/index.cgi/www/eclipse.org-common/?root=Eclipse_Website www/eclipse.org-common]
 
  cvs -d :pserver:anonymous@dev.eclipse.org:/cvsroot/org.eclipse -q co -d eclipse.org-common [http://dev.eclipse.org/viewcvs/index.cgi/www/eclipse.org-common/?root=Eclipse_Website www/eclipse.org-common]
 +
 +
* In the <b>config</b> directory, review <b>localhost.properties</b> for correct variables. More config can be done later
  
 
== Fix permissions & ownership ==
 
== Fix permissions & ownership ==
Line 246: Line 278:
 
  127.0.0.1    localhost
 
  127.0.0.1    localhost
 
  9.xx.yyy.zzz  emf.torolab.ibm.com emf
 
  9.xx.yyy.zzz  emf.torolab.ibm.com emf
 +
 +
* Note the ordering: fully qualified hostname must appear <b>before</b> the shortname
  
 
== Set up Build's Web UI==
 
== Set up Build's Web UI==
Line 405: Line 439:
 
  LockDir=/var/lock/cvs
 
  LockDir=/var/lock/cvs
  
* Create lock dir and set permissions
+
* If you get an error trying to check out files from your local CVS during a build, try this:
 +
 +
cd /var/lib/cvs/CVSROOT; chmod o+w history val-tags Emptydir
  
 +
Or this:
 +
 +
cd /var/lib/cvs/CVSROOT; chown ''someuser'' -R *
 +
 +
* Create lock dir and set permissions - create a file called /etc/init.d/cvsstartup
 +
 +
#!/bin/sh
 +
# /etc/init.d/cvsstartup: start up the cvs server
 
  mkdir /var/lock/cvs
 
  mkdir /var/lock/cvs
 
  chgrp cvs /var/lock/cvs
 
  chgrp cvs /var/lock/cvs
 +
chmod g+w /var/lock/cvs
 
  chmod g+s /var/lock/cvs
 
  chmod g+s /var/lock/cvs
  
* See also [[EMFT Build Server Setup#CVS_configuration | CVS configuration]]
+
* Make the script executable, then symlink this file as /etc/rc5.d/S98cvs and /etc/rc2.d/S98cvs
  
= See Also =
+
cd /etc/init.d; chmod +x cvsstartup
 +
cd /etc/rc5.d; ln -s ../init.d/cvsstartup S98cvs
 +
cd /etc/rc2.d; ln -s ../init.d/cvsstartup S98cvs
  
Additional info and configuration steps are listed in [[EMFT Build Server Setup]].
+
* Run it. On next boot it should run by itself.
 +
 
 +
/etc/init.d/cvsstartup
 +
 
 +
=== See also ===
 +
* [[EMFT Build Server Setup#CVS_configuration | EMFT Build Server - CVS configuration]]
 +
* [[EMF_Build_Server_Setup/Anonymous Pserver Access|Anonymous Pserver Access]]
 +
* [http://liveintellect.com/how_to/cvs_server_setup.html#setuprepository HOWTO: CVS Server - Setup Repository]
 +
* [http://www.cs.hmc.edu/qref/cvs.html#wt Walkthru: Creating a Repository]
 +
 
 +
== Verify X Server ==
 +
 
 +
<table><tr valign="top">
 +
<td>
 +
* Ensure the X server is running. You will have to have installed all the x* packages listed above in [[#Install_via_apt-get|the apt-get section]] or the [[EMFT_Build_Server_Setup#Install_via_yum_using_yum-xen.conf_file|yum section]]. You will also need [http://rpmfind.net/linux/rpm2html/search.php?query=xorg-x11-Xvfb Xvfb], if not already installed. Type:
 +
 
 +
# ps aux --forest | grep root | grep -i x
 +
 
 +
* You should see something like this:
 +
 
 +
root      3820  0.0  0.0  4456  2428 ?        Ss  Jan11  0:00 /usr/bin/xfs -daemon
 +
root    12128  0.0  0.2  14128  7304 tty2    Ss+  Jan14  0:20 X
 +
root    12264  0.0  0.0  2408  1160 ?        S    Jan14  0:00 /bin/sh /usr/bin/x-session-manager
 +
root    12289  0.0  0.0  4336  692 ?        Ss  Jan14  0:00  \_ /usr/bin/ssh-agent x-session-manager
 +
 
 +
* If not running, start X by setting runlevel to 5 in /etc/inittab:
 +
 
 +
# The default runlevel.
 +
id:5:initdefault:
 +
 
 +
* Or, to change runlevels without rebooting, type:
 +
 
 +
telinit 5
 +
 
 +
 
 +
</td>
 +
<td>&nbsp;&nbsp;&nbsp;</td>
 +
<td><table cellpadding="3"><tr><td>
 +
===Aside===
 +
 
 +
* On Fedora, you may need to tweak <code>/etc/gdm/gdm.conf</code> or <code>/etc/gdm/gdm.schemas</code> to enable TCP connections:
 +
 
 +
DisallowTCP=false
 +
 
 +
or
 +
 
 +
&lt;schema>
 +
  &lt;key>security/DisallowTCP&lt;/key>
 +
  &lt;signature>b&lt;/signature>
 +
  &lt;default>false&lt;/default>
 +
&lt;/schema>
 +
</td></tr></table>
 +
</td></tr></table>
 +
 
 +
=== Start X Server ===
 +
 
 +
Note: this may not be required.
 +
 
 +
* Then restart the server or manually start up X (as root) with one or more of these:
 +
 
 +
X &                                ''# start the X server''
 +
startx &                          ''# start the X server''
 +
startkde &                        ''# start the K Desktop Environment''
 +
kdm &                              ''# start the K Display Manager''
 +
 
 +
/etc/init.d/xfs start &            ''# start the X font server''
 +
/etc/init.d/x11-common start &    ''# start the X server and ICE socket dirs''
 +
/etc/init.d/kdm start &            ''# start the K display manager
 +
 
 +
=== Connect Headlessly ===
 +
 
 +
* To connect headlessly to an x server, try this. Ignore any font errors:
 +
 
 +
export DISPLAY=localhost:38.0;
 +
ulimit -c unlimited;                  ''# set core file size = unlimited''
 +
Xvfb :38 -screen 0 1024x768x24 -ac &  ''# start new
 +
xhost +                                ''# allow anyone to connect on :38''
 +
xterm &
 +
 
 +
* If the x server allowed you to connect to it, you can run this:
 +
 
 +
ps ax --forest | grep xterm -B5
 +
 
 +
* You should see something like this -- note that the second bash is running on a different [http://www.google.com/search?q=linux+man+page+pts+pseudo-terminal+slave pts]:
 +
 
 +
5792 ?        S      0:00      \_ sshd: nickb@pts/1
 +
5793 pts/1    Ss    0:00          \_ -bash
 +
5795 pts/1    S      0:00              \_ Xvfb :38 -screen 0 1024x768x24 -ac
 +
5801 pts/1    S      0:00              \_ twm -display localhost:38.0
 +
5807 pts/1    S      0:00              \_ '''xterm'''
 +
5809 '''''pts/2    Ss+'''''    0:00              |  \_ bash
 +
5842 pts/1    R+    0:00              \_ ps ax --forest
 +
5843 pts/1    S+    0:00              \_ grep '''xterm''' -B5
 +
 
 +
* If that worked, you can now [[Modeling_Project_Releng/Building#UI_Testing|enable UI testing in your builds]]!
 +
 
 +
=== Capture X Server Display ===
 +
 
 +
* You can also attempt to capture a screenshot of the X session with xwd:
 +
 
 +
xwd -root -out /tmp/screenshot.xwd
 +
 
 +
* If you have ImageMagick installed, you can convert the xwd file to something more friendly:
 +
 
 +
convert screenshot.xwd screenshot.png
 +
 
 +
* Or skip those 2 steps and just capture with ImageMagick's import:
 +
 
 +
import -window root /tmp/screenshot.png
 +
 
 +
* Then, scp that file somewhere that you can look at it (eg., your local dev machine), and open it with your preferred viewer. Don't have ImageMagick? You can view the file with xwud:
 +
 
 +
xwud -in screenshot.xwd
 +
 
 +
=== Connect Via Remote Desktop ===
 +
 
 +
<table><tr valign="top"><td>
 +
* To connect to the build server via a remote X connection (with ssh-tunnelling for security), you need to install a VNC viewer client on your local machine, and a VNC server on the remote machine. If you use linux, you can also use krdc + rdesktop on the client side and krfb on the remote side, or a combination thereof.
 +
 
 +
<table><tr><th>Local Client</th><th>Remote Server</th></tr>
 +
<tr><td>{{codeblock|<nowiki>apt-get install \
 +
krdc xvnc4server rdesktop</nowiki>}}</td><td>{{codeblock|<nowiki>apt-get install \
 +
krfb vnc4server fluxbox</nowiki>}}</td></tr>
 +
</table>
 +
 
 +
* Here's an alias script for connecting to a remote box, starting up a new vnc server session on :1, connecting to that session, and terminating the connection when done:
 +
 
 +
{{codeblock|<nowiki>alias bbr='ssh buildbox "vnc4server -kill :1"; \
 +
ssh buildbox "vnc4server -geometry 1300x900 :1"; \
 +
krdc buildbox:1 && ssh buildbox "vnc4server -kill :1"'</nowiki>}}
 +
 
 +
* Here's the startup script for the vnc server, in <code>~/.vnc/xstartup</code>:
 +
 
 +
{{codeblock|<nowiki>#!/bin/sh
 +
[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup
 +
[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources
 +
xsetroot -solid black
 +
vncconfig -iconic &
 +
x-terminal-emulator -geometry 750x525+225+0 -ls &
 +
fluxbox &</nowiki>}}
 +
</td><td>
 +
[[Image:Remote desktop build server.png|thumbnail|right|400px||border|Remote desktop connection to buildbox server. In this example, the local server is running the K desktop manager; the remote server is running the fluxbox window manager.]]
 +
</td></tr></table>
 +
 
 +
* To connect to the remote desktop, simply type 'bbr'. Once it loads, you can start up Eclipse, browse build server files, etc. Note that if your local machine is running Windows or Mac OS, the process above is more or less the same, but you'll need install the correct VNC viewer for your OS. See [http://www.realvnc.com/products/download.html RealVNC] or [http://www.tightvnc.com/download.html TightVNC], for example.
 +
 
 +
= See Also =
  
 +
* Additional info and configuration steps are listed in [[EMFT Build Server Setup]].
 +
* Want to set up a local mirror of your build server so that you can test .php pages before publishing them? See [[Eclipse Server Sandbox Setup]].
 +
* Want to use VMWare to run multiple servers on one box? See [[EMF Build Server Setup/VMWare]].
 +
* Want to run multiple http servers on one box? See [[EMF Build Server Setup/Lighttpd]].
 +
* Want to allow anonymous read-only CVS access? See [[EMF Build Server Setup/Anonymous Pserver Access]].
  
 
[[Category:Modeling]] [[Category:EMF]] [[Category:Releng]]
 
[[Category:Modeling]] [[Category:EMF]] [[Category:Releng]]

Latest revision as of 19:13, 12 December 2008

This install log is for an Ubuntu server. For a Xen/Fedora server, see EMFT Build Server Setup.

EMF Build Server Setup

You will need to be root for most of these tasks.

Install OS

Installation began with Ubuntu Server 6.06LTS, for x86. Very simple and straight forward. I partitioned the drive 36.4G drive like this:

sdb1   Boot, NC   Primary   Linux ext3   10000.01M
sdb2              Primary   Linux swap    1000.00M
sdb3              Primary   Linux ext3   25391.44M

Install via apt-get

apt-get update; apt-get upgrade
apt-get install apache2 perl5 curl cvs sun-java6-jdk
apt-get install php5 php5-cgi php5-cli php5-curl 
apt-get install zip unzip tar bzip2 sudo screen sendmail
apt-get install openssh-server ccrypt tofrodos
apt-get install libcvsservice0 cvs tkcvs bc
apt-get install vim vim-common vim-scripts vim-gtk
apt-get install libgtk2.0-common libgtk2.0-0 libgtk2.0-bin
apt-get install xserver-xorg-core xvfb xserver-xorg-driver-vesa xrgb \
  xserver-xorg xserver-xorg-input-mouse xfs xkill xlsfonts xlsclients \
  xinit xinitrc xkeyboard-config kbd xserver-xorg-input-kbd xkbutils \
  xset xsetroot libxp6 libxpm4 libxp-java xnest menu twm metacity \
  xkbutils xxkb xkbset xkbsel libxkbsel-dev x11-apps

Install & symlink

Set default editor to vim

  • If you prefer vim to nano, you'll have to do this:
export EDITOR=vim
  - or - 
cd /etc/alternative; rm -f editor; ln -s /usr/bin/vim editor

Disable system bell / PC speaker

Annoyed by beeps when you backspace or tab in a console? Disable the system bell by removing the pcspkr module.

modprobe -r pcspkr

Or, to kill it forever, and add this to /etc/modprobe.d/blacklist so the pc speaker will be disabled on your next reboot:

blacklist pcspkr

Recreate ssh keys

  • If upgrading from an existing server, you can copy the old server's ssh keys so that the server will "look" the same to users connecting via ssh.
cp /mnt/sda1/etc/ssh/* /etc/ssh/

Set a static IP address

/etc/network/interfaces

# The loopback interface
auto lo
iface lo inet loopback

# The first network card - this entry was created during the Debian installation
# (network, broadcast and gateway are optional)
auto eth0
iface eth0 inet static
address 9.xx.yyy.zzz
netmask 255.255.254.0
network 9.xx.yyy.0
broadcast 9.xx.yyy.255
gateway 9.xx.yyy.1

Add bootable partitions to grub loader

/boot/grub/menu.lst

default         0
timeout         5

title           Ubuntu, kernel 2.6.15-26-server
root            (hd1,0)
kernel          /boot/vmlinuz-2.6.15-26-server root=/dev/sdb1 ro quiet splash
initrd          /boot/initrd.img-2.6.15-26-server
savedefault
boot

title           Ye Old Debian on sda1, kernel 2.4.25, no initrd
root            (hd0,0)
kernel          /boot/vmlinuz-2.4.25 root=/dev/sda1
boot

Fix web user (www-data)

  • Add group 'www':
addgroup www
  • On Fedora, run groupadd www instead
  • Edit /etc/group. Add www group:

/etc/group

www:x:1008:www-data,user1,user2,user3,...
  • On Fedora, edit /etc/gshadow to contain the same line
  • Edit /etc/passwd. Change users' home directories and shell:

/etc/passwd

www-data:x:33:33:www-data:/home/www-data:/bin/sh
  • Edit /etc/sudoers to let you run commands and switch to the web user w/o needing a password. Add the following lines:

/etc/sudoers

# Members of the admin group may gain root privileges
%admin ALL=(ALL) ALL

# Members of the www group can sudo to www-data
%www            ALL = (www-data) NOPASSWD: ALL
  • Switch to the web user. You should NOT be prompted for a password.
sudo -u www-data bash
  • Create an ssh key, WITH NO PASSPHRASE. Store in ~/.ssh/id_rsa and ~/.ssh/id_rsa.pub:
ssh-keygen -b 2048 -t rsa
  • Copy contents of ~/.ssh/id_rsa.pub into ~/.ssh/authorized_keys file for user on dev.eclipse.org who will be running builds. This is so that the web user can commit changes to cvs (tagging, updating map files) for I, M, S & R builds. This is only required for EMF builds.
  • Test by ssh'ing to dev.eclipse.org, where _username_ should be replaced with your actual username:
ssh _username_@dev.eclipse.org
  • Run newgrp www so that when new files are created, they will use the group id www instead of www-data:
newgrp www
  • Set umask 022 so that files will be created with group write perms 664 (see #/etc/bashrc):
umask 022
  • Set the remote shell connection method for CVS to be ssh instead of the default rsh (see #/etc/bashrc):
export CVS_RSH=/usr/bin/ssh
  • Set an ANT_HOME and JAVA_HOME, and add ant to the PATH (see #/etc/bashrc):
export ANT_HOME=/opt/apache-ant-1.6
export JAVA_HOME=/opt/sun-java2-5.0
export PATH=${PATH}:${ANT_HOME}/bin
  • Switch to the root user.
  • Replace /etc/bashrc with this:

/etc/bashrc

[ -f ~/.alias ] && . ~/.alias

# set PATH so it includes user's private bin if it exists
if [ -d ~/bin ] ; then
    PATH=~/bin:"${PATH}"
fi

umask 022
export CVS_RSH=/usr/bin/ssh

export ANT_HOME=/opt/apache-ant-1.6
export JAVA_HOME=/opt/sun-java2-5.0
export PATH=${PATH}:${ANT_HOME}/bin

export GREP_OPTIONS='--color=auto'
export GTK2_RC_FILES=$HOME/.gtkrc-2.0
export KDEDIR=/usr
export QTDIR=/usr/share/qt3
export EDITOR=vi
export LC_ALL=C; #for perl

norm="\033[0;39m";grey="\033[1;30m";green="\033[1;32m";brown="\033[0;33m";
yellow="\033[1;33m";blue="\033[1;34m";cyan="\033[1;36m";red="\033[1;31m";

ucol=$grey;
hcol=$yellow;
rcol=$red;

if [ $UID -eq 0 ]; then # root
  prompt="\! $rcol\]\u@$hcol\h$norm:\$(pwd)\\[$norm\]\n$rcol#$norm ";  fcol=$brown;
else
  prompt="\! $ucol\]\u@$hcol\h$norm:\$(pwd)\\[$norm\]\n$ucol\$$norm "; fcol=$ucol;
fi

# If this is an xterm set the title to user@host:dir
case $TERM in
 xterm*)
  export PS1=$prompt"\[\e]30;\u@\H\a\]"
  export PROMPT_COMMAND='echo -ne "\033]0;${USER}@${HOSTNAME}: ${PWD}\007"'
  ;;
 *)
  export PS1=$prompt
  ;;
esac
  • Add the following to .bashrc and .bash_profile files:

~/.bashrc

[ -f /etc/bashrc ] && . /etc/bashrc

~/.bash_profile

[ -f ~/.bashrc ] && . ~/.bashrc

  • Copy both .bashrc and .bash_profile into /etc/skel, so that new users will get these files in their home dirs.

Change Apache's default web dir

  • For consistency w/ other servers, edit /etc/apache2/sites-enabled/000-default and set Apache's default DocumentRoot to /var/www/html instead of /var/www.
  • Not required on Fedora (config file found in /var/httpd/conf/httpd.conf).

Set up web content

  • Copy content from an existing server partition or else extract from CVS:
mkdir -p /home/www-data/build; cd /home/www-data/build;
cvs -d :pserver:anonymous@dev.eclipse.org:/cvsroot/modeling -q co -d config releng-common/server-config
mkdir -p /home/www-data/build/modeling; cd /home/www-data/build/modeling;
cvs -d :pserver:anonymous@dev.eclipse.org:/cvsroot/modeling -q co -d scripts releng-common/tools/scripts
mkdir -p /var/www/html; cd /var/www/html; 
cvs -d :pserver:anonymous@dev.eclipse.org:/cvsroot/org.eclipse -q co -d modeling www/modeling
cvs -d :pserver:anonymous@dev.eclipse.org:/cvsroot/org.eclipse -q co -d eclipse.org-common www/eclipse.org-common
  • In the config directory, review localhost.properties for correct variables. More config can be done later

Fix permissions & ownership

  • Create a script called /usr/local/bin/wwwup, which does this:
#!/bin/bash

# update cvs
cd /var/www/html/modeling; cvs -q up -Pd;
cd /var/www/html/eclipse.org-common; cvs -q up -Pd;

# fix perms/groups
cd /var/www/html; find . -type f -exec chmod 664 {} \;
cd /var/www/html; find . -type d -exec chmod 2775 {} \;
cd /var/www/html; find . -exec chown www-data:www {} \;

# fix perms/groups
cd /home/www-data/build; find . -type f -exec chmod 664 {} \;
cd /home/www-data/build; find . -type d -exec chmod 2775 {} \;
cd /home/www-data/build; find . -exec chown www-data:www {} \;

# make scripts executable
cd /home/www-data/build/modeling/scripts; find . -type f -name "*.sh" -exec chmod 755 {} \;
  • Make the script executable, then run it:
chmod 775 /usr/local/bin/wwwup;
wwwup

Fix hostname

  • Ensure your /etc/hosts file contains something a line such as this, so that the server knows its own name:
127.0.0.1     localhost
9.xx.yyy.zzz  emf.torolab.ibm.com emf
  • Note the ordering: fully qualified hostname must appear before the shortname

Set up Build's Web UI

  • Clone existing web UI, then fix server name, version, branch, JDK requirement, email address & username:
cd /var/www/html/modeling/emft/
cp -r mwe newComponent;
vi newComponent/build/_common.php 
$ cat _common.php
<?php
require_once ("../../../includes/buildServer-common.php");

$options = array (
       "BaseBuilderBranch" => "v20070614",

       "BranchAndJDK" => array (
               "emft.eclipse.org=------------,------------",
               "0.7.0=HEAD,/opt/sun-java2-5.0",

               "build.eclipse.org=------------,------------",
               "0.7.0=HEAD,/opt/public/common/ibm-java2-ppc-50"
       ),

       "Mapfile_Rule_Default" => 0, // 0: "Use Map, No Tagging=use-false" or 1:"Generate Map, No Tagging=gen-false"

       "EmailDefault" => "someone@company.net", // prefil email contact box with comma-sep'd list

       "Users" => array("username", "username", null) /* build user, eclipse cvs user, IES cvs user */
);
?>
  • Create folder where builds will be produced
mkdir /home/www-data/build/modeling/emft/newComponent/downloads/drops
  • If the above new /build/ folder was created outside the web root and committed to CVS, update web UI in web path:
cd /var/www/html/modeling/emft/;
cvs -q up -Pd newComponent;
$isBuildServer = $_SERVER["SERVER_NAME"] = "127.0.0.1" || (preg_match("/^(emft|build)\.eclipse\.org$/", $_SERVER["SERVER_NAME"])) || $isEMFserver;
  • Check the website, http://servername/modeling/emft/newComponent/build/

Password Protection w/ .htaccess

  • If you require .htaccess security, put an .htaccess file in the /build/ folder like this one:
AuthName "Build Server Password Required"
AuthUserFile /path/to/password/file
AuthGroupFile /dev/null
AuthType Basic
require user authorizedUsername
  • Then create a password file in /path/to/password/file. You can use htpasswd to generate the password.
authorizedUsername:lXfv8s4tra.qI
  • Set permissions on the password file:
 chmod 444 /path/to/password/file
 chown www-data:www-data /path/to/password/file
  • Check the website, http://servername/modeling/emft/newComponent/build/ -- this time, you should be prompted for a password. Depending on your browser, you may have to close & reopen it.

Run a build

From the build page, http://servername/modeling/emft/newComponent/build/, fire a build.

Check the build log while it's running or after it completes. If you see any messages such as permission denied or the following, something is amiss.

Debugging tips

execvp: Permission denied
  • Shell scripts must contain unix line endings. Run dos2unix to make sure, if copying them from a non-unix filesystem.
  • Shell scripts /home/www-data/build/emft/scripts/*.sh must be executable. See #Fix permissions & ownership.




  • Directories, eg. /home/www-data/build/emft/jet/downloads/drops/1.0.0 must be writable by the web user (or group www). See #Fix permissions & ownership.




eclipse.org: Connection refused
cvs [checkout aborted]: end of file from server (consult above messages if any)
  • CVS connection refusals are the result of not being able to automatically ssh as the web user to _username_@dev.eclipse.org. This

could be a problem with ssh keys (see above - #Fix web user (www-data)) or the method CVS uses to connect (rsh instead of ssh - see #/etc/bashrc).




  • For any compilation-related problems, missing file problems, or other issues not touched upon in this document, see EMFT_Procedures.

Display build logs, details & test results

Builds - including unpublished Nightly builds - are listed on the downloads page here: http://emf.torolab.ibm.com/modeling/emf/downloads/.

Published builds are located here: http://downloads.eclipse.org/modeling/emf/downloads/.

Add additional users

  • As root, run:
adduser -p newuserpassword newuser
  • Then copy .bashrc, .bash_profile, and .alias from an existing user to the new user's home dir (if /etc/skel is not used).
 cp /home/user/.bashrc /home/user/.bash_profile /home/user/.alias /home/newuser
  • Fix permissions on copied files
 chown newuser:newuser /home/newuser/.bashrc /home/newuser/.bash_profile /home/newuser/.alias

Install & configure mysql server 5

  • Install
 apt-get install mysql-server-5.0 mysql-client-5.0

CVS configuration

  • Install
apt-get install cvs
  • Create cvs group
vi /etc/group
  • Add the following line (or similar), listing all the users to have CVS access, then exit and save (ESC, :wq)
cvs:x:115:user1,user2,user3,...
  • Create CVSROOT
cd /home; ln -s /var/lib/cvs
  • Set permissions
cd /home/cvs
chgrp -R cvs .
chmod -R 2775 . CVSROOT
  • Edit /home/cvs/CVSROOT/config; add or change LockDir to be:
LockDir=/var/lock/cvs
  • If you get an error trying to check out files from your local CVS during a build, try this:
cd /var/lib/cvs/CVSROOT; chmod o+w history val-tags Emptydir

Or this:

cd /var/lib/cvs/CVSROOT; chown someuser -R *
  • Create lock dir and set permissions - create a file called /etc/init.d/cvsstartup
#!/bin/sh
# /etc/init.d/cvsstartup: start up the cvs server
mkdir /var/lock/cvs
chgrp cvs /var/lock/cvs
chmod g+w /var/lock/cvs
chmod g+s /var/lock/cvs
  • Make the script executable, then symlink this file as /etc/rc5.d/S98cvs and /etc/rc2.d/S98cvs
cd /etc/init.d; chmod +x cvsstartup
cd /etc/rc5.d; ln -s ../init.d/cvsstartup S98cvs
cd /etc/rc2.d; ln -s ../init.d/cvsstartup S98cvs
  • Run it. On next boot it should run by itself.
/etc/init.d/cvsstartup

See also

Verify X Server

  • Ensure the X server is running. You will have to have installed all the x* packages listed above in the apt-get section or the yum section. You will also need Xvfb, if not already installed. Type:
# ps aux --forest | grep root | grep -i x
  • You should see something like this:
root      3820  0.0  0.0   4456  2428 ?        Ss   Jan11   0:00 /usr/bin/xfs -daemon
root     12128  0.0  0.2  14128  7304 tty2     Ss+  Jan14   0:20 X
root     12264  0.0  0.0   2408  1160 ?        S    Jan14   0:00 /bin/sh /usr/bin/x-session-manager
root     12289  0.0  0.0   4336   692 ?        Ss   Jan14   0:00  \_ /usr/bin/ssh-agent x-session-manager
  • If not running, start X by setting runlevel to 5 in /etc/inittab:
# The default runlevel.
id:5:initdefault:
  • Or, to change runlevels without rebooting, type:
telinit 5


   

Aside

  • On Fedora, you may need to tweak /etc/gdm/gdm.conf or /etc/gdm/gdm.schemas to enable TCP connections:
DisallowTCP=false

or

<schema>
  <key>security/DisallowTCP</key>
  <signature>b</signature>
  <default>false</default>
</schema>

Start X Server

Note: this may not be required.

  • Then restart the server or manually start up X (as root) with one or more of these:
X &                                # start the X server
startx &                           # start the X server
startkde &                         # start the K Desktop Environment
kdm &                              # start the K Display Manager
/etc/init.d/xfs start &            # start the X font server
/etc/init.d/x11-common start &     # start the X server and ICE socket dirs
/etc/init.d/kdm start &            # start the K display manager

Connect Headlessly

  • To connect headlessly to an x server, try this. Ignore any font errors:
export DISPLAY=localhost:38.0; 
ulimit -c unlimited;                   # set core file size = unlimited
Xvfb :38 -screen 0 1024x768x24 -ac &   # start new 
xhost +                                # allow anyone to connect on :38
xterm &
  • If the x server allowed you to connect to it, you can run this:
ps ax --forest | grep xterm -B5
  • You should see something like this -- note that the second bash is running on a different pts:
5792 ?        S      0:00      \_ sshd: nickb@pts/1
5793 pts/1    Ss     0:00          \_ -bash
5795 pts/1    S      0:00              \_ Xvfb :38 -screen 0 1024x768x24 -ac
5801 pts/1    S      0:00              \_ twm -display localhost:38.0
5807 pts/1    S      0:00              \_ xterm
5809 pts/2    Ss+    0:00              |   \_ bash
5842 pts/1    R+     0:00              \_ ps ax --forest
5843 pts/1    S+     0:00              \_ grep xterm -B5

Capture X Server Display

  • You can also attempt to capture a screenshot of the X session with xwd:
xwd -root -out /tmp/screenshot.xwd
  • If you have ImageMagick installed, you can convert the xwd file to something more friendly:
convert screenshot.xwd screenshot.png
  • Or skip those 2 steps and just capture with ImageMagick's import:
import -window root /tmp/screenshot.png
  • Then, scp that file somewhere that you can look at it (eg., your local dev machine), and open it with your preferred viewer. Don't have ImageMagick? You can view the file with xwud:
xwud -in screenshot.xwd

Connect Via Remote Desktop

  • To connect to the build server via a remote X connection (with ssh-tunnelling for security), you need to install a VNC viewer client on your local machine, and a VNC server on the remote machine. If you use linux, you can also use krdc + rdesktop on the client side and krfb on the remote side, or a combination thereof.
Local ClientRemote Server
apt-get install \
krdc xvnc4server rdesktop
apt-get install \
krfb vnc4server fluxbox
  • Here's an alias script for connecting to a remote box, starting up a new vnc server session on :1, connecting to that session, and terminating the connection when done:
alias bbr='ssh buildbox "vnc4server -kill :1"; \
ssh buildbox "vnc4server -geometry 1300x900 :1"; \
krdc buildbox:1 && ssh buildbox "vnc4server -kill :1"'
  • Here's the startup script for the vnc server, in ~/.vnc/xstartup:
#!/bin/sh
[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup
[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources
xsetroot -solid black
vncconfig -iconic &
x-terminal-emulator -geometry 750x525+225+0 -ls &
fluxbox &
Remote desktop connection to buildbox server. In this example, the local server is running the K desktop manager; the remote server is running the fluxbox window manager.
  • To connect to the remote desktop, simply type 'bbr'. Once it loads, you can start up Eclipse, browse build server files, etc. Note that if your local machine is running Windows or Mac OS, the process above is more or less the same, but you'll need install the correct VNC viewer for your OS. See RealVNC or TightVNC, for example.

See Also

Copyright © Eclipse Foundation, Inc. All Rights Reserved.