Revision as of 11:02, 3 February 2014

Committers Charter

Committers are responsible for:

• Maintaining and enhancing the Virgo code base
• Ensuring the Hudson CI jobs pass all tests (blue ball)
• Responding to queries on virgo-dev
• Handling contributions
• Developing contributors into committers
• Complying with Eclipse IP policy
• Keeping an eye on the Virgo forum

Machine Setup

You need Sun JDK 6, Apache Ant 1.8.2 or later, and git. You'll probably want an IDE such as Eclipse and, in order for Virgo projects to build under Eclipse, the Virgo IDE tooling and AJDT (the current update site may be found on http://www.eclipse.org/ajdt/downloads/).

Don't forget to install AJDT as this is needed so that Eclipse produces the same compiled classes as the Ant build!

At the time of writing, some ant targets occasionally fail because they cannot load classes from jsch-0.1.42.jar. A workaround on Mac OS X is to copy this JAR from virgo-build's /lib/ivy directory to /opt/local/share/java/apache-ant/lib.

On Mac OS X, increase the default number of concurrently open files by adding this to .profile:

       ulimit -n 10000

To run certain scripts (don't run them unless you know what you're doing—and why), you'll need ruby, gems, and the 'choice' gem. On Mac OS you can get these by installing the XCode tools (from the Mac OS X disk) and MacPorts, then issuing:

       sudo port -v install ruby (Not required on Mountain Lion)
       sudo port -v install rb-rubygems (Not required on Mountain Lion)
       sudo gem install --remote choice

To run scripts such as Releaselor or Ripplor you'll need to:

1. Generate SSH key by calling ssh-keygen without a passphrase
2. Upload your public SSH key to build.eclipse.org
   • login with user virgoBuild
   • follow the steps described here
3. Log in (adding the host key) at least once to:
   • build.eclipse.org
   • download1.eclipse.org
4. Add your private key to ANT_OPTS:

       -Dkey.file=$HOME/.ssh/id_dsa

If you have headless environment you may want to add to ANT_OPTS:

       -Djava.awt.headless=true

To enable Clover you will need to specify in ANT_OPTS the path to the licence:

       -Dclover.license.path=/opt/clover.license

Notes for Ubuntu 10.04

JDK

I have found that the sun-java6-jdk package is not installable directly from a fresh install of this level of Ubuntu (I'm running this in a virtual machine). (I set JAVA_HOME to be /usr/lib/jvm/java-6-sun which is a symbolic link, and this doesn't need updating after the install below.)

To get the jdk to be installed I issued the following commands (which you might find useful):

   sudo add-apt-repository "deb http://archive.canonical.com/ lucid partner"

(which made the package visible to the package-manager)

   sudo apt-get update

(to reset the package-manager indexes)

   sudo apt-get install --fix-missing sun-java6-jdk

The --fix-missing option is necessary since other packages are required.

jsch library

The JCraft's jsch library needed for SSH/SCP support in ANT may need to be installed, e.g.:

Note: An annoying bug (http://sourceforge.net/p/jsch/bugs/58/) has been fixed in version 0.1.50 which is available in Orbit as: com.jcraft.jsch_0.1.50.v201310081430.jar

   sudo apt-get install libjsch-java

Check in ANT if its available:

   ant -diagnostics

If the output says under "Tasks availability" sshexec : Missing dependency com.jcraft.jsch.Logger, then you'll need to provide this library to ANT. On Linux you can do this by creating a symlink:

   sudo ln -s /usr/share/java/jsch-0.1.42.jar /usr/share/ant/lib/jsch-0.1.42.jar

or on Mac OS, place (a link to) the JAR in ~/.ant/lib:

   cp jsch-0.1.42.jar ~/.ant/lib/.

Finally, you may check if the JAR is correct (as suggested in this blog):

   jar tvf /usr/share/ant/lib/jsch*jar

or:

   jar tvf ~/.ant/lib/jsch*jar

If everything is ok you should get a list of files inside the archive. Be sure to check "ant -diagnostics" too.

Policies

In addition to detailed coding and testing policies describe later, there are some specific policies that committers need to be aware of.

Event Log Message Policy

Please avoid renumbering event log messages when possible. People sometimes write scripts that act on event log messages. Some of our tests may even be sensitive to the numbers. So renumbering an event log message can cause unnecessary disruption to our users and ourselves.

Grokking

You can grok the high level structure of Virgo by understanding the set of git repositories that comprise Virgo.

Use OpenGrok for searching the master branch of the entire Virgo code base. Always specify "Virgo" in the File Path field to avoid hits in Gemini Web and SpringSource dm Server which are also included in that index. Note that there is a link to OpenGrok on the Virgo home page for convenience.

To get an idea of the dependency structure, use OpenGrok specifying "virgo build.versions" in the File Path field and, to see dependencies on a specific component, the component name in Full Search. For example, this search shows the git repositories that depend on the kernel.

Coding

Virgo has a strong emphasis on maintainable, or "clean", code. If you need a really good introduction to coding, we recommend "Code Complete". If you are already a proficient programmer and want to write really clean code, read "Clean Code". If you are not an expert in writing Java, read "Effective Java".

Virgo code is thread safe unless specifically stated. Achieving thread safety in Java is not easy. We recommend "Java Concurrency in Practice" for a good grounding in Java concurrency.

Coding guidelines are here.

A Contextual Bash History can be very useful, guide here.

Eclipse Setup

1. Install the Virgo IDE tooling into Eclipse as described under the Tooling tab.
2. Install AJDT into Eclipse. This is necessary so that Virgo projects which use AspectJ are correctly woven when rebuilt under Eclipse.
3. Install egit into Eclipse. This step is recommended but not absolutely essential since it is possible to perform git operations on the command line or using some other git GUI tool.
4. Build the Virgo git repository you want to import into Eclipse as described under the Build tab. Ensure you see BUILD SUCCESSFUL.
5. Import the projects of the chosen git repository by using File->Import->Git->Projects from Git. Then click Next and add the git repository, select it, and click Next. Select the "Import existing projects" wizard and click Next. Check the projects you want to import (typically org.eclipse.virgo.xxx where xxx corresponds to the repository - other projects such as test bundle projects and any inside the build-xxx and virgo-build directories are unlikely to be of interest) and click Finish.
6. If you see lots of errors, you probably need to define a suitable classpath variable, e.g., KERNEL_IVY_CACHE, to point to the Ivy cache which has been built in the local copy of the git repository (actually to the ivy-cache/repository folder).
7. There should be no errors (marked in red). Occasionally, the build paths get out of step with the build.versions file and need updating.

Testing

See Test.

Hudson CI

Committers should subscribe to virgo-build@eclipse.org so they are notified when builds fail, become unstable, become stable, etc.

Build breaks should be investigated as soon as they occur and should be fixed as soon as possible.

IP Due Diligence

Handling Code Contributions

Committers are responsible for ensuring that Eclipse IP policy, summarised in the legal poster, is adhered to.

Patches must be attached to a bugzilla bug and have its iplog flag set to '+', and the contributor must confirm in the bug that: they wrote 100% of the code, they have the right to contribute the code to Eclipse, and new Java files contain the appropriate license header.

If the contribution is provided in an external git repository, it is not necessary to attach a patch to the bug: details here.

Raising "works with" CQs

"works with" CQs are required for test dependencies which are not distributed with Virgo or checked in to Eclipse version control (git, svn, cvs).

The initial set of test dependencies was determined, for repositories which are built with ant, as follows:

1. In the build-xxx directory run ant report

2. Extract a raw list of the test dependency jars

find <report directory> -name "*-test.xml" -exec grep -E \/.+\.jar  {} \; >test-jars.txt

3. Edit test-jars.txt using an editor with a regular expression global change facility and do the following global changes.

3.1 Replace regex .+\/(.+)\.jar\"> with $1.jar.

3.2 Replace regex .+\/(.+)\.jar\"\/> with $1.jar.

3.3 Replace -sources with the empty string

4. Sort test-jars.txt and remove duplicate lines.

5. Look through test-jars.txt and raise "works with" for any JARs which don't have a Virgo CQ for the correct version. Also, raise corresponding Virgo bugzillas and set the iplog flag to enter the bugzillas in the automated IP log.

Development process advice

To enable distributed and cooperative development, some simple rules and processes need to be followed. These are all subject to change and are in the nature of advice rather than laws.

The advice is here.

Agile Project Management

The basic principle is that tasks which need to be worked on are placed in the backlog. For a particular sprint, some of the tasks in the backlog are moved to the sprint and completed. So no task appears in both the backlog and a sprint.

Import bugs from bugzilla into Acunote as follows:

1. Export all the open bugs for Virgo (and Gemini Web if you like) to a CSV file using the CSV link on the bugzilla search results panel.
2. Go to the "Imported from bugzilla" sprint (not really a sprint at all) and import the CSV using the Tools link followed by "Import tasks from Bugzilla" (not "Import tasks from CSV"). Make a note of the task numbers added.
3. Go to the "Imported from bugzilla" sprint and copy the tasks added in the previous step to the backlog.
4. It is helpful to categorise bugs and features using the "More Actions" pull down. We put the bugs at the top of the backlog so we don't lose track of them.

To work on a sprint:

1. Add a sprint.
2. Move tasks from the backlog into the new sprint to work on them. If you don't complete them, move them to a new sprint or move them back to the backlog.

Since we are trying to use Acunote to manage sprints in Southampton, Sofia, and potentially elsewhere, please ensure that bugzilla is kept up to date in case the Acunote data gets corrupted.

Note that we never remove tasks from the "Imported from bugzilla" so that subsequent imports do not accidentally re-create finished tasks. The maintainers of Acunote recommended this technique.

Logos

If you have need to use the Virgo logos some example and the svg source can be found here.