Build Workshop 2: Build Harder/Report

Meeting notes

Attendees

Igor Fedorenko, Nick Boldt, Bjorn Freeman-Benson, Denis Roy, Kim Moir (morning), Pascal Rapicault (morning), DJ Houghton (morning)

Pain points

• Build and server can take up to 2 weeks to set up
• Steep learning curve (server config, .releng project, ant/bash scripting, etc.)
• Enforcing eclipse.org rules (eg., Ganymede Must Dos)
• No one wants to build, but everyone NEEDS a build

Existing build approaches

• Set of scripts that run headless eclipse with basebuilder. Builds are scheduled from WebUI but actual work is done by cron job. WebUI and the cron job communicate via tmp file.
• Maven
• Scripts + Buckminster to collect dependencies

Other considerations

• Ability to build/test projects against multiple versions of target platform. Not many projects use that, ones that do would need to setup separate build for each target platform.
• Ability to reproduce builds (needs to be tagging, but there is more to this). Some exploiters rebuild eclipse.org projects from sources, so builds should be reproducible outside of eclipse.org too.

Discussed topics

• Common build infrastructure for smaller projects. Configuration will be done via project portal. Committers will be able to trigger builds via http://build.eclipse.org/cbi/.
• As a way to promote best practices, there will be only one way of doing builds in CBI. Projects that want/need to deviate, will have to fork CBI or setup their own build from scratch.
• One build or one build per top-level project. Use build queue as a way to measure build server utilization.
• Bigger projects will likely need dedicated build person/team.
• Building Eclipse SDK -- or any product/project with platform specifics -- is outside of CBI scope. This is just for all-Java, simple projects.
• Separate virtual server for each project will require too much hardware resources, but individual projects can request virtual build server.
• Cruisecontrol or similar tool to execute builds on as needed bases or by request. For CVS, watch changes to map file(s) instead of entire source tree. Not a problem for SVN, though build system currently isn't tested w/ SVN repo.

Build Configuration Data

We're trying to move the configuration information from static files (php arrays) into project info meta-data entered in the portal. So far we've moved these parameters which are all values under the "build" meta-data item:

• ProjRelengRoot e.g. :pserver:anonymous@dev.eclipse.org:/cvsroot/tools
• BranchAndJDK e.g. 3.4.0=HEAD,/opt/public/stp/apps/IBMJava2-ppc-142 -or- 3.4.0=HEAD,/opt/public/common/ibm-java2-ppc-50
• regex e.g. I200.*/eclipse-SDK-|[SR]-.*200.*/eclipse-SDK- (optional) regular expression to be used to collect the most recent matching dependencies for running a build
• Mapfile_Rule_Default e.g. 0; 0: "Use Map, No Tagging=use-false" or 1:"Generate Map, No Tagging=gen-false"
• EmailDefault e.g. nickboldt@gmail.com,anthonyh@ca.ibm.com

The semantics of these parameters will eventually be documented - once the system is ready for general use. This is by no means a complete list -- some items may be rendered obsolete (eg., replacing EmailDefault with $project-dev@eclipse.org) and others may be added. BrandAndJDK really ought to be refactored into a more logical format.