Build Workshop 2: Build Harder/Report
Igor Fedorenko, Nick Boldt, Bjorn Freeman-Benson, Denis Roy, Kim Moir (morning), Pascal Rapicault (morning), DJ Houghton (morning)
- Build and server can take up to 2 weeks to set up
- Steep learning curve (server config, .releng project, ant/bash scripting, etc.)
- Enforcing eclipse.org rules (eg., Ganymede Must Dos)
- No one wants to build, but everyone NEEDS a build
Existing build approaches
- Set of scripts that run headless eclipse with basebuilder. Builds are scheduled from WebUI but actual work is done by cron job. WebUI and the cron job communicate via tmp file.
- Scripts + Buckminster to collect dependencies
- Ability to build/test projects against multiple versions of target platform. Not many projects use that, ones that do would need to setup separate build for each target platform.
- Ability to reproduce builds (needs to be tagging, but there is more to this). Some member companies (particularly those in the Linux world) rebuild eclipse.org projects from sources, so builds should be reproducible outside of eclipse.org too.
- Ability to generate source builds for both legal considerations and for rebuild needs
- Common build infrastructure for smaller projects. Configuration will be done via project portal. Committers will be able to trigger builds via http://build.eclipse.org/cbi/.
- As a way to promote best practices, there will be only one way of doing builds in CBI. Projects that want/need to deviate, will have to fork CBI or setup their own build from scratch.
- One build or one build per top-level project. Use build queue as a way to measure build server utilization.
- Bigger projects will likely need dedicated build person/team.
- Building Eclipse SDK -- or any product/project with platform specifics -- is outside of CBI scope. This is just for all-Java, simple projects.
- Separate virtual server for each project will require too much hardware resources, but individual projects can request virtual build server.
- Cruisecontrol or similar tool to execute builds on as needed bases or by request. For CVS, watch changes to map file(s) instead of entire source tree. Not a problem for SVN, though build system currently isn't tested w/ SVN repo.
- .releng Blog
- Cruise Control
- Buckminster Project
- Dash Project
- Maven Integration for Eclipse @ Apache & @ Eclipse
Build Configuration Data
We're trying to move the configuration information from static files (php arrays) into project info meta-data entered in the portal. So far we've moved these parameters which are all values under the "build" meta-data item:
- ProjRelengRoot e.g. :pserver:firstname.lastname@example.org:/cvsroot/tools -- defines CVSROOT for source & releng checkouts
- BranchAndJDK e.g. 3.4.0=HEAD,/opt/public/stp/apps/IBMJava2-ppc-142 -or- 3.4.0=HEAD,/opt/public/common/ibm-java2-ppc-50 -- defines version, CVS branch, and path to java
- regex e.g. I200.*/eclipse-SDK-|[SR]-.*200.*/eclipse-SDK- (optional) regular expression to be used to collect the most recent matching dependencies for running a build -- this is used by the web UI to preselect the "latest" available dependencies or by the crontab scripts to build using the latest dependencies. In time this may be replaced by the use of an RSS feed aggregator/parser, so that builds will always run with the latest deps.
- Mapfile_Rule_Default e.g. 0 or 1 == "Use Map" or "Generate Map" -- this defines whether or not to use a static mapfile in CVS, or to use a mapfile template to generate a map using either HEAD or the specified branch tag (eg., for an 0.8.1 build, R0_8_maintenance)
- EmailDefault e.g. email@example.com,firstname.lastname@example.org -- email address(es) to contact when build is done or a problem occurs
The semantics of these parameters will eventually be documented, once the system is ready for general use. This is by no means a complete list -- some items may be rendered obsolete (eg., replacing EmailDefault with a defacto $email@example.com or $firstname.lastname@example.org mailing list) and others may be added. BranchAndJDK ought to be refactored into a CSV format.