Eclipse Top-level Project Release Engineering Team
The release engineering team consists of Sonia Dimitrov and Kim Moir. We provide release engineering services the Eclipse top-level project (Platform, JDT, PDE and Equinox subprojects). We also manage builds for some of IBM's Eclipse based product offerings.
Eclipse 1.0 was released November 7, 2001. Sonia joined the eclipse release engineering team in early 2001. Kim moved to the release engineering team in late 2003.
Our relationship to other developers on the team is multi-faceted. Some of our responsibilities include
- Implementing and testing new functionality in the build process. We work closely with the PDE Build and Core/Equinox committers to test new PDE build features, for example grouped builds, versioning, conditioned and packed jars etc.
- Managing the build and test infrastructure.
- Assisting developers when they wish add new plugins, test plugins, reorganize features etc. Run many test builds to ensure that the next build will complete successfully.
- Public relations - Keep the team up to date on the build schedule, milestone schedules, changes to the build etc. Often we also serve as the unofficial "greeter" to the team and redirect people to the correct contact person for their area of interest.
- Designing and maintaining feature structure.
- Build optimization
- Implementing new automation tools in our build.
- Testing, testing and testing the build process. And then testing it again.
- Although our process is automated, we do still monitor the builds, especially during milestone weeks. We also encourage quick resolution of build problems. (Gentle prodding.)
- We often provide advice to teams who are trying to build using the "Eclipse Way" on a best effort basis.
We truly enjoy working with the other developers on the team because
- Everyone is willing to share knowledge and work together to solve problems.
- People care about the code they submit and ensuring that a build is successful.
- They feed us well. (Core team - chocolate, PDE team - cake, UI team - curry).
On a good day
- We don't have to do anything build related because it just works automatically.
- This allows us to focus of fixing our bug backlog, test and implement new build functionality.
On a bad day
- Everything that possibly can go wrong does go wrong. Our lab infrastructure has serious problems and unexpected and unavoidable build problems have occurred such as a network or power outage, hardware failure or even a small flood.
Our top priority is to get a build to developers. We also minimize risk of build failure by testing our process from end to end with any change in the scripts or infrastructure. We strive for CRISP builds (complete, repeatable, informative, schedulable, and portable) that incorporate release engineering best practices.
Since we have milestone builds every six weeks, we always feel that we are working toward a release. Of course, there isn't as much at stake with M1 as there is with M5 so the later milestones are much more stressful. During milestone week, we don't usually implement new functionality into the builder because we want to ensure that the builds run smoothly as possible. However, sometimes we need to make late breaking changes in the builder to address a serious bug during milestone week. During each milestone week, we test the newest plugins in org.eclipse.releng.basebuilder by running test builds. (org.eclipse.releng.basebuilder is a subset of eclipse SDK plugins that we use to run a headless build). This ensures that we can move the plugins to the builder as soon as the milestone is released. (See the latest Platform-releng-basebuilder version.
As we ramp down toward release, we try try to reduce the magnitude of the changes we make to the builder. Also, by the very nature of the end game, as we move toward the release it requires more approval to submit bug fixes to the build.
Overall, Callisto was a good experience for us although there is certainly a lot of room for improvement the teams communicate and share knowledge. It was also interesting that the simultaneous release process itself was embryonic and continued to develop throughout the release but ultimately it worked. I don't think that we made any special adjustments because we are at the bottom of the stack, don't compile against anyone and we usually release milestones early on to allow others to build on us. I found the mailing list to be an effective means of communication but also spoke with other teams on IM. It was also good to collaborate with other project teams and learn from the approaches they took to solving problems. It would have been good to start Callisto submissions earlier so the platform team could have more time to implement update manager changes.
Issues that arose in Callisto that I would hope that we would be addressed in Europa
- Bug triage - There needs to be more participation in bug triage of cross-platform issues. Very few people subscribed to the Callisto component for bug notification.
- Implementing new and existing build technology across projects - For instance, including packed jars could decrease the bandwidth utilization at eclipse.org and associated mirrors as well as significantly improve the update experience for users. The platform team introduced this technology very late in the development cycle which made it difficult for other teams to implement it in a timely fashion. What process should there be to adopt/encourage accepted best releng practices across teams?
- Versioning and implementing qualifiers - There were a lot of questions about implementing qualifiers and the core team's new versioning scheme Version_Numbering. Admittedly, it is a complicated topic. However, I don't know if there is a way to diffuse information such as this across projects in a more effective manner. Tutorials etc? Again, this is probably an issue of lack of release engineering resources and time.
- Build notification - Other teams would like to be notified by RSS feeds that our builds are ready so they can build against us, Nick has contributed this to basebuilder and we have implemented this for 3.3. builds and we have implemented it in the build but still troubleshooting some issues.
- Contributed ports - Currently we have teams that build Eclipse drops outside the build process and contribute them. They have the hardware, time and interest to build these drops. Currently the process for them to contribute the builds is not elegant, I would like to automate it more.
Facilities I wish we had
- More time dedicated to improving our build process and fixing bugs.
- Web page for new eclipse projects on suggested best practices for getting their builds started.
- I would trust other committers or active contributors help us.
Procedures to reduce stress and increase efficiency
- Incremental build (Ultimately a much faster build). See Eclipse 3.3 plan item http://www.eclipse.org/eclipse/development/eclipse_project_plan_3_3.html and bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=154083
- More stable lab infrastructure.
- More automated tools.
- Communication facilities - We use email and IM for routine and exceptional circumstances.
- RSS feeds would be good to automatically notify downstream teams when dependancies are available.
- Collection of release engineering best practices so new projects get their builds working and submitted to Europa. Pointers regarding update sites, creating reproducable builds, versioning, pde build best practices, feature design etc.