Jump to: navigation, search

Difference between revisions of "Platform-releng-europa-blurb"

(draft)
 
m
 
(18 intermediate revisions by the same user not shown)
Line 1: Line 1:
<h3>Top Level Eclipse project Release Engineering team</h3>
+
<h3>Eclipse Top-level Project Release Engineering Team</h3>
 +
 
  
 
<h4>About Us</h4>
 
<h4>About Us</h4>
 
Describe yourselves, your project and your relationship to other developers on the team. Consider describing ...
 
 
    * years of experience in development, doing release engineering, and specifically building Eclipse projects.
 
    * how you work, on a good day, or on a not so good day. how does your work change as you ramp-down toward release.
 
    * what automation you use and how often you change it. how late in a project you would consider automation changes and how you would know that they work.
 
  
  
 +
<p>
 
The release engineering team consists of Sonia Dimitrov and Kim Moir.  We provide release engineering services the Eclipse top-level project (Platform, JDT, PDE and Equinox subprojects).  We also manage builds for some of IBM's Eclipse based product offerings.
 
The release engineering team consists of Sonia Dimitrov and Kim Moir.  We provide release engineering services the Eclipse top-level project (Platform, JDT, PDE and Equinox subprojects).  We also manage builds for some of IBM's Eclipse based product offerings.
 +
</p>
  
 +
 +
<p>
 
Eclipse 1.0 was released November 7, 2001.  Sonia joined the eclipse release engineering team in early 2001.  Kim moved to the release engineering team in late 2003.
 
Eclipse 1.0 was released November 7, 2001.  Sonia joined the eclipse release engineering team in early 2001.  Kim moved to the release engineering team in late 2003.
 +
</p>
 +
  
 +
<p>
 
Our relationship to other developers on the team is multi-faceted. Some of our responsibilities include
 
Our relationship to other developers on the team is multi-faceted. Some of our responsibilities include
* Implement and test new functionality in the build process.  We work closely with the PDE Build and Core/Equinox committers to test new PDE build features, for example grouped builds, versioning, conditioned and packed jars etc.
+
* Implementing and testing new functionality in the build process.  We work closely with the PDE Build and Core/Equinox committers to test new PDE build features, for example grouped builds, versioning, conditioned and packed jars etc.
* Manage the build and test infrastructure.
+
* Managing the build and test infrastructure.
* Assist developers when they wish add new plugins, test plugins, reorganize features etc.  Run many test builds to ensure that the next build will complete successfully.
+
* Assisting developers when they wish add new plugins, test plugins, reorganize features etc.  Run many test builds to ensure that the next build will complete successfully.
* Public relations - keep the build team up to date on the build schedule, milestone schedules, changes to the build etc. Often we also serve as the unofficial "greeter" to the team and redirect people to the correct contact person for their area of interest.
+
* Public relations - Keep the team up to date on the build schedule, milestone schedules, changes to the build etc. Often we also serve as the unofficial "greeter" to the team and redirect people to the correct contact person for their area of interest.
* Design and maintain feature structure.
+
* Designing and maintaining feature structure.
* Constantly trying to optimize our builds.
+
* Build optimization
* Implement new automation tools in our build.  
+
* Implementing new automation tools in our build.  
* Testing, testing, testing the build process. And then testing it again.
+
* Testing, testing and testing the build process. And then testing it again.
 
* Although our process is automated, we do still monitor the builds, especially during milestone weeks.  We also encourage quick resolution of build problems.  (Gentle prodding.)
 
* Although our process is automated, we do still monitor the builds, especially during milestone weeks.  We also encourage quick resolution of build problems.  (Gentle prodding.)
* We often provide advice to teams who are trying to build in the "Eclipse Way".
+
* We often provide advice to teams who are trying to build using the "Eclipse Way" on a best effort basis.
 +
</p>
  
 +
 +
<p>
 
We truly enjoy working with the other developers on the team because
 
We truly enjoy working with the other developers on the team because
 
* Everyone is willing to share knowledge and work together to solve problems.
 
* Everyone is willing to share knowledge and work together to solve problems.
 
* People care about the code they submit and ensuring that a build is successful.
 
* People care about the code they submit and ensuring that a build is successful.
 
* They feed us well.  (Core team - chocolate, PDE team - cake, UI team - curry).
 
* They feed us well.  (Core team - chocolate, PDE team - cake, UI team - curry).
 +
</p>
  
On a good day, the builds are working, the lab infrastructure is stable and there isn't a crisis that needs immediate attention. This allows us to focus of fixing our bug backlog, testing new build functionality or implementing new functionality.  On a bad day, our lab infrastructure has serious problems, unexpected and unavoidable build problems have ocurred. ie. network or power outage, hardware failure, small floods.
 
  
Our top priority is to get a build to developersWe also minimize risk of build failure by testing our process from end to end with any change in the scripts or infrastructure.   
+
<p>
 +
On a good day
 +
* We don't have to do anything build related because it just works automatically.   
 +
* This allows us to focus of fixing our bug backlog, test and implement new build functionality.   
 +
</p>
  
  
1) Lab is down (network/hardware etc)
 
2) Build is broken
 
3) Test or implement fixes from other platform commiters.
 
4) Fix bugs from other teams.
 
  
Since we have milestone builds every six weeks, we always feel that we are working toward a releaseOf course, there isn't as much at stake with M1 as there is with M5 so the later milestones are much more stressful.  During milestone week, we don't usually implement new functionality into the builder because we want to ensure that the builds run smoothly as possible. However, sometimes we need to make late breaking changes in the builder to address a serious bug during milestone week.
+
<p>
During each milestone week, we test the newest plugins org.eclipse.releng.basebuilder.  (org.eclipse.releng.basebuilder is a subset of eclipse SDK plugins that we use to run a headless build).  This ensures that we can move the plugins to the builder as soon as the milestone is released.
+
On a bad day
 +
* Everything that possibly can go wrong does go wrongOur lab infrastructure has serious problems and unexpected and unavoidable build problems have occurred such as a network or power outage, hardware failure or even a small flood.
 +
</p>
  
As we ramp down toward release, we try try to reduce the magnitude of the changes we make to the builder.  Also, by the very nature of the end game, as we move toward the release it requires more approval to submit bug fixes to the build.
 
  
 +
<p>
 +
Our top priority is to get a build to developers.  We also minimize risk of build failure by testing our process from end to end with any change in the scripts or infrastructure.  We strive for [http://www.pragmaticprogrammer.com/starter_kit/au/PragmaticAutomationSummary.pdf#search=%22crisp%20builds%22 CRISP] builds (complete, repeatable, informative, schedulable, and portable) that incorporate release engineering best practices.
 +
</p>
  
<h4>Callisto Thoughts</h4>
 
  
Callisto was a good experience for usI don't think that we made any special adjustments because we are at the bottom of the stack, don't compile against anyone and we usually release milestones early on to allow others to build upon us :-)  I found the mailing list to be an effective means of communication but also spoke with other teams on IMIt was also good to collaborate with other Eclipse project teams and learn how they were approaching problems.
+
<p>
 +
Since we have milestone builds every six weeks, we always feel that we are working toward a releaseOf course, there isn't as much at stake with M1 as there is with M5 so the later milestones are much more stressful.  During milestone week, we don't usually implement new functionality into the builder because we want to ensure that the builds run smoothly as possible. However, sometimes we need to make late breaking changes in the builder to address a serious bug during milestone week.
 +
During each milestone week, we test the newest plugins in org.eclipse.releng.basebuilder by running test builds.  (org.eclipse.releng.basebuilder is a subset of eclipse SDK plugins that we use to run a headless build)This ensures that we can move the plugins to the builder as soon as the milestone is released. (See the latest [[Platform-releng-basebuilder]] version.
 +
</p>
  
  
 +
<p>
 +
As we ramp down toward release, we try try to reduce the magnitude of the changes we make to the builder.  Also, by the very nature of the end game, as we move toward the release it requires more approval to submit bug fixes to the build.
 +
</p>
  
Describe your Callisto experience. Consider describing ...
+
<h4>Callisto Thoughts</h4>
  
    * special adjustments you made because of your participation in the simultanious release.
 
    * tactical communication regarding inbound and outbound bits.
 
    * strategic communication with respect to the Callisto planning team.
 
  
 +
<p>
 +
Overall, Callisto was a good experience for us although there is certainly a lot of room for improvement the teams communicate and share knowledge.  It was also interesting that the simultaneous release process itself was embryonic and continued to develop throughout the release but ultimately it worked. I don't think that we made any special adjustments because we are at the bottom of the stack, don't compile against anyone and we usually release milestones early on to allow others to build on us.  I found the mailing list to be an effective means of communication but also spoke with other teams on IM.  It was also good to collaborate with other project teams and learn from the approaches they took to solving problems. It would have been good to start Callisto submissions earlier so the platform team could have more time to implement update manager changes.
 +
</p>
  
 
<h4>Europa Ambitions</h4>
 
<h4>Europa Ambitions</h4>
  
  
 +
<p>
 
Issues that arose in Callisto that I would hope that we would be addressed in Europa
 
Issues that arose in Callisto that I would hope that we would be addressed in Europa
* bug triage - There needs to be more participation in bug triage of cross-platform issues. Very few people subscribed to the Callisto component for bug notification.
+
* Bug triage - There needs to be more participation in bug triage of cross-platform issues. Very few people subscribed to the Callisto component for bug notification.
* Implementing new build technology across projects - For instance, including packed jars could decrease the bandwidth utilization and significantly improve the update experience for users.  However, since it's new technology it takes a while for it to percolate across projects, be tested in the build scripts, and implemented in productionIt would be great if more projects implement technologies such as these in an earlier timeframe.
+
* Implementing new and existing build technology across projects - For instance, including packed jars could decrease the bandwidth utilization at eclipse.org and associated mirrors as well as significantly improve the update experience for users.  The platform team introduced this technology very late in the development cycle which made it difficult for other teams to implement it in a timely fashionWhat process should there be to adopt/encourage accepted best releng practices across teams?
* versioning - There were a lot of questions about implementing qualifiers. Admittedly, it is a complicated topic.  However, I don't know if there is a way to diffuse information such as this across projects in a more effective manner. Tutorials etc?
+
* Versioning and implementing qualifiers - There were a lot of questions about implementing qualifiers and the core team's new versioning scheme [[Version_Numbering]].   Admittedly, it is a complicated topic.  However, I don't know if there is a way to diffuse information such as this across projects in a more effective manner. Tutorials etc? Again, this is probably an issue of lack of release engineering resources and time.
 
+
* Build notification - Other teams would like to be notified by RSS feeds that our builds are ready so they can build against us, Nick has contributed this to basebuilder and we have implemented this for 3.3. builds and we have implemented it in the build but still troubleshooting some issues.
Facilities I wish we had
+
* Contributed ports - Currently we have teams that build Eclipse drops outside the build process and contribute them.  They have the hardware, time and interest to build these drops.  Currently the process for them to contribute the builds is not elegant, I would like to automate it more.
* More time dedicated to improving our build process
+
</p>
 
+
  
 +
<h5>Facilities I wish we had</h5>
 +
<p>
 +
* More time dedicated to improving our build process and fixing bugs.
 +
* Web page for new eclipse projects on suggested best practices for getting their builds started.
 +
* I would trust other committers or active contributors help us.
 +
</p>
  
Describe your hopes and ambitions for Europa. Consider describing ...
+
<h5>Procedures to reduce stress and increase efficiency</h5>
 +
<p>
 +
* Incremental build (Ultimately a much faster build). See Eclipse 3.3 plan item http://www.eclipse.org/eclipse/development/eclipse_project_plan_3_3.html and bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=154083
 +
* More stable lab infrastructure.
 +
* More automated tools.
 +
</p>
 +
  
    * facilities you wish you had and how they might work and who you would trust to build them.
+
<h5>Communication Mechanisms</h5>
    * precedures that would make your life less stressful or more efficient. also when and how you would escape those precedures in an emergency.
+
<p>
    * communication mechanisms that would support both routine and exceptional circumstances that are sure to arise as more projects join the release train.
+
* Communication facilities - We use email and IM for routine and exceptional circumstances.  
 +
* RSS feeds would be good to automatically notify downstream teams when dependancies are available.
 +
* Collection of release engineering best practices so new projects get their builds working and submitted to Europa. Pointers regarding update sites, creating reproducable builds, versioning, pde build best practices, feature design etc.
 +
</p>

Latest revision as of 08:59, 11 September 2006

Eclipse Top-level Project Release Engineering Team


About Us


The release engineering team consists of Sonia Dimitrov and Kim Moir. We provide release engineering services the Eclipse top-level project (Platform, JDT, PDE and Equinox subprojects). We also manage builds for some of IBM's Eclipse based product offerings.


Eclipse 1.0 was released November 7, 2001. Sonia joined the eclipse release engineering team in early 2001. Kim moved to the release engineering team in late 2003.


Our relationship to other developers on the team is multi-faceted. Some of our responsibilities include

  • Implementing and testing new functionality in the build process. We work closely with the PDE Build and Core/Equinox committers to test new PDE build features, for example grouped builds, versioning, conditioned and packed jars etc.
  • Managing the build and test infrastructure.
  • Assisting developers when they wish add new plugins, test plugins, reorganize features etc. Run many test builds to ensure that the next build will complete successfully.
  • Public relations - Keep the team up to date on the build schedule, milestone schedules, changes to the build etc. Often we also serve as the unofficial "greeter" to the team and redirect people to the correct contact person for their area of interest.
  • Designing and maintaining feature structure.
  • Build optimization
  • Implementing new automation tools in our build.
  • Testing, testing and testing the build process. And then testing it again.
  • Although our process is automated, we do still monitor the builds, especially during milestone weeks. We also encourage quick resolution of build problems. (Gentle prodding.)
  • We often provide advice to teams who are trying to build using the "Eclipse Way" on a best effort basis.


We truly enjoy working with the other developers on the team because

  • Everyone is willing to share knowledge and work together to solve problems.
  • People care about the code they submit and ensuring that a build is successful.
  • They feed us well. (Core team - chocolate, PDE team - cake, UI team - curry).


On a good day

  • We don't have to do anything build related because it just works automatically.
  • This allows us to focus of fixing our bug backlog, test and implement new build functionality.


On a bad day

  • Everything that possibly can go wrong does go wrong. Our lab infrastructure has serious problems and unexpected and unavoidable build problems have occurred such as a network or power outage, hardware failure or even a small flood.


Our top priority is to get a build to developers. We also minimize risk of build failure by testing our process from end to end with any change in the scripts or infrastructure. We strive for CRISP builds (complete, repeatable, informative, schedulable, and portable) that incorporate release engineering best practices.


Since we have milestone builds every six weeks, we always feel that we are working toward a release. Of course, there isn't as much at stake with M1 as there is with M5 so the later milestones are much more stressful. During milestone week, we don't usually implement new functionality into the builder because we want to ensure that the builds run smoothly as possible. However, sometimes we need to make late breaking changes in the builder to address a serious bug during milestone week. During each milestone week, we test the newest plugins in org.eclipse.releng.basebuilder by running test builds. (org.eclipse.releng.basebuilder is a subset of eclipse SDK plugins that we use to run a headless build). This ensures that we can move the plugins to the builder as soon as the milestone is released. (See the latest Platform-releng-basebuilder version.


As we ramp down toward release, we try try to reduce the magnitude of the changes we make to the builder. Also, by the very nature of the end game, as we move toward the release it requires more approval to submit bug fixes to the build.

Callisto Thoughts


Overall, Callisto was a good experience for us although there is certainly a lot of room for improvement the teams communicate and share knowledge. It was also interesting that the simultaneous release process itself was embryonic and continued to develop throughout the release but ultimately it worked. I don't think that we made any special adjustments because we are at the bottom of the stack, don't compile against anyone and we usually release milestones early on to allow others to build on us. I found the mailing list to be an effective means of communication but also spoke with other teams on IM. It was also good to collaborate with other project teams and learn from the approaches they took to solving problems. It would have been good to start Callisto submissions earlier so the platform team could have more time to implement update manager changes.

Europa Ambitions


Issues that arose in Callisto that I would hope that we would be addressed in Europa

  • Bug triage - There needs to be more participation in bug triage of cross-platform issues. Very few people subscribed to the Callisto component for bug notification.
  • Implementing new and existing build technology across projects - For instance, including packed jars could decrease the bandwidth utilization at eclipse.org and associated mirrors as well as significantly improve the update experience for users. The platform team introduced this technology very late in the development cycle which made it difficult for other teams to implement it in a timely fashion. What process should there be to adopt/encourage accepted best releng practices across teams?
  • Versioning and implementing qualifiers - There were a lot of questions about implementing qualifiers and the core team's new versioning scheme Version_Numbering. Admittedly, it is a complicated topic. However, I don't know if there is a way to diffuse information such as this across projects in a more effective manner. Tutorials etc? Again, this is probably an issue of lack of release engineering resources and time.
  • Build notification - Other teams would like to be notified by RSS feeds that our builds are ready so they can build against us, Nick has contributed this to basebuilder and we have implemented this for 3.3. builds and we have implemented it in the build but still troubleshooting some issues.
  • Contributed ports - Currently we have teams that build Eclipse drops outside the build process and contribute them. They have the hardware, time and interest to build these drops. Currently the process for them to contribute the builds is not elegant, I would like to automate it more.

Facilities I wish we had

  • More time dedicated to improving our build process and fixing bugs.
  • Web page for new eclipse projects on suggested best practices for getting their builds started.
  • I would trust other committers or active contributors help us.

Procedures to reduce stress and increase efficiency


Communication Mechanisms

  • Communication facilities - We use email and IM for routine and exceptional circumstances.
  • RSS feeds would be good to automatically notify downstream teams when dependancies are available.
  • Collection of release engineering best practices so new projects get their builds working and submitted to Europa. Pointers regarding update sites, creating reproducable builds, versioning, pde build best practices, feature design etc.