Jump to: navigation, search

Difference between revisions of "Platform-releng-europa-blurb"

m
(draft)
Line 1: Line 1:
<h3>Top Level Eclipse project Release Engineering team</h3>
+
<h3>Top Level Eclipse Project Release Engineering team</h3>
  
<h4>About Us</h4>
 
  
Describe yourselves, your project and your relationship to other developers on the team. Consider describing ...
 
  
    * years of experience in development, doing release engineering, and specifically building Eclipse projects.
+
<h4>About Us</h4>
    * how you work, on a good day, or on a not so good day. how does your work change as you ramp-down toward release.
+
    * what automation you use and how often you change it. how late in a project you would consider automation changes and how you would know that they work.
+
  
  
 +
<p>
 
The release engineering team consists of Sonia Dimitrov and Kim Moir.  We provide release engineering services the Eclipse top-level project (Platform, JDT, PDE and Equinox subprojects).  We also manage builds for some of IBM's Eclipse based product offerings.
 
The release engineering team consists of Sonia Dimitrov and Kim Moir.  We provide release engineering services the Eclipse top-level project (Platform, JDT, PDE and Equinox subprojects).  We also manage builds for some of IBM's Eclipse based product offerings.
 +
</p>
  
 +
 +
<p>
 
Eclipse 1.0 was released November 7, 2001.  Sonia joined the eclipse release engineering team in early 2001.  Kim moved to the release engineering team in late 2003.
 
Eclipse 1.0 was released November 7, 2001.  Sonia joined the eclipse release engineering team in early 2001.  Kim moved to the release engineering team in late 2003.
 +
</p>
  
 +
 +
<p>
 
Our relationship to other developers on the team is multi-faceted. Some of our responsibilities include
 
Our relationship to other developers on the team is multi-faceted. Some of our responsibilities include
* Implement and test new functionality in the build process.  We work closely with the PDE Build and Core/Equinox committers to test new PDE build features, for example grouped builds, versioning, conditioned and packed jars etc.
+
* Implementing and testing new functionality in the build process.  We work closely with the PDE Build and Core/Equinox committers to test new PDE build features, for example grouped builds, versioning, conditioned and packed jars etc.
 
* Manage the build and test infrastructure.
 
* Manage the build and test infrastructure.
 
* Assist developers when they wish add new plugins, test plugins, reorganize features etc.  Run many test builds to ensure that the next build will complete successfully.
 
* Assist developers when they wish add new plugins, test plugins, reorganize features etc.  Run many test builds to ensure that the next build will complete successfully.
Line 25: Line 28:
 
* Although our process is automated, we do still monitor the builds, especially during milestone weeks.  We also encourage quick resolution of build problems.  (Gentle prodding.)
 
* Although our process is automated, we do still monitor the builds, especially during milestone weeks.  We also encourage quick resolution of build problems.  (Gentle prodding.)
 
* We often provide advice to teams who are trying to build in the "Eclipse Way".
 
* We often provide advice to teams who are trying to build in the "Eclipse Way".
 +
</p>
  
 +
 +
<p>
 
We truly enjoy working with the other developers on the team because
 
We truly enjoy working with the other developers on the team because
 
* Everyone is willing to share knowledge and work together to solve problems.
 
* Everyone is willing to share knowledge and work together to solve problems.
 
* People care about the code they submit and ensuring that a build is successful.
 
* People care about the code they submit and ensuring that a build is successful.
 
* They feed us well.  (Core team - chocolate, PDE team - cake, UI team - curry).
 
* They feed us well.  (Core team - chocolate, PDE team - cake, UI team - curry).
 +
</p>
  
On a good day, the builds are working, the lab infrastructure is stable and there isn't a crisis that needs immediate attention. This allows us to focus of fixing our bug backlog, testing new build functionality or implementing new functionality.  On a bad day, our lab infrastructure has serious problems, unexpected and unavoidable build problems have ocurred. ie. network or power outage, hardware failure, small floods.
 
  
 +
<p>
 +
On a good day, we don't have to do anything build related because it just works automatically.  This allows us to focus of fixing our bug backlog, test and implement new build functionality.  On a bad day, everything that possibly can go wrong does go wrong.  Our lab infrastructure has serious problems, unexpected and unavoidable build problems have occurred. ie. network or power outage, hardware failure or even a small flood.
 +
</p>
 +
 +
 +
<p>
 
Our top priority is to get a build to developers.  We also minimize risk of build failure by testing our process from end to end with any change in the scripts or infrastructure.  We strive for [http://www.pragmaticprogrammer.com/starter_kit/au/PragmaticAutomationSummary.pdf#search=%22crisp%20builds%22 CRISP] builds (complete, repeatable, informative, schedulable, and portable) that incorporate release engineering best practices.
 
Our top priority is to get a build to developers.  We also minimize risk of build failure by testing our process from end to end with any change in the scripts or infrastructure.  We strive for [http://www.pragmaticprogrammer.com/starter_kit/au/PragmaticAutomationSummary.pdf#search=%22crisp%20builds%22 CRISP] builds (complete, repeatable, informative, schedulable, and portable) that incorporate release engineering best practices.
 +
</p>
  
 +
 +
<p>
 
Since we have milestone builds every six weeks, we always feel that we are working toward a release.  Of course, there isn't as much at stake with M1 as there is with M5 so the later milestones are much more stressful.  During milestone week, we don't usually implement new functionality into the builder because we want to ensure that the builds run smoothly as possible. However, sometimes we need to make late breaking changes in the builder to address a serious bug during milestone week.
 
Since we have milestone builds every six weeks, we always feel that we are working toward a release.  Of course, there isn't as much at stake with M1 as there is with M5 so the later milestones are much more stressful.  During milestone week, we don't usually implement new functionality into the builder because we want to ensure that the builds run smoothly as possible. However, sometimes we need to make late breaking changes in the builder to address a serious bug during milestone week.
During each milestone week, we test the newest plugins org.eclipse.releng.basebuilder.  (org.eclipse.releng.basebuilder is a subset of eclipse SDK plugins that we use to run a headless build).  This ensures that we can move the plugins to the builder as soon as the milestone is released.
+
During each milestone week, we test the newest plugins in org.eclipse.releng.basebuilder by running test builds.  (org.eclipse.releng.basebuilder is a subset of eclipse SDK plugins that we use to run a headless build).  This ensures that we can move the plugins to the builder as soon as the milestone is released.
 +
</p>
 +
 
  
 +
<p>
 
As we ramp down toward release, we try try to reduce the magnitude of the changes we make to the builder.  Also, by the very nature of the end game, as we move toward the release it requires more approval to submit bug fixes to the build.
 
As we ramp down toward release, we try try to reduce the magnitude of the changes we make to the builder.  Also, by the very nature of the end game, as we move toward the release it requires more approval to submit bug fixes to the build.
 +
</p>
 +
  
  
 
<h4>Callisto Thoughts</h4>
 
<h4>Callisto Thoughts</h4>
 
Overall, Callisto was a good experience for us.  I don't think that we made any special adjustments because we are at the bottom of the stack, don't compile against anyone and we usually release milestones early on to allow others to build upon us :-)  I found the mailing list to be an effective means of communication but also spoke with other teams on IM.  It was also good to collaborate with other Eclipse project teams and learn how they were approaching problems.
 
 
  
  
Describe your Callisto experience. Consider describing ...
+
<p>
 +
Overall, Callisto was a good experience for us although there is certainly a lot of room for improvement the teams communicate and share knowledge. It was also interesting that the simultaneous build process itself was embryonic and continued to develop throughout the release but ultimately it worked. I don't think that we made any special adjustments because we are at the bottom of the stack, don't compile against anyone and we usually release milestones early on to allow others to build on us.   I found the mailing list to be an effective means of communication but also spoke with other teams on IM. It was also good to collaborate with other project teams and learn from the approaches they took to solving problems. It would have been good to start Callisto submissions earlier so the platform team have a greater timeframe to implement update manager changes.
 +
</p>
  
    * special adjustments you made because of your participation in the simultanious release.
 
    * tactical communication regarding inbound and outbound bits.
 
    * strategic communication with respect to the Callisto planning team.
 
  
  
Line 57: Line 73:
  
  
 +
<p>
 
Issues that arose in Callisto that I would hope that we would be addressed in Europa
 
Issues that arose in Callisto that I would hope that we would be addressed in Europa
 
* Bug triage - There needs to be more participation in bug triage of cross-platform issues. Very few people subscribed to the Callisto component for bug notification.
 
* Bug triage - There needs to be more participation in bug triage of cross-platform issues. Very few people subscribed to the Callisto component for bug notification.
* Implementing new and existing build technology across projects - For instance, including packed jars could decrease the bandwidth utilization and significantly improve the update experience for users.  However, since it's new technology it takes a while for it to percolate across projects, be tested in the build scripts, and implemented in productionIt would be great if more projects implement technologies such as these in an earlier timeframe.
+
* Implementing new and existing build technology across projects - For instance, including packed jars could decrease the bandwidth utilization at eclipse.org and associated mirrors as well as significantly improve the update experience for users.  The platform team introduced this technology very late in the development cycle which made it difficult for other teams to implement it in a timely fashionWhat process should there be to adopt/encourage accepted best releng practices across teams?
* versioning - There were a lot of questions about implementing qualifiers. Admittedly, it is a complicated topic.  However, I don't know if there is a way to diffuse information such as this across projects in a more effective manner. Tutorials etc?
+
* Versioning and implementing qualifiers - There were a lot of questions about implementing qualifiers and the core team's new versioning scheme [[Version_Numbering]].   Admittedly, it is a complicated topic.  However, I don't know if there is a way to diffuse information such as this across projects in a more effective manner. Tutorials etc? Again, this is probably an issue of lack of release engineering resources and time.
 +
* Build notification - Other teams would like to be notified by RSS feeds that our builds are ready so they can build against us, Nick has contributed this to basebuilder and we are testing this facility.
 +
</p>
  
Facilities I wish we had
 
* More time dedicated to improving our build process
 
  
 +
<h5>Facilities I wish we had</h5>
 +
<p>
 +
* More time dedicated to improving our build process and fixing bugs.
 +
* Place to point to new projects to to help them get started with their build processes.
 +
* I would trust other committers or active contributors to build them.
 +
</p>
  
  
Describe your hopes and ambitions for Europa. Consider describing ...
+
<h5>Procedures to reduce stress and increase efficiency</h5>
 +
<p>
 +
* Incremental build (Ultimately a much faster build). See Eclipse 3.3 plan item http://www.eclipse.org/eclipse/development/eclipse_project_plan_3_3.html and bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=154083
 +
</p>
 +
  
    * facilities you wish you had and how they might work and who you would trust to build them.
+
<h5>Communication Mechanisms</h5>
    * precedures that would make your life less stressful or more efficient. also when and how you would escape those precedures in an emergency.
+
<p>
    * communication mechanisms that would support both routine and exceptional circumstances that are sure to arise as more projects join the release train.
+
* Communication facilities - We use email and IM for routine and exceptional circumstances.  
 +
* RSS feeds would be good to automatically notify downstream teams when dependancies are available.
 +
* Collection of release engineering best practices so new projects get their builds working and submitted to Europa. Pointers regarding update sites, creating reproducable builds, versioning, pde build best practices, feature design etc.
 +
</p>

Revision as of 16:25, 6 September 2006

Top Level Eclipse Project Release Engineering team


About Us


The release engineering team consists of Sonia Dimitrov and Kim Moir. We provide release engineering services the Eclipse top-level project (Platform, JDT, PDE and Equinox subprojects). We also manage builds for some of IBM's Eclipse based product offerings.


Eclipse 1.0 was released November 7, 2001. Sonia joined the eclipse release engineering team in early 2001. Kim moved to the release engineering team in late 2003.


Our relationship to other developers on the team is multi-faceted. Some of our responsibilities include

  • Implementing and testing new functionality in the build process. We work closely with the PDE Build and Core/Equinox committers to test new PDE build features, for example grouped builds, versioning, conditioned and packed jars etc.
  • Manage the build and test infrastructure.
  • Assist developers when they wish add new plugins, test plugins, reorganize features etc. Run many test builds to ensure that the next build will complete successfully.
  • Public relations - keep the build team up to date on the build schedule, milestone schedules, changes to the build etc. Often we also serve as the unofficial "greeter" to the team and redirect people to the correct contact person for their area of interest.
  • Design and maintain feature structure.
  • Constantly trying to optimize our builds.
  • Implement new automation tools in our build.
  • Testing, testing, testing the build process. And then testing it again.
  • Although our process is automated, we do still monitor the builds, especially during milestone weeks. We also encourage quick resolution of build problems. (Gentle prodding.)
  • We often provide advice to teams who are trying to build in the "Eclipse Way".


We truly enjoy working with the other developers on the team because

  • Everyone is willing to share knowledge and work together to solve problems.
  • People care about the code they submit and ensuring that a build is successful.
  • They feed us well. (Core team - chocolate, PDE team - cake, UI team - curry).


On a good day, we don't have to do anything build related because it just works automatically. This allows us to focus of fixing our bug backlog, test and implement new build functionality. On a bad day, everything that possibly can go wrong does go wrong. Our lab infrastructure has serious problems, unexpected and unavoidable build problems have occurred. ie. network or power outage, hardware failure or even a small flood.


Our top priority is to get a build to developers. We also minimize risk of build failure by testing our process from end to end with any change in the scripts or infrastructure. We strive for CRISP builds (complete, repeatable, informative, schedulable, and portable) that incorporate release engineering best practices.


Since we have milestone builds every six weeks, we always feel that we are working toward a release. Of course, there isn't as much at stake with M1 as there is with M5 so the later milestones are much more stressful. During milestone week, we don't usually implement new functionality into the builder because we want to ensure that the builds run smoothly as possible. However, sometimes we need to make late breaking changes in the builder to address a serious bug during milestone week. During each milestone week, we test the newest plugins in org.eclipse.releng.basebuilder by running test builds. (org.eclipse.releng.basebuilder is a subset of eclipse SDK plugins that we use to run a headless build). This ensures that we can move the plugins to the builder as soon as the milestone is released.


As we ramp down toward release, we try try to reduce the magnitude of the changes we make to the builder. Also, by the very nature of the end game, as we move toward the release it requires more approval to submit bug fixes to the build.


Callisto Thoughts


Overall, Callisto was a good experience for us although there is certainly a lot of room for improvement the teams communicate and share knowledge. It was also interesting that the simultaneous build process itself was embryonic and continued to develop throughout the release but ultimately it worked. I don't think that we made any special adjustments because we are at the bottom of the stack, don't compile against anyone and we usually release milestones early on to allow others to build on us. I found the mailing list to be an effective means of communication but also spoke with other teams on IM. It was also good to collaborate with other project teams and learn from the approaches they took to solving problems. It would have been good to start Callisto submissions earlier so the platform team have a greater timeframe to implement update manager changes.


Europa Ambitions


Issues that arose in Callisto that I would hope that we would be addressed in Europa

  • Bug triage - There needs to be more participation in bug triage of cross-platform issues. Very few people subscribed to the Callisto component for bug notification.
  • Implementing new and existing build technology across projects - For instance, including packed jars could decrease the bandwidth utilization at eclipse.org and associated mirrors as well as significantly improve the update experience for users. The platform team introduced this technology very late in the development cycle which made it difficult for other teams to implement it in a timely fashion. What process should there be to adopt/encourage accepted best releng practices across teams?
  • Versioning and implementing qualifiers - There were a lot of questions about implementing qualifiers and the core team's new versioning scheme Version_Numbering. Admittedly, it is a complicated topic. However, I don't know if there is a way to diffuse information such as this across projects in a more effective manner. Tutorials etc? Again, this is probably an issue of lack of release engineering resources and time.
  • Build notification - Other teams would like to be notified by RSS feeds that our builds are ready so they can build against us, Nick has contributed this to basebuilder and we are testing this facility.


Facilities I wish we had

  • More time dedicated to improving our build process and fixing bugs.
  • Place to point to new projects to to help them get started with their build processes.
  • I would trust other committers or active contributors to build them.


Procedures to reduce stress and increase efficiency


Communication Mechanisms

  • Communication facilities - We use email and IM for routine and exceptional circumstances.
  • RSS feeds would be good to automatically notify downstream teams when dependancies are available.
  • Collection of release engineering best practices so new projects get their builds working and submitted to Europa. Pointers regarding update sites, creating reproducable builds, versioning, pde build best practices, feature design etc.