Jump to: navigation, search

Difference between revisions of "Europa Build Workshop Breakout Results"

(Build Best Practices Breakout)
m (Actions)
 
(21 intermediate revisions by 4 users not shown)
Line 1: Line 1:
 
==Build Best Practices Breakout==
 
==Build Best Practices Breakout==
 
(computer crashed in the middle of typing in the result)
 
(computer crashed in the middle of typing in the result)
# Proper version numbering
+
# Proper versioning
 +
## We discovered there are three projects that have created tooling for checking version numbers conform to this specification [[Version_Numbering]] (BIRT,WTP, [http://www.eclipse.org/tptp/home/downloads/version.php?version=4.3.0&buildID=TPTP-4.3.0-200609130100 TPTP] and Platform).  Before anyone creates anymore tools :-) - please contact one of the teams to use one of the existing tools.  Links to be added.  Consolidation or pick best one to reduce overlap.
 
# [[Making_Builds_Reproducible|Making builds reproducible]]
 
# [[Making_Builds_Reproducible|Making builds reproducible]]
 
# Tag files from each release with the same human-readable tag
 
# Tag files from each release with the same human-readable tag
 
# Track API changes
 
# Track API changes
 +
## Hubert Leung uses Jdiff http://jdiff.org
 +
## WTP also provides a tool (Jeff Liu).
 
# avoid circular dependencies
 
# avoid circular dependencies
 
# documentation up-to-date; current link available on website; remove old contents
 
# documentation up-to-date; current link available on website; remove old contents
Line 10: Line 13:
 
# separate publishing script from “build” (compilation)
 
# separate publishing script from “build” (compilation)
 
# able to turn functions on the build on or off e.g. turn off publishing
 
# able to turn functions on the build on or off e.g. turn off publishing
# regularly scheduled builds
+
# [[Regularly_Scheduled_Builds|regularly scheduled builds]]
## public humiliation
+
## [[Gentle_Public_Humiliation|Gentle public humilitaion]]
## send email to mailing list
+
## [[Email_to_PDL|Send email to public mailing list or wiki/web page]]
## send email to plug-in owner
+
## [[Email_to_Owners|send email to responsible parties]]
# have an update site; optimize update sites
+
# have an update site; [[Update_Site_Optimization|optimize update sites]]
# pack 200
+
# pack 200 [[Pack200]].  How to implement in builds - see
# feature design – group plug-ins based on functional grouping
+
##
 +
# feature design – group plug-ins based on functional grouping, optimize feature design for PDE build. 
 +
## Hubert wrote a tool that generates a [http://download.eclipse.org/tptp/4.2.0.2/TPTP-4.2.0.2-200608021100/featuredoc/ report] that shows the feature structure and the dependancies that may be useful to other teams.  Also displays the feature that each plugin resides in and additional information.  Also, it displays whether a tool is shipped as a jar and has a plugin.xml or MANIFEST.MF. 
 +
## He also has an additional tool that conducts dependancy analysis.
 +
 
 +
 
 +
Go to [[Europa Build Workshop]] Home page
  
 
==Buckminster/Maven==
 
==Buckminster/Maven==
* Common model
+
 
** Backbone
+
'''Day 1 Discussions'''
** Graphical model input
+
* Common Model for Project Artifacts/Components
** Buckminster and Buckminster model
+
** Development of a Backbone
 +
** Graphical Model Input
 +
** Buckminster and Buckminster model for Eclipse-based Model of a Project
 
** Maven in Eclipse and Maven model
 
** Maven in Eclipse and Maven model
** Maven
+
** [http://www.maven.apache.org Maven]
* Suggest that they adopt a Europa project and help revise that project's build. Nick volunteered one of the smaller EMFT projects; another candidate is BIRT (Sue); don't want to take on too much - want one/few good success stories rather than broad swath of mediocrity
+
 
* Document the process as a part of working with one of the projects.
+
'''Day 1 Actions'''
 +
** Closer integration of Maven and Buckminster (Thomas/John)
 +
** Adoption of an existing Callisto project to help revise that project's build by the Europa release (group)
 +
** Document the migration/conversion process to Buckminster/Maven (group)
 +
** Agreed that BIRT will be the test case project (Sue/Thomas/John)
 +
 
 +
'''Day 2 Discussions'''
 +
 
 +
* BIRT Build Review
 +
** Denis offered build.eclipse.org build server for BIRT to run its builds (providing shell access to Eclipse committers)
 +
** (awaiting notes from Thomas)
 +
* Adoption of ECF Project (to have both a large and small build project converted for Europa)
 +
 
 +
'''Day 2 Actions'''
 +
* Subscription to the Users and Dev lists at Buckminster
 +
* Maven/Buckminster
 +
* BIRT
 +
* ECF
 +
** CVS location to be sent to Buckminster team (Pete)
 +
** Set up ECF locally w/ Buckminster (Henrik)
 +
 
 +
Future activities/planning will be coordinated by Natalie.
  
 
==RSS==
 
==RSS==
Line 126: Line 158:
  
  
[[Category:Releng]]
+
[[Category:Releng]] [[Category:Europa]] [[Category:Coordinated]]

Latest revision as of 01:37, 18 December 2006

Build Best Practices Breakout

(computer crashed in the middle of typing in the result)

  1. Proper versioning
    1. We discovered there are three projects that have created tooling for checking version numbers conform to this specification Version_Numbering (BIRT,WTP, TPTP and Platform). Before anyone creates anymore tools :-) - please contact one of the teams to use one of the existing tools. Links to be added. Consolidation or pick best one to reduce overlap.
  2. Making builds reproducible
  3. Tag files from each release with the same human-readable tag
  4. Track API changes
    1. Hubert Leung uses Jdiff http://jdiff.org
    2. WTP also provides a tool (Jeff Liu).
  5. avoid circular dependencies
  6. documentation up-to-date; current link available on website; remove old contents
  7. backup redundant systems
  8. separate publishing script from “build” (compilation)
  9. able to turn functions on the build on or off e.g. turn off publishing
  10. regularly scheduled builds
    1. Gentle public humilitaion
    2. Send email to public mailing list or wiki/web page
    3. send email to responsible parties
  11. have an update site; optimize update sites
  12. pack 200 Pack200. How to implement in builds - see
  13. feature design – group plug-ins based on functional grouping, optimize feature design for PDE build.
    1. Hubert wrote a tool that generates a report that shows the feature structure and the dependancies that may be useful to other teams. Also displays the feature that each plugin resides in and additional information. Also, it displays whether a tool is shipped as a jar and has a plugin.xml or MANIFEST.MF.
    2. He also has an additional tool that conducts dependancy analysis.


Go to Europa Build Workshop Home page

Buckminster/Maven

Day 1 Discussions

  • Common Model for Project Artifacts/Components
    • Development of a Backbone
    • Graphical Model Input
    • Buckminster and Buckminster model for Eclipse-based Model of a Project
    • Maven in Eclipse and Maven model
    • Maven

Day 1 Actions

    • Closer integration of Maven and Buckminster (Thomas/John)
    • Adoption of an existing Callisto project to help revise that project's build by the Europa release (group)
    • Document the migration/conversion process to Buckminster/Maven (group)
    • Agreed that BIRT will be the test case project (Sue/Thomas/John)

Day 2 Discussions

  • BIRT Build Review
    • Denis offered build.eclipse.org build server for BIRT to run its builds (providing shell access to Eclipse committers)
    • (awaiting notes from Thomas)
  • Adoption of ECF Project (to have both a large and small build project converted for Europa)

Day 2 Actions

  • Subscription to the Users and Dev lists at Buckminster
  • Maven/Buckminster
  • BIRT
  • ECF
    • CVS location to be sent to Buckminster team (Pete)
    • Set up ECF locally w/ Buckminster (Henrik)

Future activities/planning will be coordinated by Natalie.

RSS

  • One artifact of an EMF build (Thu 00:00 EDT) being promoted (Thu 01:00 EDT) is that an RSS file is updated and scp'd to download.eclipse.org
  • The UML project consumes the RSS feed to trigger a new build (Thu 03:00 EDT)
  • Platform will be publishing RSS soon (very soon)
  • UML assumes a linear ordering (reads just one feed, not many feeds, since it depends on EMF which in turn depends on Eclipse)
  • RSS feed supports a variety of status codes: build done, tests done, etc. though a definitive list has yet to be stated.
    • This is a TODO for Nick to define the spec, though I'm of course open to suggestions to make the spec useful to all
  • RSS is on a download site so that mirrors will be updated with files and then the RSS - as such, you can watch a given mirror's copy of the feed in order to know when the mirror's got new upstream project bits on which you depend
  • RSS is committed to CVS for history/auditing
  • Dennis and Nick recommend this should be adopted for cross-project notifications (and optionally response) by Europa projects to try it, use it and improve it
  • Nick is willing to be the point person for initial communication, documentation, assistance
  • Which builds to publish for? nightly? integration? good? failed?
    • Kim brings up "how to define what 'good' is - what if it was a test fixture failure (as determined by a human) instead of a test failure - then the human says 'this is a good build' and yet the RSS says 'bad'".
    • Bjorn argues for more information rather than less - let the downstream consumers decide what to do with the extra information (e.g., ignore nightly builds, ignore bad builds, ...)
    • As such, we would suggest that if you publish a build, it should includes an RSS entry (be it of type N or I/M or S or R). Build status in the feed (PASSED, FAILED, IN PROGRESS, PENDING, UNKNOWN, etc.) can default to a given state and be updated as needed by Ant tasks

Bjorn, Ward, Nick (and others) would like to see everyone adopting the process of publishing feeds within the next month, and listening to feeds (and determining how to respond) the month thereafter. Ideally, this means we could try some RSS build cascades (or at least RSS information cascades) by early November. There was a general consensus that this was a good idea, so I'm hoping everyone can commit to this schedule.

--nickb

Nick's Tools

Nick shows off new Search CVS tool for browsing CVS and Bugzilla. (This tool is also linked from our generated release notes.) If you'd like your project indexed in this database, add your name and /cvsroot/path/to/project(s) here. I'll update this when you're indexed so you can play with it.

  • org.eclipse.corona
  • org.eclipse.emf, org.eclipse.emf.releng.build, org.eclipse.emf.ecore.sdo, org.eclipse.xsd
  • org.eclipse.emft
  • org.eclipse.gmf
  • org.eclipse.uml2, org.eclipse.uml2.releng
  • www
    • emf
    • emft
    • uml2

Buckminster Demo

The Buckminster team gave a demo.

Common Build Infrastructure

Goals:

  • Shorter learning curve / quicker startup
  • Consistency
  • Best practices
  • Outreach Activity

Scope:

  • What is provided / what form
  • Benefits to Foundation / Projects
  • Committers and projects need not worry about so much setup/infrastructure

Wishlist

Bjorn advises to ask for the moon, so...

  • vservers / vmware instances to start builds from clean states
  • add build infrastructure to vserver image file so more consistency



  • PDE?: decouple java compilation from native building from packaging for projects with platform dependent code (eg., TPTP)


  • staff at eclipse.org to maintain infrastructure
  • staff at eclipse.org to manage release train

Eclipse.org Current Build Infrasctructure Offerings

  • Option 1: unmanaged vserver (*.eclipse.org)
    • users get root access, running Xen
    • no management provided by Eclipse
    • two physical servers to share load
    • currently limited by IP address pool
    • hardware limited by budget
    • requires extra maintenance / security / duplication of effort


  • Option 2: managed "massive massive quad-CPU beast": build.eclipse.org
    • Backups provided, no root access

Best Practices

  • Let projects choose whether to publish builds from build server to mirrored, so as to minimize bandwidth impact at Eclipse
  • Protect access to web UI to kick builds using LDAP so only project committers can run their own builds

Actions

  • Set up a vserver for Buckminster/BIRT builds
  • Assist in migrating ECF to web UI for kicking builds & running tests on their vserver using EMFT as example
  • Explore Buckminster as wrapper for running EMF/EMFT builds
  • Explore creating a web UI for running Buckminster builds (PHP, XMLRPC?)