Skip to main content
Jump to: navigation, search

Difference between revisions of "Kepler Project Plan"

(Build Management)
Line 8: Line 8:
  
 
''Commented and Edited by: [[User:slewis.composent.com|slewis at composent.com]] 12:25, 26 December 2006 (PST)''
 
''Commented and Edited by: [[User:slewis.composent.com|slewis at composent.com]] 12:25, 26 December 2006 (PST)''
 +
 +
''Commented by: [[User:thomas.tada.se|thomas at tada.se]] 17:19, 26 December 2006 (EST)''
  
 
=== Kepler Project Proposal ===
 
=== Kepler Project Proposal ===

Revision as of 17:19, 26 December 2006

Proposed by: Jcasey.mergere.com 14:35, 8 December 2006 (EST)

Modified by: slewis at composent.com 14:55, 11 December 2006 (PST)

Commented by: henrik at cloudsmith.com 08:38, 12 December 2006 (EST)

Commented by: slewis at composent.com 12:48, 12 December 2006 (PST)

Commented and Edited by: slewis at composent.com 12:25, 26 December 2006 (PST)

Commented by: thomas at tada.se 17:19, 26 December 2006 (EST)

Kepler Project Proposal

Proposed Schedule

NOTE: Notation 'd' signifies the date of Kepler project approval with EMO, since this will keep us from having a code repository, etc. @EF.

NOTE: This proposed schedule is DRAFT and subject to change based upon input from other participants about related project plans and schedules

Proposed schedule for Kepler milestones.
Milestone Start Date Target Date Comments
#Milestone 1 d + 1 week d + 7 weeks 1-week delay anticipated to get CVS, other infrastructure setup @EF.
#Milestone 2 d + 9 weeks d + 15 weeks 1-week delay expected to review/fix process using lessons from first dev cycle.
#Milestone 3 d + 16 weeks d + 22 weeks
#Milestone 4 d + 23 weeks d + 29

Notes and Conventions

  • This project should be divided into a series of relatively short milestones, each of which will trigger a new release of the Kepler project once completed. Milestones should be achievable in six weeks, plus or minus one week.
  • Files formatted using the (new) native Kepler project model syntax as kmodel-extension files, since they are meant to catch project metadata that cannot be stored in the original metadata files for the project.

Kepler Integration Strategy

NOTE: The general strategy for implementing Kepler deliverables listed below will be integration of existing technologies from both Eclipse projects (e.g. Buckminster, PDE, ALF, Corona, EMFT, ECF, Phoenix, etc), and other open source community resources (e.g. Apache Maven, Bugzilla, Php, contributed new technologies, Continuum, Eclipse Wiki, etc). The desire is to leverage as much of existing efforts as possible, avoid reimplementation and rework, and cooperatively use all available expertise and technology for Community Lifecycle Mangagement (you heard it hear first :).

Here is a comment. Please feel free to insert your own comments inline below by reusing this markup. --Scott Lewis

<div id="Milestone 1"/>


Milestone 1

Headless Eclipse-Plugin and OSGi Builds

This is what Buckminster already does. I propose that Buckminster is used as is to provide this functionality. There is not need to create yet another abstraction layer for building - Buckminster is already this abstraction layer. If/when somethin is missing it is much faster to add it to Buckminster than to create something new. -- Henrik Lindberg
I agree...we should/will use what Buckminster already does. If we identify required additions to Buckminster to meet these goals then we will jointly address them -- Scott Lewis
  • Build at least 3 Eclipse plugin projects. Support building multiple plugins with both internal and external dependencies, multiple features, deployable zip files, and update sites for three existing Eclipse projects.
    • EXIT CRITERIA: BIRT, GEF, ECF projects (and others willing to participat) building deployable full-project plugins, features, zips, and update sites with existing technologies integrated-by-Kepler.

Common Project Model

  • Define extensible initial Kepler project model API
    • Define initial scope of project model: What to initially exclude?
    • Define target scope of project model (i.e. in Kepler 1.0 timeframe): What to eventually exclude?
Corona provides a collaboration context container. --Dennis O'Flynn
  • Corona's exemplary implementation of a context container is a project model called a Project Context Container
    • Used to define repository descriptors that reference artifact repositories associated with the project context
    • Defined and managed in a server-side Eclipse environment accessible via web services
    • Project Context Container events are distributed to all users of the container
Seems to me we can use Corona-identified elements (e.g. collaboration context container)

as candidate additions to the Common Project Model. I suggest, however, that we add to the Common Project Model incrementally, after getting working experience with proposed additions...e.g. using Corona's event notification to notify about build events. --Scott Lewis

  • Read (at least) the following native Eclipse project files using existing APIs, and construct a runtime model instance in memory
    • .project
    • .classpath
    • MANIFEST.MF
    • plugin.xml
    • feature.xml
Buckminster reads these and instantiates a model for them already. -- Henrik Lindberg
Excellent that Buckminster reads these and instantiates a model for them. I would suggest we also add site.xml to the list above -- Scott Lewis
Buckminster has a site.xml reader too but we've put it on hold since we don't feel that a site.xml accurately describes a component per se. A site.xml is more like a description of a repository. It may contain several versions of different components and it has no defined relationship between the components that it includes. It might still be a good idea to include it but we need to work more on the semantics -- Thomas Hallgren
  • Define Kepler native file format and schema, for capturing augmented model information not captured by the original project metadata files
  • Support generating project metadata in one of many specific metadata formats, regardless of metadata origin
    • Mark each generated metadata file as GENERATED in a header comment, to warn users against modifying it
    • Mark original set of project metadata files as authoritative, plus kmodel-extras file
I think the following two requirements should be added to the first milestone. -- Henrik Lindberg
  • Tag all metadata with its origin to enable popping domain specific editors and other functionality.
  • Include support for Corona collaboration from the start; weaving in collaboration events late in the game is much harder.
See questions below. -- Scott Lewis
  • Tag all metadata with its origin to enable popping domain specific editors and other functionality. Henrick, for me can you provide an example for what you mean here? What would be the origin for manifest.mf for example?
  • Include support for Corona collaboration from the start; weaving in collaboration events late in the game is much harder. +1. I think that having some notion of collaboration in project object model makes a lot of sense...I'm just not clear on how far to go with it initially (as workflow/etc can become very complex...and I would like to get the 'simple' stuff right first).
The origin is the manifest.mf, i.e. the file as such (relative to the component root). The domain agnostic model can declare that dependencies origins from the Require-Bundle header of that file and that generated actions that performs for instance the plugin export stems from the build.properties file. -- Thomas Hallgren

Build Management

  • Initial Framework to orchestrate external builds from Kepler using native Kepler project model.
This is covered by Buckminster - although "to orchestrate" sounds like something elaborate like a distributed build. -- Henrik Lindberg
Since the Kepler project model isn't yet defined I don't know what you mean that it's already covered by Buckminster...but I accept and am very glad that Buckminster already has access/is using all of the Eclipse plugin/feature/update site meta info to do complete builds...so the execution of a Kepler project model will be straightforward. -- Scott Lewis

<div id="Milestone 2"/>


Milestone 2

Headless Eclipse-Plugin and OSGi Builds

  • Maven, Buckminster, PDE, and web site integration to automate/orchestrate the steps which are currently explicit or custom when performing headless plugin builds
Concerning Headless Eclipse-Plugin and OSGi builds. We already do this based on PDE (and the pde-build). Putting Maven, Buckminster and PDE alongside is wrong. Buckminster represents a generic build model and we already have great support for building PDE headless. Where does Maven come in? -- Thomas Hallgren
Thomas asked: Where does Maven come in? I (Scott Lewis) believe Maven comes in when dealing with
  • Artifacts that are already stored, or 'want' to be stored in remote repositories. e.g. Jars/libs that are to be used/reused throughout an organization in a consistent, managed way.
  • Other build artifact creation well-supported in headless builds via Maven plugins: for example reporting, automated testing, style checking, documentation/help generation, deployment management -- Scott Lewis
Build Server
  • Introduce build-server using Kepler-integrated technologies
Why introduce a "build-server" at this stage? All benefits from it, web-based interface etc. arrives in M3. -- Thomas Hallgren
I think there are some practical benefits of having a build server even short of web-based interface. One (for projects using Kepler tech) is doing continuous builds and forcing the participating projects to be rigorous about always having a functioning build. Plus the experience that comes (to Kepler) from continuously running these technologies together to support actual project needs. -- Scott Lewis

Common Project Model

  • Refinements and extensions of project model to incorporate Corona, Buckminster, ALF, PDE, and other relevant technologies (NOTE: this depends on discussions with other projects to determine alignment, etc.)
I think that Buckminster and Corona are not something that is "accomodated", but rather the technologies used to implement a large portion of Kepler. -- Henrik Lindberg
Wording changed. -- Scott Lewis
The refinements to accommodate Corona and Buckminster must be done already in M1 since those technologies are central to what is done in Kepler. Again, PDE is just a domain specific model. Its covered by Buckminster. -- Thomas Hallgren
Corona's context container is designed for community collaboration. --Dennis O'Flynn
  • Can be used to accommodate Kepler's project model, or any other collaboration context.
I understand the points about context container and domain specific model. The point about 'refinements and extensions' is simply that I expect the common project model to evolve and be extended during Kepler development rather than being completed entirely 'up front' (M1). This is not meant to exclude collaboration, non-plugin/PDE domains, or other things (e.g. testing) from incorporation into the common project model, but rather just convey the realization that introduction of things into the common project model will probably have to be phased -- Scott Lewis
  • Tracking of the origin for each model element (for eventual write-back purposes)
This should be in milestone 1. It must be there from the beginning. -- Henrik Lindberg
Agreed. -- Will be moved. -- Scott Lewis
  • Selection of searchable artifact sources to be used for current project (using configured sources, above)
  • Search for project dependencies using artifact sources (see configuration, above)
This is within Buckminster's scope - although there is much that can be done; such as examining source code rather than meta data. A framework where language specific (Java, C, PHP, etc.) plugins can be added is perhaps the right way to go here. Scanning source in this way, is tricky to do "on the fly" and keep in sync - it depends on the underlying project model and what is persisted. Hence this is something that probably should be provided by each project (java, C, PHP) with help from Kepler. As en example, a refactoring of Java code would perhaps alter the dependencies. -- I could be missing the point here as I cant say I fully understood the proposed search functionality.
    • Use generic dialog provider base for common search elements.
    • Use extension point to augment search form elements.
  • Simple graphical editor for Kepler project model editing
    • Editor synchronization with .project/.classpath/plugin.xml/feature.xml files
Fine if this means using the existing editors. Anything beyond that should be done in cooperation with each project. -- Henrik Lindberg
Should/will use existing editors. -- Scott Lewis

Build Management

Build Management should be based on Buckminster since we already cover 80% of the functionality. -- Thomas Hallgren
I agree. I think, though, that what is the 80% that is the covered by Buckminster and what is the remaining 20% not covered by Buckminster is not clear to everyone (me). Could you (Thomas and/or Henrick) provide us with pointers to description of what you see as there and what's missing WRT Buckminster build management? -- Scott Lewis
Buckminster defines the build process in terms of actions, dependencies, etc. It then orchestrates the retrieval and invocation of actions in accordance with the definition. A more detailed explanation can be found on our wiki. The 20% I feel that we're lacking now relates mostly to variability. It all depends on what you put into the term Build Management of course. We do not have a build server. Nor do we provide some specific framework for compiling reports from test output (like TPTP). -- Thomas Hallgren
  • Support automated builds targeting multiple platform versions simultaneously (e.g. Eclipse 2.0, 3.0, 3.1, 3.3M3, current release, etc).
Why do we need to support target environments like Eclipse 2.0? I think we should start at 3.1 and not support targets older then that. Very few (if any) will use it and it just generates a lot of overhead. -- Thomas Hallgren
I think we need some further feedback/discussion about commercial (and other community) needs here. In other words, is 3.0+ or 3.1+ support enough? -- Scott Lewis
  • Builder bindings: Ability to execute "main build" for any integrated build tool whenever the Eclipse project is built in the IDE (via integration with existing Eclipse builder mechanisms as well as Kepler-defined extension points).
    • Consolidation of Europa Common Build Infrastructure Ant scripts inside a PDE-specific Kepler-Ant integration point. (NOTE: this requires help from those who know PDE/CBI more thoroughly)
      • Using Kepler, allow augmentation of standard PDE builds using these CBI scripts/scriptlets.
  • Accessory builds (using the build tool to execute a non-default build activity, such as generating javadocs)
    • Support for calling specific Maven lifecycle phases/goals
    • Support for calling specific Buckminster actions.
    • Support for calling specific Ant targets

Configuration (currently global-only, not per-project)

  • Model
    • Preferred external project metadata format
      • Eclipse .project/.classpath + kmodel-externals files
  • Searchable artifact sources
    • Ordering of these sources

<div id="Milestone 3"/>


Milestone 3

NOTE: This milestone will remain incomplete until we know more from the outcomes of previous milestones.

Headless Eclipse-Plugin and OSGi Builds

Build Server
Has someone looked at TPTP (build output etc.). Seems some of the coming functionality is covered there.
I have not. Who is the contact for TPTP release engineering? and I will contact. -- Scott Lewis
  • Web-based interface for control of build server and build log output
What is the difference between this and "Support for graphical management of build server" (m4)
Duplication. M4 item removed. -- Scott Lewis
  • Support for Add/Remove/Edit of project definitions on the build server
  • Support for Triggered builds that starts browser window to monitor build results using native build-server interface.

Build Management

  • Support graphical configuration helper-classes per plugin
    • Search for graphical editor helper-class for each plugin; use default XML-editor if not found.
  • Search for plugins
    • Add plugin repository (select from searchable artifact source list, filtered for entry with Maven capabilities)
  • Project-level integration for Problems/Errors view
    • Filtering by type of Kepler problem
  • User input/prompt dialog framework (to allow external build tool to prompt the user for information)

Configuration

  • Support per-project configuration of all existing Kepler options
  • Build Server
    • Global build-server list

<div id="Milestone 4"/>


Milestone 4

NOTE: This milestone will remain incomplete until we know more from the outcomes of previous milestones.

Headless Eclipse-Plugin and OSGi Builds

Build Server
  • Replace build monitoring/results browser window with integrated notifications framework for remote events.
    • Track elapsed time
    • Track accumulated output
    • Support multiple monitor types
      • Polling monitor: Configurable polling period
      • Async monitor: monitor abstract event connection (can be JMS/socket/etc.)
    • Event/Process management interface for gaining access to server-side controls available for that process.

Configuration

  • Build Server
    • Per-project configuration of build server(s) to be used
      • Allow selection from global list
      • Allow one-off configuration of build server(s) for that project only

<div id="Milestone 5"/>


Milestone 5

NOTE: This milestone will remain incomplete until we know more from the outcomes of previous milestones.

Headless Eclipse-Plugin and OSGi Builds

Build Server
  • Support backing up/restoring build server configuration to/from file.
  • Tool Detection
    • Support abstract detection mechanism for use in various detection scenarios (below).
    • Support detection of build servers; add to the global build-server list
    • Support detection of remote dependency sources

Back to the top