Skip to main content
Jump to: navigation, search

Difference between revisions of "Kepler Project Plan"

(Build Management)
 
(17 intermediate revisions by 4 users not shown)
Line 6: Line 6:
  
 
''Commented by: [[User:slewis.composent.com|slewis at composent.com]] 12:48, 12 December 2006 (PST)''
 
''Commented by: [[User:slewis.composent.com|slewis at composent.com]] 12:48, 12 December 2006 (PST)''
 +
 +
''Commented and Edited by: [[User:slewis.composent.com|slewis at composent.com]] 12:25, 26 December 2006 (PST)''
 +
 +
''Commented by: [[User:thomas.tada.se|thomas at tada.se]] 17:19, 26 December 2006 (EST)''
 +
 +
''Updated by: [[User:carlos.sanchez.devzuz.com|carlos.sanchez at devzuz.com]] 19:37, 25 August 2007 (CEST)''
  
 
=== Kepler Project Proposal ===
 
=== Kepler Project Proposal ===
Line 46: Line 52:
 
<div style="border: 2px solid #8E87EB; padding: 6px;">Here is a comment.  Please feel free to insert your own comments inline below by reusing this markup. ''--Scott Lewis''</div>
 
<div style="border: 2px solid #8E87EB; padding: 6px;">Here is a comment.  Please feel free to insert your own comments inline below by reusing this markup. ''--Scott Lewis''</div>
  
<div id="Milestone 1"/>
+
<div id="Milestone 1">
  
 
----
 
----
=== Milestone 1 ===
 
  
==== Headless Eclipse-Plugin and OSGi Builds ====
+
=== Milestone 1 : Core Model Definition ===
{{CommentBox|This is what Buckminster already does. I propose that Buckminster is used as is to provide this functionality. There is not need to create yet another abstraction layer for building - Buckminster is already this abstraction layer. If/when somethin is missing it is much faster to add it to Buckminster than to create something new. -- Henrik Lindberg}}
+
  
{{CommentBox|I agree...we should/will use what Buckminster already does.  If we identify required additions to Buckminster to meet these goals then we will jointly address them -- Scott Lewis}}
+
==== EMF Models for the core model and base extensions ====
  
* Build at least 3 Eclipse plugin projectsSupport building multiple plugins with both internal and external dependencies, multiple features, deployable zip files, and update sites for three existing Eclipse projects.
+
Provide the core schema for handling the storage of project meta-data.   
  
** '''EXIT CRITERIA:''' BIRT, GEF, ECF projects (and others willing to participat) building deployable full-project plugins, features, zips, and update sites with existing technologies integrated-by-Kepler.
+
For more information on the core structure see [[Proposed_Kepler_Collaboration_Model_Structure]].
  
==== Common Project Model ====
+
Also define a set of core project, version and artifact facets that can be used to capture common information.
  
* Define extensible initial Kepler project model API
+
==== Definition of meta-data around artifacts ====
**Define initial scope of project model:  What to initially exclude?
+
**Define target scope of project model (i.e. in Kepler 1.0 timeframe):  What to eventually exclude?
+
{{CommentBox|Corona provides a collaboration context container. ''--Dennis O'Flynn''
+
* Corona's exemplary implementation of a context container is a project model called a Project Context Container
+
** Used to define ''repository descriptors'' that reference artifact repositories associated with the ''project'' context
+
** Defined and managed in a server-side Eclipse environment accessible via web services
+
** Project Context Container ''events'' are distributed to all users of the container
+
}}
+
{{CommentBox|Seems to me we can use Corona-identified elements (e.g. collaboration context container)
+
as candidate additions to the Common Project Model.  I suggest, however, that we add to the Common Project Model incrementally, after getting working experience with proposed additions...e.g. using Corona's event notification to notify about build events. ''--Scott Lewis''
+
}}
+
* Read (at least) the following native Eclipse project files using existing APIs, and construct a runtime model instance in memory
+
** ''.project''
+
** ''.classpath''
+
** ''MANIFEST.MF''
+
** ''plugin.xml''
+
** ''feature.xml''
+
{{CommentBox|Buckminster reads these and instantiates a model for them already. -- Henrik Lindberg}}
+
{{CommentBox|Excellent that Buckminster reads these and instantiates a model for them. I would suggest we also add ''site.xml'' to the list above -- Scott Lewis}}
+
* Define Kepler native file format and schema, for capturing augmented model information not captured by the original project metadata files
+
* Support generating project metadata in one of many specific metadata formats, regardless of metadata origin
+
** Mark each generated metadata file as '''GENERATED''' in a header comment, to warn users against modifying it
+
** Mark original set of project metadata files as authoritative, plus kmodel-extras file
+
{{CommentBox|I think the following two requirements should be added to the first milestone. -- Henrik Lindberg
+
* Tag all metadata with its origin to enable popping domain specific editors and other functionality.
+
* Include support for Corona collaboration from the start; weaving in collaboration events late in the game is much harder.
+
}}
+
{{CommentBox|See questions below. -- ''Scott Lewis''
+
* Tag all metadata with its origin to enable popping domain specific editors and other functionality.  ''Henrick, for me can you provide an example for what you mean here?  What would be the origin for manifest.mf for example?''
+
* Include support for Corona collaboration from the start; weaving in collaboration events late in the game is much harder.  ''+1.  I think that having some notion of collaboration in project object model makes a lot of sense...I'm just not clear on how far to go with it initially (as workflow/etc can become very complex...and I would like to get the 'simple' stuff right first).''
+
}}
+
  
==== Build Management ====
+
Provide a clearer definition around the artifacts side of the project model. Understanding how to represent the artifacts (files) that are part of the project and are key to collaborating and consuming the project.
     
+
* Initial Framework to orchestrate external builds from Kepler using native Kepler project model.
+
{{CommentBox|This is covered by Buckminster - although "to orchestrate" sounds like something elaborate like a distributed build. -- Henrik Lindberg}}
+
  
{{CommentBox|Since the Kepler project model isn't yet defined I don't know what you mean that it's already covered by Buckminster...but I accept and am very glad that Buckminster already has access/is using all of the Eclipse plugin/feature/update site meta info to do complete builds...so the execution of a Kepler project model will be straightforward. -- Scott Lewis}}
+
==== Maven2 and PDE model adapters (integration with Buckminster) ====
  
<div id="Milestone 2"/>
+
Work with Buckminster to define how we can leverage their ability to parse different project types and gather project information that can be used in collaboration.
  
----
+
==== Collaboration Model Viewer/Editor ====
  
=== Milestone 2 ===
+
Provide an Eclipse editor that can be used to create/view or edit a collaboration model.  The core editor will provide access to the core model,  through the use of extension points you will be able to register editor sections to represent different facets of the collaboration model.  This will allow people to create extensions to the editor when new facets are defined.
  
==== Headless Eclipse-Plugin and OSGi Builds ====
+
See [[Kepler_Collaboration_Model_Editor]]
  
* Maven, Buckminster, PDE, and web site integration to automate/orchestrate the steps which are currently explicit or custom when performing headless plugin builds
+
</div>
{{CommentBox|Concerning Headless Eclipse-Plugin and OSGi builds. We already do this based on PDE (and the pde-build). Putting Maven, Buckminster and PDE alongside is wrong. Buckminster represents a generic build model and we already have great support for building PDE headless. Where does Maven come in? -- Thomas Hallgren}}
+
{{CommentBox|''Thomas asked:  Where does Maven come in?''  I (Scott Lewis) believe Maven comes in when dealing with
+
* Artifacts that are already stored, or 'want' to be stored in remote repositories.  e.g. Jars/libs that are to be used/reused throughout an organization in a consistent, managed way.
+
* Other build artifact creation well-supported in headless builds via Maven plugins:  for example reporting, automated testing, style checking, documentation/help generation, deployment management -- ''Scott Lewis'' }}
+
  
===== Build Server =====
+
<div id="Milestone 2">
  
* Introduce build-server using Kepler-integrated technologies
+
----
{{CommentBox|Why introduce a "build-server" at this stage? All benefits from it, web-based interface etc. arrives in M3. -- Thomas Hallgren}}
+
  
{{CommentBox|I think there are some practical benefits of having a build server even short of web-based interface.  One (for projects using Kepler tech) is doing continuous builds and forcing the participating projects to be rigorous about always having a functioning build.  Plus the experience that comes (to Kepler) from continuously running these technologies together to support actual project needs. -- ''Scott Lewis''}}
+
=== Milestone 2 : Adapters and UI ===
  
==== Common Project Model ====
+
==== Definition of integration with component model from Buckminster ====
 
+
==== Definition of extension points for extending collaboration model ====
* Refinements and extensions of project model to incorporate Corona, Buckminster, ALF, PDE, and other relevant technologies ''(NOTE: this depends on discussions with other projects to determine alignment, etc.)''
+
==== Definition of extension points for extending collaboration model viewer/editor ====
{{CommentBox|I think that Buckminster and Corona are not something that is "accomodated", but rather the technologies used to implement a large portion of Kepler. -- Henrik Lindberg}}
+
==== Integration of Corona Event notifications in Collaboration model ====
{{CommentBox|Wording changed. -- ''Scott Lewis''}}
+
{{CommentBox|The refinements to accommodate Corona and Buckminster must be done already in M1 since those technologies are central to what is done in Kepler. Again, PDE is just a domain specific model. Its covered by Buckminster. -- Thomas Hallgren}}
+
{{CommentBox|Corona's context container is designed for community collaboration. ''--Dennis O'Flynn''
+
* Can be used to accommodate Kepler's project model, or any other collaboration context.
+
}}
+
{{CommentBox|I understand the points about context container and domain specific model.  The point about 'refinements and extensions' is simply that I expect the common project model to evolve and be extended during Kepler development rather than being completed entirely 'up front' (M1).  This is not meant to exclude collaboration, non-plugin/PDE domains, or other things (e.g. testing) from incorporation into the common project model, but rather just convey the realization that introduction of things into the common project model will probably have to be phased -- ''Scott Lewis''}}
+
* Tracking of the origin for each model element ''(for eventual write-back purposes)''
+
{{CommentBox|This should be in milestone 1. It must be there from the beginning. -- Henrik Lindberg}}
+
{{CommentBox|Agreed. -- Will be moved. -- ''Scott Lewis'' }}
+
* Selection of searchable artifact sources to be used for current project ''(using configured sources, above)''
+
* Search for project dependencies using artifact sources ''(see configuration, above)''
+
{{CommentBox|This is within Buckminster's scope - although there is much that can be done; such as examining source code rather than meta data. A framework where language specific (Java, C, PHP, etc.) plugins can be added is perhaps the right way to go here. Scanning source in this way, is tricky to do "on the fly" and keep in sync - it depends on the underlying project model and what is persisted. Hence this is something that probably should be provided by each project (java, C, PHP) with help from Kepler. As en example, a refactoring of Java code would perhaps alter the dependencies. -- I could be missing the point here as I cant say I fully understood the proposed search functionality.}}
+
** Use generic dialog provider base for common search elements.
+
** Use extension point to augment search form elements.
+
* Simple graphical editor for Kepler project model editing
+
** Editor synchronization with .project/.classpath/plugin.xml/feature.xml files
+
{{CommentBox|Fine if this means using the existing editors. Anything beyond that should be done in cooperation with each project. -- Henrik Lindberg}}
+
{{CommentBox|Should/will use existing editors. -- ''Scott Lewis''}}
+
  
==== Build Management ====
+
</div>
{{CommentBox|Build Management should be based on Buckminster since we already cover 80% of the functionality. -- Thomas Hallgren}}
+
{{CommentBox|I agree.  I think, though, that what is the 80% that is the covered by Buckminster and what is the remaining 20% not covered by Buckminster is not clear to everyone (me).  Could you (Thomas and/or Henrick) provide us with pointers to description of what you see as there and what's missing WRT Buckminster build management? -- ''Scott Lewis'' }}
+
* Support automated builds targeting multiple platform versions simultaneously (e.g. Eclipse 2.0, 3.0, 3.1, 3.3M3, current release, etc).
+
{{CommentBox| Why do we need to support target environments like Eclipse 2.0? I think we should start at 3.1 and not support targets older then that. Very few (if any) will use it and it just generates a lot of overhead. -- Thomas Hallgren}}
+
{{CommentBox| I think we need some further feedback/discussion about commercial (and other community) needs here. In other words, is 3.0+ or 3.1+ support enough? -- Scott Lewis}}
+
* Builder bindings: Ability to execute "main build" for any integrated build tool whenever the Eclipse project is built in the IDE (via integration with existing Eclipse builder mechanisms as well as Kepler-defined extension points).
+
** Consolidation of Europa Common Build Infrastructure Ant scripts inside a PDE-specific Kepler-Ant integration point. ''(NOTE: this requires help from those who know PDE/CBI more thoroughly)''
+
*** Using Kepler, allow augmentation of standard PDE builds using these CBI scripts/scriptlets.
+
* Accessory builds (using the build tool to execute a non-default build activity, such as generating javadocs)
+
** Support for calling specific Maven lifecycle phases/goals
+
** Support for calling specific Buckminster actions.
+
** Support for calling specific Ant targets
+
  
==== Configuration (currently global-only, not per-project) ====
+
<div id="Milestone 3">
  
* Model
+
----
** Preferred external project metadata format
+
*** Eclipse .project/.classpath + kmodel-externals files
+
* Searchable artifact sources
+
** Ordering of these sources
+
  
<div id="Milestone 3"/>
+
=== Milestone 3 : Definition of Collaboration Storage Extensions ===
  
----
+
==== Definition of concept of Project Store and Artifact Store ====
=== Milestone 3 ===
+
==== API definitions for Project Store and Artifact Store (WSDL) ====
 +
==== UI components for searching Project Store and Artifact Store ====
 +
==== Integration of Q4E for generating model for Maven ====
 +
==== Integration of PDE/Java for generating model from Eclipse projects ====
  
''NOTE: This milestone will remain incomplete until we know more from the outcomes of previous milestones.''
+
</div>
  
==== Headless Eclipse-Plugin and OSGi Builds ====
+
<div id="Milestone 4">
  
===== Build Server =====
+
----
{{CommentBox|Has someone looked at TPTP (build output etc.). Seems some of the coming functionality is covered there.}}
+
  
* Web-based interface for control of build server and build log output
+
=== Milestone 4 : Project Store and Integration Extensions ===
{{CommentBox|What is the difference between this and "Support for graphical management of build server" (m4)}}
+
* Support for Add/Remove/Edit of project definitions on the build server
+
* Support for Triggered builds that starts browser window to monitor build results using native build-server interface.
+
  
==== Build Management ====
+
==== Prototype of local Project Store and Artifact Store ====
 +
==== Extension point definition to allow tooling to integrate based on Collaboration Model ====
  
* Support graphical configuration helper-classes per plugin
+
</div>
** Search for graphical editor helper-class for each plugin; use default XML-editor if not found.
+
* Search for plugins
+
** Add plugin repository (select from searchable artifact source list, filtered for entry with Maven capabilities)
+
* Project-level integration for Problems/Errors view
+
** Filtering by type of Kepler problem
+
* User input/prompt dialog framework (to allow external build tool to prompt the user for information)
+
  
==== Configuration ====
+
<div id="Milestone 5">
 
+
* Support per-project configuration of all existing Kepler options
+
* Build Server
+
** Global build-server list
+
 
+
<div id="Milestone 4"/>
+
  
 
----
 
----
=== Milestone 4 ===
 
  
''NOTE: This milestone will remain incomplete until we know more from the outcomes of previous milestones.''
+
=== Milestone 5 : Integration ===
  
==== Headless Eclipse-Plugin and OSGi Builds ====
+
==== Integration of Corona ====
 +
==== Integration of ECF (IRC/Jabber) ====
 +
==== Integration of SCM tooling ====
 +
==== Integration of Mylyn ====
  
===== Build Server =====
+
</div>
  
* Support for graphical management of build server
+
<div id="Milestone 6">
* Replace build monitoring/results browser window with integrated notifications framework for remote events.
+
** Track elapsed time
+
** Track accumulated output
+
** Support multiple monitor types
+
*** Polling monitor: Configurable polling period
+
*** Async monitor: monitor abstract event connection (can be JMS/socket/etc.)
+
** Event/Process management interface for gaining access to server-side controls available for that process.
+
 
+
==== Configuration ====
+
 
+
* Build Server
+
** Per-project configuration of build server(s) to be used
+
*** Allow selection from global list
+
*** Allow one-off configuration of build server(s) for that project only
+
 
+
<div id="Milestone 5"/>
+
  
 
----
 
----
=== Milestone 5 ===
 
 
''NOTE: This milestone will remain incomplete until we know more from the outcomes of previous milestones.''
 
  
==== Headless Eclipse-Plugin and OSGi Builds ====
+
=== Milestone 6 : Integration of Build/CI Servers ===
  
===== Build Server =====
+
==== Definition of Build/CI server API (Buckminster) ====
 +
==== UI Components for Build Servers ====
 +
==== Meta-data extensions for builds servers ====
  
* Support backing up/restoring build server configuration to/from file.
+
</div>
* Tool Detection
+
** Support abstract detection mechanism for use in various detection scenarios ''(below)''.
+
** Support detection of build servers; add to the global build-server list
+
** Support detection of remote dependency sources
+
[[Category:Kepler]]
+

Latest revision as of 16:46, 1 February 2012

Proposed by: Jcasey.mergere.com 14:35, 8 December 2006 (EST)

Modified by: slewis at composent.com 14:55, 11 December 2006 (PST)

Commented by: henrik at cloudsmith.com 08:38, 12 December 2006 (EST)

Commented by: slewis at composent.com 12:48, 12 December 2006 (PST)

Commented and Edited by: slewis at composent.com 12:25, 26 December 2006 (PST)

Commented by: thomas at tada.se 17:19, 26 December 2006 (EST)

Updated by: carlos.sanchez at devzuz.com 19:37, 25 August 2007 (CEST)

Contents

Kepler Project Proposal

Proposed Schedule

NOTE: Notation 'd' signifies the date of Kepler project approval with EMO, since this will keep us from having a code repository, etc. @EF.

NOTE: This proposed schedule is DRAFT and subject to change based upon input from other participants about related project plans and schedules

Proposed schedule for Kepler milestones.
Milestone Start Date Target Date Comments
#Milestone 1 d + 1 week d + 7 weeks 1-week delay anticipated to get CVS, other infrastructure setup @EF.
#Milestone 2 d + 9 weeks d + 15 weeks 1-week delay expected to review/fix process using lessons from first dev cycle.
#Milestone 3 d + 16 weeks d + 22 weeks
#Milestone 4 d + 23 weeks d + 29

Notes and Conventions

  • This project should be divided into a series of relatively short milestones, each of which will trigger a new release of the Kepler project once completed. Milestones should be achievable in six weeks, plus or minus one week.
  • Files formatted using the (new) native Kepler project model syntax as kmodel-extension files, since they are meant to catch project metadata that cannot be stored in the original metadata files for the project.

Kepler Integration Strategy

NOTE: The general strategy for implementing Kepler deliverables listed below will be integration of existing technologies from both Eclipse projects (e.g. Buckminster, PDE, ALF, Corona, EMFT, ECF, Phoenix, etc), and other open source community resources (e.g. Apache Maven, Bugzilla, Php, contributed new technologies, Continuum, Eclipse Wiki, etc). The desire is to leverage as much of existing efforts as possible, avoid reimplementation and rework, and cooperatively use all available expertise and technology for Community Lifecycle Mangagement (you heard it hear first :).

Here is a comment. Please feel free to insert your own comments inline below by reusing this markup. --Scott Lewis

Milestone 1 : Core Model Definition

EMF Models for the core model and base extensions

Provide the core schema for handling the storage of project meta-data.

For more information on the core structure see Proposed_Kepler_Collaboration_Model_Structure.

Also define a set of core project, version and artifact facets that can be used to capture common information.

Definition of meta-data around artifacts

Provide a clearer definition around the artifacts side of the project model. Understanding how to represent the artifacts (files) that are part of the project and are key to collaborating and consuming the project.

Maven2 and PDE model adapters (integration with Buckminster)

Work with Buckminster to define how we can leverage their ability to parse different project types and gather project information that can be used in collaboration.

Collaboration Model Viewer/Editor

Provide an Eclipse editor that can be used to create/view or edit a collaboration model. The core editor will provide access to the core model, through the use of extension points you will be able to register editor sections to represent different facets of the collaboration model. This will allow people to create extensions to the editor when new facets are defined.

See Kepler_Collaboration_Model_Editor


Milestone 2 : Adapters and UI

Definition of integration with component model from Buckminster

Definition of extension points for extending collaboration model

Definition of extension points for extending collaboration model viewer/editor

Integration of Corona Event notifications in Collaboration model


Milestone 3 : Definition of Collaboration Storage Extensions

Definition of concept of Project Store and Artifact Store

API definitions for Project Store and Artifact Store (WSDL)

UI components for searching Project Store and Artifact Store

Integration of Q4E for generating model for Maven

Integration of PDE/Java for generating model from Eclipse projects


Milestone 4 : Project Store and Integration Extensions

Prototype of local Project Store and Artifact Store

Extension point definition to allow tooling to integrate based on Collaboration Model


Milestone 5 : Integration

Integration of Corona

Integration of ECF (IRC/Jabber)

Integration of SCM tooling

Integration of Mylyn


Milestone 6 : Integration of Build/CI Servers

Definition of Build/CI server API (Buckminster)

UI Components for Build Servers

Meta-data extensions for builds servers

Back to the top