Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "EclipseLink/Build/NextGen"

(Issues needing resolution)
(Ongoing evaluation)
 
(5 intermediate revisions by the same user not shown)
Line 4: Line 4:
 
In investigating solutions that meet the [http://wiki.eclipse.org/EclipseLink/Build/Requirements requirements outlined], many strengths and weaknesses of existing systems were uncovered. Below is a summary of each system as I understand it currently.
 
In investigating solutions that meet the [http://wiki.eclipse.org/EclipseLink/Build/Requirements requirements outlined], many strengths and weaknesses of existing systems were uncovered. Below is a summary of each system as I understand it currently.
  
- The current build system, based upon Ant running standalone, does not utilize any of the existing PDE settings configured for use within the eclipse IDE. This was intentional, and has strengths and weaknesses. First, it allows the 'official' builds to be completely independent of any IDE, or development tooling, since it relies soley upon ANT and the JDK within the OS environment.
+
- The current build system, based upon Ant running standalone, does not utilize any of the existing PDE settings configured for use within the eclipse IDE. This was intentional, and has strengths and weaknesses. First, it allows the 'official' builds to be completely independent of any IDE, or development tooling, since it relies soley upon ANT and the JDK within the OS environment. It also therefore relies upon minimal tooling and has a small number of "points-of-failure" dependent upon external tooling/resources.
However, it also means that any development builds in the IDE are not reflective of the true 'state of the project'. In addition, most advanced functionality provided by the IDE needs to either be emulated, or duplicated by the build system.
+
However, it also means that any development builds in the IDE are not reflective of the true 'state of the project'. In addition, most advanced functionality provided by the IDE needs to either be emulated, or duplicated by the build system. For true OSGi-like builds this would mean extending Ant's capabilities (manifest parsing).
  
 
- PDE (Plugin Development Platform) takes care of many OSGi related build and dependency issues, but its scope is limited. It is aware of zips and jars, but was not intended to be run outside an the Eclipse IDE.  
 
- PDE (Plugin Development Platform) takes care of many OSGi related build and dependency issues, but its scope is limited. It is aware of zips and jars, but was not intended to be run outside an the Eclipse IDE.  
  
- PDEBuild has been created to extend PDE functionality to these areas, and allow PDE to be used 'headlessly' (without a gui), but has similar drawbacks to Ant alone, in that the entire build is separate from the IDE - though it is much closer. It also has several hard-coded project assumptions.
+
- PDEBuild has been created to extend PDE functionality outside the IDE and allow PDE to be used 'headlessly' (without a gui), but has similar drawbacks our current dual system, in that the entire build is separate from the IDE - though it is much closer. It also has several hard-coded project assumptions, and is difficult to extend.
  
- Athena is an Eclipse build tool that utilizes PDE and PDEBuild to handle the project setup portion, and drive automation.
+
- Athena is an Eclipse build tool that utilizes PDE and PDEBuild to handle the project setup portion and drive automation. Use seems to be waning.
  
- Buckminster is another tool that has come into use for builds recently (Ganymede/Galileo) at Eclipse. It is tightly integrated with several Eclipse technologies - P2, PDE, Equinox, and others (CVS, SVN, Git, Maven) as well as integrating with ANT. It is intended to provide a standardized project provisioning/build system that could be used by development during development, yet also funtion through the entire release cycle (perform the necessary tasks of provisioning, building, testing, and publishing automatically during the all phases of the development cycle). In addition, it advertises the capability of generating Maven Metadata for P2 repositories.
+
- Buckminster is another tool that has come into use for builds recently (Ganymede/Galileo) at Eclipse. It is tightly integrated with several Eclipse technologies - P2, PDE, Equinox, and others (CVS, SVN, Git, Maven) as well as integrating with ANT. It is intended to provide a standardized project provisioning/build system that could be used by development during development, yet also funtion through the entire release cycle (perform the necessary tasks of provisioning, building, testing, and publishing automatically during the all phases of the development cycle). In addition, it advertises the capability of generating Maven Metadata for P2 repositories. Many Eclipse Projects use, and are adopting use of this tool.
  
- B3 is an iteration upon Buckminster. Refining what was learned and strengthening its deficiencies. It is, however, in incubation and only the P2 aggregation portion of the tool is currently available. I have been unable to evaluate its full capabilites.
+
- B3 is an iteration upon Buckminster. Refining what was learned and strengthening its deficiencies. It is, however, in incubation and only the P2 aggregation portion of the tool is currently available. The Aggregator is now the standard for generating the "release train" build repositories (Indigo). I have been unable to evaluate its full capabilites.  
  
- Maven is a build tool intended to manage complex dependencies. It was initially created before OSGi, and in some ways competes with it. It also operates in the same domain as ANT, and its use would replace the existing system in entirety. MAven 2 is widely adopted, while Maven 3 is new on the scene.
+
- Maven is a build tool intended to manage complex dependencies. It was initially created before OSGi, and in some ways competes with it. It also operates in the same domain as ANT, and its use would replace the existing system in entirety. Maven 2 is widely adopted, while Maven 3 is new on the scene. There has been extensive colaboration between the Eclipse P2, Buckminster and Maven3 teams. Extensibility does not appear to have been a goal for Maven 2, and I'm unaware if that has changed with Maven 3.  
  
-Tyco is a set of Eclipse extensions designed to allow an OSGi-based, Maven build strategy within the IDE. (replacement for PDE).
+
-Tyco is a set of Eclipse extensions designed to allow an OSGi-based, Maven build strategy within the IDE (replacement for PDE). It has a similar scope to PDE and seems targeted for Eclipse plugins. It is in incubation.
+
 
Below is a summary table:
+
==== Below is a summary table: ====
  
 
{|{{BMTableStyle}}
 
{|{{BMTableStyle}}
Line 31: Line 31:
 
|-
 
|-
 
| Ant
 
| Ant
| flexable
+
| flexable, extensible
| Not designed for OSGi; Needs extension; High Maintenance cost; constantly 'reinventing the wheel'; partially separate system than used in Dev
+
| Not designed for OSGi; Needs extension; Moderate Maintenance cost; constantly 'reinventing the wheel'; partially separate system than used in Dev
 
|  
 
|  
 
|-
 
|-
Line 41: Line 41:
 
|-
 
|-
 
| Buckminster
 
| Buckminster
| Flexible; Designed to provide dynamic dependency mapping; is end-to-end build integration system; Collaboration between Maven and P2 and Buckminster teams; can utilize Ant, PDE, Maven 2/3 and or Tycho; headless and IDE use same processes
+
| Flexible; Claims high extensibility; Designed to provide dynamic dependency mapping; is end-to-end build integration system; Collaboration between Maven and P2 and Buckminster teams; can utilize Ant, PDE, Maven 2/3 and or Tycho; headless and IDE use same processes
| closely tied to Eclipse; Complex - uses multible languages (learning cost); Requires interaction with multple teams
+
| closely tied to Eclipse; Complex - uses multible languages (learning cost)(large investment in initial setup); Requires interaction with multple teams; Light Documentation;
 
|  
 
|  
 
|-
 
|-
Line 57: Line 57:
 
| Tycho
 
| Tycho
 
| Eclipse plugin to use Maven build, p2 aware, Maven2 and Maven3 aware, Collaboration between Maven and P2 teams
 
| Eclipse plugin to use Maven build, p2 aware, Maven2 and Maven3 aware, Collaboration between Maven and P2 teams
| Incubation; Requires interaction with multple teams; Only aware of maven, p2, OSGi, and features; complete system re-write
+
| Incubation; Requires interaction with multple teams; Only aware of maven, p2, OSGi, and features (need separate solution for jar/zip creation); complete system re-write
 
|
 
|
 
|}
 
|}
 
  
 
=== Ongoing evaluation ===
 
=== Ongoing evaluation ===
Upon review of the [http://wiki.eclipse.org/EclipseLink/Build/Requirements requirements] and the tools available it has been decided to go forward with a proof-of-concept based upon Buckminster/PDE/Ant. Since buckminster seems to have the desired combination of scope, capabilities and flexability; while PDE is the same tool used by developement; and Ant can easily fill in any gaps. While it is possible that the combination of Buckminster/Tycho/Maven may be the ultimate iteration to fully replace the current shell/Ant system.
+
See [http://wiki.eclipse.org/EclipseLink/Build/NextGen/BuckminsterEval Buckminster] for the results of the evaluation from 11/10-01/11.
 
+
==== Issues needing resolution ====
+
Items found that need technical resolution:
+
 
+
* How to specify inclusion of jar artifacts?
+
**  Separate P2 repo (Still resolving "below threshold")?
+
**  URL Reader (matcher appears to be buggy)?
+
 
+
* How is a component jar actually created?
+
** PDE may do it. How to specify it must be done? Need .project regen?
+
 
+
* What issues are there with running our tests?
+
** Combination of Non-standard frameworks using junit, junit tests, and ant fixup of result "path" strings.
+
** Test suites can be run as simple junit test (Tom does regularly)
+
** Also theoretically possible to invoke using Ant.
+
 
+
* How to specify/generate standalone zips and POJO jars?
+
** Presume using Ant or Maven. Need driver mechanism.. mspec?
+
 
+
* How to setup to only build component if "necessary"?
+
** Will this require further breakup of the build, or can intelligence be built in?
+
** Can we assume component code change won't require downstream component recompile?
+
 
+
Items found that will need group resolution/discussion include:
+
 
+
* PDE/SVN integrations assume a flat project structure (trunk/org.eclipse.persistence.core). Fully integrating with these tools will require a SVN restructuring
+
 
+
* Once development is in Eclipse exclusively, we will NEED to be able to develop (edit code) via SVN through the firewall.
+
**  How to set this up will need to be resolved and clearly documented.
+
 
+
* Buckminster SVN integration assumes trunk/tags/branches followed by component structure (trunk/org.eclipse.persistence.core).
+
** Also the branches/<branch>/trunk structure we are using is non-standard and will need to change.
+
** It may be possible to write a custom resolver to deal with our current system though.
+
  
* Buckminster and PDE appear to allow date or revision qualifiers, but not both (.v<DATE>-r<REVISION> not supported).
+
See [http://wiki.eclipse.org/EclipseLink/Build/NextGen/TychoEval Tycho] for the results of the evaluation from 01/11-03/11
** It may be possible to write a custom versioning scheme though.
+
  
* PDE doesn't allow a newer compiler to generate older style classes (there is no ability to have 1.6 generate 1.5 classes).
+
See [http://wiki.eclipse.org/EclipseLink/Build/NextGen/TychoAgenda TychoAgenda] for the Agenda of the meeting to discuss the Tycho Eval Resuts.

Latest revision as of 12:11, 4 April 2011

NextGen Build Investigation

Results of preliminary investigations

In investigating solutions that meet the requirements outlined, many strengths and weaknesses of existing systems were uncovered. Below is a summary of each system as I understand it currently.

- The current build system, based upon Ant running standalone, does not utilize any of the existing PDE settings configured for use within the eclipse IDE. This was intentional, and has strengths and weaknesses. First, it allows the 'official' builds to be completely independent of any IDE, or development tooling, since it relies soley upon ANT and the JDK within the OS environment. It also therefore relies upon minimal tooling and has a small number of "points-of-failure" dependent upon external tooling/resources. However, it also means that any development builds in the IDE are not reflective of the true 'state of the project'. In addition, most advanced functionality provided by the IDE needs to either be emulated, or duplicated by the build system. For true OSGi-like builds this would mean extending Ant's capabilities (manifest parsing).

- PDE (Plugin Development Platform) takes care of many OSGi related build and dependency issues, but its scope is limited. It is aware of zips and jars, but was not intended to be run outside an the Eclipse IDE.

- PDEBuild has been created to extend PDE functionality outside the IDE and allow PDE to be used 'headlessly' (without a gui), but has similar drawbacks our current dual system, in that the entire build is separate from the IDE - though it is much closer. It also has several hard-coded project assumptions, and is difficult to extend.

- Athena is an Eclipse build tool that utilizes PDE and PDEBuild to handle the project setup portion and drive automation. Use seems to be waning.

- Buckminster is another tool that has come into use for builds recently (Ganymede/Galileo) at Eclipse. It is tightly integrated with several Eclipse technologies - P2, PDE, Equinox, and others (CVS, SVN, Git, Maven) as well as integrating with ANT. It is intended to provide a standardized project provisioning/build system that could be used by development during development, yet also funtion through the entire release cycle (perform the necessary tasks of provisioning, building, testing, and publishing automatically during the all phases of the development cycle). In addition, it advertises the capability of generating Maven Metadata for P2 repositories. Many Eclipse Projects use, and are adopting use of this tool.

- B3 is an iteration upon Buckminster. Refining what was learned and strengthening its deficiencies. It is, however, in incubation and only the P2 aggregation portion of the tool is currently available. The Aggregator is now the standard for generating the "release train" build repositories (Indigo). I have been unable to evaluate its full capabilites.

- Maven is a build tool intended to manage complex dependencies. It was initially created before OSGi, and in some ways competes with it. It also operates in the same domain as ANT, and its use would replace the existing system in entirety. Maven 2 is widely adopted, while Maven 3 is new on the scene. There has been extensive colaboration between the Eclipse P2, Buckminster and Maven3 teams. Extensibility does not appear to have been a goal for Maven 2, and I'm unaware if that has changed with Maven 3.

-Tyco is a set of Eclipse extensions designed to allow an OSGi-based, Maven build strategy within the IDE (replacement for PDE). It has a similar scope to PDE and seems targeted for Eclipse plugins. It is in incubation.

Below is a summary table:

System Pros Cons Comments or Wiki link
Ant flexable, extensible Not designed for OSGi; Needs extension; Moderate Maintenance cost; constantly 'reinventing the wheel'; partially separate system than used in Dev
PDE Build Less extenstion needed than ant, get many features now have to 'hack' Complex, tied to Eclipse, uses own implimentation of ant, may still need extensions, high maintenance cost, IDE/headless build separate systems, tied to someone else's process, our versioning will need to change; headless and IDE use different processes
Buckminster Flexible; Claims high extensibility; Designed to provide dynamic dependency mapping; is end-to-end build integration system; Collaboration between Maven and P2 and Buckminster teams; can utilize Ant, PDE, Maven 2/3 and or Tycho; headless and IDE use same processes closely tied to Eclipse; Complex - uses multible languages (learning cost)(large investment in initial setup); Requires interaction with multple teams; Light Documentation;
B3 Next Gen of Buckminster, Collaboration between Maven and P2 and Buckminster teams Incubation, Requires interaction with multple teams
Maven 2 Absolute Dependency stability (reproducability) Complete system rewrite, Non-OSGi aware, Not full solution? "Provisioning, building, testing, publishing" only provides 2-4, Maven 3 coming
Tycho Eclipse plugin to use Maven build, p2 aware, Maven2 and Maven3 aware, Collaboration between Maven and P2 teams Incubation; Requires interaction with multple teams; Only aware of maven, p2, OSGi, and features (need separate solution for jar/zip creation); complete system re-write

Ongoing evaluation

See Buckminster for the results of the evaluation from 11/10-01/11.

See Tycho for the results of the evaluation from 01/11-03/11

See TychoAgenda for the Agenda of the meeting to discuss the Tycho Eval Resuts.

Back to the top