Difference between revisions of "EclipseLink/Build/Infrastructure"

From Eclipsepedia

Jump to: navigation, search
(Continuous Build)
(Continuous Build)
Line 19: Line 19:
 
The purpose of the continuous build process is to insure that the codebase remains as 'clean' as possible by alerting committers as soon as possible to regressions. This is done by having cron kick-off a build of type "continuous", the autobuild checks to see if a transaction has been committed, and if so updates the local code 'view', initiates a build (test and product jars only), then runs the "SRG" (Short ReGression) Testsuite. Any test or build failures are reported to the EclipseLink dev mailing list.
 
The purpose of the continuous build process is to insure that the codebase remains as 'clean' as possible by alerting committers as soon as possible to regressions. This is done by having cron kick-off a build of type "continuous", the autobuild checks to see if a transaction has been committed, and if so updates the local code 'view', initiates a build (test and product jars only), then runs the "SRG" (Short ReGression) Testsuite. Any test or build failures are reported to the EclipseLink dev mailing list.
  
Due to the Automated Build architecture each branch is capable of running continuous builds. However, to allow the required window of time (approximately 15-20 minutes) only three branches can be alloted continuous build time. Therefore the three most active branches usually have crontab entries to kick-off every 20 minutes starting at the top of, 20 past, and 40 past the hour. Since trunk is always the most active it holds the top of the hour spot.
+
Due to the architecture of the Automated Build system, every branch is capable of running continuous builds. However, to allow the required window of time (approximately 15-20 minutes) only three branches can be alloted continuous build time. Therefore the three most active branches usually have crontab entries to kick-off every 20 minutes starting at the top of, 20 past, and 40 past the hour. Since trunk is always the most active it holds the top of the hour spot.
  
 
==== Nightly Build ====
 
==== Nightly Build ====

Revision as of 13:32, 18 June 2010

This page is a landing page dedicated to Infrastructure related topics for the EclipseLink project.

Contents

Build Infrastructure

Build Machines

All downloadable builds of EclipseLink are generated on the Eclipse Foundations build machine. Currently even classes which cannot be compiled at Eclipse are assembled with each build. An example of this is the Oracle extension which requires access to jars that cannot be stored on the Foundations hardware due to licencing restrictions. In this case, the extension is built on Oracle hardware from EclipseLink's SVN repository and local dependancy libraries. The generated classes are contributed by Oracle into the EclipseLink repository (in the form of source and bundle jars). The nighlty process takes these jars and generates appropriately packaged bundles for each build.

Automated Build Process

EclipseLink builds are all produced on the Eclipse Foundation build machine using a cron initiated process which kicks off a shell script that sets-up the environment, cleans the test database, and invokes the automated ant build.

The Automated Ant Build is setup in three layers:

bootstrap.xml: This ant script, as its name implies, establishes the ant environment necessary for builds on the Eclipse build machine. Its content remains fairly constant, and is shared by all active development branches.

autobuild.xml: The next level of the build is branch specific, and regulates the automated processes (nightly and continuous builds), it also includes targets for milestone promotion, and the high level target for the oracle extension). This script determines if a build is necessary (has the code changed?), and then root build script to do the work.

build.xml: This is the root build script for a development branch. In addition to general use by those wanting to build EclipseLink. It is used by the automated process to generate all deliverables.

Process Flow

Continuous Build

The purpose of the continuous build process is to insure that the codebase remains as 'clean' as possible by alerting committers as soon as possible to regressions. This is done by having cron kick-off a build of type "continuous", the autobuild checks to see if a transaction has been committed, and if so updates the local code 'view', initiates a build (test and product jars only), then runs the "SRG" (Short ReGression) Testsuite. Any test or build failures are reported to the EclipseLink dev mailing list.

Due to the architecture of the Automated Build system, every branch is capable of running continuous builds. However, to allow the required window of time (approximately 15-20 minutes) only three branches can be alloted continuous build time. Therefore the three most active branches usually have crontab entries to kick-off every 20 minutes starting at the top of, 20 past, and 40 past the hour. Since trunk is always the most active it holds the top of the hour spot.

Nightly Build

Nightly builds are unsigned, and routinely have issues. However, in most cases these are minor because the continuous build process (cb) regularly builds and tests the codebase as transactions are merged (between 6am-8pm EDT daily). Thus most major issues (regressions) are fixed within hours of the cause. Issues do slip through the cracks though, because only limited testing can be done with the cb (it is limited to a 20 minute run duration), regressions caught by the more extensive nightly build process are usually fixed by the next nightly build. Because of this, milestone and even release builds are directly promoted from nightly builds (after various levels of additional testing).

Build Promotion

Issues and Limitations

Download Infrastructure

P2 Update Sites

EclipseLink organizes its update sites into three different sites; signed "release builds", signed "milestone builds", and unsigned "nightly builds". Where "released builds" consist of product releases and thier associated bugfix maintenance patches, "milestone builds" are tested stable builds released regularly during development, and "nightly builds" are the latest and (hopefully) greatest build of the development codebase.

EclipseLink now utilizes "compound" repositories to make all the builds available in one site available for each type.

The releases can be found at:

Milestone releases are stored similarly. However, it should be noted that due to space restrictions only milestones for versions under development are retained, and due to limitations with P2 the milestone a build belongs to can take some research. It is usually best to simply update to the latest if you choose to use milestone builds. They can be found at:

Nightly builds have similar restrictions. They are only only retained for about a week. The builds are unsigned, and maynot even be stable. It is best not to use them unless you are trying to get past a specific issue you know is resolved with a particular nightly build. The URL to the nightly build update site is:

It should be noted that we have opted to use composite repositories in spite of a major drawback: a user can simply select the category checkbox, and inadvertently select *all* available builds. Thus cluttering , and causing a maintenance head-ache on thier own system. No solution to this issue is currently available, however, not using a composite repository for incremental builds also has issues led to issues that effected all users, all the time.

P2 organization

The Project update site is now centered around a single catagorized feature. The SDK. In the past we made available several features, and corresponding "source" features. Those features are still avalailable, but to select them you will need to unselect "show categorozed items". Currently, only the SDK feature is categorized under "EclipseLink". All past release repositories have been regenerated to use the SDK (binary bundles have not been modified).

The EclipseLink SDK is currently made up of the other six (6) features. They are:

  • EclipseLink JPA (org.eclipse.persistence.jpa)
  • EclipseLink MOXy (org.eclipse.persistence.moxy)
  • EclipseLink SDO (org.eclipse.persistence.sdo)
  • EclipseLink JPA Source (org.eclipse.persistence.jpa.source)
  • EclipseLink MOXy Source (org.eclipse.persistence.moxy.source)
  • EclipseLink SDO Source (org.eclipse.persistence.sdo.source)


Though the jar name of the feature is similar to an EclipseLink bundle of the same name, the two do not conflict because the feature (or in p2 parlance: the Installable Unit) is stored separately from the bundles it installs/includes.

Feature content

With the exception of the SDK (which includes all the other features), EclipseLink features are not 'nested', but rather are designed as completely autonomous units specific to the utilization of specific EclipseLink functionality. So though every 'feature' depends upon the EclipseLink core, there is no "Core feature". Instead "EclipseLink JPA" groups together all libraries (both project and external) that are distributable and are needed for developing and running an EclipseLink JPA application. The same is true of the MOXy and SDO features. This decision was originally made because it was felt to be more 'developer-centric', more easily understood and resulted in a less cluttered 'update' display.

Now all the features are made available in a single SDK, and nesting may make more conceptual sense, but for backward compatability the original structure has been maintained. In addition there is no real need to change it since, becasue of the nature of OSGi, a specific bundle will only be installed once, no matter how many other bundles (or features) may depend upon it.

It has been requested that a breakdown be made available of the bundles included in each feature outside of the xml projects used to create them (thus sparing the reader from having to understand the "feature.xml" layout). Given the changing nature of these dependencies I will outline the dependencies for the latest release currently under development (2.1.0), with the caveat that the ultimate autority is the feature.xml stored in the "features" subproject of EclipseLink's SVN repository.

EclipseLink SDK (org.eclipse.persistence.sdk):

  "includes" the other six features directly.

EclipseLink JPA (org.eclipse.persistence.jpa):

  javax.activation_1.1.0.v200906290531.jar
  javax.mail_1.4.0.v200804091730.jar
  javax.persistence_2.0.0.v201002051058.jar
  javax.xml.stream_1.0.1.v201001131653.jar
  org.eclipse.persistence.antlr_2.1.0
  org.eclipse.persistence.asm_2.1.0
  org.eclipse.persistence.core_2.1.0
  org.eclipse.persistence.jpa_2.1.0
  org.eclipse.persistence.jpa.equinox_2.1.0
  org.eclipse.persistence.jpa.equinox.weaving_2.1.0
  org.eclipse.persistence.oracle_2.1.0

EclipseLink MOXy (org.eclipse.persistence.moxy):

  javax.activation_1.1.0.v200906290531.jar
  javax.mail_1.4.0.v200804091730.jar
  javax.persistence_2.0.0.v201002051058.jar
  javax.xml.bind_2.1.12.v200912160909
  javax.xml.stream_1.0.1.v201001131653
  org.eclipse.persistence.asm_2.1.0
  org.eclipse.persistence.core_2.1.0
  org.eclipse.persistence.moxy_2.1.0

EclipseLink SDO (org.eclipse.persistence.sdo):

  commonj.sdo_2.1.1.v200905221342
  javax.activation_1.1.0.v200906290531.jar
  javax.mail_1.4.0.v200804091730.jar
  javax.persistence_2.0.0.v201002051058.jar
  javax.xml.bind_2.1.12.v200912160909
  javax.xml.stream_1.0.1.v201001131653
  org.eclipse.persistence.asm_2.1.0
  org.eclipse.persistence.core_2.1.0
  org.eclipse.persistence.moxy_2.1.0
  org.eclipse.persistence.sdo_2.1.0

EclipseLink JPA Source (org.eclipse.persistence.jpa.source):

  org.eclipse.persistence.antlr.source_2.1.0
  org.eclipse.persistence.asm.source_2.1.0
  org.eclipse.persistence.core.source_2.1.0
  org.eclipse.persistence.jpa.source_2.1.0
  org.eclipse.persistence.jpa.equinox.source_2.1.0
  org.eclipse.persistence.jpa.equinox.weaving.source_2.1.0
  org.eclipse.persistence.oracle.source_2.1.0

EclipseLink MOXy Source (org.eclipse.persistence.moxy.source):

  org.eclipse.persistence.asm.source_2.1.0
  org.eclipse.persistence.core.source_2.1.0
  org.eclipse.persistence.moxy.source_2.1.0

EclipseLink SDO Source (org.eclipse.persistence.sdo.source):

  org.eclipse.persistence.asm.source_2.1.0
  org.eclipse.persistence.core.source_2.1.0
  org.eclipse.persistence.moxy.source_2.1.0
  org.eclipse.persistence.sdo.source_2.1.0

Downloadable Content

Since classic zip archives of content are made available via Webpages, it makes little sense to discuss directory hierarchies. It may be of benefit to talk about the web organization, and actual content of the archives will be related below. The latest release available for download can be found on the main download page

Installer Zip

Bundle Zip

  eclipselink-jpa-modelgen_2.1.0.v20100220-r6632.jar
  javax.activation_1.1.0.v200906290531.jar
  javax.jms_1.1.0.jar
  javax.mail_1.4.0.v200804091730.jar
  javax.persistence_1.0.0.v200905011740.jar
  javax.resource_1.5.0.jar
  javax.servlet_2.4.0.v200806031604.jar
  javax.transaction_1.1.0.v201002051055.jar
  javax.xml.bind.source_2.1.12.v200912160909.jar
  javax.xml.bind_2.1.12.v200912160909.jar
  javax.xml.soap_1.3.0.jar
  javax.xml.stream_1.0.1.v201001131653.jar
  javax.xml.ws_2.0.0.v200902170419.jar
  org.eclipse.persistence.antlr.source_2.1.0.v20100220-r6632.jar
  org.eclipse.persistence.antlr_2.1.0.v20100220-r6632.jar
  org.eclipse.persistence.asm.source_2.1.0.v20100220-r6632.jar
  org.eclipse.persistence.asm_2.1.0.v20100220-r6632.jar
  org.eclipse.persistence.core.source_2.1.0.v20100220-r6632.jar
  org.eclipse.persistence.core_2.1.0.v20100220-r6632.jar
  org.eclipse.persistence.dbws.source_2.1.0.v20100220-r6632.jar
  org.eclipse.persistence.dbws_2.1.0.v20100220-r6632.jar
  org.eclipse.persistence.jpa.equinox.source_2.1.0.v20100220-r6632.jar
  org.eclipse.persistence.jpa.equinox.weaving.source_2.1.0.v20100220-r6632.jar
  org.eclipse.persistence.jpa.equinox.weaving_2.1.0.v20100220-r6632.jar
  org.eclipse.persistence.jpa.equinox_2.1.0.v20100220-r6632.jar
  org.eclipse.persistence.jpa.modelgen.source_2.1.0.v20100220-r6632.jar
  org.eclipse.persistence.jpa.modelgen_2.1.0.v20100220-r6632.jar
  org.eclipse.persistence.jpa.source_2.1.0.v20100220-r6632.jar
  org.eclipse.persistence.jpa_2.1.0.v20100220-r6632.jar
  org.eclipse.persistence.moxy.source_2.1.0.v20100220-r6632.jar
  org.eclipse.persistence.moxy_2.1.0.v20100220-r6632.jar
  org.eclipse.persistence.oracle.source_2.1.0.v20100220-r6632.jar
  org.eclipse.persistence.oracle_2.1.0.v20100220-r6632.jar
  org.eclipse.persistence.sdo.source_2.1.0.v20100220-r6632.jar
  org.eclipse.persistence.sdo_2.1.0.v20100220-r6632.jar
  about.html
  license.html
  readme.html
  javax.persistence.source_2.0.0.v201002051058.jar
  javax.persistence_2.0.0.v201002051058.jar
  commonj.sdo_2.1.1.v200905221342.jar

Source Zip

Maven Repository

EclipseLink maintains a maven repository for nightly, milestone and release builds. Nightly builds are kept to the latest ten builds per release version. There is a bug being looked into to split the incremental from the release repo. There is also work to determine if recent p2 support for Maven could be utilized to replace the creation of additional Maven repositories.