Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Difference between revisions of "Automated ECF Build"
(New page: <P><BR><BR> </P> <H1 CLASS="western">Introduction</H1> <P>Here's Part 3 of my notes on setting up our automated build system. There are a number of questions (open for discussion) interspe...) |
|||
(3 intermediate revisions by one other user not shown) | |||
Line 1: | Line 1: | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | OLD INFORMATION | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | <br> | ||
+ | <br> | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
<P><BR><BR> | <P><BR><BR> | ||
</P> | </P> | ||
Line 60: | Line 99: | ||
indicates when they were made. Also, the feature directories contain | indicates when they were made. Also, the feature directories contain | ||
a Feature Version Suffix (randomly generated) in their names.</P> | a Feature Version Suffix (randomly generated) in their names.</P> | ||
− | + | [[Image:P3DLSDK.png]] | |
− | < | + | <P> |
− | + | [[Image:P3DLFeatures.png]] | |
</P> | </P> | ||
+ | |||
<P>When should these Feature Version Suffixes be used? You can | <P>When should these Feature Version Suffixes be used? You can | ||
determine whether to generate them or not by setting the property | determine whether to generate them or not by setting the property | ||
Line 70: | Line 110: | ||
the update within Eclipse. (Help --> Software Updates.) Note that | the update within Eclipse. (Help --> Software Updates.) Note that | ||
the features have the Feature Version Sufffix.</P> | the features have the Feature Version Sufffix.</P> | ||
− | + | [[Image:P3UPFeatures.png]] | |
+ | <P> | ||
</P> | </P> | ||
<H1 CLASS="western">Files Available for Download</H1> | <H1 CLASS="western">Files Available for Download</H1> | ||
Line 123: | Line 164: | ||
</P> | </P> | ||
<P>Note the directory <CODE CLASS="western">A-20070610-1724</CODE>.</P> | <P>Note the directory <CODE CLASS="western">A-20070610-1724</CODE>.</P> | ||
− | + | [[Image:P3AList.png]] | |
− | + | ||
<P>The zip contains jars that have HEAD in their names. So does jars | <P>The zip contains jars that have HEAD in their names. So does jars | ||
under <CODE CLASS="western">updateSite</CODE>.</P> | under <CODE CLASS="western">updateSite</CODE>.</P> | ||
− | + | [[Image:P3ACore.png]] | |
− | + | ||
<P>The Auto Build should sends no mail when it fails; When it | <P>The Auto Build should sends no mail when it fails; When it | ||
− | succeeds, it should send mail to | + | succeeds, it should send mail to ecf-build@eclipse.org. |
Right now it appears to be sending both success and failure mail. I'm | Right now it appears to be sending both success and failure mail. I'm | ||
− | not sure why. I | + | not sure why. I think CruiseControl wants to send mail to whoever |
modified the CVS repository, and I've aliased contributors to | modified the CVS repository, and I've aliased contributors to | ||
<CODE CLASS="western">ecf-build</CODE>.</P> | <CODE CLASS="western">ecf-build</CODE>.</P> | ||
+ | |||
<H2 CLASS="western">Daily Build</H2> | <H2 CLASS="western">Daily Build</H2> | ||
<P>CruiseControl attempts a Daily build once a day at (for now) 15:00 | <P>CruiseControl attempts a Daily build once a day at (for now) 15:00 | ||
Line 169: | Line 211: | ||
</P> | </P> | ||
<P>Note the directory <CODE CLASS="western">D-20070610-1500</CODE>.</P> | <P>Note the directory <CODE CLASS="western">D-20070610-1500</CODE>.</P> | ||
− | + | [[Image:P3DList.png]] | |
− | + | ||
<P>If the build is successful, the zip files and update site are | <P>If the build is successful, the zip files and update site are | ||
ftp'ed to a download site. Currently,this is just another directory | ftp'ed to a download site. Currently,this is just another directory | ||
Line 228: | Line 270: | ||
<CODE CLASS="western">/opt/apache-ant-1.7.0/lib</CODE>. Get them from | <CODE CLASS="western">/opt/apache-ant-1.7.0/lib</CODE>. Get them from | ||
the Apache Jakarta project.</P> | the Apache Jakarta project.</P> | ||
− | + | [[Image:P3UpdateZip.png]] | |
− | </P> | + | <P></P> |
− | + | [[Image:P3UpdateFTP.png]] | |
− | + | ||
<P>The resulting files in <CODE CLASS="western">/opt/ecf_deploy</CODE> | <P>The resulting files in <CODE CLASS="western">/opt/ecf_deploy</CODE> | ||
do not show HEAD but rather the datetime. | do not show HEAD but rather the datetime. | ||
</P> | </P> | ||
− | + | [[Image:P3ECFdeploy.png]] | |
− | + | ||
<P>The <CODE CLASS="western">updateSite</CODE> is there, but should | <P>The <CODE CLASS="western">updateSite</CODE> is there, but should | ||
be named <CODE CLASS="western">update</CODE>, not <CODE CLASS="western">updateSite</CODE>. | be named <CODE CLASS="western">update</CODE>, not <CODE CLASS="western">updateSite</CODE>. | ||
Here is a screenshot of files on <CODE CLASS="western">dev.eclipse.org</CODE>.</P> | Here is a screenshot of files on <CODE CLASS="western">dev.eclipse.org</CODE>.</P> | ||
− | + | [[Image:P3DevEclipse.png]] | |
− | + | ||
<P>The Daily Build sends both success and failure mail to | <P>The Daily Build sends both success and failure mail to | ||
ecf-build@eclipse.org. It | ecf-build@eclipse.org. It |
Revision as of 10:53, 4 December 2012
OLD INFORMATION
Contents
Introduction
Here's Part 3 of my notes on setting up our automated build system. There are a number of questions (open for discussion) interspersed throughout this document.
The ECF automated build system automates two builds. We called one Auto Build and the other Daily Build. Current docs spec out another automated build called Nightly which is similar but not identical to the Daily Build I'm describing. We'll merge the operation of Daily and Nightly into one type of build and whether we call it Daily or Nightly at this point does not matter.
This system will be deployed on ecf2
.
The user interface will do more than automated builds. It will also
manage user-initiated builds. These user-initiated builds may include
Integration, Stable, Release, and Maintenance builds.
User-Initiated Builds
The downloads page on http://www.eclipse.org/ecf defines these user-initiated (non-automated) build types in detail. Here's a summary.
Integration Builds are builds that have been judged stable enough to be used for integration with other eclipse projects.
Stable Builds are like Integration Builds but they have been judged to be stable enough for most people to use ,where most people means a greater user set than those integrating with other Eclipse projects. Do we still need this build type? We should consider merging Integration and Stable builds into the same build.
Release Builds are major releases, like R1.0.0. ECF currently now has a Release Candidate 2 (RC2), but not yet an official release.
Maintenance Builds are fixes to a Release Build. They are cumulative as bugs are fixed. They are merged into the next release.
A release is tagged in the CVS repository as a release. What is the exact string used?
CVS Tags
How do these user-initiated builds relate to CVS tags? Here's a screenshot of current CVS tags for ECF. Note that the topmost tag is v20070606-1500, meaning that the files associated with that tag define a build made on 6/6/2007 at 15:00 hours.
This latest tag is older than the HEAD, which defines the very latest ECF files available. The HEAD should always successfully build. This latest tag corresponds to the files presented for download. The CVS tag is embedded in the name of the zip. The text on the download page identifies this download as RC2.
If you download one of those files
(org.eclipse.ecf.sdk-20070606-1500.zip
)
and unzip it you see the following. Note that the jars have a version
string (for example v20070606-1500
) that
indicates when they were made. Also, the feature directories contain
a Feature Version Suffix (randomly generated) in their names.
When should these Feature Version Suffixes be used? You can
determine whether to generate them or not by setting the property
genFVSuffix
to true or false.
The zips provided on the download page are also available through the update within Eclipse. (Help --> Software Updates.) Note that the features have the Feature Version Sufffix.
Files Available for Download
It appears that what we offer for download is the top of the CVS tag list. Should the ECF download page also offer Integration/Stable Builds, Maintenance Builds, and Daily Builds? And if so, how should they be designated?
The Two Automated Builds
These are the Auto Build and the Daily Build. Both are controlled by CruiseControl. Each is a separate project.
Other than scheduling, the differences are that the Daily Build after a successful build ftp's the built files and that it uses different criteria for sending notification email.
I don't know whether either of these builds should set the Feature Version Suffix for the feature directories.
Also, I noted that the Nightly Build does not include the Update Site. The Update Site (Help --> Software Updates) is what is used to update from within Eclipse. It makes more sense for this to be the latest release rather than the Daily Build. However, having two Update Sites is a possibility.
Auto Build
CruiseControl attempts an Auto Build every 30 minutes with a
quietperiod
of 5 minutes. I gave it a
buildType
of A for Auto. The only place
this buildType
is used is to name the
directory that contains the built files. I also chose to set
genFVSuffix
.
<modificationset quietperiod="300">
<cvs localworkingcopy="/home/ted/workanon/org.eclipse.ecf/plugins" />
</modificationset>
<schedule interval="1800">
<ant buildfile="cc-build.xml" target="ecf.build" >
<property name="buildType" value="A" />
<property name="genFVSuffix" value="true" />
</ant>
</schedule>
Note the directory A-20070610-1724
.
The zip contains jars that have HEAD in their names. So does jars
under updateSite
.
The Auto Build should sends no mail when it fails; When it
succeeds, it should send mail to ecf-build@eclipse.org.
Right now it appears to be sending both success and failure mail. I'm
not sure why. I think CruiseControl wants to send mail to whoever
modified the CVS repository, and I've aliased contributors to
ecf-build
.
Daily Build
CruiseControl attempts a Daily build once a day at (for now) 15:00
hours. I gave it a buildType
of D for
Daily. I chose for the sake of variety not to include a Feature
Version Suffix.
<modificationset quietperiod="300">
<cvs localworkingcopy="/home/ted/workanon/org.eclipse.ecf/plugins" />
</modificationset>
<schedule>
<ant buildfile="cc-build.xml"
time="1500"
target="ecf.copy" >
<property name="buildType" value="D" />
<property name="genFVSuffix" value="false" />
</ant>
</schedule>
Note the directory D-20070610-1500
.
If the build is successful, the zip files and update site are
ftp'ed to a download site. Currently,this is just another directory
on halibut
called /opt/ecf_deploy
.
Note that the <schedule/>
for
the Daily Build calls the ant target ecf.copy
which does the ftp and depends on the target ecf
.
The target ecf is the same one called by the Auto Build, except is is
given a different buildType
(D) in the
config.xml
file. The target ecf
checks for this buildType
and sets
forceContextQualifier
accordingly.
<condition property="forceContextQualifier" value="${timestamp}">
<equals arg1="${buildType}" arg2="D" />
</condition>
<java classname="org.eclipse.core.launcher.Main" classpath="${eclipse.home}/startup.jar"
In config.xml
, here is how the ftp is
done.
<onsuccess>
<antpublisher
antscript="/opt/apache-ant-1.7.0/bin/ant"
antWorkingDir="/opt/build.ecf"
buildfile="uploadZIP.xml"
target="deploy" />
<antpublisher
antscript="/opt/apache-ant-1.7.0/bin/ant"
antWorkingDir="/opt/build.ecf"
buildfile="uploadUPDATE.xml"
target="deploy" />
</onsuccess>
The ftp is just calling ant twice with to different ant files that
do the ftp. <ftp/> is an optional ant task, and for it to work,
you have to download and install two libraries: commons-net-1.4.1.jar
and jakarta-oro-2.0.8.jar
. Put them in
/opt/apache-ant-1.7.0/lib
. Get them from
the Apache Jakarta project.
The resulting files in /opt/ecf_deploy
do not show HEAD but rather the datetime.
The updateSite
is there, but should
be named update
, not updateSite
.
Here is a screenshot of files on dev.eclipse.org
.
The Daily Build sends both success and failure mail to
ecf-build@eclipse.org. It
also sends mail to ecf-dev@eclipse.org.
During this test period, I'm only sending mail to ecf-build
.
The User Interface
Remember we have two builds going on: Auto Build (the project is
called called ecf
) and the Daily Build
(the project is called called ecfDaily
).
Here's how it looks using CruiseControl's web reporting tool. I've
used this tool to force either of the builds.
To use the web reporting tool, you have to create a war
file and install it in the webapps
folder for TomCat and run TomCat. If you click on the project name
you go into a tabbed window that also displays the JMX console. To
run the JMX console, you must have started CruiseControl with -port
8000.
I've also used the CruiseControl GUI. To use this GUI you must
start CruiseControl with -rmiport 1099
.
/opt/cruisecontrol-2.6.2/main/bin/cruisecontrol.sh -configfile config.xml -port 8000 -rmiport 1099
Then, download cruisecontrol-gui-2.jnlp
and open it up with FireFox. Configure a server to monitor.
You can force a build by selecting a project, dropping down the Projects list and choosing ForceBuild.