|
|
| (24 intermediate revisions by 3 users not shown) |
| Line 1: |
Line 1: |
| − | This page is to collect general information and references about the WTP build process and procedures. The idea is that any WTP committer can update it, pretty much "on the fly" or "as you go", or "as needed", so that there will often be a bit of a disorganized, stream of consciousness flavor to it. Hopefully, occasionally, some kind-hearted committer will stop and organize all the miscellaneous notes and tips and references that are provided, into fun-to-read treasure that you just can't put down.
| |
| | | | |
| − | Note: since only committers can edit these pages, if anyone from the community has contributions or suggestions for additions, please open a [[https://bugs.eclipse.org/bugs/enter_bug.cgi?product=Web%20Tools;component=releng feature request]] on our releng component.
| |
| − |
| |
| − | Thanks, [[User:David williams|David Williams]] 01:42, 2 February 2006 (EST)
| |
| − |
| |
| − | == Build Schedules ==
| |
| − |
| |
| − | As of February 1, 2006
| |
| − |
| |
| − | Our WTP 1.0.1 contributions are due by EOD on Monday's, we smoke test that build on Tuesday's, and plan to declare it by Wednesday's at noon (eastern time).
| |
| − |
| |
| − | Our WTP 1.5 contributions are due by EOD Wednesday's, we smoke test that build on Thursday's, and plan to declare it by Friday's at noon (eastern time).
| |
| − |
| |
| − |
| |
| − | == Build mechanics ==
| |
| − |
| |
| − | The platform's releng tool should be used to "release" projects to the map files.
| |
| − | Be sure to have the appropriate branch of releng project (which has the map files)
| |
| − | loaded in your workspace.
| |
| − | R1_0_maintenance for 1.0.x builds, R1_5_maintenance for 1.5.x builds, and HEAD for 2.0 builds.
| |
| − |
| |
| − | As projects are versioned, please use the "standard" format, in UTC time, following
| |
| − | vYYYYMMDDHHMM.
| |
| − | This is important as these cvs tags become the qualifier field of the plugin's version.
| |
| − |
| |
| − | Note: do NOT use underscores in the CVS version, as there are some issues with Eclispe tooling when it finds an underscore in a plugin or version qualifier, see [[https://bugs.eclipse.org/bugs/show_bug.cgi?id=89428 89428]].
| |
| − |
| |
| − | The build files are in a CVS project called
| |
| − | /releng.control
| |
| − |
| |
| − | The idea is that anything that is "machine specific" or "cruise control" specific is in releng.control. The CVS version should be in synch with the eclipse.org PPC machine, but other machines can be kept in "parallel" files, e.g. see
| |
| − | cruise-lomboz.properties
| |
| − | and rename them when deployed.
| |
| − |
| |
| − | Please note, one implication is that "releng.builder" isn't downloaded and checked automatically by CruiseControl, in other words,
| |
| − | changes in it won't trigger a build. So tools such as API checkers will have to be downloaded or triggered by some other means.
| |
| − |
| |
| − | The "whatisfixed" tools are off by default because it takes almost as long as the build itself.
| |
| − |
| |
| − |
| |
| − | == How to watch the builds ==
| |
| − |
| |
| − | Do you enjoy watching screens of console logs fly by as much as I? Waiting for that joyous final result?
| |
| − | Well then you too can do that with the following information.
| |
| − |
| |
| − | We are currently doing our builds on <b>build.eclipse.org</b>. Any committer can logon to that server
| |
| − | using their committer ID and password. I use something like
| |
| − |
| |
| − | ssh david_williams@build.eclipse.org
| |
| − |
| |
| − | Once there the main work we do is in <b>/shared/webtools/</b>. If you want to "watch" the console log while the compiiler and ant files are running, you can use
| |
| − |
| |
| − | tail -F /shared/webtools/releng.control/out.txt -n200
| |
| − |
| |
| − | to watch the last part of the main log (during compile, etc.). This is the fastest way to find out if the
| |
| − | build failed. Once it does fail, there's some diagnostic messages that might be informative in the last 20 or 50 lines. (Do be careful, though, the build always says "Build Successful" as its very very last message, and that message just means all the ant scripts were any to completely execute (not that they did so successfully).
| |
| − |
| |
| − | During tests, you can watch test results console at,
| |
| − |
| |
| − | tail -F /shared/webtools/test-wtp-S/results/consolelogs/wtptestlog.txt -n200
| |
| − |
| |
| − |
| |
| − | In fact, I do this so much, I define these aliases in my .bash_proflie
| |
| − |
| |
| − | alias tailout='tail -F /shared/webtools/releng.control/out.txt -n200'
| |
| − | alias tailtest-i='tail -F /shared/webtools/test-wtp-I/results/consolelogs/wtptestlog.txt -n200'
| |
| − | alias tailtest-m='tail -F /shared/webtools/test-wtp-M/results/consolelogs/wtptestlog.txt -n200'
| |
| − | alias tailtest-r='tail -F /shared/webtools/test-wtp-R/results/consolelogs/wtptestlog.txt -n200'
| |
| − | alias tailtest-s='tail -F /shared/webtools/test-wtp-S/results/consolelogs/wtptestlog.txt -n200'
| |
| − |
| |
| − | Here the handy thing to watch for is if a test doesn't seem to make any progress at all, after 10-20 minutes.
| |
| − | That's a sure sign of a hang. And, here's how to get some diagnotic information on Linux.
| |
| − | First, "break out" of the tail program with ctrl-c. Then get a list of processes that are running by using
| |
| − |
| |
| − | ps -ef | grep java
| |
| − |
| |
| − | Usually, torwards the end, you'll see a task that mentions, in the long (wrapped) command line, a recognizable JUnit suite. For that process, looking toward the left of the long (wrapped) line, get its process ID ... usually 4 or 5 digits. Then (almost there!) type "kill -3 <process-id>". I forget exactly what the '-3' means to Unix, but to Java, it means to take a dump of current data and thread states, You'll see a few messages to the console that a java dump wsa greated. That's the file that's handy to attach to a bugzills. (use sftp copy, if you do not have handier tools). That file is usually located in the /eclipse directory in use during the test.
| |
| − |
| |
| − | Feel free to edit this document with further tips (or, improve the directions above is I left out some steps, or, you've noticed things are slightly different than written here.
| |
| − |
| |
| − |
| |
| − | == Use of Archives Site ==
| |
| − |
| |
| − | For reference:
| |
| − |
| |
| − | Out of date milestones, releases, and translations can be found at
| |
| − | http://archive.eclipse.org/webtools/downloads/
| |
| − |
| |
| − | Or, if you are navigating there directly, see
| |
| − | /home/data/httpd/archive.eclipse.org/webtools/downloads/drops
| |
| − |
| |
| − | There are also out of date non-released pre-req drivers at
| |
| − | /home/data/httpd/archive.eclipse.org/webtools/downloads/drivers
| |
| − |
| |
| − | Please note, there is a bug with the download pages on the archive site. The links still point to the "downloads"
| |
| − | area instead of the "archived" area. Manually edit the URL to go to the archive area to download the zip file.
| |
| − |
| |
| − | == Managing builds for multiple WTP build streams ==
| |
| − |
| |
| − | Following the 1.5 release, we started building WTP for 1.0, 1.5 and 2.0 streams concurrently. This required some modififications:
| |
| − |
| |
| − | First, we added a property named "buildBranch" to the ant scripts in releng.wtpbuilder. The value of the buildBranch is provided by the parent script that triggers the build. Automated builds are triggered by cruisecontrol, so we also organized cruise projects by build type (i.e. R, S, I, N) and build branch (e.g. R1.0, R1.5, R2.0).
| |
| − |
| |
| − | Second, the download areas for declared builds, committers and archives have been re-organized. Previosly, all builds were kept under the same folder (e.g. webtools/downloads/drops). Now there is a seperate folder for each build branch under drops. It goes like
| |
| − | drops/
| |
| − | + R1.5
| |
| − | | ...
| |
| − | + R1.0/
| |
| − | | + R-1.0.2...
| |
| − | | + I-I200607081130...
| |
| − | + R1.5
| |
| − | | ...
| |
| − | + R2.0
| |
| − | | ...
| |
| − |
| |
| − | The translations are still left in the same place in the archives area.
| |
| − |
| |
| − | We also updated the declare script in committers area. Previosly the script required only one parameter, the full build stamp (e.g. ./declare.sh I-I200607130237-200607130237 ). Now, we provide two parameters to this build script, the build branch and the build name as usual (e.g. ./declare.sh R2.0 I-I200607130237-200607130237)
| |
| − |
| |
| − |
| |
| − | == A Quick Guide to Installing a WTP Build for Testing or as Development Target ==
| |
| − |
| |
| − | WTP developers install WTP builds that are available at committers or downloads areas frequently for testing or using these as plugin development targets. Since each build has its own set of prerequisites, the installation process involves downloading WTP and its prereqs such as emf, gef, jem and eclipse sdk. To simplify this process I have created a scripts that uses the install scripst from the actual releng builder that was used to create a particular build. To use it create simple eclipse project or just copy these scripts into a folder and run "ant -f install.xml get". The script should do all the work for you.
| |
| − |
| |
| − | To use these scripts you should have apache ant (v1.6 or better), a JDK (v1.4.2 ro higher), and CVS installed on your machine. They should be on your path.
| |
| − |
| |
| − |
| |
| − | First file you should create is the build.properties. You modify this file to change the build you want to install or where you want files to be installed etc. You should review these properties before each install:
| |
| − |
| |
| − |
| |
| − | <pre>
| |
| − | baseos=win32
| |
| − | basews=win32
| |
| − | basearch=x86
| |
| − |
| |
| − | #-------------------------------------------------
| |
| − | # Build Stream are one of R0.7, R1.0, R1.5, R2.0
| |
| − | #-------------------------------------------------
| |
| − | wtp.buildStream=R1.5
| |
| − |
| |
| − | #-------------------------------------------------
| |
| − | # Build area is one downloads or committers
| |
| − | #-------------------------------------------------
| |
| − | wtp.buildArea=committers
| |
| − |
| |
| − | #-------------------------------------------------
| |
| − | # label of the Build to install
| |
| − | # e.g. M-1.5.1-200607141912
| |
| − | #
| |
| − | # cvstag is a derived label that you must provide
| |
| − | # in order to automtaically access the correct
| |
| − | # maps and wtp builder
| |
| − | # it is "v"+"buildType"+"buildTimeStamp"
| |
| − | # e.g. vM200607141912
| |
| − | #-------------------------------------------------
| |
| − | wtp.buildLabel=M-1.5.1-200607141912
| |
| − | wtp.cvsTag=vM200607141912
| |
| − |
| |
| − | #-------------------------------------------------
| |
| − | # This is the home dir for build tasks
| |
| − | # i.e. downloads folder etc.
| |
| − | #-------------------------------------------------
| |
| − | build.home=C\:/dev/build-home
| |
| − |
| |
| − | #-------------------------------------------------
| |
| − | # This is where the deriver will be installed
| |
| − | # i.e. downloads folder etc.
| |
| − | #-------------------------------------------------
| |
| − | targetDir=${build.home}/drivers
| |
| − |
| |
| − | #-------------------------------------------------
| |
| − | # This is the folder under build.home where the
| |
| − | # scripts will look for previosly doenloaded files
| |
| − | # i.e. downloads folder etc.
| |
| − | #-------------------------------------------------
| |
| − | build.local.repository=downloads
| |
| − | </pre>
| |
| − |
| |
| − |
| |
| − | Second file you should create is the ant script for installation, install.xml. You do not need to modify anything in this this file, just change the properties we described before:
| |
| − |
| |
| − |
| |
| − | <pre>
| |
| − | <?xml version="1.0"?>
| |
| − | <!-- ======================================================================
| |
| − | Properties that must be passed to this script can be found in
| |
| − | build.properties
| |
| − |
| |
| − | ====================================================================== -->
| |
| − | <project name="wtp-installer" default="get">
| |
| − |
| |
| − |
| |
| − |
| |
| − | <property file="build.properties" />
| |
| − | <property file="./tmp/releng/maps/dependencies.properties" />
| |
| − | <property name="dependencyTargets" value="./tmp/releng.wtpbuilder/scripts/dependency/build.xml" />
| |
| − | <property name="local.cache.dir" value="${build.home}/${build.local.repository}" />
| |
| − | <property name="base.install.dir" value="${targetDir}/${wtp.buildLabel}" />
| |
| − |
| |
| − | <!-- We form these properties dynamically to download the latest build -->
| |
| − | <property name="wtp.url" value="http://download.eclipse.org/webtools/${wtp.buildArea}/drops/${wtp.buildStream}/${wtp.buildLabel}" />
| |
| − | <property name="wtp.releng.url" value="http://download.eclipse.org/webtools/downloads/drivers" />
| |
| − | <property name="wtp.file" value="wtp-sdk-${wtp.buildLabel}.zip" />
| |
| − | <property name="wtp.name" value="WTP sdk ${wtp.buildLabel}" />
| |
| − | <property name="wtp.description" value="WTP sdk ${wtp.buildLabel}" />
| |
| − |
| |
| − | <target name="get" depends="releng">
| |
| − | <delete dir="${base.install.dir}" failonerror="false" />
| |
| − | <mkdir dir="./tmp" />
| |
| − | <antcall target="releng" />
| |
| − | <antcall target="getAndInstall">
| |
| − | <param name="groupId" value="emf" />
| |
| − | </antcall>
| |
| − | <antcall target="getAndInstall">
| |
| − | <param name="groupId" value="gef" />
| |
| − | </antcall>
| |
| − | <antcall target="getAndInstall">
| |
| − | <param name="groupId" value="jem" />
| |
| − | </antcall>
| |
| − | <antcall target="getAndInstall">
| |
| − | <param name="groupId" value="eclipse" />
| |
| − | </antcall>
| |
| − | <antcall target="getAndInstall">
| |
| − | <param name="groupId" value="wtp" />
| |
| − | </antcall>
| |
| − |
| |
| − | </target>
| |
| − |
| |
| − |
| |
| − | <target name="getAndInstall">
| |
| − | <ant antfile="${dependencyTargets}" target="checkDependency">
| |
| − | <property name="groupId" value="${groupId}" />
| |
| − | </ant>
| |
| − | <ant antfile="${dependencyTargets}" target="installDependency">
| |
| − | <property name="groupId" value="${groupId}" />
| |
| − | <property name="install.destination" value="${base.install.dir}" />
| |
| − | </ant>
| |
| − | </target>
| |
| − |
| |
| − |
| |
| − | <target name="releng">
| |
| − |
| |
| − | <available file="./tmp/releng/maps/dependencies.properties" property="map.exists" />
| |
| − | <available file="./tmp/releng.wtpbuilder/scripts/dependency/build.xml" property="releng.exists" />
| |
| − | <antcall target="checkout-maps" />
| |
| − | <antcall target="getwtpbuilder" />
| |
| − | </target>
| |
| − |
| |
| − | <target name="checkout-maps" unless="map.exists">
| |
| − |
| |
| − | <cvs quiet="true" cvsRoot=":pserver:anonymous@dev.eclipse.org:/cvsroot/webtools" package="releng/maps/build.cfg" dest="./tmp" tag="${wtp.cvsTag}" />
| |
| − | <cvs quiet="true" cvsRoot=":pserver:anonymous@dev.eclipse.org:/cvsroot/webtools" package="releng/maps/dependencies.properties" dest="./tmp" tag="${wtp.cvsTag}" />
| |
| − |
| |
| − | </target>
| |
| − |
| |
| − | <target name="getwtpbuilder" unless="releng.exists">
| |
| − | <property file="./tmp/releng/maps/build.cfg" />
| |
| − | <delete dir="./tmp/releng.wtpbuilder" failonerror="false" />
| |
| − | <echo message="Version tag for releng.wtpbuilder is: ${wtpBuilderVersion}" />
| |
| − | <cvs quiet="true" cvsRoot=":pserver:anonymous@dev.eclipse.org:/cvsroot/webtools" package="releng.wtpbuilder/scripts/dependency" dest="./tmp" tag="${wtpBuilderVersion}" />
| |
| − |
| |
| − | </target>
| |
| − |
| |
| − |
| |
| − | </project>
| |
| − |
| |
| − |
| |
| − | </pre>
| |
| − |
| |
| − | If you are behind a proxy you may need to make a few changes to the install.xml script above. First, you will need to add your proxy settings to each of the cvs tasks. The proxy information is of the form:
| |
| − |
| |
| − | :pserver;proxy=<host>;proxyport=<port>;proxyuser=<user>;proxypassword=<password>:anonymous@dev.eclipse.org:/home/webtools
| |
| − |
| |
| − | So you can adjust the "cvsRoot" attribute of the cvs tags accordingly.
| |
| − |
| |
| − | The other thing you may need to do is add a setproxy task to the get recipe. Let's say that your proxy is called "proxy" and its port is 82, then you could add:
| |
| − |
| |
| − | <pre><setproxy proxyhost="proxy" proxyport="82"/></pre>
| |
| − |
| |
| − | to the "get" target above. See http://ant.apache.org/manual/OptionalTasks/setproxy.html for setproxy options.
| |
| − |
| |
| − | == How to get a JUnit test project into our builds ==
| |
| − |
| |
| − | #NORMAL (nearly) PROJECT CREATION
| |
| − | ##put in cvs
| |
| − | ##:directory should be created first, creating a folder in the 'tests' directory of the component.
| |
| − | ##:then using eclipse wizard to 'share', but don't use default name (that'd create a top level project)
| |
| − | ##:but instead use the 'browse' function to put in the folder created.
| |
| − | ##add to appropriate .map file (using the project name, and full cvs name).
| |
| − | ##:it must be added "by hand" to a map file before it can be released with release tool.
| |
| − | ##add the project to a features xml file.
| |
| − | ##:For test plugins, the test plugin should NOT be packed (there are some ant tasks that use file functions).
| |
| − | ##:And, this means to omit the unpack attribute.
| |
| − | ##send note to webmaster to have project and its location added to the modules file in CVS
| |
| − | #PROJECT test.xml file
| |
| − | ##copy a test.xml file from another project,
| |
| − | ##:and edit the 2 or 3 things that are project specific
| |
| − | ###near the top the plugin name
| |
| − | ###near the bottom the one, main classname that runs all the tests
| |
| − | ###the ant target to run (in the element that contains the tests classname) is either
| |
| − | ####target="core-test" (if there is no UI required), or
| |
| − | ####target="ui-test" (if there is UI required).
| |
| − | ##add the test.xml file (using check box in properties editor)
| |
| − | ##:to the build.properties file so it'll be included in the "bin" build.
| |
| − | #WTP test.xml file
| |
| − | ##there's a test.xml file in wtpBuilder that needs updating. Check two spots.
| |
| − | ###one is that the target needs to be defined in top part of file or else the plugin included in another existing target, if there is a "family" of tests).
| |
| − | ###Then near the bottom the "all" target must be updated to run the test target (unless the new one was just included in an existing one).
| |
| − |
| |
| − | == How to create an official patch ==
| |
| − |
| |
| − | See [[WTP Release 1.5 Patches]].
| |
| − |
| |
| − | == Background and Further Reading References ==
| |
| − |
| |
| − | We in WTP following the basic process and recommendations for versioning as the base Eclipse platform.
| |
| − | [http://www.eclipse.org/equinox/documents/plugin-versioning.html Plugin Versioning]
| |
| − |
| |
| − | Very helpful guide to builds and automatic testing.
| |
| − | [http://www.eclipse.org/articles/Article-PDE-Automation/automation.html Build and Test Automation for plug-ins and features]
| |
| − |
| |
| − | Good step-by-step on how to do updates. [http://www.eclipse.org/articles/Article-Update/keeping-up-to-date.html How To Keep Up To Date]
| |
| − |
| |
| − | We base our builds on the Eclipse platform's "basebuilder".
| |
| − | [http://wiki.eclipse.org/index.php/Platform-releng Platform-releng]
| |
| − |
| |
| − | With our WTP specific needs addressed in the WTP project called releng.wtpbuilder.
| |
| − | While slightly out of date, see the [[https://bugs.eclipse.org/bugs/attachment.cgi?id=26584 attachment]] to [[https://bugs.eclipse.org/bugs/show_bug.cgi?id=108259 bug 108259]]
| |
| − |
| |
| − | Our basic server configuration and cruise control triggers is handled by the WTP project called releng.builder (see especially the tools/cruise directory in that project).
| |
| − |
| |
| − | And ... never forget [http://help.eclipse.org/help31/index.jsp Eclipse Help] ... search for things related to update manager, PDE, features, site.xml, etc.
| |
