Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "Building ECF Release 3.2"

(The mspec Files)
(The cspex Files)
Line 125: Line 125:
  
 
====The cspex Files====
 
====The cspex Files====
 +
"It is possible to extend/decorate an automatically generated CSPEC by placing a file called buckminster.cspex in the component’s root." (Bucky Book, page 63).
 +
 +
If you open our buckminster.cspex file with the Buckminster CSPEX editor, you just see what is essential a color-coded xml editor. There is no graphical assistance.
 +
 +
Our buckminster.cspex file i sjust the default with some specified deendencies.
 +
 
The links below go to discussions about the cspex files. If you look in our repository, you see the following cspex files. Go to the link for information about each file. Both called buckminster.cspex. They are identical.
 
The links below go to discussions about the cspex files. If you look in our repository, you see the following cspex files. Go to the link for information about each file. Both called buckminster.cspex. They are identical.
  

Revision as of 16:42, 20 January 2010

Please use this page for planning and discussing build issues related to the upcoming ECF 3.2 Release.
Back to ECF.

We're targeting February 19, 2010 for the ECF 3.2 release. What does this mean?

It means on the morning (PST which is -8 hours UTC) of Feb 19, we have a built, signed, and tested set of bundles. So actually this means Feb 18 for Europe.

How to follow what's building

Builds (success and failure) are tweeted to ecfbuildmonkey. Failures and failure-fixes are mailed to ecf-build@eclipse.org.

Where does the release build go?

The build is uploaded (by our build script) to dev.eclipse.org. It goes to downloads/rt/ecf/3.2Test/3.6/site.p2. Currently, it really goes to downloads/rt/ecf/3.2Test/3.5/site.p2, but that will be changed shortly. What's key to note here is that the directory is 3.2Test, not 3.2. The Test here does not mean that there is actually any ECF testing performed on dev.eclipse.org. When we're making release builds, we want to first test that the build is made correctly and gets uploaded correctly. Scott looks at the 3.2Test directory; and if he likes what he sees, he moves the files over to 3.2.

Past experience has shown that sometimes the release build just doesn't work. The network may have timed out and only a subset of the files got uploaded. Or the permissions on the dev.eclipse.org directories were not correct. Or the certificate we use to upload can't be accessed. Or a variety of reasons that have nothing to do with the correctness of the ECF software. And we don't want to clutter up the real 3.2 directory (and overwrite the good but old stuff that may be there) until we know that we have a complete build.

When does the release build happen?

It's forced. It does not occur on a schedule. You have to use the Hudson web interface to ecf2.osuosl.org to kick off a build. And you have to have an account there. https://ecf2.osuosl.org/hudson/

What needs to be done

We need to attach tests to the build

We have the tests. This task is basically figuring out how to modify the Hudson/Buckminster configuration to perform the tests. Currently we know how to link Hudson projects to each other. By that I mean when a build is started and completes, another (the linked build, which contains the tests also builds). We need to understand how to make those tests run.

Which builds should we attach tests to?

We have lots of Hudson projects, but they fall into four categories: Continuous, Nightly, Platform, and Release.

  • Continuous builds occur whenever there is a change to our CVS repository. These builds are not uploaded to dev.eclipse.org. We don't make them available to users; they are considered throwaway builds.
  • Nightly (or daily depending on your time zone) occur once a day. The build is uploaded to dev.eclipse.org and available for users who want the very latest.
  • Platform builds occur once a week. They are currently forced, but we need to change this so that the build occurs automatically once a week.
  • Release builds occur when forced.

We could test all or a subset of these builds. Each of these builds may not have the same set of tests.

We need to get the OSU builds up and working

The OSU builds are plugins that have not yet gone through the eclipse IP process. This means that their CVS repositories are on ecf1.osuosl.org and not dev.eclipse.org. I think eventually they will move to dev.eclipse.org, but this transition is not our current concern. The current concern is making a Hudson project for each of these builds. Scott, Markus, and myself are currently figuring out how to do this, but help and advice from the owners of these projects is needed.

List of OSU projects and their owners

This means making a Hudson project for each of the OSU builds. There are several issues with the OSU builds.

One important issue is the fact that these projects require ECF to be present to be built. I think this is all of the OSU projects, although I believe it is only a subset of ECF that these projects need. Is it the same subset for all projects and if not, what are the subsets?

Should we rebuild ECF before building each of these projects?

Then, how do we ensure that the OSU project uses the ECF that we just built? Note that our Hudson/Buckminster configuration uses a headless eclipse that comes with Buckminster. Is there a way of updating this eclipse, that is, of adding plugins to it or updating its version. In fact, there does not seem to be a way of even finding out what eclipse version this is, even by looking in its config.ini. The inability to determine the version of eclipse used by Buckminster is bug: https://bugs.eclipse.org/bugs/show_bug.cgi?id=298241

Should we look into using the delta pack?

There seems to be some issue about running the delta pack in a headless eclipse. Can it be done? http://aniefer.blogspot.com/2009/06/using-deltapack-in-eclipse-35.html Why are we interested in the delta pack? Our Skype plugin requires that we build it with a 32-bit version of java. Our build machine (ecf2.osuosl.org) uses 64-bit java. So that's why we used to build it on ecf1.osuosl.org, which is a 32-bit machine. And we have to build different versions of the Skype plugin for Linux and Windows. There is no Mac version.

Thomas Hallgren does not recommend using the delta pack with Buckminster. See the forum entry, http://www.eclipse.org/forums/index.php?t=msg&goto=505226& Instead he recommends that we

"set up our target platform for the build so that it contains the multi-platform setup that you use. This is normally the case when you use target.os, target.arch, and target.ws set to '*'. You can also achieve this by installing this feature into your TP using a 'Software Site'.

The buckminster launcher doesn't care about the delta-pack at all unless you mix your runtime with your target platform (not recommended). Buckminster comes with two launchers, one shell script for Linux and a bat file for Windows. You normally don't need an executable to launch Buckminster."

Can someone provide more details on how we would do this?

What we need to put a project under Hudson/Buckminster control

Should we change our builds to check out code using extssh instead of pserver?

There's now a time lag between extssh and pserver. Although we can't be 100% sure, we do think we may have seen some build failures that resulted from recent commits not being picked up by the build.

Make the Release_3_2 branch

Currently the release build is made from HEAD. We have to create a branch called Release_3_2. Making the branch doesn't mean that we cannot make changes and fix bugs, but it does mean that we are not adding new features. When should we do this? When the branch is created, we'll change the build scripts to use the branch.

The Buckminster configuration files

The links here go to a discussion of Buckminster configuration files that we currently use. The intent is to understand these files in enough detail so that we can construct similar files for all our ECF/OSU projects. The intent is not to show the very latest version of these files (although they may very well be the latest) but rather to illustrate the concepts governing the file's definition.

The rmap File

For a Buckminster build, the first thing to set up is an rmap. Here is the rmap file that we use with the ECF SDK.

ecf.rmap

The cquery Files

The cquery files work with the rmap file. For your queries to resolve against repositories (source code or binary), you need to construct a Buckminster Resource Map (rmap) (Bucky Book, page 46).

What's inside the cquery file? You can use the Component query editor (part of Eclipse's Buckminster plugin) to view the cquery file. Or you can just read and edit the xml.

  • Identify the rmap if you are using one (we are).
  • Give the component a name (org.eclipse.ecf.sdk) and a type (eclipse.feature).
  • We are not specifying a version designator. I mean we specify ?==Version but then we do not identity Version.
  • Note that target.arch, target.os, and target,.ws have values of *. These values may become important later when we figure out how to use different target platforms.
  • Lots of advisor nodes. These provide advice to the Buckminster resolution process. Order is important for the advisor nodes. An advisor node is identified by a name pattern, which “is a regular expression pattern that is used to select the components that should receive the advice provided in the advisor node. “ (Bucky Book, page 50).

The cquery creates a bill of materials and materializes them. What is materialization? Materializing means creating material (files?) in your workspace, installing material (plugins?) into your Eclipse IDE or target platform, or placing files in your file system.

Consider the two projects: C-HEAD-sdk_feature and C-HEAD-platform_feature. These are continuous projects, meaning they get fired off whenever there is a committed change to our source repository.

Note that the sdk project also has an N version (nightly) and and R version (released). Note that the platform project also has an I version (integration bundles provided to the platform) and is linked to an L version (test bundles linked to the platform builds).

The links below go to discussions about the cquery files. If you look in our repository, you see the following cquery files. Go to the link for information about each file.

releng/features/org.eclipse.ecf.sdk_feature/org.eclipse.ecf.sdk.cquery
releng/features/org.eclipse.ecf.platform-feature/org.eclipse.ecf.platform.cquery
releng/features/org.eclipse.ecf.tests_feature/org.eclipse.ecf.tests.cquery
releng/features/org.eclipse.ecf.tests-platform_feature/org.eclipse.ecf.tests-platform.cquery

The cspex Files

"It is possible to extend/decorate an automatically generated CSPEC by placing a file called buckminster.cspex in the component’s root." (Bucky Book, page 63).

If you open our buckminster.cspex file with the Buckminster CSPEX editor, you just see what is essential a color-coded xml editor. There is no graphical assistance.

Our buckminster.cspex file i sjust the default with some specified deendencies.

The links below go to discussions about the cspex files. If you look in our repository, you see the following cspex files. Go to the link for information about each file. Both called buckminster.cspex. They are identical.

releng/features/org.eclipse.ecf.core-feature/buckminster.cspex
releng/features/org.eclipse.platform-feature/buckminster.cspex

The mspec Files

The links below go to discussions about the mspec files. If you look in our repository, you see the following mspec files. Go to the link for information about each file

releng/features/org.eclipse.ecf.platform-feature/org.eclipse.ecf.platform.mspec
releng/features/org.eclipse.ecf.sdk-feature/org.eclipse.ecf.sdk.mspec
releng/features/org.eclipse.ecf.tests-feature/org.eclipse.ecf.tests.mspec
releng/features/org.eclipse.ecf.tests-platform-feature/org.eclipse.ecf.tests-platform.mspec

Back to the top