Revision as of 02:18, 16 September 2006

< To: Buckminster Project

Welcome to Buckminster

If you've already read this introduction, you can skip straight to the rest of the Buckminster Project/Documentation.

In Its Simplest Form

In its simplest form, Buckminster solves the following problem:

• Your colleague say "ok, project Overland is ready for you to work on - just check out com.megacorp.overland from CVS and get started.
• You checkout com.megacorp.overland and discover that it has unresolved project dependencies.
• "Oh yeah," he says, "you have to checkout project overwater as well". Of course that doesn't completely solve the problem...
• "Also you have to get the latest undersea.jar from the central FTP server."
• Eventually, after a few rounds of this, you have the whole set of resources and you are ready to go to work.

In its simplest form, Buckminster is a tool for materializing a workspace. As a component developer, your workspace is the set of components you are working on plus all of their prerequisite components. Under Buckminster, components are projects. In pictures, the workspace (the cloud) is initially empty, then you ask Buckminster to materialize the component (the box), after which the workspace contains the component:

http://www.eclipse.org/buckminster/drafts/images/image1.gif
(Bear with us on these pictures - right now they are trivial, but
soon Buckminster's complexity will make the pictures useful.)

Because component A has pre-requisites B & C which have prerequisites themselves, the actual result is something like this:

http://www.eclipse.org/buckminster/drafts/images/image2.gif
Component A is the one you are working on;
components B, C & D are pre-reqs that must be in the workspace.

Where Does Buckminster Get Things?

Once you've grasped that Buckminster materializes components, the next big questions in your mind is probably "what are components?" and "where does Buckminster get the components?". Components are collections of files (a.k.a. resources) and are represented in Eclipse by projects. Components can be retrieved from a variety of different repositories. The most common repositories are a CVS repository or an FTP server, but almost anything (databases, file systems, auto-generator programs, etc) can be used via the org.eclipse.buckminster.core.readerTypes extension point.

http://www.eclipse.org/buckminster/drafts/images/image3.gif
Component A being retrieved from a CVS repository 
and prerequisite component B from an FTP server.

Of course the story is a bit more complicated than the simple picture because the component retrieval takes into account various version naming schemes, but we'll get into that later. For now, this picture is good enough.

How Does Buckminster Find The Repositories?

So the next question you're asking yourself is how does Buckminster know which component is stored in which repository? Knowing what you know about Eclipse, you know the projects are linked to Team (CVS) repositories. However there are actually two concepts: the component and its storage location. Separating these two concepts is like separating interface and implementation in good software design. Buckminster does this separation through a level of indirection known as a resource map. The resource map (rmap) provides location information for families of components. When Buckminster needs to load component A, it looks in the rmap to determine the repository containing A, then goes to that repository and loads A. Then because A requires B and C, Buckminster looks in the rmap for information on B and C, goes to those repositories, loads those components, and so on.

http://www.eclipse.org/buckminster/drafts/images/image4.gif
The resource map maps components to storage locations.

Why Is The RMAP Useful?

This separation of component and storage location is useful because it allows Buckminster to retrieve the same component from potentially many different storage locations. The rmap has a number of areas of flexibility. First, the rmap defines storage locations for families of components rather than single components. Thus one can say "find all the Apache components on the Apache ftp server" without having to specify "find Apache Struts on the Apache ftp server and find Apache Cocoon on the Apache ftp server and find ..." ad nausea.  Second, the rmap defines a search path of storage locations to search for a component. Thus one can say "first look in my local repository, then look in my team's repository, then look in my corporate group repository, then look in the Eclipse public CVS repository".

http://www.eclipse.org/buckminster/drafts/images/image5.gif

One More Useful RMAP Feature

Last, but not least, the rmap allows different search paths for different distributed development sites. Thus the rmap for Team Stockholm would list the same component families as the rmap for Team Winnipeg, but with different local repository servers.  TCP round-trips from Canada to Sweden are slower than TCP round-trips from Canada to Canada, thus the Winnipeg team maintains a replicated CVS repository for certain components.

Buckminster allows both teams to use the same rmap - the exact same file - but to define a different site, and thus a different set of search paths. Obviously the central servers (probably the last entry in each search path) would point to the same central Swedish servers so that if the components were not cached locally, they would be fetched across the Atlantic.

http://www.eclipse.org/buckminster/drafts/images/image6.gif
Components A,B, and C mapped to multiple repositories.

Back to Simplicity

We have described many useful features of the resource map, but to avoid cluttering the picture, let us retreat to the basics: the resource map (rmap) is mapping from components to storage locations. 

http://www.eclipse.org/buckminster/drafts/images/image4.gif

For details of the rmap XML schema, see <a href="rmap.html">the reference manual</a>.

What is a Component?

A component, in the Buckminster parlance, is a collection of files or sometimes one single file. For most Buckminster users, a component is an Eclipse project, but components are a larger concept than that - components can be external to Eclipse, components can have sub-components, etc.

Components Do Not Stand Alone: Dependencies

Components may depend on other components but a fundamental concept in Buckminster is that it's not until you do something with a component that the dependencies arise. Depending on what you do, you might need different things. As an example, for a complilation-action of component A you may need interfaces (or headerfiles of some sort) from component B, but you do not need the rest of B, and later, when running the system you may need B in compiled binary loadable library form (but you are not interested in headers or the source of B). Buckminster allows you to separate out the dependencies for different actions (what is needed for a particular action) from the overall dependencies (every dependency). The action related dependencies are described in an Action Specification (aspec). This will be covered later. The dependencies are described in the Dependency Specification (dspec). The dspec describes all dependencies without respect to actions.

http://www.eclipse.org/buckminster/drafts/images/image8.gif
A component has zero or more dependencies

Obviously Buckminster finds the closure of the dependency graph when it materializes a workspace. The closure will always be in the form of a Directed Acyclic Graph (DAG). In our pictorial example, component A requires components B and C, and component B requires component D, and component C requires component D, so when Buckminster will load all four into the workspace.

http://www.eclipse.org/buckminster/drafts/images/image9.gif

Versioned Dependencies

Software has always been a challenging field because, unlike the continuous (or analog) nature of the real world, software is discrete (or digital). Thus a feature that worked in version 4.5 of the system may no longer work in version 5.0 or even in 4.6 or even in 4.5.1. Thus cspec dependencies include a version matching clause.

There are two pieces of a version match clause:

1. The version string
2. The version type

Normally, the version string would be something simple like "4.5". However, because Buckminster is designed for and by software developers, and because software is not developed in a linear fashion, the version string is more complex (and more useful) than the simple "4.5". Specifically, the version string can denote a range. The characters '[' and ']' are used to denote an inclusive range and the characters '(' and ')' will denote an exclusive range. Here are some examples:

Examples of version ranges.

Buckminster is not limited to the major.minor.micro number notation used in this example. That notation is just one of several possible notations. Each notation corresponds to a version type.  The type interprets the version token(s) of a range and assings a magnitude to the result so that versions can be compared. Buckminster allow new types to be added dynamically.

Mapping Versions to a Revision Control System

The versions used in the dependency specification will often be meaningless to a revision control system. Buckminster will therefore use a Version Converter. It is the responsability of the converter to create something that a revision control system can understand. The converter will also be able to convert in the other direction and it plays a very important role when the best match for a version string is to be found in a revision control system. The version will typically be converted into branches and tags.

http://www.eclipse.org/buckminster/drafts/images/image10.gif
Branches and versions of component C that one might find in a revision control system.

http://www.eclipse.org/buckminster/drafts/images/image11.gif
Component A depends on the specific
version "main/2.0" of component C

TODO: Fix this image. Branch/Tag combinations are not typcial in the dspec.

http://www.eclipse.org/buckminster/drafts/images/image12.gif
The component DAG with explicit branch specifiers on each depedency.

The Simplest Case of Versions

The simplest version numbering (branch specifications) is the linear version numbers: 1.0, 1.1, 1.2, 2.0, etc. Linear version numbers are the base case: a revision control system with just one main branch.

http://www.eclipse.org/buckminster/drafts/images/image13.gif
Component B has one main branch, the <default> branch.

When the component dependencies have no branch specifier, Buckminster uses <default>/LATEST which translates to "the latest version of the default branch". The default branch varies by revision control system: CVS uses "HEAD", SVN uses "trunk", ClearCase uses "main", etc.

http://www.eclipse.org/buckminster/drafts/images/image9.gif
Component A's dependency on component B has no branch 
specifier, thus Buckminster uses B:<default>/LATEST

If the cspec dependency includes a version number, Buckminster uses a version converter to translate the version number into a branch specification. The most common translation is "version" into "<default>/version". For example:

http://www.eclipse.org/buckminster/drafts/images/image14.gif
Component A's dependency on component B is version "2.0"
which gets translated to "<default>/2.0". Similarly, the dependency
on component C is "1.1" which gets translated as "<default>/1.1".
Looking at the branching diagram above, we see that "<default>" is
"main", thus Buckminster will load "main/1.1" to satisfy that dependency.

What If There Isn't a Revision Control System?

What if the component is not stored in a revision control system? For example, what if the component is stored in an FTP directory? The answer is that the providers and component readers defined in the rmap handle reading versioned components from the appropriate repository. If the repository does not provide versions, then the component reader treats the repository as if it has just one version: <default>/LATEST. Requesting any other branch specification will fail.

Or, to put the situation more positively, Buckminster treats a "no revision control system" situation as a simple revision control system that provides only one branch and one version.

The Flexible Case of Versions

MOREMORE other tag naming

MOREMORE resource maps, search-paths, providers.

MOREMORE ref earlier pictures

MOREMORE picture of A, B, C, D with complex labels

MOREMORE org.eclipse.buckminster.core.componentTypes and dynamic cspecs

MOREMORE component queries

MOREMORE action specs