Buckminster Quick Tech Overview

< To: Buckminster Project
The purpose of this document is to provide a concise overview of the Buckminster framework. It will outline the main concepts, usage patterns and application scenarios. More detailed information is provided in other documents.

What is Buckminster?

Buckminster is a component resolution & materialization framework. Its purpose is to get software components for you and materialize them in a context of choice, typically a workspace or file system. This applies whether you are looking at what's available on your local machine, within your development organization or in the public open source cloud. Buckminster removes ambiguity from component descriptions, enables component sharing and increases productiveness when applied in development, build, assembly and deploy scenarios.

Buckminster is an extensible meta-framework, comprising a component metadata language and a runtime, that can execute either headless or as a plug-in within the Eclipse IDE. Although Buckminster can be used to solve common build, assemble & deploy problems, it is not a build or component management system per se. Buckminster can reuse existing investments in a wide range of build and source management tools - Maven, ANT, CVS, SVN, PDE, etc. - and is designed to place as few restrictions as possible on the services these external tools can provide.

Buckminster understands how other systems express information about components, their dependencies and the actions that transform them over the component lifecycle. Buckminster automatically translates this information in a unified format and uses it to materialize a set of components in a consistent way. If the components and their dependencies are properly mapped out by a supported system, Buckminster reduces the resolution and materialization of a complex assembly to a few mouse clicks.

We will look into the process, the artifacts and the application scenarios in more detail below.

Key Concepts

The key concept in Buckminster is that of a component. Software components can vary along many dimensions - location, version, fastest mirror, version range, packaging type, deployment target, release cycle status, etc. At the most basic level, Buckminster sees a component as just a "name for a thing." A "component" can as easily refer to a physical artifact (a source file, binary package, project, etc.) as a set of actions that produces a physical artifact.

Complexity and variability increase exponentially as components are combined into larger assemblies. Determining which form of which component a given assembly requires is one of the hardest problems in working with components, and one that Buckminster greatly simplifies.

Core to Buckminster is a very flexible approach to describing components and configurations. (We use the terms "configuration" and "assembly" interchangeably.) For many configurations alternative choices can be made for individual items in the assembly. One assembly may specify that it needs version "2.x or later" whereas another may specifically require a single version or even build of the same component.

With Buckminster, producers of a component - which can be a single component or a complex configuration - express those requirements as a set of "component constraints". Buckminster automatically resolves all of those constraints for a whole assembly in its "component resolution" routine. This ensures the consumer can reuse and share the configuration and can materialize it in exactly the form intended.

Buckminster defines a set of artifacts to represent the metadata that is used to express and resolve component dependencies and variability. Next, we will take a brief look at these artifacts and how Buckminster uses them.

Buckminster artifacts and lifecycle

The following artifacts are covered by the Buckminster resolution and materialization process. The Buckminster meta-data language provides an XML vocabulary to describe these artifacts. All of these are XML files and the listed identifiers are typically used as file extensions to instances of these artifacts.

• CQUERY - an XML file which describes a component query.
• RMAP - the resource map tells us where components are and how to search for them.
• CSPEC - a component specification provides meta-data about the components including its dependencies.
• BOM - the bill of materials is like a shopping list of components with detailed descriptions of the component's location.
• MSPEC - a materialization specification refers to a BOM and describes what, where and how components should be materialized.

Buckminster integrates with a wide range of build and component management systems and shields the user from the actual creation of many of those artifacts. The Buckminster Eclipse IDE plugin provides custom editors to assist the user in creating artifacts where required. One of those is the CQUERY.

Buckminster artifacts and basic component resolution and materialization process

The Buckminster component resolution process requires a CQUERY as top-level input. As indicated earlier, the resolution process allows to incorporate a set of complex constraints which can be expressed as part of the CQUERY. In its simplest form a CQUERY points to an RMAP and names the component that should be resolved.

The RMAP provides both information on how to locate and match required components as well as how to get access to them. A flexible regular expression syntax is used to accommodate different naming schemes or capture groups of related resources. The included access details will allow Buckminster to perform a repository login and component retrieval for the various supported systems.

Assuming availability of a CQUERY and an RMAP this is in many cases all that Buckminster requires to start the resolution and materialization process. The Buckminster runtime will start with the top-level component referenced in the CQUERY. If this component is mapped in the RMAP - that is its location and retrieval is ensured - Buckminster will start by retrieving component meta-data from the relevant location. If the component is for example an Eclipse plugin or a Maven component, Buckminster will be able to retrieve sufficient information through plugin.xml, feature.xml or POM files to automatically generate its own component description in the form of CSPECs. Those may be persisted for later consumption. The process is similar for all supported systems.

Dependencies expressed in the CSPECs represent the next level in the resolution process. Buckminster will recursively continue with the resolution until all dependencies are resolved. In order to fully grasp the flexibility of this mechanism it is important to reiterate that a component may be described as a set of actions. Buckminster may for example have to retrieve source files and run a build process on them in order to resolve outstanding dependencies. The notion of actions represents itself another extension point of Buckminster.

Having successfully resolved all dependencies in the component query Buckminster produces a bill of materials which serves as an input to the actual materialization process. The BOM is the complete list of components and actions required to materialize and use the requested component assembly in a consistent way. Optionally, a materialization specification may be provided. The MSPEC references a BOM and allows to specify details of the materialization process such as the location in your filesystem to which certain components should be materialized.

This provides an overview of the complete set of artifacts and their role in the Buckminster resolution and materialization process. Buckminster offers a wide range of advanced options which are however out of scope for this introductory document. Next, we will have a brief look at how and where Buckminster is used.

Application scenarios

The potential set of Buckminster application scenarios is probably as large as the world of components itself. Two scenarios for which Buckminster is commonly used should however be included here.

Workspace sharing & Project build systems

Setting up a workspace with recent versions of a product under development can be a time-consuming experience. It often requires extensive communication between multiple developers to arrive at a common starting point in order to become productive. This may happen again whenever a new member joins the team or even if developers are switching between different development branches. The complexity lies in the dependencies of components as well as issues relating to the setup of workspaces. Buckminster is commonly used to describe dependencies and materialize a development workspace in the Eclipse IDE. Initially, Buckminster can assist in the discovery process when setting up the artifacts (CQUERY and RMAP) for a consistent resolution process. Once this is established teams can share those artifacts. Together with an optional MSPEC, workspaces can be materialized in no time ensuring a common ground.

With regard to complex projects like for example the Eclipse SOA Tools project, Buckminster can not only serve to support the setup of development workspaces but also in the build and deploy process. Buckminster's component model ensures that complex dependencies on external projects are captured correctly. Changes in dependencies will be flagged up by the Buckminster runtime during the resolution process shortening the time spent on the build & deploy process.

Virtual distributions & Component stacks

It is a common scenario that the broad range of available products and components from diverse sources (proprietary and/or open source) is stacked in order to address specific needs. The diverse Eclipse stacks as well as broader assemblies such as ObjectWeb Lomboz can serve as typical examples. With regard to the former the Eclipse Enterprise track/stack may be mentioned: a set of recommended Eclipse plugins which have to satisfy certain dependencies in order to work as a consistent stack. Lomboz on the other hand is an example for a stack which incorporates components/products from a large number of different sources (Eclipse STP, Apache, Java, JBoss, etc.). Again various dependencies have to be observed. Once a stack has been found to be useful and reliable, it is easy to capture it using Buckminster. Sharing a combination of BOM/MSPEC is all that is required to get Buckminster to materialize a stack to a location of choice.

Advanced concepts

The detailed explanation of advanced concepts in Buckminster is out of scope for this document. However, one of the most powerful concepts is that of abstract components and should thus be mentioned briefly. An abstract component describes certain properties or constraints on a component without specifying a concrete instance of such a component. If a component should for example require a SQL database this may be specified as an abstract component. The concrete instance would only have to be discovered at resolution time and pending on the availability of either i.e. MySQL, PostgreSQL or Oracle without having to specify one of those in advance.

Extensibility & Supported Technologies

Again the details of Buckminster extension points is out of scope for this introduction. With regard to extending the coverage of Buckminster the componentType and componentReader extension points are the most important to mention. The latter allows to introduce custom handling of physical or conceptual component types whereas the latter deals with ensuring correct access, retrieval and interpretation of component meta-data. Currently, Buckminster provides Eclipse, CVS, SVN, Maven, P4, URL readers and supports ANT build.

Conclusion

The Buckminster framework formalizes the way components of different types, sources and locations and their interdependencies are described removing the ambiguity and fragmentation commonly found in large component assemblies. As a result, Buckminster helps in exposing required and/or broken dependencies in the build, assembly and deploy process and ensures a consistent and reusable description of component assemblies. The Buckminster specification and runtime implementation serves as a component resolution and materialization framework which may be targeted at populating workspaces as well as delivering large-scale application stacks. Buckminster is extensible and provides a range of advanced concepts that helps to address complex assembly problems allowing to explore the variability of the component space in an efficient manner.