Buckminster Project FAQ

(Converted from the old material at Tigris...)

What is the Buckminster Project about?

Developers are increasingly working with complex component structures comprising code artifacts from a variety of sources implemented in disparate technologies. The Eclipse Plug-In Development Environment (PDE) currently provides an outstanding framework for developing componentized plug-ins and feature-sets. However, Eclipse provides only limited support for materializing a workspace—i.e. so that all component dependencies are fulfilled—a problem which is particularly acute in team-based development. Moreover, the PDE’s component management support is available only to components implemented as Eclipse plug-ins, which limits its applicability to other types of development projects.

The objective of the Buckminster, Component Assembly Project is to leverage and extend the Eclipse platform to bring mixed-component development comparable to Eclipse plug-in development in terms of efficiency and ease of use, as well as to extend the Eclipse component dependency model to allow component materialization based on match rules. Buckminster can accomplish this objective by introducing a project-agnostic way of describing the component structure and dependencies of an arbitrarily complex development project, plus a mechanism for materializing all corresponding source and binary artifacts.

Buckminster is implemented as a set of Eclipse plug-ins that complement existing Eclipse infrastructure. Functionality within the project’s initial functional scope includes the following:

• Complex dependency resolution, providing recursive resolution of dependencies leveraging existing Eclipse "Team Providers," with the addition of new retrievers, for exemplary purposes, covering source and binary artifacts that are not version-controlled in a traditional sense. Resolution will use a variety of versioning schemes and will be based on match rules similar to those found in the Eclipse plug-in framework. This allowes comparison of current and prior dependency resolutions to support update impact analyses.
• Uniform component dependency format, using a component-type agnostic mechanism for describing components and their respective targets and dependency requirements. Most Eclipse projects, and many other component types, have some level of dependency information that can be leveraged. Extensions can be added to provide additional strategies for dependency pattern recognition.
• Intelligent retrieval mechanisms, leveraging the Eclipse "Team Project Set" mechanism by separating the bill of material needed for a given configuration from its actual materialization. This is important since dependencies may appoint software that is locally installed on one machine but lacking on another. The bill of materials is typically shared between team members, although the materialization information may vary.
• Flexible project workspace binding, allowing components materialized on disc to be bound to a workspace in different ways, including "Proxy Projects" consisting of links to physical artifacts and auto-generated Eclipse project information. These capabilities are helpful when sharing code or other artifacts that are not eclipse projects.

The Buckminster Project’s initial scope focuses primarily on materialization of components based on dependencies and rules, although this scope may be expanded over time, to the extent appropriate and consistent with its objectives. One of a number of requirements that may be addressed in the future is tighter integration with Eclipse build systems. For example, this would allow fulfillment of pre-built artifacts by actually building source artifacts and then caching the results for group access.

What is there to know about Buckminster?

Background

What is Buckminster?

Buckminster is a set of frameworks and tools for automating build, assemble & deploy (BA&D) development processes in large, complex or distributed component-based development projects. Buckminster allows development organizations to define fine-grained "production lines" for the various stages in software production - for example, unit testing, integration testing, field deployment, staged migration, etc. - and to automate the execution of the relevant BA&D processes.

Why does Buckminster exist?

Manual execution of BA&D processes is hugely resource-intensive in large and/or distributed development organizations. The burden expands exponentially with the size and complexity of the development organization and component base and as the frequency of system change increases. In the past, resource costs associated with BA&D processes have acted as a check on the ability of development organizations to respond to changing technical, market or customer requirements. Moreover, BA&D overhead has made true component-based development costly to the point of impracticality, forcing development organizations to collapse componentized code bases back into monolithic structures.

Existing IDEs and source code management (SCM) systems take a fragmented approach to managing BA&D processes in component-based development, leaving ISVs and large development organizations to either smith their own tools or do without.

Buckminster exists to address these problems. Its goal is to achieve manageable reuse of software in distributed development environments where components originate in diverse, independent and frequently changing code streams.

What is the history of the Buckminster Project?

Buckminster results from well over a decade of developing component-based systems in multi-site environments and a deep interest in software configuration management. Initially, we just set out to solve the typical process problems we encountered working with monolithic code bases' patch problems, lengthy builds, version incompatibilities, difficulties tracing bugs and design flaws.

Perhaps because we assigned more senior architectural resources to the problem, or perhaps because we just developed a "passion" for process perfection (or both), we went farther and deeper with these issues than the typical development organization. Software CM products generally did a poor job when it came to managing code bases comprising merging component streams. So we essentially built our own component configuration management (CCM) domain' comprising some code and lots of process-on top of the leading commercial CM product.

The real breakthrough was when we started a venture in early 2000 to build a platform for next-generation software production that has evolved into The Frameworx Project. We wanted to simplify and rationalize the process of producing and changing applications that were a specialization of a more generic set of functional components. For example, this would allow an application vendor to deeply customize its application for individual customers without having to manage each customer as a separate code base. Similarly, an ISV could swap out components in a SOA in a controlled fashion, without undue disruption to its installed base.

We realized that the investment we had made in automating our own internal development processes - sophisticated BA&D frameworks, software CM and a heavy investment in our own CCM - provided the blueprint of the solution. Buckminster is essentially a further generalization and productization of that investment.

What exactly is Component Configuration Management?

Component Configuration Management (CCM) is traditional configuration management interwoven with component-based thinking.

Service-oriented (or component-based) development requires version and configuration control not just of individual files, as in traditional CM, but also of the components - structured groupings of files - comprising the larger product or system. Components undergo repeated transformations or "state changes" throughout the development and maintenance lifecycle. The larger system or product is repeatedly reassembled and deployed as components are transformed. The result is configuration complexity and organizational and operational overhead.

The CCM discipline addresses this complexity by promoting recurrent values of architectural quality, reuse & repeatability, flexibility and scalability.

What is the scope and ambition of the Buckminster Project?

The Buckminster Project's mission is to create open source frameworks and tools that:

• complement the existing IDE and software CM domains
• to support an integrated, comprehensive and low-overhead approach to BA&D aspects of software production
• providing support for large, distributed development organizations producing complex and frequently changing component-based systems.

How does Buckminster relate to The Frameworx Project?

Buckminster has no technical dependencies on other components originating in the Frameworx Project, and can be used to build, assemble and deploy nearly any multi-component system.

However, the objective of the Frameworx Project is to develop frameworks and tools for producing service-oriented applications. So true SOA and BA&D efficiently are inseparable in practice. Service-oriented architecture implies fine-grained componentization, which is impractical without BA&D efficiency.

Functionality Scope

What functional units does Buckminster comprise?

Buckminster currently comprises two key functional elements which work together to automate complex, distributed component-based BA&D processes.

The first is the Assembly Manager (AM) framework, an elaborated build tool that implements component CM ideas and processes and uses the Ant tool for the build-related tasks. The key concept in AM is an "assembly line" approach to maintaining a stepwise build process, with support for persistent and reusable component states after each step. The AM framework is intended to support automation of nearly any conceivable BA&D process, from unit test to staged production migration.

The second functional element is Component Configuration Broker (CCB), which is essentially an intelligent repository of versioned component dependencies and compatibilities used to generate valid configurations of components (i.e. those that are known or tested to result in buildable and fully function systems). Configuration Broker manages the complexity that arises when assembling frequently changing, n-dimensional component streams, typically from multiple sources, into one or more production configurations.

What exactly does Buckminster do?

Following is a summary list of the basic functionality Buckminster can support:

Assembly Manager

• Concise handling of configuration descriptions — AM has a succinct, extendable and very flexible way to describe a given mix of components. It can describe use of volatile component mixes directly in a file system, as stable/unstable versions of components residing in an SCM tool, or as highly secure components available through the web.
• Version control tool integration — AM can help extract components from an SCM tool based on the configuration description.
• Unique assembly line paradigm — AM can manage a build/transformation of many components through an "assembly line", or sequence of steps. The process can be stopped after any step and restarted a different time or site, without loss of component state or other context. (One example would be a continuous process that pauses when an installation CD is burned and resumes when field installation begins.) Moreover, tracking information can be recorded to help backtracking in case of later problems.
• Minimized component coupling — Components follow simple rules that control their behavior. They can express dependencies, and be assured that AM will run them separately in the correct order. By adherence to build rules, and support for build instructions by external reference, components remain location-independent. This allows the user/AM to switch states in the background - for example, to present a pre-built state instead of a source state. This cuts build times, and makes repeatability much more achievable.

What is OUTSIDE the scope of the Buckminster project?

Buckminster does not try to be a traditional SCM tool. For all version-handling, Buckminster relies on the user having a regular SCM tool available for that function, as well as an established process to follow.

However, there are a few exceptions:

• AM can help extract components from the CM system based on information in the configuration description.
• Buckminster defines a specialized secure component state format called CSAR that offers functionality for persisting a component state.

What products or technologies are most comparable to Buckminster?

A large number of both commercial and open source build tools are available. Some are narrower in scope and ambition than Buckminster, other are larger, but overlap Buckminster partially.

A newer tool, Maven, raises the abstraction level above that of the typical build tool, and adds development process management support. Recently, the Maven project and Buckminster has started to cooperate as the two are complementary.

However, we believe that Buckminster is more ambitious in architectural and functional scope and extensibility in the area it covers – from tight integration in the Eclipse IDE to management of unattended builds - and will in many cases be more adaptable for larger scale development projects.

Technolocy

What are the key technologies on which Buckminster depends?

Buckminster is implemented in Java on the Eclipse platform, and thus depends on a Java environment - with JDK 1.5 currently required as a minimum.

Important to note is that AM has NO dependency on any specific implementation technology (i.e. language) for the components on which it operates.

Buckminster embeds two other open source projects: Ant, as one build engine along side any Eclipse internal builders; and Log4j, for logging purposes. Depending on Ant tasks used, builds may have other dependencies.

What external components or systems does Buckminster require?

Depending on viewpoint, Buckminster requires a lot...or a little.

First, Buckminster requires some effort from users to become truly useful, although it is possible to start off small & simple. A detailed understanding is required in order to design and produce reusable components and create relevant assembly lines, etc.

Although not a formal dependency, use of some SCM system will be essential to attaining practical benefits from Buckminster in any significant development effort.

What SCM products does Buckminster support?

Buckminster uses a plug-in implementation that handles integration with SCM system. Currently, AM comes with handlers for a plain file system, CVS, Subversion, Perforce, as well as also handling Eclipse repositories (update sites), and Maven repositories.

Specialized handlers may be needed for development organizations with proprietary wrappers around CM systems that would otherwise be supported, although such handlers generally should be straightforward to write.

Benefits

Who is Buckminster for?

Buckminster was designed for large or complex development projects, measured in terms of number of developers, number of development sites, number of production installations, size of the component base and frequency and degree of component change, as well as repetition of more complex build and configuration tasks. In theory, anyone could use it, although the benefits to individual developers on quick one-off self contained development projects would probably not justify the effort.

Note that our definition of "large or complex" is qualitative, rather than tied to any specific numeric thresholds. The best litmus test for whether Buckminster is a fit is whether the development organization believes that BA&D process efficiency is an issue. If it is, then we would hope that Buckminster can help.

How is Buckminster licensed?

Buckminster is available under the Eclipse EPL license.

Why is the project called "Buckminster"?

Like other projects originated by the Frameworx Project, Buckminster is named after a "famous dead architect". In Buckminster's case, this is R. Buckminster Fuller, an architect, engineer and visionary best known for inventing the geodesic dome. In contrast to conventional architects, Fuller was more interested in structural efficiency, mass-producibility and utility than aesthetics. We felt that this best fit the scope and vision of the Buckminster Project.

"When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." R. Buckminster Fuller.

For more on Buckminster Fuller, see http://www.bfi.org.