Revision as of 18:35, 21 September 2006

What is there to know about Buckminster?

What is the Buckminster Project about?

Developers are increasingly working with complex component structures comprising code artifacts from a variety of sources implemented in disparate technologies. The Eclipse Plug-In Development Environment (PDE) currently provides an outstanding framework for developing componentized plug-ins and feature-sets. However, Eclipse provides only limited support for materializing a workspace — i.e. so that all component dependencies are fulfilled — a problem which is particularly acute in team-based development. Moreover, the PDE’s component management support is available only to components implemented as Eclipse plug-ins, which limits its applicability to other types of development projects.

The objective of the Buckminster, Component Assembly Project is to leverage and extend the Eclipse platform to bring mixed-component development comparable to Eclipse plug-in development in terms of efficiency and ease of use, as well as to extend the Eclipse component dependency model to allow component materialization based on match rules. Buckminster can accomplish this objective by introducing a project-agnostic way of describing the component structure and dependencies of an arbitrarily complex development project, plus a mechanism for materializing all corresponding source and binary artifacts.

Buckminster is implemented as a set of Eclipse plug-ins that complement existing Eclipse infrastructure. Functionality within the project’s initial functional scope includes the following:

• Complex dependency resolution, providing recursive resolution of dependencies leveraging existing Eclipse "Team Providers," with the addition of new retrievers, for exemplary purposes, covering source and binary artifacts that are not version-controlled in a traditional sense. Resolution will use a variety of versioning schemes and will be based on match rules similar to those found in the Eclipse plug-in framework. This allows comparison of current and prior dependency resolutions to support update impact analyses.
• Uniform component dependency format, using a component-type agnostic mechanism for describing components and their respective targets and dependency requirements. Most Eclipse projects, and many other component types, have some level of dependency information that can be leveraged. Extensions can be added to provide additional strategies for dependency pattern recognition.
• Intelligent retrieval mechanisms, leveraging the Eclipse "Team Project Set" mechanism by separating the bill of material needed for a given configuration from its actual materialization. This is important since dependencies may appoint software that is locally installed on one machine but lacking on another. The bill of materials is typically shared between team members, although the materialization information may vary.
• Flexible project workspace binding, allowing components materialized on disc to be bound to a workspace in different ways, including "Proxy Projects" consisting of links to physical artifacts and auto-generated Eclipse project information. These capabilities are helpful when sharing code or other artifacts that are not eclipse projects.

The Buckminster Project’s initial scope focuses primarily on materialization of components based on dependencies and rules, although this scope may be expanded over time, to the extent appropriate and consistent with its objectives. One of a number of requirements that may be addressed in the future is tighter integration with Eclipse build systems. For example, this would allow fulfillment of pre-built artifacts by actually building source artifacts and then caching the results for group access.

Background

What is Buckminster?

Buckminster is a set of frameworks and tools for automating build, assemble & deploy (BA&D) development processes in large, complex or distributed component-based development projects. Buckminster allows development organizations to define fine-grained "production lines" for the various stages in software production - for example, unit testing, integration testing, field deployment, staged migration, etc. - and to automate the execution of the relevant BA&D processes.

Why does Buckminster exist?

Manual execution of BA&D processes is hugely resource-intensive in large and/or distributed development organizations. The burden expands exponentially with the size and complexity of the development organization and component base and as the frequency of system change increases. In the past, resource costs associated with BA&D processes have acted as a check on the ability of development organizations to respond to changing technical, market or customer requirements. Moreover, BA&D overhead has made true component-based development costly to the point of impracticality, forcing development organizations to collapse componentized code bases back into monolithic structures.

Existing IDEs and source code management (SCM) systems take a fragmented approach to managing BA&D processes in component-based development, leaving ISVs and large development organizations to either smith their own tools or do without.

Buckminster exists to address these problems. Its goal is to achieve manageable reuse of software in distributed development environments where components originate in diverse, independent and frequently changing code streams.

What is the history of the Buckminster Project?

Buckminster results from well over a decade of developing component-based systems in multi-site environments and a deep interest in software configuration management. Initially, we just set out to solve the typical process problems we encountered working with monolithic code bases' patch problems, lengthy builds, version incompatibilities, difficulties tracing bugs and design flaws.

Perhaps because we assigned more senior architectural resources to the problem, or perhaps because we just developed a "passion" for process perfection (or both), we went farther and deeper with these issues than the typical development organization. Software CM products generally did a poor job when it came to managing code bases comprising merging component streams. So we essentially built our own component configuration management (CCM) domain comprising some code and lots of process on top of the leading commercial CM product.

The real breakthrough was when we started a venture in early 2000 to build a platform for next-generation software production that has evolved into The Frameworx Project. We wanted to simplify and rationalize the process of producing and changing applications that were a specialization of a more generic set of functional components. For example, this would allow an application vendor to deeply customize its application for individual customers without having to manage each customer as a separate code base. Similarly, an ISV could swap out components in a SOA in a controlled fashion, without undue disruption to its installed base.

We realized that the investment we had made in automating our own internal development processes - sophisticated BA&D frameworks, software CM and a heavy investment in our own CCM - provided the blueprint of the solution, and was something that resonated with almost everyone that we presented it to. Buckminster is essentially a further generalization and productization of that investment.

What exactly is Component Configuration Management?

Component Configuration Management (CCM) is traditional configuration management interwoven with component-based thinking.

Service-oriented (or component-based) development requires version and configuration control not just of individual files, as in traditional CM, but also of the components - structured groupings of files - comprising the larger product or system. Components undergo repeated transformations or "state changes" throughout the development and maintenance lifecycle. The larger system or product is repeatedly reassembled and deployed as components are transformed. The result is configuration complexity and organizational and operational overhead.

The CCM discipline addresses this complexity by promoting recurrent values of architectural quality, reuse & repeatability, flexibility and scalability.

What is the scope and ambition of the Buckminster Project?

The Buckminster Project's mission is to create open source frameworks and tools that:

• complement the existing IDE and software CM domains
• to support an integrated, comprehensive and low-overhead approach to BA&D aspects of software production
• providing support for large, distributed development organizations producing complex and frequently changing component-based systems.

How does Buckminster relate to The Frameworx Project?

Buckminster has no technical dependencies on other components originating in the Frameworx Project, and can be used to build, assemble and deploy nearly any multi-component system.

However, Buckminster goes hand in hand with the objectives of SOA frameworks such as the Frameworx Project. True SOA and BA&D efficiently are inseparable in practice. Service-oriented architecture implies fine-grained componentization, which is impractical without BA&D efficiency.

Functionality Scope

What functional units does Buckminster comprise?

Buckminster currently comprises two key functional elements which work together to automate complex, distributed component-based BA&D processes; Materialization and Action management.

Together they form an elaborated build tool that implements component CM ideas and processes and uses internal Eclipse build tools and the Ant tool for the build-related tasks. The key concept in Buckminster is an "assembly line" approach to maintaining an action based stepwise build process, with support for persistent and reusable component states after each step. The Buckminster framework is intended to support automation of nearly any conceivable BA&D process, from unit test to staged production migration.

What exactly does Buckminster do?

Following is a summary list of the basic functionality Buckminster can support:

• Concise handling of configuration descriptions — Buckminster has a succinct, extendable and very flexible way to describe a given mix of components. It can describe use of volatile component mixes directly in a file system, as stable/unstable versions of components residing in an SCM tool, or as components available through the web.
• Version control tool integration — Buckminster can help extract components from an SCM tool based on the configuration description.
• Repository integration — In addition to integrating with version control tools, Buckminster also handles a growing and expandable set of other types of repositories such as download sites, Maven repositories, etc. This leverages existing repositories and technologies without having to impose a new structure on used components.
• Unique assembly line paradigm — Buckminster can manage a build/transformation of many components through an "assembly line", or sequence of steps. The process can be stopped after any step and restarted a different time or site, without loss of component state or other context. (One example would be a continuous process that pauses when an installation CD is burned and resumes when field installation begins.) Moreover, tracking information can be recorded to help backtracking in case of later problems.
• Minimized component coupling — Components follow simple rules that control their behavior. They can express dependencies, and be assured that Buckminster will run them separately in the correct order. By adherence to build rules, and support for build instructions by external reference, components remain location-independent. This allows the user/Buckminster to switch states in the background - for example, to present a pre-built state instead of materializing a source state and compiling. This cuts build times, and makes repeatability much more achievable.
• Minimize impedance mismatch between IDE and the rest of the software factory — while the IDE is an excellent tool for the developer, the needs of the rest of the software factory; automated (headless) builds and assembly, autmatic (nightly) testing and other batch oriented tasks often require two separately maintained structures. Buckminster brides the two - the same materialization and build steps can be executed both in the IDE and in headless mode. When using separate environments it is difficult to catch such errors that are normally captured by the Eclipse internal build process when using general purpose batch oriented tools.
• Integration with Ant and Ant plugins — leverages the plethora of plugins for the popular Ant build tool, while providing an additional layer of management and control on top. Buckminster seamlessly integrates Ant tasks and plugins with Eclipse internal build tasks.

What is OUTSIDE the scope of the Buckminster project?

Buckminster tries to be as agnostic as possible to other environments and tools that already provide valuable functionality. Buckminster will not include functionality for:

• repository management (i.e. the storage of components) - Buckminster reads or writes (via actions) in repositories but do not provide a repository.
• development process management - here Buckminster is intended to be used as an integral part, or indeed be used to drive an assembly line as part of an overall (business) software development process
• tracking issues or bugs - however if you need to include release documentation as part of your build process, you can use Buckminster (or Ant) plugins to retrieve information, and the Buckminster action based assembly line to transform it to the shape wanted in the end result.
• versioned storage of artifacts (such as CVS or SVN) - but Buckminster works well with these tools, adding an additional layer of management, control, and ease of use.

What products or technologies are most comparable to Buckminster?

A large number of both commercial and open source build tools are available. Some are narrower in scope and ambition than Buckminster, other are larger, but overlap Buckminster partially.

A newer tool, Maven, raises the abstraction level above that of the typical build tool, and aims to add development process management support throughout the entire project. Recently, the Maven project and Buckminster has started to cooperate as the two technologies are complementary.

However, we believe that Buckminster is more ambitious in architectural and functional scope and extensibility in the area it covers – from tight integration in the Eclipse IDE to management of unattended builds - and will in many cases be more adaptable for larger scale development projects.

Technology

What are the key technologies on which Buckminster depends?

Buckminster is implemented in Java on the Eclipse platform, and thus depends on a Java environment - with JDK 1.5 currently required as a minimum.

Important to note is that Buckminster has NO dependency on any specific implementation technology (i.e. language) for the components on which it operates.

Buckminster embeds two other open source projects: Ant, as one build engine along side Eclipse internal builders; and Log4j, for logging purposes. Depending on Ant tasks used, builds may have other dependencies.

What external components or systems does Buckminster require?

Depending on viewpoint, Buckminster requires a lot...or a little.

First, Buckminster requires some effort from users to become truly useful, although it is possible to start off small & simple. A detailed understanding is required in order to design and produce reusable components and create relevant assembly lines, etc.

Although not a formal dependency, use of some SCM system will be essential to attaining practical benefits from Buckminster in any significant development effort.

What SCM products does Buckminster support?

Buckminster uses a plug-in implementation that handles integration with SCM system. Currently, Buckminster comes with handlers for a plain file system, URL-based repositories, CVS, Subversion, Perforce P4, as well as also handling Eclipse repositories (update sites), and Maven repositories.

Specialized handlers may be needed for development organizations with proprietary wrappers around CM systems that would otherwise be supported, although such handlers generally should be straightforward to write.

Benefits

Who is Buckminster for?

Buckminster was designed for large or complex development projects, measured in terms of number of developers, number of development sites, number of production installations, size of the component base and frequency and degree of component change, as well as frequency of running more complex build and configuration tasks. In theory, anyone could use it, although the benefits to individual developers on quick one-off self contained development projects (where everything can be built by the Eclipse IDE itself) would probably not justify the effort.

Note that our definition of "large or complex" is qualitative, rather than tied to any specific numeric thresholds. The best litmus test for whether Buckminster is a fit is whether the development organization (or indeed the individual developer) believes that BA&D process efficiency is an issue. If it is, then we would hope that Buckminster can help.

Legal and Stuff

How is Buckminster licensed?

Buckminster is available under the Eclipse EPL license.

Why is the project called "Buckminster"?

Like other projects originated by the Frameworx Project, Buckminster is named after a "famous dead architect". In Buckminster's case, this is R. Buckminster Fuller, an architect, engineer and visionary best known for inventing the geodesic dome. In contrast to conventional architects, Fuller was more interested in structural efficiency, mass-producibility and utility than aesthetics. We felt that this best fit the scope and vision of the Buckminster Project.

"When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." R. Buckminster Fuller.

For more on Buckminster Fuller, see http://www.bfi.org.