GPW Lightning Retrospectives
This page collects notes for Lightning Retrospectives prepared for the Ganymede Provisioning Workshop. You won't need to make slides to present at the workshop. We'll provide access to these pages and a big font. Use the following template and digress as you see fit.
(your project name here)
- what code do you have
- how good is it from a reusable framework perspective
- what experience do you have
- what can we learn from your experience that should influence our discussion at the workshop
- what do we want
- freely speculate about what you want from us or we should want from you in the Ganymede timeframe
See also: Lightning Retrospective Guide were similar notes were collected for the build workshop.
Equinox Provisioning Incubator
The evolution of Eclipse running in a wide range of runtime scenarios (e.g., tooling, RCP applications, cell phones and devices, and server) coupled with the increasingly dynamic nature of these computing environments is driving new requirements in the area of provisioning. Eclipse Update Manager has served well in relatively static, large-block, non-OSGi, tooling world but these additional requirements are exceeding the flexibility designed into Update Manager.
The main goals of the incubator are to create an "extensible provisioning platform" that provides the base infrastructure and building blocks for provisioning systems that cover a wide range of usecases. Provisioning solutions lie along several intersecting axes.
- fully client managed to fully server managed
- coarse-grained "product" delivery to fine-grained "function" delivery
- static component/version lineups to dynamically computed lineups
- component-side component isolation to component pooling
The Equinox provisioning platform seeks to facilitate (though not necessarily directly provide) provisioning solutions at multiple points along each axis by separating the base provisioning mechanisms from the policies that drive those mechanisms.
The initial scope of this work is to create the base platform and enough concrete implementations to viably replace the current Eclipse Update Manager.
Maya Incubator Project
Maya is a new project to Eclipse being contributed from Cisco. While new to Eclipse, Maya is a technology developed within Cisco based on past provisioning experiences including working with technology used to manage thousands of Eclipse desktops. Maya is our ground-up rewrite to do managed provisioning the "right" way (disclaimer: Maya is our approximation of "right"; we look forward to further refining and evolving our understanding here at the workshop).
The Maya technology is being contributed to Eclipse as part of the new Maya project.
- Maya is currently being used in a limited rollout within Cisco.
- Maya was designed from day one with the intention of releasing to open source.
- Maya architectural design is robust; certain individual components still maturing.
- The technology is developed following Eclipse conventions for modularity and extensibility.
In approaching Maya, our experiences were based on past experiences using both Update as well as another internally developed provisioning platform used for a certain Eclipse-based tool stack.
- End-user empowerment is key -- but not the base requirement. Many users of Eclipse tooling couldn't care about how the software gets provisioned, instead, they are interested in accessing the software they need.
- Many users within an organization need to share a base stack of Eclipse tooling. In Maya we abstracted this notion into Profiles which can be shared by users. Users can then extend (add) additional software onto a Profile.
- Discovery of software sources in an enterprise has been problematic; in Maya, we took the burden central for managing the list of update sites to be scanned and made available through Maya.
- Users need to be able have access to multiple Eclipse Profiles ("stacks") at the same time.
- Update technology when looking to address delivery of complex RCP tools needs to be significantly easier than what we would expect a developer to go through.
- Provide a base framework that allows different vendors to offer value-add services to be deployed both server and client side (such as licensing, tool stack validation, etc.).
From a selfish point of view, we want to be able to pick up software from vendors and quickly deploy on our network without requiring changing out our provisioning or distribution infrastructure, or requiring users to manually install software on our system.
As a technology, Maya addresses some standard problems for Eclipse provisioning (in no particular order).
- Single entry-point into multiple Eclipse profiles including software updates.
- Automatic selection and failure/retry between download mirrors.
- Ability to override mirror lists for Enterprise-internal download sites.
- Plug-in and feature caching to be shared over multiple Eclipse instances.
- Automation of dependency resolution when defining software for an Eclipse instance (Profile).
- No need for end users to find or specify update site URLs as handled by administrator.
We are looking at the following for Ganymede:
- An extensible model for provisioning in Eclipse whether single end-user deployments or centralized managed deployments.
- A much simplified model for distribution better suited to do RCP applications in addition to the mode developer centric provisioning model of today.
- A common paradigm for products to be defined and deployed without requiring customized installers or custom client-side technology for each system.
In addition to the notes above, we recommend reading over the following links and potentially browse the Maya newsgroup for some additional architecture discussions.
IBM Installation Manager
In Q4 2006 IBM released the IBM Rational Software Delivery Platform 7.0 family of Eclipse-based products employing a new internally developed installation technology, the IBM Installation Manager. Some of the high level goals of the Installation Manager were
- A simplified, unified user experience for software provisioning
- Support for delivery of componentized software packages into shared environments
- Support for enterprise management and governance of the distribution of developer tools
The Installation Manager (IM) is an RCP application which is installed via bootstrap on client machines.
Architectural principles in the IM include the separation of decisions about what is be provisioned (Director function) from the steps taken to do the provisioning (Engine function) and the separation of the description of provisionable components (metadata) from the data to be provisioned (artifacts). We have implementations of metadata, transports, metadata and artifact repositories, extensibility paradigms, adapters for eclipse and native provisioning, and other parts of the IM which may be appropriate for contribution (either as code or concept) to Eclipse provisioning (particularly to the Equinox provisioning incubator).
Our unique experience comes from the larger end of the provisioning scale both in size and diversity of software packages (~7500 bundles, fragments, and features; lots of native artifacts) and in number of deployed managed clients (many 1000's). Some random areas related to our experience are build integration; consistency of install, update, rollback, fix, and uninstall scenarios; support of constrained, tested lineups; shell-sharing; qualification of install configuration (install into existing eclipse); silent install; media repositories; licensing and fulfillment system integration; and compatibility of provisioning with open source, partner, and third party components.
We want the provisioning in the Ganymede timeframe to provide an extensible, functional framework on which open source and commercial specialized provisioning systems can be built that will work reliably and consistently in a heterogeneous enterprise environment.
Smartup software update
Smartup is the provisioning solution for our non Eclipse based Smart Client technology. These technologies are the basis for large client/server sales representatives systems. Our largest system is within the financial/assurances area and has about 30.000 clients.
- the end user is (and shall) in no way concerned by updating the software. This includes keeping track of installed versions and the versions to upgrade to as well as automatic detection of the communication properties such as internet settings, HTTP proxies, LAN, mobile,..
- ensure that the end user is always working with the latest version of the client software. Update cycles are very frequent (at least one per month).
- for identifying software ´artifacts´ (jars, dlls, documents, ..) versioning schemes and/or MD5 hashes are used.
- works well with low bandwidth connections (cell phone connections/ISDN). To minimize transport data the software update is capable of delivering updates as deltas between the outdated and the current version.
- updating of all components (application, software update itself, the os-specific native launcher and JRE)
- the software update server maintains all the applications and their different versions within a repository. Deliverables are packaged and handled similar to servlet war files. On server start and/or when requested by a client, deltas will be created between the current version on the client and the target version on the server.
- with profiles it is possible to divide an application into parts with limited capabilities. This allows e.g. focusing on specific tasks and obviously it minimizes the data needed for updating the software. Each of those parts (they can overlap) is in itself a working client.
We are currently working on extending our existing technology into the Eclipse/RCP ecosystem. Therefore we would like to share our experience and code if possible/feasible. We are also willing to support the eclipse project with man power.
Eclipse Packaging Project (EPP)
The Eclipse Packaging Project is a new project at eclipse.org and will provide its first results with the Europa Release. Based on the assumption that the Eclipse SDK download package is the most popular download at eclipse.org, but that package is not what most users want, the Packaging Project provides ready-to-install packages for different user profiles.
A nice special feature of the Packaging Project is the installer: The project will provide not only zip or tar packages, but also native platform executables which contain an installer bundled with the install package.
Initial packages will be build for
- Java Developer
- Java EE Developer
- C/C++ Developer
- RCP Developer
The Packaging Project can provide package definitions and installers for the Ganymede Release. We have to decide where we can share our experience and which parts are of particular interest for the other projects.