Ganymede Provisioning Workshop Breakout Results
Day One Results
The provisioning mechanism needs to be able to handle and be aware of non-eclipse pieces (see list below). It needs to be able to reason about those non-eclipse pieces even though it has not installed them (e.g. check their presence, get their path, etc.). It needs to be able to install those non-eclipse pieces, which means that the metadata needs to be able to handle those requirements. In addition to that, the provisioning mechanism should support the execution of non-java code while provisioning (e.g. run an ismp installer)
All operations against eclipse and non-eclipse pieces should be structured to help an installation author to keep things consistent, therefore limiting the cases where the system is put in an inconsistent state.
It is clear that the metadata needs to be extensible to describe how to install/configure/discover non-eclipse pieces. Eclipse pieces should be able to express dependencies onto non-eclipse pieces, and get configuration information about them. In the current equinox provisioning design, installing/configuring another runtime can be done by a touchpoint.
Non-eclipse pieces target a "runtime".
Like OSGi/eclipse is seen as a "Runtime" that needs to be provisioned, the "Runtime" analogy can be carried on to talk about the non-eclipse pieces. For example, an operating system can be seen as a Runtime in which dlls are provisioned. The same goes of WAR files for a web app server, or .net asseblies.
- On some systems installing non-eclipse pieces requires access rights - We need to be able to know whether the system needs to be restarted
Non-eclipse pieces of interest (non-exhaustive)
eclipse launcher, JREs, Dlls, SOs, Registry entries, Script files, Java jars, System env variable, Browser, MSI, ISMP, WARs
Co-existence of multiple provisioners
- multiple sources
- consistent UI / operations story
- multiple users
- Linux experience but Vista issues, too
- metadata compatibility - common basic metadata (install registry) plus source specific properties/extensions
- how to avoid duplicate information/properties?
- interaction of multiple sources (priority, precedence, preference)
- validation of profiles
- licensing issues
- shared runtimes
- shared binaries
- repository discovery (RSS? Web servcies?)
- disconnected cases
- media (including metadata and/or artifacts)
- pending connection - future installation
- on-the-fly creation of native packages (in the background)
- add dependencies on native bits (ex. adding a dependency on GTK for SWT so that if GTK is going to be removed, SWT will either also be removed or at least the potential for the removal will be presented)
- admin interaction
- touchpoints for native installation methods (ex. MSI, RPM)
Spectrum of Governance : Local control with central governance
- initial provisioning
- ongoing provisioning
- points of control
- "I am the governance provider" -- ensure that at the OSGi layer we can control who can activate bundles
- potential for ongoing communication between provisioning server and client
- license revocation or mandatory changes for other reasons
- points of control
- hooks for governance both server and client side
- validation at provisioning server
- based on not only resolver but also other policies including software configuration settings
- validation at provisioning server
- spectrum of governance
- fully managed
- no or almost no client-side ui
- user has no choice over what is installed and can not install additional software
- partially managed
- roles of user can differ (can they choose to install software)
- governance may be required while end user is installing software
- governance may be restricted to a certain set of "critical" components
- potential for some user interface but may be restricted
- no management
- complete provisioning ui (comparable to existing update manager UI)
- fully managed
metadata for user
- end user and admins are users
- what is good for me to consume? Ready for prime time?
- What does this person/role have
- what is this person/roll allowed to consume? (permissions/access controls)
- what can people change in the profiles
- license management ( what licenses have been agreed to, which must be agreed to, license server locations, ...)
- what profiles can I manage
- what does XX depend on?
- consistency checking (can I add XX)
- what do my friends have?
- what function is available
- search for things that fulfill a need
- categorization of function
- What is recommended. Ranking? interests? what did other people like like?
- what repositories are available?
- something like Mugshot Application usage http://mugshot.org/applications ?
- has this lineup been tested?
- are users happy/confident
- known good / known bad
- what NL languages are available
- what are the system requirements?
- can this run on platform X
- cross platform validation/qualification
- machine capabilities (cores, memory, cpu speed, connectivity, ....)
- download size
- install branding
- who (how many people) are using a particular lineup
- what other configurations do I have
- what other configurations are running
- how do I report bugs, related user forums
- settings and preferences
- shared by role, organization, user, templates, ...
- install history
- can I roll back
- does this require a system restart or reboot
- what is needed so that I can run disconnected (whatever)
- what are the external constraints for this system (client/server version, ...)
- where do you enter the system?
- product id? default perspective?
- Enabled Capabilities, Welcome info?
- source used to build this IU
- extensible attributes of the metadata
- are there any post actions required (e.g., populate workspace, ...)
metadata for system
Resource/runtime/service requirements Platform information
state of installable units What is installed?
Qualification What touch points do you need
transports available Internet connection? Speed? Local connection vs end to end What transports available to get this artifact? What repositories contain this metadata/artifact?
What metadata is new since xyzzy?
Who provides beer when the developers make a mistake
What are install operations that will be done?
What installable units will be installed/replaced/uninstalled?
Repository questions - binary deltas, latency, fastest mirrors
Security - is there a proxy
Authorization level Restart required
Will configuration be valid
time/size(download, memory) to perform
system locale languages to install
languages supported what are your dependencies power consumption
system view/all users view/private view
vendor signed hashes/md5 downloaded correctly
Does xyzzy support emacs shortcuts?
Day Two Morning Results
These rough notes were taken during the workshop. The project intersections have since been split into a separate Provisioning Project Intersections page.
(italics means interested party, but not necessarily an active participant)
- Installation manager
- Installation Manager
- Installation Manager
Integration with other mechanisms
- Linux Distros
Cross cutting concerns
- Integration with other mechanisms
- Multiple directors
- Two aspects of metadata: system-level data, that needs to be processed for installing something, and user-level data, necessary for showing meaningful information to the user (e.g. groups of install units, etc.)
- Metadata has to handle both system-level and user-level concerns.
- Metadata has to be extensible.
- Core Metadata: the system-level metadata providing the information necessary for putting the system into a target state
- A grouping mechanism is needed, but cannot be mandatory for installing things. It should avoid duplicating dependency information.
- Allow to express cross-runtime dependencies and requirements (like needing a certain web server, dll, compiler, etc.)
- Cross-runtime dependency: a dependency between two different kinds of runtimes. Example: a bundle in an eclipse-runtime needs a certain web server
- "Installable Units" - something that gets installed, contains dependencies and has a descriptor of what the payload is
- "Grouping" - groups a number of installable units. Groups are not mandatory.
- "Constraint modification" - allows to express limitations or expand limitations. Should allow to declaratively overwrite and change metadata to fix "screw-ups".
- lots of different workflows for products
- we like the Firefox simple and low-key model
- Maya workflow:
- 1. start bootstrap
- 2. pick profile from most popular
- OR customize profile with list of available stuff
- 3. launch profile
- create / modify profiles
- distribute profiles to others
- New user workflow:
- potential for many configuration screens (ex. location, licence, information collection, password)
- progress dialogue during download/installation (same for media-based installation)
- Update workflow:
- ideally there will be no need to "start over" with complete new zip
- do away with infinitely growing footprint
- look into keeping old / keeping n versions
- require API or per-product policy
- being referred to as "garbage colletion policy"
- compared with today, many fewer choices presented to user
- potential preferences:
(on by default ideally)
[ ] Automatically download
(not on by default)
[ ] Automatically install
- Areas of concern:
- updating of non-user-writable bundles
- interaction between Eclipse-based updater and system-wide updater
- policy surrounding updating shared installs
- support some sort of admin proxy (ex. PAM dialogue, admin password prompt)
- profiles shared between machines
- workspace, metadata (describing "profile"), user site (optional)
- take some combination of above to a new machine and replicate system
- perhaps some Import/Export mechanism
- updating of non-user-writable bundles
Day Two Afternoon Results
Engine/Director deep dive
Context on IBM install manager terms:
- A "Job" is either install, update, rollback, fix, uninstall
- A list of jobs is provided to the director
- Multiple jobs can be part of a single interaction with the director
Common work flow:
- Some piece of software has been installed
- User initiates an install/update interaction
- Question to director: what is installed right now?
- Question to discoverer: What other things are available to be installed?
- Question to discoverer: What is this user allowed to install?
- Show user what is installed right now and what can be installed
- User selects pieces of software to install/uninstall/replace
- Question to director: What are the prerequisites for things to be installed? Are the selected actions valid?
- Question to director: What are the resource implications of performing this set of jobs? (disk space usage, etc)
- Question to engine: How much work is it to do these actions?
- Answer comes back from director, and consequences are displayed to user
- User says ok and asks for the work to start
- Question to director: Please install/uninstall/update the following things...
- Question to engine: Please install/uninstall/update the following things...
- Engine goes through install phases:
- Ask touchpoint what needs to be done for each artifact
- Touchpoint may return a request to be performed (e.g., mirror request)
- Download manager is told to perform the mirror request
Common Maya workflow:
- Startup Maya
- Question to director: What profiles are currently installed?
- Question to director: What profiles are available to be installed?
- User selects/creates a profile
- Question to ???: Please launch this profile
- Governance settings may dictate that some things cannot be modified, but other things can be installed/uninstalled at the user's perogative.
- Can we have one director for a subset of the system, and another director for other parts?
Breaking down the Director
As we fleshed out the concrete role of the director, we found it useful to break down the director into three different areas:
- Contains the main API to query/perform provisioning
- Well-behaved director consults Governor for permission to do things
- Director consults some "Discovery" service to find other parts of the provisioning system
- License issues
- User access control
- Security model
- Governor may be local or remote
- Governor is optional - in simple cases you might not have one
- Governor is a policy integration point
- How to find governor
- How to find director
- How to find install registry
- Create provisioning-dev mailing list. Announce availability of email list and wiki pages. -- Jeff due 4/11
- Manage mailing list conversation by announcing specific topics every few days and folding conversation into refactored wiki pages. -- Sharanya, Markus, Elias (ongoing every few days)
- Metadata catalog, or what ever turns out to be appropriate. -- Dave, Filip 4/27
- Update system diagram and develop glossary of terms (add to GPW Lexicon). -- Tim, Jeff 4/20
- Flesh out 'project intersections' Consider annotating each item with full sentences. Consult the community. -- John, Filip 4/20 (gated by system diagram) Update: page created: Provisioning Project Intersections, added description of current Buckminster solution
- Develop an initial list of potential work items based on project intersections identified above and cross-cutting concerns. These will include specific contribution intentions. Prioritize these in some open way. These will be eventually cast a bugs and assigned to people. -- Tim, Jeff 4/27
- Collect requirements for multi-user installs -- James, Andrew 4/20 Update: see Multi-User_Install_Concerns
- Draft more detailed requirements of Governor and Director (decomposition). Possibly include exploratory implementations. -- Jeff, Tim, Pascal, James 4/25
- Encourage cross-committers within the space of provisioning. Count the number of such as a measure of cooperation between projects. Relate this to the new concept of mentors. -- Ward 4/20
- Draft a timeline that at least shows various decision timeframes and commit dates. Index each project's roadmaps as they are developed and adjusted. Indicate the degree of confidence for each item. -- Sharanya, Pascal 4/20
- Develop bugzilla query for all provisioning related bugs for all projects. The goal is to enable tracking of detailed development across projects. Post this query on this wiki page. Update it when provisioning projects change. -- Elias 4/20 Update: see Bugzilla Query (Provisioning)
- Develop technical plan for alignment of Maya and Equinox. -- Dennis, Pascal 4/27