Architecture Council/Minutes May 15 2008

From Eclipsepedia

Jump to: navigation, search
Meeting Title: Architecture Council meeting on Architectural Foundations for E4
Date & Time: Thursday May 15, 2008 at 1500 UTC / 0800 SFO / 1100 Ottawa / 1600 London / 1700 Berlin
Dial-in: (+1) 613.287.8000 (Ottawa and international) or
866.362.7064 (toll-free North America)
passcode 874551#



  • Invited attendees are all EAC members as well as interested architects who know Eclipse well and want to shape its future foundations
  • Signed-up:
    • Bjorn Freeman-Benson
    • Martin Oberhuber (notestaker)
    • Boris Bokowski
    • Kevin McGuire
    • Dave Orme
    • Michael Scharf
    • Mike Wilson (McQ)
    • Paul Webster
    • Paul Coltron
    • Ed Merks
    • Darin Wright
    • Anand
    • Mik Kersten
  • There were certainly more attendees, but I didn't catch all the names. Add your names on the Wiki if you'd like to get mentioned.

PIN Holders for EAC Calls

  • Currently: Bjorn, Doug Schaefer, Doug Gaff, Martin Oberhuber
  • New Volunteers:
    • Ed Merks, Boris Bokowski, Kevin McGuire

Meeting Goals

  • The primary goal of this meeting is to collect topics for the upcoming E4/Summit, and gather initial opinions.
    • Raise tough questions to be discussed at the Summit

Discussion Notes

  • Attendees are welcome to edit the agenda on the Wiki. Place a +1 / -1 next to an item that you would like / would not like to discuss.
  • While the meeting is ongoing and one hour after, the Wiki should not be edited except by a notes-taker to be designated during the meeting
  • After initial notes have been posted, everybody is free to edit the Wiki for corrections

Eclipse Application Model

  • +1 MartinO, BorisB, KevinM, MichaelS
  • See E4/Eclipse Application Model
  • Seeding Questions:
    • Why are Workbench / PlatformUI / Platform not part of the initial Application Model?
      • Boris: Because they are not services in the sense that there could be alternative implementations
    • What about SWT, Views, Widgets, Trees, Labels, ... JFace Dialogs, Fonts, Text, Editors... Perspectives... Databinding
      • Boris: SWT is a library, not a service that could have a different implementation. The same is true (mostly) for JFace and data binding.
    • Capabilities
    • Content Types, Associations
    • Markers
    • Workspace, Resources (related to Team Support)
    • Launching Framework
    • Core Variables / Launch String Substitution
    • What about EFS (core.filesystem)
    • What about Secure Preferences
    • What about the rest of OSGi (NLS, Services, Bundle States / Activator / Classloader; Log; State Location)
    • What about standard bundle layout (related to Provisioning / Update / Plugins, Products, Features)
  • Boris: Goals for this topic:
    • Compile a central list of the Services
    • Currently, several of the Services are Singletons - make it tough to write plugins, especially in other languages; find ideas for how to refactor them
  • McQ: Important to have a strong way of describing the Services - as a medium for newcomers to understand Eclipse
  • Mik Kersten: Need a Service to add focus ("Focus Service") - currently, works only with Reflection
    • McQ: if Mik not attending the Summit, Mik will need to file a Position Paper somewhere on the Wiki
  • Mik: Notification Service - made one in Mylyn
  • Jeff: Every Service should have a Wiki page describing it -- use cases, design etc;
    • Martin: Should we invent a new Documentation Tool?
    • Jeff: Wiki allows for rapid evolution, need to put it on the Critical Path
    • Ed: Don't confuse the engineering tool (while services are evolving) with the final end-user documentation (which might warrant a new kind of tooling)

Too Many Listeners

  • +1: MartinO, BorisB, KevinM, DaveO
  • Boris: Two separate Problems:
    • (1) Many separate Listener interfaces -- lots of different event classes, add/removeListener methods etc; this is one of the causes for bloat, try to cut down the number of Java classes
    • (2) Lots of things going on at the instance level -- cut down the number of instances listening to things (Performance, simplification)
  • Martin: EMF could be a solution for generic listeners
  • Boris: DataBinding "observable Maps" could be another candidate; problem with EMF is that it's tightly coupled to everything being an EObject
  • Ed: Notifiers just depend on Notifiable, not EObject; one can adapt to it
  • David Orme: sit together at the Summit to merge concepts - champions DavidO, EdM, BorisB
  • Probably related to Macro Recording, a general Command Bus / Event Bus would simplify things

Becoming More Asynchronous

  • +1: MartinO, BorisB, MichaelS
  • Seeding questions:
    • What parts of Eclipse are Thread-safe? Is thread-safety documented everywhere?
    • Related to Jobs and Scheduling Rules issues, see below
    • Modal Context Dialog; SWT Dispatch Thread; WorkspaceRunnables; Executor pattern
    • Should there be other patterns for Concurrency than we know now (Jobs, Runnables, ISchedulingRule)
      • Boris: Have a look at data binding's Realm class - it's a generalization of the 'UI thread' pattern: to have access to a resource belonging to a realm, you have to call it from within the realm, and the only way to get into the realm is by posting a runnable using Realm.asyncExec. This makes access to resources safe while avoiding deadlocks altogether, at the cost of more complex code if your operation needs to access resources from more than one realm.
  • Martin: Need more documentation: What is thread-safe; what needs a particular lock / SchedulingRule;
  • Boris: Two aspects here:
    • (a) fine-grained Java Threading (talking about locking, thread safety etc)
    • (b) coarse-grained: don't do so much on the UI thread -- post Runnables asynchronously, or use Jobs
    • Don't spin the Event Loop deep down in the Stack
  • McQ: An important outcome of this has to be a consistent overall picture; need to get above the level where we deal with these things one by one;
    • Mik: Agree.
  • Martin: Have 3 concepts for concurrency: Threads, Runnables, Jobs. I personally like Runnables, should we move more to this pattern? Are these concepts sufficient?
  • McQ: Imagine multiple consumers talking to a particular Eclipse on a Server. In this scenario, everything needs to be very clear about when it can be called and under what circumstances -- need to be completely independent of the Event Loop
  • Martin: One question is Callbacks; how to request additional information from the client
  • Jeff: In Distributed Computing (Riena), it's not just Concurrency but also high Latency; suggest Futures as a potential solution
    • McQ: One caveat - is this making things too complex?
    • Jeff: Concurrency is never simple.
    • Ed: People often ask "Is this xxx thread-safe?" -- even Vector is not really thread-safe depending on its use (get an index and delete the Object later on). A consistent story simplifies things.
  • Martin: Simplest way of Concurrency seems Runanbles / Executor / Realms. Single un-separable Unit of Operation to be scheduled in a Realm, very simple to understand. Is it hard when multiple Realms are accessed?
    • Boris: Synchronization between 2 realms can only be one-way (risk of race conditions). It's not very fine-grained.
  • Kevin: We have too many different Runnables.
    • Martin: Don't we just have so many Runnables because we declare a separate Class for separate Realms (WorkspaceRunnable, ...)? Could this be simplified with a general concept of a single type of Runnable, running in different Realms?
    • Boris: Not sure. Now we have SafeRunnable which does a different kind of job.
  • We are currently not as concurrent as we could be; programming model is difficult because we have many different ways of doing things.
  • Martin: So, a simpler Application Model would also help getting more asynchronous.

Sessions, Role-based Access and Security

  • +1: MartinO, KevinM, DaveO
  • Seeding Questions
    • Role-based Access: Hide unavailable functionality for a user who has access to certain roles only, rather than just denying access
    • Capabilities are a relatively coarse-granular means of access control, need something more fine granular
    • Related to Preference Levelling, see below
    • In RCP, this needs to integrate with firm-wide single sign-on services (SSO)
  • Ed: Is Security the same thing as Java Security?
    • Martin: In this context, yes -- talking about Roles, and Privileges of seeing menus, accessing Services or executing code. Not talking here about security holes, authentication, authorization etc
  • Jeff: If we ask people to turn on Java2 Security it's a non-starter, too much performance overhead. Role-based access and ability to open Files are two different things. On application level, role-based things can be dealt with in many different ways.
  • Martin: Really talking about two things here:
    • (a) role-based hiding of UI elements with finer granularity than Capabilities, but not necessarily secure;
    • (b) role-based denial of services in non-UI on a relatively coarse (per Service / per Request) granularity. Not necessarily using Java Security.
  • Boris: Some work went into 3.4: Capabilities were invented just for hiding elements (make the UI simpler). People have been mis-using this for role-based access. In 3.4, Capabilities have been worked on, using Expressions to express when a certain activity is enabled.
  • Jeff: That's great from a UI perspective, but there's lots of stuff to be managed in non-UI
  • We don't even have a "user" in Eclipse right now!
  • Jeff: Another aspect of this is when mixing in multiple sessions: connecting to Eclipse on a Server, it gets very pervasive because now the Programming Model needs to support "a particular view" on the Services that are available
  • Right now, people can easily construct a Command (that's not in the UI) in order to hack around limitations
  • Jeff: would map Roles onto Sessions, since under the covers the same mechanisms need to be in place
  • Martin: should there be a Session on roles / security at the Summit?
  • Talk about it in the context of Sessions. Right now, we just have "the running workbench", ensuring that the bundles one hasn't access to don't load.
  • Jeff: Matt Flaherty will be at the Summit and will have lots of input
  • Martin: OK, will add a Session to the Summit Wiki

Macro Recording and Playback

  • +1: MartinO, BorisB, DaveO
  • Pervasive macro recording, see bug 8519 vs. Editor macro recording vs. Debugger macro recording
  • Seeding Questions:
    • Should everything be a command / handler / workspace operation? Or do we need a different kind of coupling / bus?
    • DaveO: This problem is possibly related to several others:
      • Too many listeners. A richer command bus makes it possible to have everybody listen to the command bus rather than having a hodgepodge of listener types.
      • Non-UI scripting: If everything is a Command, then the set of Commands that mutate state (or some subset of them that can be made API) can become the set of verbs available in the scripting API. This scripting API can be made available trivially in many scripting languages (ie: using methodNotFound in JRuby). What remains is exposing a DOM for the scripting language to interrogate and for the commands to manipulate.
  • Paul Webster: Command stuff as now gets only part of the way - it's currently serializable already, but doesn't get into any kind of state handling. Commands plus the Undo stack gets a lot but not yet macro recording. Depends on how Actions can be executed.
  • Martin: don't think about backward compatibility here. Need to lay the architectural foundation to enable macro recording in the future. Define what's the "right way" of plugins to be written such that they are scriptable.
    • Jeff: Concerned about the Application Model -- if everybody needs to write code in terms of commands it won't work
    • Ed: Yes, seen this in EMF -- foundations for fine-grained commands are there, but cannot force people to do everything in the context of commands. Invented a "Change Recorder" to listen to changes, and still have it undoable
  • Security is another concern here -- a system that records everything going on is a security hole
  • It's a lot of work getting buy-in from all plugins into some Scripting architecture.
    • If this is part of the "too many listeners" solution, it'll be enough incentive for people to move to it
  • Martin: then we need some kind of "command bus" or "event bus".
  • Martin: Application Model needs a common way of exposing state
  • Kevin: Macros help us drive commonality in our architecture, that we should have anyways -- that's good. But hesitant about complicating our programming model by over-architecting stuff for Macro Recording. Don't want to set this as a requirement.
  • Boris: For this to succeed, we need somebody to look at this in concrete terms. How much do I get today if I record everything? What needs to be recorded or what's not as important? Do we need to record resizing views, expanding nodes etc? Another topic is having something like a DOM for the interesting pieces in Eclipse, but that's a separate kind of discussion
  • Martin: Fear that if this is not a core driver, it won't happen.
  • Kevin: If shooting for 100% recordability, the weight of it is likely unbearable for plugins. Have actions / commands
    • Martin: Agree, we should push on a "90%" recording solution where user can edit the recorded script afterwards to fill in the remaining bits
  • Boris: Discussion is too abstract right now -- need somebody who was an expert Emacs user to say how Emacs is better than Eclipse, or an Expert who knows how a different App does macro recording
    • Kevin: Talked to Tim Wagner -- at MS, a lot of work went into Macro recording and it's hardly ever used
    • Boris: On the Mac, scripting is probably also not very much used?
  • Mik: in usage studies at Mylyn, found that the command framework is used by SDK and WTP; made a mapping of all menu Actions that are not a command and map it back to a class; commands are doing really well with the SDK
  • Ed: very interested in the topic because EMF Change Recorder is very similar
  • Mik: Mylyn has an interaction event log -- can actually play forward and back, will add to the Notes Wiki

Dependency Injection

  • +1: BorisB
    • Closely related to Application Model -- If components (view, editor) shouldn't know about the context in which they appear, dependency injection is a technique for this. Instead of PlatformUI.getWorkbench.getHelpSystem(), have the HelpSystem constructor injected.
    • Jeff: One of my favorites, has a fundamental impact on the entire Platform in terms of programming model, ease of testing etc.

Preference Leveling

  • We currently have (installation, instance, project) levels, what about user level / role-level Preferences? What about multiple levels of nested projects (see below)?
    • KevinM: I'd like to see this topic be the wider subject of preference management as they are already almost unmanageable for a user. For leveling we have user level ones stored outside the workspace, team based ones shared somehow, skinning level ones likely in a CSS style sheet. To reduce clutter we need a better policy on essential preferences v.s. "it's prudent that there is a key that can be set" level ones which perhaps should go elsewhere such as advanved name/value table. Finally if we're interested in role based then it needs to be a topic onto itself since it spans not just preferences but activities etc..
  • Kevin: Currently just one Preference Store: Don't do a good job separating out User Preferences (to drag around on a USB key) vs Team Preferences (which Jazz is very interested in); others should go into a Stylesheet
    • Right now, Preferences are almost unusable -- look at this top-down before adding more complexity
  • Ed: Another problem is that if Prefs are exported, there is no categorization at all; Some Prefs should apply to 100 projects, currently these are duplicated; a way of sharing preferences between projects would be a big win
  • Martin: Not on the Summit Agenda yet -- should it be merged with others?
    • Boris: Don't merge yet, better have more smaller topics
    • Tricky since some of these are cross-category...
    • Martin: Will Copy today's discussions into separate Wiki pages linked to the Summit Agenda

Java 5 Usage

  • Ed: using "Java 5 Usage" be a separate Category?
    • Can we ignore it or do we have to think about it?
    • Martin: This is a clearly defined topic, can be discussed at the Summit

Related Topics (Discussed in Other Context)

Jobs and Scheduling Rules

  • +1: MartinO, MichaelS
  • Seeding Questions:
    • Many calls into (workspace, or other contributed) API require downstream locks via ISchedulingRule
    • It's often not documented what rules / locks a call requires
    • If the "master Job" that's responsible for an operation requires an ISchedulingRule itself (e.g. for guaranteed sequential execution), it cannot call API functions that require other ISchedulingRule locks (because it cannot acquire them while it runs -- locks must be known before the Job runs, but it's often not known what locks will be required)
    • In some situations a job want to use rules that are not known before the job starts. It should be (deadlock) safe, if a job can acquire *one* rule at a time.

Getting rid of Singletons

Non-UI Scripting

  • +1 DaveO (see keyboard macros above)
  • Loading a file into an Editor from the Commandline; other macros executed from commandline
  • Martin: Already discussed this as part of #Macro Recording and Playback

Verboten: Spinning the event loop

  • +1: BorisB
  • I regularly see hard-to reproduce bugs that are caused because client code spins the event loop (e.g. by opening a blocking dialog). It would be good to have a set of rules that govern when it is ok to spin the event loop.

Avoid Modal Dialogs

  • +1 MichaelS
  • Too many dialogs are modal, like preferences or the new wizards which often make it impossible to look something up while you enter data into the dialog.

Other Topics (Not Discussed on the Phone)

Workspace, Resources, Content Types

  • +1: MartinO, KevinM, DaveO
  • Seeding Questions
    • Content Types:
      • Pattern Matching for content types rather than just file extensions + stream evaluation
      • Case sensitive patterns for content types
      • Project-specific content types
    • Eclipse Projects
      • Plumbing or user artifact?
      • KevinM: For many RCP apps, the notion of the project at the user level is itself a problem. We need to separate our need for them as a mechanism for hanging off builders etc, vs. when to surface as a first class user notion.
      • DaveO: Agreed. See 'Build Granularity' below for one proposal.
    • KevinM: Project type/nature, adding/removing
    • bug 35973 Physically nested projects
    • Namespace Resolution for projects with same name in a single workspace
    • Inclusion of files from anywhere (liked linked resource), plus exclusion of elements by pattern (exclude from resource tree rather than just hide) Maybe we should stop assuming that the files are in the same directory tree. Most people think of projects as "the set of things I am working on" not as a physical space on the drive. Linked resources should be the design norm. -- KevinM
    • Full native support for symbolic links (better algorithms for detecting link cycles and avoiding OutOfMemoryError)
    • Multiple projects in the same folder?
    • Logical Project Hierarchy Preference Levelling (like Apache Maven) - inherit settings from parent project?
    • KevinM: Plug-in loading and listeners: how to participate dynamically without forcing plug-in loading
  • Martin: Not discussed due to lack of time. Already a separate session at the Summit, will merge points into Summit agenda.

Build Granularity

  • +1: DaveO

Currently, the bits we build break down as:

  • SWT
  • Eclipse Platform Project SDK
  • Eclipse SDK | Other Ganymatic Builds

A lot of RCP applications don't need to be as generic as SWT or RCP. They want a generic container that looks and works like the Workbench. The Platform Project SDK would be ideal for this purpose, but the Platform Project SDK surfaces Build and Debug bits in the UI. I propose the following factoring:

  • SWT
  • Workbench SDK (Does not include Resource Perspective)
  • Resource SDK (Includes Resource Perspective)
  • Eclipse Platform Project SDK (Adds debug and build bits we currently have or maybe this goes away?)
  • Eclipse SDK.....
  • Martin: Not discussed due to lack of time. Will move this into a separate Summit topic.

Launch / Debug Enhancements

  • Seeding Questions:
    • Need a Launch Stepper to execute fine-grained operations, rather than monolithic Launches
    • Need extendable / modifiable Launches
    • Multicore Debug Workflow (Pin&Clone, Colored Contexts)
  • Martin: Not discussed due to lack of time. Probably not an Architectural / E4 topic but a matter of evolution.

Executive Summary of the Discussion

  • #Eclipse Application Model
    • Besides the List of Services we're compiling now by hand, we'll need a strong way of describing the Services for E4 -- as a medium for newcomers to understand Eclipse -- perhaps Tooling to make Service Descriptions machine readable / usable
    • Especially when Services go on the Server (Sessions, Macros Recording), need a consistent Story for exposing a Service, its state and commands, need to be #Getting rid of Singletons
    • Mik to write Position Papers on new Focus Service and Notification Service
    • Every Service should have a Wiki page describing it and the Architecture behind it (probably linked from the E4/Eclipse Application Model Summary Page
    • Application Model is related to almost all other architectural topics (#Too Many Listeners, #Becoming More Asynchronous, #Dependency Injection, Sessions, Macro Recording)
  • #Too Many Listeners
    • For too many different listener classes problem, the Champions will sit together at the Summit to merge generic Listener concepts from (EMF, DataBinding, others)
    • For too many listener instances problem, a general Command Bus / Event Bus may help. Related to Application Model.
    • Champions to sit together at Summit and merge concepts from (EMF, DataBinding, Others)
  • #Sessions, Role-based Access and Security
    • For role-based hiding of UI elements, Capabilities got better in 3.4
    • For role-based denial of services in non-UI, need some basic concepts ("user", "role", "session") - coarse grained, not necessarily using Java Security
    • Mapping Roles onto Sessions seems the right thing
    • Matt Flaherty will be at the Summit and have Lots of Input -- schedule a Session
  • #Macro Recording and Playback
    • Commands plus Undo Stack together are good but unsure how far they will get us -- at least also need #Non-UI Scripting
    • Cannot force all plugins to use Commands -- EMF introduced ChangeListener
    • Need to make the discussion more concrete, need experts to look at how Macro Recording is done in other apps
    • Shoot for a "90%" solution where editable Scripts are recorded and users can fine-tune them by editing
    • Strongly Related to #Eclipse Application Model (Service / Command Bus)
  • #Preference Leveling
    • Too many unstructured Preferences, need a consistent top-down Story
    • Exportable User Prefs (carry around on an USB key); Sharable Team Prefs; Style Sheets; Export with more Categories; Share Settings between multiple Projects rather than Copying
    • Separate Session at the Summit
  • #Workspace, Resources, Content Types
    • Project Nesting (physically and logically); Namespace Resolution (multiple projects with same name in a workspace); Inclusion of Files from Anywhere; Add/remove project type/nature; Listeners and plug-in Loading; Getting rid of Project for RCP (see #Build Granularity)
    • To be discussed at the Summit
  • #Build Granularity
    • New flavors of SDK, RCP-with-resources, RCP-with-workbench-but-without-resources
    • To be discussed at the Summit
  • #Launch / Debug Enhancements
    • Launch Stepper, extendable Launches, Multicore Debug Enhancements -- Not an E4 topic, can be improved evolutionary

Action Items

  • Martin to send conference bridge PIN to new PIN Holder volunteers
  • Martin to finish Meeting Notes and merge discussions into E4 Summit Agenda
  • Mik to add notes about Mylyn interaction event log
  • Mik to write a position paper about Focus Service and Notification Service for the E4 Summit
  • All: Find an Expert who knows Macro Recording in other Apps (Emacs, OpenOffice? KDE? Others?)
  •  ??? to start creating Wiki Pages for Services

Next Meeting