Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "Equinox Extension Registry Work Objects"

(Open questions)
 
(3 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
'''WORK IN PROGRESS – SUBJECT TO CHANGE'''
 
'''WORK IN PROGRESS – SUBJECT TO CHANGE'''
  
== Problem ==
+
Ver.2
  
When developers use extension registry, the most common first step is to translate registry artifacts (IExtension, IConfigurationElement) into user-specific classes. Even in a most trivial case that results in a few lines of extra code. The "proper" implementation that takes into account multiplicity, dynamic registry events, and error handling takes a bit of knowledge and effort.
+
== Changes from Ver.1 ==
 
+
As a result we have to deal with:
+
* Extra code that has to be written by each developer for parsing and walking extension registry;
+
* Inconsistent treatment of extension information not conforming to the schema;
+
* Problems in synchronizing user model and registry.
+
  
 +
After going through the feedback for the original proposal and some code prototyping, the Ver.1 proposal is transformed into the Tier II of this proposal. Schemas are removed from the picture and class type information is specified in #getObjects(). The mapping overrides are to be supported with Java annotations; only exact name matches will be supported on pre-1.5 VMs. The original proposal can be found at [http://wiki.eclipse.org/Equinox_Extension_Registry_Work_Objects1].
  
 
== Proposed solution ==
 
== Proposed solution ==
  
Let's assume that extension point schemas can specify Java classes corresponding to schema elements. When such information can be used to construct Java classes and inject them with the values specified in the plugin.xml files.
+
After reviewing extension registry usage patterns and the feedback for the original proposal, it seems that there are three common patterns on how the registry information is used:
  
 +
* (A) typed objects are created based on the configuration elements,
  
 +
* (B) portions of the configuration elements trees are transformed into hash maps, or
  
== Details ==
+
* (C) values from attributes are retrieved and combined depending on location in the configuration elements tree
  
In order to construct user-specific objects, extension registry will use constructor injection to propagate context to the objects and setter injection to propagate arguments from the plugin.xml.
 
  
 +
To make those scenarios easier we can:
  
=== Consumer-facing APIs ===
+
* (C) present an extension registry as a tree supporting a subset of XPath expression to retrieve values; provide overrides for get() method for primitive Java types (Tier I)
  
'''IExtensionRegistry''' will get a new method
+
* (A) provide methods to adapt nodes on the registry tree into a Java objects of a consumer-specified type (Tier II)
  
 +
* (B) provide methods to adapt nodes on the registry tree into a hash maps constructed based on the registry nodes and their relative positions (Tier III)
 +
 +
 +
The new methods are going to be be a part of the new IRegistryNode interface. There will be adaptors allowing switch from IExtensionPoint / IExtension / IConfigurationElement and a factory to get IRegistryNode element corresponding to a given registry at the given path.
 +
 +
As you'll see some of the proposed functionality makes extension registry look like a read-only preferences.
 +
 +
 +
=== Tier I: Registry as a tree. Getting typed values, navigation ===
 +
 +
==== Group 1. Getting typed values ====
 +
 +
Registry elements would get a set of typed method to retrieve attribute values, something like:
 
<source lang="java">
 
<source lang="java">
Object[] getObjects(String path, String scope, Object context, Class ofClass)
+
interface IRegistryNode {
 +
....
 +
boolean getBoolean(String path, boolean default)
 +
....
 +
}
 
</source>
 
</source>
  
where
+
The "path" could be:
 
+
"name" - attribute name for the current configuration element
* path: ConfEelementName1[/.../ ConfEelementName] [@attrName]
+
" element_name:attributeName" - attribute with a given name under the configuration element
* scope: registry/extensionPointID[/extensionID]
+
"..:attributeName" - value of parent's attribute
* context: a Java object to be passed to the constructor, may be null
+
* ofClass: expected Java class of the result
+
  
The path will be constructed with XPath syntax in mind (although it is unlikely that full XPath will be needed).
+
The method overrides will be provided for:
The scope will be constructed with the view of potential merge of the extension registry and preferences into one mechanism
+
* int, long
It is likely that multiple variations of this method will be provided to account for optional arguments, multiplicity, and typing of the result.
+
* float, double
 +
* boolean
  
'''IExtension''' will get a new method
+
Two versions of the method will be provided for override - one that returns one value and one that returns an array:
 
<source lang="java">
 
<source lang="java">
Object[] getObjects(Object context, Class ofType)
+
interface IRegistryNode {
 +
....
 +
int getInt(path, int default);
 +
int[] getIntValues(path);
 +
....
 +
}
 
</source>
 
</source>
  
The extension registry will cache the objects for the duration of the session. Consumers should not cache the objects but rather feel free to ask the extension registry whenever necessary.
+
The path's exact syntax will be fleshed out as we go, but it need to support hierarchical navigation (absolute and relative), multiplicity qualifiers, and attribute names. It is aprobably going to assume a format similar to a subset of XPath:
 +
 
 +
 
 +
[/extension_point_ID/][/extensionID/]ConfigurationElement_name1/.../ConfigurationElement_nameN:attributeName
  
  
=== Provider-facing APIs ===
+
For multiplicity qualifiers, the only supported qualifier at this time will be the "[1]" to indicate that the first matching element is selected.
  
A new interface will be added:
+
In addition, convenience methods will be added to get a resource URL based on the location of the contributing bundle:
 
<source lang="java">
 
<source lang="java">
public interface IRegistryObject {
+
interface IRegistryNode {
public boolean init();
+
....
public void dispose();
+
URL getResource(nodePath);
 +
URL getResources(nodePath);
 +
....
 
}
 
}
 
</source>
 
</source>
 +
(the nodePath would be expected to point to an attribute(s) containing relative resource path).
  
On objects implementing this interface, the init() method will be called as a final step of the object creation (after all injections are processed). The displose() method will be called to indicate that the corresponding extension has been removed from the framework.
+
==== Group 2. Navigating registry nodes ====
  
 +
The following methods will be added to the IRegistryNode:
  
==== Extension point schema: class ====
+
<source lang="java">
 +
public interface IRegistryNode {
 +
....
 +
public IRegistryNode parent();
  
The extension point schema will be used to pass additional information. Elements can have the Java class specified:
+
public IRegistryNode[] children();
  
<source lang="xml">
+
public String[] attributes();
<element name ="myElement">
+
 
<class name = "org.abc.MyClass"/>
+
public IRegistryNode node(String path);
</element>
+
 
 +
public IRegistryNode[] nodes(String path);
 +
 
 +
public boolean nodeExists(String path);
 +
 
 +
public String name();
 +
 
 +
public String absolutePath();
 +
....
 +
}
 
</source>
 
</source>
  
The Java class specified in this way would have to have either
+
Most methods are self-explanatory. For the initial version the #node() won't be creating new nodes, but rather only returning existing nodes or null of no such node was found.
* a default no-argument constructor (if objects created from the extensions are context-free), or
+
* a single argument constructor with the type corresponding to the context (if objects are to be created in a context)
+
  
The "context" here is any Java object that can be passed to the extension registry at the time it is prompted for extension information.
+
Unlike Preferences, some nodes will have no names and some nodes will have multiple children with the same name. Nodes with no names will match "*" in XPath expressions.
  
''TBD consider an optional attribute multiplicity="singletonAll|singletonInContext|...".''
 
  
 +
=== Tier II. Adapting registry nodes into user-specified objects ===
  
==== Default setters ====
+
Two methods will be provided to adapt a given node into a user-specified object (or objects, or hash map).
  
The generated classes will use pre-defined method and field injection.
+
<source lang="java">
 +
public interface IRegistryNode {
 +
....
 +
Object toObject(Object[] context, Map classNames);
 +
 +
Object[] toObjectArray(String path, Object[] context, Map classNames);
 +
....
 +
}
 +
</source>
  
When an attribute named "abc" is found in the XML, the user class will be polled first for the method "setABC(abc.Class)" when for the field named "abc". (The method name will be case-insensitive).
+
The classNames map is passed in as an argument and maps element names to the java.lang.Class objects to be created for those elements.
  
Similar processing will be employed for the contained elements. If a sequence of elements is described in the XML, the "set…" method will be called multiple times.
+
The new object will be specified using the constructor with the best fit for the "Object[] context" arguments. Best fit will be defined as the constructor with the largest number of arguments whose type is assignable from the context[i]. The order of the context[] will be preserved as much as possible; arguments not specified will be passed in as null.
  
 +
After the object is constructed, it will be injected with values specified in the XML file. First method injection will be tried with method name "set" + attribute_name; then field injection will be tried for the field "attribute_name".
  
==== Extension point schema: special setters ====
+
If multiple sub-elements are present in the XML, the "set" method will be called multiple times.
  
Several special arguments can be injected using explicit setter instructions: extension ID, extension name, extension contributor.
+
Expansion possibility: for Java 1.5 classes will be able to override mapping of attributes and elements to method names and field names using annotations.  
  
<source lang="xml">
+
If the created object implements IInitializable#init(), the init() method will be called to give the object an opportunity to finalize its initialization.
<element name ="myElement">
+
<class name = "org.abc.MyClass"/>
+
<set method="setID" special="extension:id"/>
+
<set field="extensionName" special="extension: name"/>
+
</element>
+
</source>
+
  
The supported values are:
+
If the created object implements IDisposable#deleted(), then deleted() method will be called when corresponding registry element is removed.
* "extension:id",
+
* "extension:name",
+
* "extension:contributor".
+
  
 +
''TBD: clarify caching of the created objects. Should they be cached by the caller or will the registry cache them? Soft references vs. getting the same object vs. memory consumption.''
  
=== TBD Sateless objects vs. objects with state ===
 
  
<source lang="xml">
+
=== Tier III. Hash maps ===
<element name ="myElement">
+
 
<class name = "org.abc.MyClass" [stateAware="true|false"]/>
+
In Tier II methods are added to create an object or array of objects corresponding to a registry node. Sometimes such objects are used to construct hash maps. We can add an extra method to perform this work:
</element>
+
 
 +
<source lang="java">
 +
public interface IRegistryNode {
 +
....
 +
public Map toMap(String keyPath, String realtiveValuePath, Object[] context, Map classNames);
 +
....
 +
}
 
</source>
 
</source>
  
The user-objects will be cached as soft references. As such they might be purged from the memory and re-constructed. This works fine for stateless objects. We need to investigate if there is a need for state-aware objects and, if so, add a way to specify this in the schema.
+
The method will use keyPath to find key elements for the hash map, then will find corresponding nodes based on the relativeValuePath, will construct objects for those values, and will add (key, value) pairs in the hash map.
  
This would address state-aware objects for the duration of the session. Is there a need to persist such objects between sessions? ISerializable?
 
  
 +
== Why don't we just use EMF ==
  
=== TBD typed values in the extension point schema ===
+
Several people asked about using EMF in this work. Let's consider where we could use EMF: external view of the extension registry (APIs) and internal implementation.
  
How about creating setter overrides specific to primitive types, arrays, and so on? It might save some more code (for instance, developer would not have to convert from String to int) at the expense of added complexity of the APIs.
+
* External view: I don't think it would be right for us to push EMF on our consumers by returning EObject-derivatives. Consumers might not be familiar with EMF, and #toObject() methods by design create objects of a user-specified classes which are not limited to EObjects.
  
 +
* Internal implementation: EMF could be used to implement IRegistryNode tree. However, it is likely that we'll want to use binary format for cache persistence, not XML (both for the CPU performance and for the memory footprint). We also very likely to use different notification approach - either via OSGi's EventAdmin or by providing our own event bus in equinox.common.
  
=== TBD Implementation: on top of the current registry or independent? ===
 
 
The first inclanation is to do the implementation on top of the existing extension registry. However, this creates an obvious duplication in memory usage and some extra processing.
 
  
It might be worth while to investigate creation of an independent mechanism based on the contents of the plugin.xml. Such mechanism would avoid creation of registry artifacts altogether and only expose user-specific objects.
+
== Open questions ==
  
Also, for such "separate" solution an optional Java-1.5+ processing can be added where injection is done completely via constructors. (Java 1.5 gives annotations that make possible to match constructor arguments to the attributes specified in plugin.xml.) (Splitting existing registry into 1.4 and 1.5 parts won't be practical.)
+
* Event Notifications
  
Also, for such "separate" implementation explore if EMF helps with creating a model from XML.
+
For objects created using #toObject() it is probably their containers that would be interested in events which makes IDisposable less than ideal. On the other hand, people who use only Tier I methods still going to need to have registry event notifications.
  
My inclination would be to go with the "current registry + objects on top" for 3.5 stream as it is way more practical. Having this implementation in the 3.5 stream would allow us to get feedback earlier. If it goes well, we can use what we learn from 3.5 to create a "separate" implementation for 4.0 and, potentially, relegate current implementation to the compatibility layer.
+
It might be a good time to see if we can get away from the listeners and start relying on an event bus, or, maybe on OSGi's EventAdmin.
 +
 
 +
The great deal of the difficulty in dealing with events in the current extension registry comes from the events being asynchronous. As a result, listeners have to deal with a strange data state where some data corresponds to the moment when event was fired, and some data is current. I think we need to reconsider if we really want registry events to remain asynchronous. (Registry events are not generated during usual everyday work, but rather when a bundle is installed or removed.) If we decide to support both synchronous and asynchronous notifications, we should consider sending the path to modified nodes rather then IRegistryNode objects.
 +
 
 +
* Permissions
 +
 
 +
Do we need to add some sort of "permission to modify my extension"?
  
''Pre-requisite for this item:'' get a clear understanding on caching of user objects and modified registry.
 
  
  
 
'''WORK IN PROGRESS – SUBJECT TO CHANGE'''
 
'''WORK IN PROGRESS – SUBJECT TO CHANGE'''
 +
  
 
===Links===
 
===Links===

Latest revision as of 16:32, 15 October 2008

WORK IN PROGRESS – SUBJECT TO CHANGE

Ver.2

Changes from Ver.1

After going through the feedback for the original proposal and some code prototyping, the Ver.1 proposal is transformed into the Tier II of this proposal. Schemas are removed from the picture and class type information is specified in #getObjects(). The mapping overrides are to be supported with Java annotations; only exact name matches will be supported on pre-1.5 VMs. The original proposal can be found at [1].

Proposed solution

After reviewing extension registry usage patterns and the feedback for the original proposal, it seems that there are three common patterns on how the registry information is used:

  • (A) typed objects are created based on the configuration elements,
  • (B) portions of the configuration elements trees are transformed into hash maps, or
  • (C) values from attributes are retrieved and combined depending on location in the configuration elements tree


To make those scenarios easier we can:

  • (C) present an extension registry as a tree supporting a subset of XPath expression to retrieve values; provide overrides for get() method for primitive Java types (Tier I)
  • (A) provide methods to adapt nodes on the registry tree into a Java objects of a consumer-specified type (Tier II)
  • (B) provide methods to adapt nodes on the registry tree into a hash maps constructed based on the registry nodes and their relative positions (Tier III)


The new methods are going to be be a part of the new IRegistryNode interface. There will be adaptors allowing switch from IExtensionPoint / IExtension / IConfigurationElement and a factory to get IRegistryNode element corresponding to a given registry at the given path.

As you'll see some of the proposed functionality makes extension registry look like a read-only preferences.


Tier I: Registry as a tree. Getting typed values, navigation

Group 1. Getting typed values

Registry elements would get a set of typed method to retrieve attribute values, something like:

interface IRegistryNode {
	....
	boolean getBoolean(String path, boolean default)
	....
}

The "path" could be: "name" - attribute name for the current configuration element " element_name:attributeName" - attribute with a given name under the configuration element "..:attributeName" - value of parent's attribute

The method overrides will be provided for:

  • int, long
  • float, double
  • boolean

Two versions of the method will be provided for override - one that returns one value and one that returns an array:

interface IRegistryNode {
	....
	int getInt(path, int default);
	int[] getIntValues(path);
	....
}

The path's exact syntax will be fleshed out as we go, but it need to support hierarchical navigation (absolute and relative), multiplicity qualifiers, and attribute names. It is aprobably going to assume a format similar to a subset of XPath:


[/extension_point_ID/][/extensionID/]ConfigurationElement_name1/.../ConfigurationElement_nameN:attributeName


For multiplicity qualifiers, the only supported qualifier at this time will be the "[1]" to indicate that the first matching element is selected.

In addition, convenience methods will be added to get a resource URL based on the location of the contributing bundle:

interface IRegistryNode {
	....
	URL getResource(nodePath);
	URL getResources(nodePath);
	....
}

(the nodePath would be expected to point to an attribute(s) containing relative resource path).

Group 2. Navigating registry nodes

The following methods will be added to the IRegistryNode:

public interface IRegistryNode {
	....
	public IRegistryNode parent();
 
	public IRegistryNode[] children();
 
	public String[] attributes();
 
	public IRegistryNode node(String path);
 
	public IRegistryNode[] nodes(String path);
 
	public boolean nodeExists(String path);
 
	public String name();
 
	public String absolutePath();
	....
}

Most methods are self-explanatory. For the initial version the #node() won't be creating new nodes, but rather only returning existing nodes or null of no such node was found.

Unlike Preferences, some nodes will have no names and some nodes will have multiple children with the same name. Nodes with no names will match "*" in XPath expressions.


Tier II. Adapting registry nodes into user-specified objects

Two methods will be provided to adapt a given node into a user-specified object (or objects, or hash map).

public interface IRegistryNode {
	....
	Object toObject(Object[] context, Map classNames);
 
	Object[] toObjectArray(String path, Object[] context, Map classNames);
	....
}

The classNames map is passed in as an argument and maps element names to the java.lang.Class objects to be created for those elements.

The new object will be specified using the constructor with the best fit for the "Object[] context" arguments. Best fit will be defined as the constructor with the largest number of arguments whose type is assignable from the context[i]. The order of the context[] will be preserved as much as possible; arguments not specified will be passed in as null.

After the object is constructed, it will be injected with values specified in the XML file. First method injection will be tried with method name "set" + attribute_name; then field injection will be tried for the field "attribute_name".

If multiple sub-elements are present in the XML, the "set" method will be called multiple times.

Expansion possibility: for Java 1.5 classes will be able to override mapping of attributes and elements to method names and field names using annotations.

If the created object implements IInitializable#init(), the init() method will be called to give the object an opportunity to finalize its initialization.

If the created object implements IDisposable#deleted(), then deleted() method will be called when corresponding registry element is removed.

TBD: clarify caching of the created objects. Should they be cached by the caller or will the registry cache them? Soft references vs. getting the same object vs. memory consumption.


Tier III. Hash maps

In Tier II methods are added to create an object or array of objects corresponding to a registry node. Sometimes such objects are used to construct hash maps. We can add an extra method to perform this work:

public interface IRegistryNode {
	....
	public Map toMap(String keyPath, String realtiveValuePath, Object[] context, Map classNames);
	....
}

The method will use keyPath to find key elements for the hash map, then will find corresponding nodes based on the relativeValuePath, will construct objects for those values, and will add (key, value) pairs in the hash map.


Why don't we just use EMF

Several people asked about using EMF in this work. Let's consider where we could use EMF: external view of the extension registry (APIs) and internal implementation.

  • External view: I don't think it would be right for us to push EMF on our consumers by returning EObject-derivatives. Consumers might not be familiar with EMF, and #toObject() methods by design create objects of a user-specified classes which are not limited to EObjects.
  • Internal implementation: EMF could be used to implement IRegistryNode tree. However, it is likely that we'll want to use binary format for cache persistence, not XML (both for the CPU performance and for the memory footprint). We also very likely to use different notification approach - either via OSGi's EventAdmin or by providing our own event bus in equinox.common.


Open questions

  • Event Notifications

For objects created using #toObject() it is probably their containers that would be interested in events which makes IDisposable less than ideal. On the other hand, people who use only Tier I methods still going to need to have registry event notifications.

It might be a good time to see if we can get away from the listeners and start relying on an event bus, or, maybe on OSGi's EventAdmin.

The great deal of the difficulty in dealing with events in the current extension registry comes from the events being asynchronous. As a result, listeners have to deal with a strange data state where some data corresponds to the moment when event was fired, and some data is current. I think we need to reconsider if we really want registry events to remain asynchronous. (Registry events are not generated during usual everyday work, but rather when a bundle is installed or removed.) If we decide to support both synchronous and asynchronous notifications, we should consider sending the path to modified nodes rather then IRegistryNode objects.

  • Permissions

Do we need to add some sort of "permission to modify my extension"?


WORK IN PROGRESS – SUBJECT TO CHANGE


Links

  1. Inversion of Control: "Inversion of Control Containers and the Dependency Injection pattern" by Martin Fowler
  2. Bug 248340: Improve usability of the extension registry
  3. Bug 221603: Provide a public, reusable RegistryReader

Top: All Extension Registry Work Items

Back to the top