View source for Higgins Configuration Management

{{#eclipseproject:technology.higgins}}
[[Image:Higgins_logo_76Wx100H.jpg|right]]

== Configuration Management ==
''' Disclaimer '''
''This page on the topic of configuration management is a compilation of higgins dev-list (Greg Byrd, Tom Doman, Daniel Sanders, Michael McIntosh,Markus Sabadello), off-line discussions (Greg Byrd, Rajalakshmi Iyer, David Kuehr-McLaren), and a prototype for the Registry done by Rajalakshmi Iyer. Bugzilla 238374 [https://bugs.eclipse.org/bugs/show_bug.cgi?id=238374] was opened to track the requirement. Please let me know if there are any errors or comments. -dkm''

== Overview ==

Applications need the ability to manage the run time configuration and persisted configuration files of various Higgins components. This example will focus on IdAS Registry as the use case, but applies to any IConfigurableComponent.  

The IdAS registry can be initialized from an XML file through the configuration APIs. The 'IdentityAttributeService' section in the configuration XML file that corresponds to the IdAS registry typically contains the following information: 

A section of settings for context factories, e.g. 
<source lang="xml">
<Setting Name="ContextFactoryInstancesList" Type="htf:list">
	<Setting Name="JNDIContextFactory" Type="htf:map">
		<Setting Name="Instance" Type="xsd:string">JNDIContextFactory</Setting>
		<Setting Name="ContextTypes" Type="htf:list">
			<Setting Name="JNDIContextType" Type="xsd:string">$context+ldap</Setting>
		</Setting>
	</Setting>
</Setting> 
</source>

A section of settings for the context IDs, e.g. 
<source lang="xml">
<Setting Name="ContextIdsList" Type="htf:list">
	<Setting Name="urn:ldap@ltoyota" Type="htf:map">
		<Setting Name="ContextId" Type="xsd:string">urn:ldap@ltoyota</Setting>
			<Setting Name="ContextTypes" Type="htf:list">
				<Setting Name="JNDIContextType" Type="xsd:string">$context+ldap</Setting>
			</Setting>
			<Setting Name="Connection" Type="htf:map">
				<Setting Name="Address" Type="xsd:string">ldap://ltoyota:389</Setting> 
				<Setting Name="SearchBase" Type="xsd:string">o=Higgins</Setting>
			</Setting>
		</Setting>
	</Setting>
</Setting>
</source>

Following APIs exist to update the IdASRegistry: 
*registerContextFactory
*removeContextFactory
*registerContextId
*removeContextId

All the above APIs currently update only the in-memory data-structures corresponding to the list of context factories and contextIds. There is no mechanism to persist updates through these APIs to the configuration XML file.

== API Proposal == 

=== Using IConfigurationHandler and ISettingDescriptor ===

The Configuration API defines:
*An IConfigurableComponent interface that must be implemented by all components that must be configured through the Configuration API 
*An IConfigurationHandler interface that must be implemented by all methods of configuring the IdAS components like XML/XRDS etc.

Currently, an IConfigurationHandler instance is used to read configuration data from persistent storage (e.g., a file).  The proposal is to use the '''IConfigurationHandler''' interface for ''writing configuration data'' to persistent storage, as well.  The format of the stored data will be determined solely by the IConfigurationHandler used to store the data.

In order to store the data for later use, the type of each configuration setting is needed.  This information is present in the configuration file, but is lost when the data is converted into Java objects.  (Multiple configuration setting types can map to the same Java object type.)  The proposal is to use the '''ISettingDescriptor''' to hold the configuration setting type information, separately from the configuration data (which is represented as a java.util.Map).

ISettingDescriptor was designed as a way for a component to describe its own settings, in order to drive a management interface.  We have not yet developed code that uses this capability, or code which generates an ISettingDescriptor for a configurable component.  For the purposes of this proposal, the IConfigurationHandler will generate the ISettingDescriptor as it reads settings from the configuration file.  As new settings are added to the configuration data, the global ISettingDescriptor must be changed to reflect the changes.  

To write configuration data to persistent storage, the IConfigurationHandler is given the data (java.util.Map) and a descriptor (ISettingDescriptor), and then the handler's '''applyUpdates''' method is called.



=== Using ISettingDescriptor ===

This option was an option discussed on the mailing list by Greg Byrd, Tom Doman and Daniel Sanders in August 2007 at [http://dev.eclipse.org/mhonarc/lists/higgins-dev/msg03029.htmlAdd]. Rajalakshmi Iyer opened the subject again on this thread [http://dev.eclipse.org/mhonarc/lists/higgins-dev/msg04721.html]. Greg started a summary thread after a dev call to reinvigorate the topic[http://dev.eclipse.org/mhonarc/lists/higgins-dev/msg04821.html].

Greg added the ISettingDescriptor interface to org.eclipse.higgins.configuration.api as an approach to define the structure, type and constraints of the data in a configuration store. 
<source lang="java">
public interface ISettingDescriptor {
	public String getName();
	public Class getType();
	public String getDisplayString();
	public String getDocString();
	public List getSubSettings();
	public Object getConstraint(String key);
	
	public void setName(String name);
	public void setType(Class type);
	public void setDisplayString(String display);
	public void setDocString(String doc);
		
	public void addConstraint(String name, Object value);
	public void removeConstraint(String name);
	public void addSubSetting(ISettingDescriptor sub);
	public void addSubSetting(int index, ISettingDescriptor sub);
	public boolean removeSubSetting(ISettingDescriptor sub);
	public ISettingDescriptor removeSubSetting(int index);

}
</source>

Different constraints are relevant for different setting types.  E.g., minValue makes sense for int/float, but not stream.  So the manager can query for a particular constraint, if null returned, then that constraint was not specified.  Or can iterate through all constraints, etc.  Constraint is a string/value pair. There are some "standard" constraint strings defined as static variables in package org.eclipse.higgins.configuration.common.SettingDescriptor, but any implementation could always do more.

When persisting an updated configuration, the component's ConfigurationHandler will use the updated in-memory Map and the SettingDescriptor to write the format to the store.

If the management aspect of the SettingDescriptor is not needed at first, then the ConfigurationHandler can actually build the SettingDescriptor as it reads the configuration file.  It has the name, type, and value, which are the things needed for writing the setting back.  

Later, we can do a more complete job with the descriptor, including display name and constraints.  But these may not be needed for the first round of requirements.

=== Proposed Configuration API Changes ===

'''''IConfigurableComponent'''''

The configure() method now takes two additional parameters: the ISettingDescriptor for the component itself, and the ISettingDescriptor for the "global" settings.  This is needed because the ConfigurationHandler creates the setting descriptor for each component as it's read.  Any changes to the component will be reflected in its descriptor.

<source lang="java">
	public abstract void configure
		(final java.util.Map mapGlobalSettings,
		final String strComponentName,
		final java.util.Map mapComponentSettings,
		final ISettingDescriptor componentDescriptor,
		final ISettingDescriptor globalDescriptor)
		throws Exception;
</source>

Also, a new method is added to return the descriptor.

<source lang="java">
public abstract ISettingDescriptor getComponentDescriptor();
</source>


'''''IConfigurationHandler'''''

As described above, applyUpdates() is used to persist the configuration.

<source lang="java">
public abstract void applyUpdates() throws Exception;
</source>

In the past, a ConfigurationHandler creates its own global settings (e.g., by reading from a file.)  Now, we're also using ConfigurationHandler to '''write''' a configuration.  So we can create a new ConfigurationHandler from scratch, and give it the settings to store.  We also need to give it the setting descriptor.

<source lang="java">
public abstract void setGlobalSettings(java.util.Map settings) throws Exception;
public abstract void setGlobalSettingDescriptor(ISettingDescriptor settingDescriptor) throws Exception;
</source>

We can query a ConfigurationHandler for its setting descriptor.  This will be used after the configuration file has been read, so that the global descriptor can be changed to reflect any new settings that may be added.

<source lang="java">
public abstract ISettingDescriptor getSettingDescriptor() throws Exception;
</source>

The following two methods are used to initialize an IConfigurationHandler's settings and descriptor.  These are useful if the IConfigurationHandler is created specifically to write configuration data to persistent storage.  (In other words, the handler doing the writing is different from the handler that read the configuration data from the original file.)

<source lang="java">
public abstract void setGlobalSettings(java.util.Map settings) throws Exception;
public abstract void setGlobalSettingDescriptor(ISettingDescriptor descriptor) throws Exception;
</source>

== Changes to XML Configuration ==

The XML ConfigurationHandler is changed to reflect the IConfigurationHandler changes described above.

In addition, the ISettingHandler interface is changed to pass in an ISettingDescriptor for the parent setting, and an ISettingDescriptor for the global settings.

<source lang="java">
public abstract Object getSetting
		(String strName,
		String strConfigurationBase,
		java.util.Map mapSettingHandlers,
		java.util.Map mapSettingsGlobal,
		Object settingsParent,
		ISettingDescriptor globalDescriptor,
		ISettingDescriptor parentDescriptor,
		org.apache.axiom.om.OMElement omSetting)
		throws Exception;
</source>

This means that classes that implement ISettingHandler must make this change, as well.  This affects classes in the following packages:

*org.eclipse.higgins.configuration.xml
*org.eclipse.higgins.idas.util.cp.jscript
*org.eclipse.higgins.sts.server.mapper.appliesto
*org.eclipse.higgins.sts.server.mapper.extension

== Changes to IdAS and IdAS Registry ==

The short-term motivation for this proposal is the ability to persist changes to the IdAS Registry.  A prototype has been done for the XML-configured registry, incorporating the following changes.

'''''IContextFactory'''''

A method is added to org.eclipse.higgins.idas.api.IContextFactory to retrieve an ISettingDescriptor for the context type created by the factory.  A typical use case would be an IdAS application getting a handle to the context factory based on the type of context to create and then using this API to get the setting descriptor for a context id to be created based on this factory.

<source lang="java">
public ISettingDescriptor getContextDescriptor(String contextIdName) throws IdASException;
</source>

'''''IdASRegistry'''''

Because IdASRegistry implements IConfigurableComponent, its ''configure'' method must be changed to include the global and component descriptors, as described above.

Changes are also made to the ''registerContextFactory'' and ''registerContextID'' methods.  First, an ISettingDescriptor is passed in, to allow the new component to be added to the registry descriptor.  Second, a boolean argument (bSave) is passed in to indicate whether the new factory or context should be added to the IdASRegistry settings (and descriptor).

<source lang="java">
public void registerContextFactory(String[] types, 
                                   String factoryClassName, 
                                   ISettingDescriptor factoryDescriptor, 
                                   boolean bSave ) throws IdASException {...}


public void registerContextFactory(String type, 
                                   String factoryClassName, 
                                   ISettingDescriptor factoryDescriptor, 
                                   boolean bSave ) throws IdASException {...}

public void registerContextFactory(IContextFactory factory, 
                                   boolean bSave) throws IdASException {...}

public void registerContextId(String contextId, 
                              Map contextIdConfiguration, 
                              ISettingDescriptor contextDescriptor, 
                              boolean bSave) throws Exception {...}
</source>

Methods are added to retrieve the ISettingDescriptor for a particular context factory or context ID.

<source lang="java">
public ISettingDescriptor getContextFactoryDescriptor(String factoryClassName, 
                                                      String[] factoryTypes) {...}

public ISettingDescriptor getContextIdDescriptor(String contextIdName, 
                                                 String[] types) throws IdASException {...}
</source>

=== Usage ===

Registering a new context or factory does not automatically make those changes persistent, even with the ''bSave'' parameter set to '''true'''.  To accomplish this, the application using IdAS would create use an IConfigurationHandler instance, calling its ''applyUpdates'' method when changes are to be made persistent.

== Option 2: Represent each setting in the configuration file using the ISetting interface ==

''[NOTE: This option was not implemented, due to significant changes needed in existing code.  It is kept here for historical purposes.]''

Currently, the in-memory configuration is represented by the Map data structure. Thus the 'IdentityAttributeService' section is represented by a Map with the keys 'ContextFactoryInstancesList' and 'ContextIdsList' and whose values are the lists of context factories and context ids respectively. This structure does not represent the 'type' of the value of each setting, which becomes necessary while writing back information to the file. 

The change of this option is to represent each element in the XML configuration through the com.ibm.higgins.configuration.api.ISetting interface. The global settings is an ISetting whose value could be any of the types represented in the 'SettingHandlers' section of the configuration file. If the type of a setting is a List, then the value of that setting would be a list of ISetting(s). Thus each setting in the XML file has a corresponding in-memory ISetting object. 

=== Change XML based configuration API ===

A new class com.ibm.higgins.configuration.xml.Setting that implements the com.ibm.higgins.configuration.api.ISetting interface has been introduced. It contains the setting name, setting type and setting value. Additionally it also contains the in-memory XML Object Model (OMElement) representation for that setting. It will have methods to add child settings and remove settings. 

Here is a prototype implementation of the addChildSetting() method: 

<source lang="java">
public void addChildSetting(String name, String type, Object value) { 

	/* Create an OMElement corresponding to the child setting to add */
	OMFactory factory = OMAbstractFactory.getOMFactory();
	OMElement addElement = factory.createOMElement("Setting", null);
	addElement.addAttribute("Name", name, null);
	addElement.addAttribute("Type", type, null);

	/* Create the new child setting to add to this setting */
	ISetting setting = new Setting(name, type, value, addElement);

	/* Depending on the type of the value of a child setting, create 
	 * sub-settings for the child setting.
	 */
	if ( type.equalsIgnoreCase("htf:list")) { 
		List listValues = (List)value;
		for (Iterator itr = listValues.iterator(); itr.hasNext(); ) { 
			ISetting childSetting = (ISetting)itr.next();
			setting.addChildSetting(childSetting.getName(), childSetting.getType(),
			childSetting.getValue());
		}
	} else if ( setting.getType().equalsIgnoreCase("htf:map")) { 
		Map mapValues = (Map)setting.getValue();
		for (Iterator itr = mapValues.keySet().iterator(); itr.hasNext(); ) { 
			ISetting childSetting = (ISetting)itr.next();
			setting.addChildSetting(childSetting.getName(), childSetting.getType(),
			childSetting.getValue());
		}
	} else { 
		addElement.setText(setting.getValue().toString());
	}

	/* Add the child OMElement to this setting's OMElement */
	_element.addChild(addElement);

	/* Update this setting's values to include the child setting */
	if ( _type.equalsIgnoreCase("htf:map")) { 
		((Map)_value).put(name, setting);
	} else if (_type.equalsIgnoreCase("htf:list")) { 
		((List)_value).add(setting);
	}
}
</source>

An ISetting represents the global settings in the com.ibm.higgins.configuration.xml.ConfigurationHandler instead of a Map. 

The signature for the getSetting() method in com.ibm.higgins.configuration.xml.ISettingHandler has changed from:
<source lang="java">
		public abstract Object getSetting
		(String strName,
		String strConfigurationBase,
		java.util.Map mapSettingHandlers,
		java.util.Map mapSettingsGlobal,
		Object settingsParent,
		org.apache.axiom.om.OMElement omSetting)
		throws Exception;
</source>
to:
<source lang="java">
		public abstract ISetting getSetting
		(String strName,
		String strConfigurationBase,
		java.util.Map mapSettingHandlers,
		ISetting globalSetting,
		ISetting parentSetting,
		org.apache.axiom.om.OMElement omSetting,
		org.eclipse.higgins.configuration.api.IConfigurationHandler configHandler)
		throws Exception;
</source>

'''All implementations of ISettingHandler in the com.ibm.higgins.configuration.xml package will be modified to conform to this new method signature.'''

All IConfigurableComponent implementatations (i.e. The IdASRegistry, all IContextFactory implementations) will adhere to consistent use of ISetting for representing and manipulating configuration settings.
 
Example: Register a new context factory with the IdASRegistry

The IdAS client application will call the following code to register a new context factory: 

<source lang="java">
ConfigurationHandler handler = new ConfigurationHandler();
handler.setConfigurationBase(".");
handler.setFileName("TestConfiguration.xml");
Handler.configure();

IdASRegistry registry = (IdASRegistry)((ISetting)((Map)handler.getSettings().getValue()).get("IdentityAttributeService")).getValue();

String[] types = {"$context+file"};
registry.registerContextFactory(types, "FileContextFactory");

The registerContextFactory method in IdASRegistry must persist this change in the XML file under 'ContextFactoryInstancesList' setting as: 

<Setting Name="FileContextFactory" Type="htf:map">
	<Setting Name="Instance" Type="xsd:string">FileContextFactory</Setting>
	<Setting Name="ContextTypes" Type="htf:list">
		<Setting Name="FileContextType" Type="xsd:string">$context+file</Setting>
	</Setting>
</Setting>
</source>

To do this, the registerContextFactory method will be updated as follows:
 
<source lang="java">
/* Get the list of ISettings for 'ContextFactoryInstancesList' */
List factoryList = (List)contextFactoryInstanceSettings.getValue();

/* Add a child setting to the context factories list setting. The child
 * setting is a map containing details about the new context factory */
Map mapCtxFactory = new HashMap();
this.contextFactoryInstanceSettings.addChildSetting(factoryClassName, "htf:map", mapCtxFactory);

for ( Iterator itr = factoryList.iterator(); itr.hasNext(); ) { 
	ISetting factorySetting = (ISetting)itr.next();
	if ( factorySetting.getName().equalsIgnoreCase(factoryClassName) ) { 
		/* Add the 'Instance' setting to the new context factory setting */
		factorySetting.addChildSetting("Instance", "xsd:string", factoryClassName);

		/* Add the 'ContextTypes' setting of type list to the new context factory setting */
		List typeList = new ArrayList(types.length);
		factorySetting.addChildSetting("ContextTypes", "htf:list", typeList);
		ISetting typeSetting = 	(ISetting)((Map)factorySetting.getValue()).get("ContextTypes");
		for ( int j = 0; j < types.length; j++ ) {
			typeSetting.addChildSetting(factoryClassName + "Type", "xsd:string", types[j]);
		}
	}
}

/* Save the update */
_config_handler.applyUpdates();

</source>

Constraints for UI management would need to be added in the future.

== Other considerations ==

Some items for follow-on investigation, 

*how to change a configured components settings at run-time, and save those new settings for later use. Could call getSettings(), change map, and call configure() again with the new settings. Or could be a separate method for changing (e.g., setSettings()), so that the component can distinguish a full configuration from an update.
*Applications may need a way to compare the in memory configuration with what is in the configuration file to detect race conditions with other applications or threads. Once updating the file, the application could check the file contents to make sure that the changes were not overwritten by another application calling applyUpdates.