Difference between revisions of "COSMOS Design 208593"

Revision as of 00:13, 4 January 2008 (view source) Jim.gyng.com (Talk | contribs) (→‎''' Implementation Details ''') ← Older edit		Latest revision as of 14:12, 15 January 2008 (view source) Jim.gyng.com (Talk | contribs) (→‎'''Task Breakdown''')
(21 intermediate revisions by 2 users not shown)
Line 86:		Line 86:
	# ANY OTHER COMPONENTS ???		# ANY OTHER COMPONENTS ???
−	Dojo widgets do support i18n, as documented on http://dojo.jot.com/WikiHome/Internationalization?revision=18	+	Dojo widgets do support i18n, as documented on http://dojo.jot.com/WikiHome/Internationalization?revision=18.
	For each of these components there are best practices to support internationalization; however, a common methodology is needed.		For each of these components there are best practices to support internationalization; however, a common methodology is needed.
	</p>		</p>
−	<p>
−	CMDBfQueryService.jar will contain the necessary code for processing a query that conforms to CMDBf.  This set will allow the user to provide implementation of selector handlers that are specific to a data provider.  CMDBfQueryService.jar cannot be used as-is and will require a minimum API set to be implemented.
−	Similarly, CMDBfRegistrationService.jar can be used by contributors to help in processing a registration request.
−	</p>
−
−	== ''' Requirements ''' ==
−	The following is the list of requirements that this enhancement will address:
−
−	# Checking the conformance of a CMDBf query against its schema
−	# Checking the conformance of a CMDBf registration request against its schema
−	# Transformation of CMDBf XML query from/to a POJO representation
−	# Transformation of CMDBf XML registration from/to a POJO representation
−	# A common method of processing a CMDBf query
−	# A common method of processing a registration request
−	== ''' Implementation Details ''' ==	+	== ''' Codebase Considerations ''' ==
−	The COSMOS code must meet the following considerations to support i18n:	+	The COSMOS code must consider the following guidelines to support i18n:
	# Externalize all translatable elements(i.e. no hardcoding)		# Externalize all translatable elements(i.e. no hardcoding)
Line 115:		Line 101:
	# Generalize design to support any language		# Generalize design to support any language
	# Dynamically select and load language module at run time		# Dynamically select and load language module at run time
		+	# Separate build process for language modules from executable
		+	# Gracefully handle missing language resources
		+	# Handle regional settings
		+	## Date Format and Time Formats
		+	## Number Formats
		+	## Currency
		+	## Paper Size
		+	## Sort Order
		+	# Clearly identify which strings should or should not be translated
		+	# Avoid string concatenation
		+	# Not embed text in icons and bitmaps
−	== Transformations ==	+	== '''GUI Considerations''' ==
−	=== Query ===	+
−	[http://cmdbf.org/CMDBf-v0.95.pdf CMDBf] specifies an XML-based syntax by which queries are submitted to the query service, which reports back a query result from the available MDRs, also using XML syntax.  As part of our implementation of the query service, we internally accept POJOs to represent a graph of the query, and return the query result as a set of POJOs.  The job of the transformers is to convert POJOs to and from the XML syntax detailed in the CMDBf specification.	+	The COSMOS GUI must consider the following guidelines to support i18n:
−	As part of [[COSMOS Design 200222]], a set of query transformations and an SML-repository-specific query service were designed and implemented.  In an effort to make these components reusable, the query transformations will be refactored to share common components needed by other libraries in this design.  The initial candidate classes to be moved to the common module are <tt>IXMLWritable</tt>, <tt>QueryServiceUtil</tt> (which will be renamed) and some of the fields in <tt>IQueryTransformerConstants</tt> (the ones pertaining to character constants and general SAX parsing).	+	# Dynamic user elements should be used when displaying simple layouts and text messages
		+	# Dynamic user elements should not be used when special and complicated layout configurations are needed
		+	# Allocating extra space as translated text may be much longer than the English text
		+	# Do not split multi-byte characters
−	In addition, the previously implemented packages will be renamed to include the word "provisional", which in the [http://www.eclipse.org/tptp/home/documents/process/development/api_contract.html API contract specifications] we are reusing from the TPTP project, indicates that the APIs are experimental and will be evolved into stable public APIs.	+	== '''Message Handling Considerations''' ==
−	Here is a simple example that illustrates the incoming query input transformation:	+	The COSMOS messages must consider the following guidelines to support i18n:
−	<tt>InputStream inputStream = new ByteArrayInputStream(example1().getBytes());<br>	+	# Prefix each message with unique message-ids
−	Query query = QueryInputTransformer.transform(inputStream);</tt>	+	# Label 3rd party messages when shown from COSMOS
−	Assume that <tt>example1()</tt> returns a String containing the XML syntax for a <query> as specified in the CMDBf spec.	+	== '''i18n Checklist''' ==
−	=== Registration ===	+	Here is a check list to determine whether COSMOS is i18n-ready or not:
		+	# Menus, dialogs and web layouts can tolerate text expansion
		+	# Development language strings are reviewed for meaning and spelling to reduce user confusion and lessen translation errors
		+	# Third-party software used in the product is examined for i18n support
		+	# Consistent terminology is used in messages
		+	# COSMOS runs properly in its base language in all target locales
		+	# Strings are not assembled by concatenation of fragments
		+	# Source code does not contain hard-coded character constants, numeric constants, screen positions, filenames or pathnames that assume a particular language
		+	# String buffers are large enough to handle translated words and phrases
		+	# East Asian editions support line-breaking rules
		+	# All international editions of the program are compiled from one set of source files
		+	# Localizable items are stored in resource files, message tables or message catalogues
		+	# All language editions share a common file format
		+	# Program handles non-homogeneous network environments where machines are operating with different encodings
		+	# No assumptions are made that one character storage element represents one linguistic character
		+	# Code does not use embedded font names or make assumptions about particular fonts being available
		+	# Program displays and prints text using appropriate fonts
		+	# Sorting and case conversion are culturally correct
		+	# Application works correctly on localized editions of the target operating system(s)
		+	# Specific for web internationalization:
		+	## Check middle-tier components for internationalization compliance
		+	## Ensure that information about encoding and locale of data is passed correctly between presentation and backend tiers
−	CMDBf specifies a registration service by which MDRs register items and relationships with the federating CMDB.  The process for registration involves an XML-based transaction, similar to how the query service works.  This enhancement will provide a transformer to convert a registration request from XML to a POJO graph structure, and will also provide the reverse of that transformation.  In addition, it will provide another transformer that will convert a registration response output by the MDR from a POJO graph structure to the XML-based syntax expected by the CMDBf spec, and will again provide a reverse of that transformation.  The reverse transformations are needed by some clients of the registration service, and also are useful in unit testing of the implementation.	+	== '''Task Breakdown''' ==
−	The following new classes will provide these transformations:	+	The following section includes the tasks required to complete this enhancement.
−	org.eclipse.cosmos.dc.cmdbf.services.provisional.registration.transform.RegistrationInputTransformer	+	# Get this design approved
−	org.eclipse.cosmos.dc.cmdbf.services.provisional.registration.transform.RegistrationOutputTransformer	+	# Determine the follow in ERs needed for i9
−		+
−	The following are the transformation APIs on the RegistrationInputTransformer class:	+
−		+
−	static public RegisterRequest transform(InputStream)	+
−	static public InputStream transform(RegisterRequest)	+
−		+
−	The following are the transformation APIs on the RegistrationOutputTransformer class:	+
−		+
−	static public RegisterResponse transform(InputStream)	+
−	static public InputStream transform(RegisterResponse)	+
−		+
−	RegisterRequest and RegisterResponse POJOs will consist of APIs and classes that match or closely mirror the corresponding element names in the CMDBf data model.  RegisterRequest will reside in the <tt>org.eclipse.cosmos.dc.cmdbf.services.provisional.registration.transform.input.artifacts</tt> package and RegisterResponse will live in the <tt>org.eclipse.cosmos.dc.cmdbf.services.provisional.registration.transform.response.artifacts</tt> package.	+
−		+
−	Here is a simple example that illustrates the incoming registration input transformation:	+
−		+
−	<tt>InputStream inputStream = new ByteArrayInputStream(example1().getBytes());<br>	+
−	RegisterRequest registerRequest = RegistrationInputTransformer.transform(inputStream);</tt>	+
−		+
−	Assume that <tt>example1()</tt> returns a String containing the XML syntax for a <registerRequest> as specified in the CMDBf spec.	+
−		+
−	== Services ==	+
−	=== Query ===	+
−		+
−	The class diagram below displays the interfaces and default implementations that will be included for the query service.  All that a contributor is required to do is to register the classes that will be used as handlers for the selectors.  The default implementation of the CMDBf query operation will look up the appropriate handlers and will invoke them accordingly.	+
−	<br/><br/><br/>	+
−	<p>	+
−	[[Image:QueryService-v1.png]]	+
−	</p>	+
−		+
−	<p>	+
−	The following is a snippet of code that illustrates how a contributor can register a set of selector handlers with the default implementation of <code>ISelectorHandlerFactory</code>.:	+
−		+
−	<code>	+
−	<pre>	+
−	SelectorHandlerFactory.getInstance().addHandlers	+
−	(	+
−	new URI ("http://cosmos.org/mdrId"),	+
−	ISelectorHandlerFactory.ITEM_TEMPLATE,	+
−	new String[]	+
−	{	+
−	"org.eclipse.cosmos.sample.handlers.InstanceIdHandler",	+
−	"org.eclipse.cosmos.sample.handlers.RecordTypeHandler",	+
−	"org.eclipse.cosmos.sample.handlers.PropertyValueHandler",	+
−	"org.eclipse.cosmos.sample.handlers.XPath1Handler"	+
−	}	+
−	);	+
−	</pre>	+
−	</code>	+
−		+
−	Once the selectors are registered a client can make use of the CMDBf query service using the default implementation of <code>ICMDBfQueryOperation</code>.  The argument passed into the <code>initialize</code> method represents a data provider that will be passed to each of the selector handlers.  The selector handlers can cast the argument to a specific type that can process native queries that is understandable by a specific data provider.  The following snippet of code illustrates how the query operation can be invoked:	+
−		+
−	<code>	+
−	<pre>	+
−	// Create a CMDBf query operation object and have it initialized	+
−	CMDBfQueryOperation cmdbfQueryOperation = new CMDBfQueryOperation();	+
−	cmdbfQueryOperation.initialize(new SampleDataProvider());	+
−		+
−	// Execute the query	+
−	Query query = retrieveQuery();	+
−	cmdbfQueryOperation.execute(query);	+
−	</pre>	+
−	</code>	+
−		+
−	Assuming <code>retrieveQuery()</code> will return a POJO representation of a CMDBf query.	+
−		+
−	=== Registration ===	+
−	The diagram below shows the relationship between the classes of the registration service.  The APIs are very symmetric to the ones provided for the query service.  An adopter will need to subscribe register handlers for specific MDR IDs.  The subscribed register handlers will be invoked if the MDR ID of the registration request matches that of the ID associated with the handler.	+
−		+
−	[[Image:RegistrationService-v2.png]]	+
−		+
−	The snippet of code below shows how a client can subscribe a registration handler.  <code>CustomRegistrationHandler</code> is expected to be an implementation of <code>IRegisterHandler</code>.	+
−		+
−	<code>	+
−	<pre>	+
−		+
−	RegisterHandlerSubscription.getInstance().addHandler	+
−	(	+
−	new URI("http://cosmos.org/mdrId"),	+
−	new CustomRegistrationHandler()	+
−	);	+
−	</pre>	+
−	</code>	+
−		+
−	The default implementation of <code>ICMDBfRegistrationOperation</code> can be used to process a registration request.  The snippet below shows how a client can use the operation:	+
−		+
−	<code>	+
−	<pre>	+
−	// Perform the initialization	+
−	CMDBfRegistrationOperation registrationOperation = new CMDBfRegistrationOperation();	+
−	registrationOperation.initialize(new SampleDataProvider());	+
−		+
−	// Execute the operation	+
−	Registration registration = retrieveRegistration();	+
−	registrationOperation.execute(registration);	+
−	</pre>	+
−	</code>	+
−		+
−	Assuming <code>retrieveRegistration()</code> will return a POJO representation of a CMDBf registration request.	+
−		+
−	== ''' Test Coverage ''' ==	+
−		+
−	Unit tests will need to perform the following tests:	+
−		+
−	* Transform a RegisterRequest POJO to XML and back to a POJO, and compare the start and end POJOs	+
−	* Transform a RegisterResponse POJO to XML and back to a POJO, and compare the start and end POJOs	+
−	* Process an empty query	+
−	* Processing a query with just one item	+
−	* Processing a query with a relationship and no items	+
−	* Processing a query with matching items and an unmatched relationship	+
−	* Processing a query with matching items and a matched relationship	+
−	* Processing a query with a relationship that has cardinality	+
−	* Processing a query with unregistered selector handlers	+
−	* Processing an empty registration request	+
−	* Processing a registration request with one item	+
−	* Processing a registration request with matched items and missing relationship	+
−	* Processing a registration request with matched items and a relationship	+
−	* Processing a registration request with a relationship and no items	+
−	* Processing a registration request with missing registration handlers	+
−		+
−	== ''' Task Breakdown ''' ==	+
−		+
−	The following section includes the tasks required to complete this enhancement <!--along with a PERT chart that displays the dependencies of each task-->.	+
−		+
−	[[image:CMDBf-PertChart-v1.png]]	+
−		+
−		+
−	# Refactor plugin, package and class names in cmdbf.query plugin to represent the 5 different jar configurations	+
−	# Refactor the JUnits for the query transformation	+
−	# Create graph representation of registration request and response	+
−	# Create the registration input transformer	+
−	# Create the registration output transformer	+
−	# Add in JUnits for the registration transformer	+
−	# Add in the APIs for processing a CMDBf query	+
−	# Implement a generic implementation of the CMDBf query processor	+
−	# Add in JUnits for the CMDBf query APIs	+
−	# Modify the SML repository to make use of the new API set	+
−	# Add in the APIs for processing a CMDBf registration request	+
−	# Implement a generic implementation of the CMDBf registration processor	+
−	# Add in JUnits for the registration APIs	+
	== '''Open Issues/Questions '''==		== '''Open Issues/Questions '''==
Line 286:		Line 167:
	* Should this ER address the Language Groups issue? Or should this be deemed out of scope since COSMOS 1.0 will not address localization?		* Should this ER address the Language Groups issue? Or should this be deemed out of scope since COSMOS 1.0 will not address localization?
	* Where does XLIFF fit into the i18n considerations? http://docs.oasis-open.org/xliff/v1.2/cs02/xliff-core.html		* Where does XLIFF fit into the i18n considerations? http://docs.oasis-open.org/xliff/v1.2/cs02/xliff-core.html
		+	* How about i18n issues OUTSIDE the code, e.g. what if COSMOS has to deal with two MDRs, each from a different continent?
		+
		+	--[[User:Popescu.ca.ibm.com|Popescu.ca.ibm.com]] 13:23, 4 January 2008 (EST)XLIFF is one option for localization support in xml documents.
	----		----
	[[Category:COSMOS_Bugzilla_Designs]]		[[Category:COSMOS_Bugzilla_Designs]]

---

Latest revision as of 14:12, 15 January 2008

Internationalization Support - programming model

Change History

Name:	Date:	Revised Sections:
Jimmy Mohsin	01/3/2008	Initial version

Workload Estimation

Rough workload estimate in person weeks

Process	Sizing	Names of people doing the work
Design	1	Jimmy Mohsin
Code (not part of this ER)	4	Dev Team
Test (not part of this ER)	4	QA Team
Documentation (not part of this ER)	1
Build and infrastructure (not part of this ER)	1
Code review, etc. (not part of this ER)	1
TOTAL	12

Terminologies/Acronyms

The terminologies/acronyms below are commonly used throughout this document. The list below defines each term regarding how it is used in this document:

Term	Definition
Internationalization	The process of generalizing the design of a product so that it can handle multiple languages and cultural conventions without the need for making code changes for each language. COSMOS 1.0 will be internationalized for its eventual adoption and deployment
Localization	The process of taking an internationalized product and making it linguistically and culturally appropriate to the target locale (country/region and language) where it will be used. COSMOS 1.0 will not be localized for any non-US English languages, this is something that the adopers will need to do.
Translatable Elements	Everything intended to be locale and language dependent. These items may need to be modified for different locales, e.g. date formats, time formats, word lists for content controls, and so on.
Non-user elements	Everything not intended for end-users to see and understand. This fits into two categories, proper names and internal elements. Examples of internal elements are command line commands and options, debug strings, registry keys, program file names, comments, and so on. Examples of proper names are product name, company name, and so on.
User Elements	Everything intended for end-users to see and understand, e.g. menus and menu items, dialog boxes, windows, program groups, error messages, installation scripts, icons, buttons, "Display Name" and "Description" fields for services we create on Windows, and so on.

Purpose

This enhancement is associated with bugzilla 208593.

The data visualization component does not follow a consisten design pattern to support internationalization. There are serveral components within the visualization project that require internationalization support:

1. Dojo widgets
2. Servlet data feeds
3. XML configuration files
4. ANY OTHER COMPONENTS ???

Dojo widgets do support i18n, as documented on http://dojo.jot.com/WikiHome/Internationalization?revision=18. For each of these components there are best practices to support internationalization; however, a common methodology is needed.

Codebase Considerations

The COSMOS code must consider the following guidelines to support i18n:

1. Externalize all translatable elements(i.e. no hardcoding)
2. Put translatable elements in language modules separated from executables
3. Generalize design to support any language
4. Dynamically select and load language module at run time
5. Separate build process for language modules from executable
6. Gracefully handle missing language resources
7. Handle regional settings
   1. Date Format and Time Formats
   2. Number Formats
   3. Currency
   4. Paper Size
   5. Sort Order
8. Clearly identify which strings should or should not be translated
9. Avoid string concatenation
10. Not embed text in icons and bitmaps

GUI Considerations

The COSMOS GUI must consider the following guidelines to support i18n:

1. Dynamic user elements should be used when displaying simple layouts and text messages
2. Dynamic user elements should not be used when special and complicated layout configurations are needed
3. Allocating extra space as translated text may be much longer than the English text
4. Do not split multi-byte characters

Message Handling Considerations

The COSMOS messages must consider the following guidelines to support i18n:

1. Prefix each message with unique message-ids
2. Label 3rd party messages when shown from COSMOS

i18n Checklist

Here is a check list to determine whether COSMOS is i18n-ready or not:

1. Menus, dialogs and web layouts can tolerate text expansion
2. Development language strings are reviewed for meaning and spelling to reduce user confusion and lessen translation errors
3. Third-party software used in the product is examined for i18n support
4. Consistent terminology is used in messages
5. COSMOS runs properly in its base language in all target locales
6. Strings are not assembled by concatenation of fragments
7. Source code does not contain hard-coded character constants, numeric constants, screen positions, filenames or pathnames that assume a particular language
8. String buffers are large enough to handle translated words and phrases
9. East Asian editions support line-breaking rules
10. All international editions of the program are compiled from one set of source files
11. Localizable items are stored in resource files, message tables or message catalogues
12. All language editions share a common file format
13. Program handles non-homogeneous network environments where machines are operating with different encodings
14. No assumptions are made that one character storage element represents one linguistic character
15. Code does not use embedded font names or make assumptions about particular fonts being available
16. Program displays and prints text using appropriate fonts
17. Sorting and case conversion are culturally correct
18. Application works correctly on localized editions of the target operating system(s)
19. Specific for web internationalization:
    1. Check middle-tier components for internationalization compliance
    2. Ensure that information about encoding and locale of data is passed correctly between presentation and backend tiers

Task Breakdown

The following section includes the tasks required to complete this enhancement.

1. Get this design approved
2. Determine the follow in ERs needed for i9

Open Issues/Questions

All reviewer feedback should go in the Talk page for 208593.

• Should this ER address the Language Groups issue? Or should this be deemed out of scope since COSMOS 1.0 will not address localization?
• Where does XLIFF fit into the i18n considerations? http://docs.oasis-open.org/xliff/v1.2/cs02/xliff-core.html
• How about i18n issues OUTSIDE the code, e.g. what if COSMOS has to deal with two MDRs, each from a different continent?

--Popescu.ca.ibm.com 13:23, 4 January 2008 (EST)XLIFF is one option for localization support in xml documents.

---