Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Difference between revisions of "COSMOS CMDBf Framework XSLT Redesign"
(Post simple but complete XSLT-based CMDBf registration server) |
|||
| Line 1: | Line 1: | ||
| + | == Proposal == | ||
While looking at some performance and robustness concerns that came up when applying the current CMDBf framework, it occurred to me that one way of looking at what we're trying to do is to recognize some patterns in an XML input stream, invoke an application-specific function upon recognizing a pattern, and then format the function responses back into XML. Looked at in this light, an approach based on XSLT has much to recommend it. Recognizing patterns in XML input and generating XML output is precisely what it is intended to accomplish, and implementations are generally efficient and robust at doing so. By dealing with the XML directly you deal with a standards-based artifact, not a derivative, and a whole layer of code and a whole class of potential difficulties can be avoided. Moreover, XSLT has been a w3c standard since 1999. There are numerous commercial and open-source implementations compatible with a range of programming environments, and there are a variety of development tools as well. | While looking at some performance and robustness concerns that came up when applying the current CMDBf framework, it occurred to me that one way of looking at what we're trying to do is to recognize some patterns in an XML input stream, invoke an application-specific function upon recognizing a pattern, and then format the function responses back into XML. Looked at in this light, an approach based on XSLT has much to recommend it. Recognizing patterns in XML input and generating XML output is precisely what it is intended to accomplish, and implementations are generally efficient and robust at doing so. By dealing with the XML directly you deal with a standards-based artifact, not a derivative, and a whole layer of code and a whole class of potential difficulties can be avoided. Moreover, XSLT has been a w3c standard since 1999. There are numerous commercial and open-source implementations compatible with a range of programming environments, and there are a variety of development tools as well. | ||
| + | == Registration Handling == | ||
To illustrate the proposed approach, consider the following templates, to process registerRequest messages: | To illustrate the proposed approach, consider the following templates, to process registerRequest messages: | ||
<xsl:template match="cmdbf:registerRequest"> | <xsl:template match="cmdbf:registerRequest"> | ||
| Line 49: | Line 51: | ||
The first template is triggered by a registerRequest message. It loops through each item and relationship in the message, passing information to the extension functions registerItem and registerRelationship, and then formats each reply into an instanceResponse in responseTemplate. Running this transform parses the incoming message, recognizes the item and relationship registration patterns, invokes application-specific functions to process the requests, and formats the response -- without any further code than that required to invoke the transform and implement the application-specific functions. The transform can be invoked from a SOAP request in a JAX-WS environment with just a few dozen lines of code, or it can be driven directly from something like a servlet input stream. I've run standalone tests using a file for the message source, and with the open-source Apache Xalan processor this approach processes large (1-100MB) input messages 4-5 times faster than the current COSMOS framework, with similar memory requirements. | The first template is triggered by a registerRequest message. It loops through each item and relationship in the message, passing information to the extension functions registerItem and registerRelationship, and then formats each reply into an instanceResponse in responseTemplate. Running this transform parses the incoming message, recognizes the item and relationship registration patterns, invokes application-specific functions to process the requests, and formats the response -- without any further code than that required to invoke the transform and implement the application-specific functions. The transform can be invoked from a SOAP request in a JAX-WS environment with just a few dozen lines of code, or it can be driven directly from something like a servlet input stream. I've run standalone tests using a file for the message source, and with the open-source Apache Xalan processor this approach processes large (1-100MB) input messages 4-5 times faster than the current COSMOS framework, with similar memory requirements. | ||
| + | == Query Handling == | ||
The query interface is more complex, but I think the gains to be made are greater, too. I suspect that most repositories are going to be based on a relational database, generally accessed indirectly through application logic. Even for a well-engineered application, reducing the number of trips you take through the stack to satisfy a CMDBf query can offer substantial performance benefits. I would propose a similar structure to the registerRequest approach outlined above: use XSLT to break down the CMDBf query message, and invoke application-specific functions to synthesize a query from the constraints contained in the templates. The queries would then be evaluated to produce the XML response. XSL has import/include features that I would envision being used to format the application-specific record as well as the standard-specified portion of the message. This was my initial reason to look at the details of the framework, by the way, and I have measured 10X and greater improvement on complex queries using this general approach. | The query interface is more complex, but I think the gains to be made are greater, too. I suspect that most repositories are going to be based on a relational database, generally accessed indirectly through application logic. Even for a well-engineered application, reducing the number of trips you take through the stack to satisfy a CMDBf query can offer substantial performance benefits. I would propose a similar structure to the registerRequest approach outlined above: use XSLT to break down the CMDBf query message, and invoke application-specific functions to synthesize a query from the constraints contained in the templates. The queries would then be evaluated to produce the XML response. XSL has import/include features that I would envision being used to format the application-specific record as well as the standard-specified portion of the message. This was my initial reason to look at the details of the framework, by the way, and I have measured 10X and greater improvement on complex queries using this general approach. | ||
| + | == Example Application == | ||
I've posted a simple application [[Media:XsltRegistrationService.zip|here]] that uses the above approach to implement a CMDBf-compliant registration service. It doesn't do much besides accepting all register and deregister requests, but it's useful as a test server and it serves as an example. I'll try to post a prototype query implementation within the next couple days. | I've posted a simple application [[Media:XsltRegistrationService.zip|here]] that uses the above approach to implement a CMDBf-compliant registration service. It doesn't do much besides accepting all register and deregister requests, but it's useful as a test server and it serves as an example. I'll try to post a prototype query implementation within the next couple days. | ||
Revision as of 20:58, 25 July 2008
Proposal
While looking at some performance and robustness concerns that came up when applying the current CMDBf framework, it occurred to me that one way of looking at what we're trying to do is to recognize some patterns in an XML input stream, invoke an application-specific function upon recognizing a pattern, and then format the function responses back into XML. Looked at in this light, an approach based on XSLT has much to recommend it. Recognizing patterns in XML input and generating XML output is precisely what it is intended to accomplish, and implementations are generally efficient and robust at doing so. By dealing with the XML directly you deal with a standards-based artifact, not a derivative, and a whole layer of code and a whole class of potential difficulties can be avoided. Moreover, XSLT has been a w3c standard since 1999. There are numerous commercial and open-source implementations compatible with a range of programming environments, and there are a variety of development tools as well.
Registration Handling
To illustrate the proposed approach, consider the following templates, to process registerRequest messages:
<xsl:template match="cmdbf:registerRequest">
<cmdbf:registerResponse>
<xsl:for-each select="cmdbf:itemList/cmdbf:item">
<xsl:variable name="itemId" select="InstanceId:new(cmdbf:instanceId/cmdbf:mdrId, cmdbf:instanceId/cmdbf:localId)"/>
<xsl:variable name="errMsg" select="CMDBfRegistrationHandler:registerItem(cmdbf:record, $itemId)"/>
<xsl:call-template name="responseTemplate">
<xsl:with-param name="mdrId" select="cmdbf:instanceId/cmdbf:mdrId"/>
<xsl:with-param name="localId" select="cmdbf:instanceId/cmdbf:localId"/>
<xsl:with-param name="errMsg" select="$errMsg"/>
</xsl:call-template>
</xsl:for-each>
<xsl:for-each select="cmdbf:relationshipList/cmdbf:relationship">
<xsl:variable name="sourceId" select="InstanceId:new(cmdbf:source/cmdbf:mdrId, cmdbf:source/cmdbf:localId)"/>
<xsl:variable name="targetId" select="InstanceId:new(cmdbf:target/cmdbf:mdrId, cmdbf:target/cmdbf:localId)"/>
<xsl:variable name="relationshipId" select="InstanceId:new(cmdbf:instanceId/cmdbf:mdrId, cmdbf:instanceId/cmdbf:localId)"/>
<xsl:variable name="errMsg" select="CMDBfRegistrationHandler:registerRelationship(cmdbf:record, $sourceId, $targetId, $relationshipId)"/>
<xsl:call-template name="responseTemplate">
<xsl:with-param name="mdrId" select="cmdbf:instanceId/cmdbf:mdrId"/>
<xsl:with-param name="localId" select="cmdbf:instanceId/cmdbf:localId"/>
<xsl:with-param name="errMsg" select="$errMsg"/>
</xsl:call-template>
</xsl:for-each>
</cmdbf:registerResponse>
</xsl:template>
<xsl:template name="responseTemplate">
<xsl:param name="mdrId"/>
<xsl:param name="localId"/>
<xsl:param name="errMsg"/>
<cmdbf:instanceResponse>
<cmdbf:instanceId>
<cmdbf:mdrId><xsl:value-of select="$mdrId"/></cmdbf:mdrId>
<cmdbf:localId><xsl:value-of select="$localId"/></cmdbf:localId>
</cmdbf:instanceId>
<xsl:choose>
<xsl:when test="$errMsg = ">
<cmdbf:accepted/>
</xsl:when>
<xsl:otherwise>
<cmdbf:declined><xsl:value-of select="$errMsg"/></cmdbf:declined>
</xsl:otherwise>
</xsl:choose>
</cmdbf:instanceResponse>
</xsl:template>
The first template is triggered by a registerRequest message. It loops through each item and relationship in the message, passing information to the extension functions registerItem and registerRelationship, and then formats each reply into an instanceResponse in responseTemplate. Running this transform parses the incoming message, recognizes the item and relationship registration patterns, invokes application-specific functions to process the requests, and formats the response -- without any further code than that required to invoke the transform and implement the application-specific functions. The transform can be invoked from a SOAP request in a JAX-WS environment with just a few dozen lines of code, or it can be driven directly from something like a servlet input stream. I've run standalone tests using a file for the message source, and with the open-source Apache Xalan processor this approach processes large (1-100MB) input messages 4-5 times faster than the current COSMOS framework, with similar memory requirements.
Query Handling
The query interface is more complex, but I think the gains to be made are greater, too. I suspect that most repositories are going to be based on a relational database, generally accessed indirectly through application logic. Even for a well-engineered application, reducing the number of trips you take through the stack to satisfy a CMDBf query can offer substantial performance benefits. I would propose a similar structure to the registerRequest approach outlined above: use XSLT to break down the CMDBf query message, and invoke application-specific functions to synthesize a query from the constraints contained in the templates. The queries would then be evaluated to produce the XML response. XSL has import/include features that I would envision being used to format the application-specific record as well as the standard-specified portion of the message. This was my initial reason to look at the details of the framework, by the way, and I have measured 10X and greater improvement on complex queries using this general approach.
Example Application
I've posted a simple application here that uses the above approach to implement a CMDBf-compliant registration service. It doesn't do much besides accepting all register and deregister requests, but it's useful as a test server and it serves as an example. I'll try to post a prototype query implementation within the next couple days.