Difference between revisions of "Stardust/Knowledge Base/Transaction Management/Understanding Transaction Management in Stardust"

Revision as of 02:07, 14 August 2013

Introduction

There is often a requirement to define units of work in a business process. The analyst may want a set of operations to be executed atomically (all-or-nothing). It is important to understand the transaction management process in Stardust to be able to define such execution semantics. It is relevant for the execution of a process to specify where a transaction starts and where it is committed. Specification of where a transaction begins and where it is committed is referred to as transaction demarcation. Transaction demarcation in Stardust is done implicitly and is compliant with the mechanisms of the EJB standard:

Normally, each call to start or continue a process demarcates the boundaries of a transaction. A two-phase commit is performed against all involved data sources after this call returns. If the client submitting this call has already demarcated a transaction, the commit may even be postponed until that point.

This article explores various transaction management scenarios under resource local and distributed transactions contexts involving the Stardust Audit Trail and external database. All artifacts referenced in the article may be downloaded from here. Please refer to the "readme.txt" file contained within the zip for information on how to build, deploy and test the application.

Scenario

We consider a scenario where a Stardust process is kicked off and the user is prompted for some basic Employee information (Emp Id, First Name and Last Name). This data is persisted in a subsequent activity to the external data store. We simulate a TX Rollback exception by generating a PK violation from the database if the Emp Id provided already exists in the database. A Non-TX exception is generated if the Emp Id provided exceeds the value 1000.We execute this process repeatedly for the following conditions:

• Stardust Exception Propagation mode - never, onRollback, always
  
• Transaction Management configuration - Resource Local, JTA
  
• Exception type: TX Rollback, Application Exception
  
• Service Exception - First service call, Second service call
  

We thus end up testing 24 scenarios.

Process Model

The following process model is considered for the following discussion.

As shown in the model above, we initially accept some user input pertaining to an employee.We can generate exceptions in the Emp Sevice App or Emp Service 2 App under the following conditions:

• TX Exception - If the Emp Id specified for the Employee matches an existing Emp Id in the Employee table, a duplicate PK exception is thrown by the database and results in a TX Rollback. For Emp Service 2 we generate the Emp Id using the following logic in the preceeding MTA activity - If Emp Id entered by user <10, subtract 1 and set that as the Emp Id for the  Emp Service 2 invocation, if the Emp Id entered  >=10, set the Emp Id to 1001. Emp Id values >1000 result in Application Exceptions as detailed below.
• Application Exception - If the Emp Id specified by the user >1000 it results in an Application Exception on the first sevice invocation. If the Emp Id specified >=10, it results in an Application Exception on the second service invocation.

The message transformation logic for the activity "MTA for Emp" is shown below:

Employee Service

We use two variations of an Employee Service bean to persist the Employee information into the external data store. The first variation is used to test Resource Local transaction management scenarios while the second one is used to test JTA (distributed) transaction management. The two service beans are shown below:

We explicitly invoke the flush() method on the Resource local bean only so that we can study the transactional behavior more easily. The flush forces a database write and any  DB exceptions will be thrown right away. Likewise setting the transactional propagation mode to MANDATORY is not a strict requirement. We only do it to ensure that the service method is being invoked within the scope of a global transaction. If it weren't (which shouldn't really be the case with a JTA configuration where the TX has already been initiated by the process engine), we would see a TX related exception prior to the invocation of the bean method.

Resource Local Transaction Management

We analyze the following scenarios:

   Stardust Exception Propagtion Mode - never, onRollback, always
 Exception type: TX rollback, application exception
 Service exception: First service, Second service

Exception Propagation Mode - never

This scenario requires the following entry in carnot.properties

Carnot.Engine.ErrorHandling.ApplicationExceptionPropagation = never

The Stardust documentation states the following:

never - The exception is consumed by the engine and not propagated to the client. The engine sets both, the activity 
instance and the process instance, to interrupted state and logs the exception to the audit trail. After that, the 
engine attempts to commit the transaction and stops the execution of the process. The activity (and process) execution 
may be resumed after running the process instance recovery. Please note that the engine does not check the status of 
the transaction, and therefore if the transaction is already set to rollback then all the actions performed in this 
transaction will be lost. 

Scenario - TX rollback exception on first service

Although this scenario requires a TX exception to be raised we don't expect the process to roll back since the transaction is not global. Deploy the process model and start the process. Enter an Emp Id that already exists in the Employee table. This will ensure that a PK violation is thrown by the database and result in a TX rollback in the first service invocation but the process instance will only reflect the fact that it has been interrupted. We can verify this by checking the process history as shown below:

The server log appears similar to the following:

11:17:54 INFO - [http-8080-5    ] ActivityInstanceBean      - State change for Activity instance 'CallEmployeeService',  oid: 1584 (process instance = 112): Application-->Interrupted.
11:17:54 WARN - [http-8080-5    ] AuditTrailLogger          - Activity thread interrupted at 'Activity: Call Employee Service', reason: javax.persistence.PersistenceException: Exception 
[EclipseLink-4002] (Eclipse Persistence Services - 2.1.2.v20101206-r8635): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry '5' for key 'PRIMARY'

Scenario - Application Exception on first service

Since this scenario requires an application exception to be raised we expect the process to be interrupted. Enter the Emp Id 1001. Per the service definition described earlier this ought to result in an application exception. As shown from the process history below, the process is interrupted at activity "Call Employee Service". Note that the exception raised is however different from the one raised in the previous scenario.

The server log shows the following exception:

java.lang.Exception
 at com.sungard.jpa.service.EmployeeService.addEmployee(EmployeeService.java:18)

Distributed (JTA) Transaction Management

We analyze the following scenarios:

• Stardust Exception Propagtion Mode - never, onRollback, always
• Exception type: TX rollback, application exception
• Service exception: First service, Second service
  

Exception Propagation Mode - never

This scenario requires the following entry in carnot.properties

Carnot.Engine.ErrorHandling.ApplicationExceptionPropagation = never

The Stardust documentation states the following:

never - The exception is consumed by the engine and not propagated to the client. The engine sets both, the activity 
instance and the process instance, to interrupted state and logs the exception to the audit trail. After that, the 
engine attempts to commit the transaction and stops the execution of the process. The activity (and process) execution 
may be resumed after running the process instance recovery. Please note that the engine does not check the status of 
the transaction, and therefore if the transaction is already set to rollback then all the actions performed in this 
transaction will be lost. 

Scenario - TX rollback exception on first service

Since this scenario requires a TX exception to be raised we expect the process to be rolled back all the way. Deploy the process model and start the process. Enter an Emp Id that already exists in the Employee table. This will ensure that a PK violation is thrown by the database and result in a TX rollback exception in the first service invocation.As shown below, the process is rolled back to the manual activity:

The server log shows the following exception:

org.springframework.transaction.UnexpectedRollbackException: JTA transaction unexpectedly rolled back (maybe due to a 
timeout); nested exception is javax.transaction.RollbackException: Unable to commit: transaction marked for rollback

The process history shows that it is currently suspended again in the manual activity after the rollback:

The Employee database does not contain an entry for this Emp Id.

Scenario - Application Exception on first service

Since this scenario requires an application exception to be raised we expect the process to be interrupted. Enter the Emp Id 1001. Per the service definition described earlier this ought to result in an application exception. As shown from the process history below, the process is interrupted at activity "Call Employee Service":

The server log shows the following exception:

14:27:36 WARN - [http-8080-6    ] ngBeanApplicationInstance - Failed to invoke Spring Bean method 
'addEmployee(com.sungard.jpa.test.Employee)'.
java.lang.Exception

The Employee database does not contain an entry for this Emp Id.

Scenario - TX rollback exception on second service

Since this scenario requires a TX exception to be raised we expect the process to be rolled back all the way to the previous commit point. Enter an Emp Id that is greater by one than the largest current entry in the Employee table (for e.g. if the Employee table currently has Emp Ids 1 thru 5, enter the value 6). This will ensure that the first service call executes without any exception while a PK violation is thrown by the database for the second service call when it tries to insert Emp Id 5.This will result in a TX rollback exception in the second service invocation.As shown below, the process is rolled back to the previous commit point which happens to be right after the first service call. This happens due to the "Fork on Traversal" flag set on the outgoing transition from activity "Call Employee Service". This ensures that a unit of work is defined for all activities from the first manual activity to the service call. Therefore the external data store now shows a new entry for Emp Id 6.There is no exception message shown on the UI in this case since the roll back isn't upto the manual activity. Rather we would have to look through the server log. If the user desired to execute both service calls atomically they would have to turn off the "Fork on Traversal" flag. In this case the process would rollback all the way to first manual activity. The reader may also refer to the model "JTAWorkflowModel2" in the attached zip. The process history view in the portal shows the following:

As seen above, the process state is still Active and there is no interrupted activity due to the rollback. The server log reveals the following exception message indicating that there has been a  rollback and that the default Stardust mechanism has attempted to retry the service call 10 times (default).

15:05:47 WARN - [Thread8        ] MultipleTryInterceptor    - Unexpected exception : JTA transaction unexpectedly rolled 
back (maybe due to a timeout); nested exception is javax.transaction.RollbackException: Unable to commit: transaction 
marked for rollback.
15:05:47 WARN - [Thread8        ] MultipleTryInterceptor    - Retrying 1 time.

Scenario - Application exception on second service

Since this scenario requires an application exception to be raised we expect the process to be interrupted. Enter the Emp Id 11. This PK does not currently exist in the database. So it would result in the first service call executing without any exceptions. Per the service definition described earlier this ought to result in an application exception only in the second service. As shown from the process history below, the process is interrupted at activity "Call Employee Service Again":

The server log shows the following exception:

15:19:44 WARN - [Thread10       ] ngBeanApplicationInstance - Failed to invoke Spring Bean method 
'addEmployee(com.sungard.jpa.test.Employee)'.
java.lang.Exception

Exception Propagation Mode - onRollback

This scenario requires the following entry in carnot.properties

Carnot.Engine.ErrorHandling.ApplicationExceptionPropagation = onRollback

The Stardust documentation states the following:

onRollback - The behavior depends on the status of the current transaction. If the transaction is set to RollbackOnly, 
similarly with "always", the engine attempts to log the original exception to the audit trail in a separate transaction 
and after that, the exception is propagated to the client (re-thrown). If the transaction is not set to RollbackOnly, 
then it will behave like "never" and it will attempt to interrupt the activity instance and the process instance. 

Scenario - TX rollback exception on first service

Since this scenario requires a TX exception to be raised we expect the process to be rolled back all the way. Enter an Emp Id that already exists in the Employee table. This will ensure that a PK violation is thrown by the database and result in a TX rollback exception in the first service invocation. As shown below, the process is rolled back to the manual activity:

The server log shows the following exception:

org.springframework.transaction.UnexpectedRollbackException: JTA transaction unexpectedly rolled back (maybe 
due to a timeout); nested exception is javax.transaction.RollbackException: Unable to commit: transaction marked for rollback

The process history shows that it is currently suspended again in the manual activity after the rollback:

The Employee database does not contain an entry for this Emp Id.

Scenario - Application Exception on first service

Since this scenario requires an application exception to be raised we expect the process to be interrupted. Enter the Emp Id 1001. Per the service definition described earlier this ought to result in an application exception. The process history looks similar to the following and the process is interrupted at activity "Call Employee Service":

The server log shows an exception similar to the following:

14:27:36 WARN - [http-8080-6    ] ngBeanApplicationInstance - Failed to invoke Spring Bean method 
'addEmployee(com.sungard.jpa.test.Employee)'.
java.lang.Exception

The Employee database does not contain an entry for this Emp Id.

Scenario - TX rollback exception on second service

Since this scenario requires a TX exception to be raised we expect the process to be rolled back all the way to the previous commit point. Enter an Emp Id that is greater by one than the largest current entry in the Employee table (for e.g. if the Employee table currently has Emp Ids 1 thru 6, enter the value 7). This will ensure that the first service call executes without any exception while a PK violation is thrown by the database for the second service call when it tries to insert Emp Id 6.This will result in a TX rollback exception in the second service invocation. As shown below, the process is rolled back to the previous commit point which happens to be right after the first service call. This happens due to the "Fork on Traversal" flag set on the outgoing transition from activity "Call Employee Service". This ensures that a unit of work is defined for all activities from the first manual activity to the service call. Therefore the external data store now shows a new entry for Emp Id 7.There is no exception message shown on the UI in this case since the roll back isn't upto the manual activity. Rather we would have to look through the server log. If the user desired to execute both service calls atomically they would have to turn off the "Fork on Traversal" flag. In this case the process would rollback all the way to first manual activity. The reader may also refer to the model "JTAWorkflowModel2" in the attached zip. The process history view in the portal looks similar to thee following:

As seen above, the process state is still Active and there is no interrupted activity due to the rollback. The server log reveals an exception message similar to the following indicating that there has been a  rollback and that the default Stardust mechanism has attempted to retry the service call 10 times (default).

15:05:47 WARN - [Thread8        ] MultipleTryInterceptor    - Unexpected exception : JTA transaction unexpectedly rolled 
back (maybe due to a timeout); nested exception is javax.transaction.RollbackException: Unable to commit: transaction 
marked for rollback.
15:05:47 WARN - [Thread8        ] MultipleTryInterceptor    - Retrying 1 time.

Scenario - Application exception on second service

Since this scenario requires an application exception to be raised we expect the process to be interrupted since there is no Rollback. Enter the Emp Id 11. This PK does not currently exist in the database. So it would result in the first service call executing without any exceptions. Per the service definition described earlier this ought to result in an application exception only in the second service. The process history view looks similar to the one below and indicates that the process is interrupted at activity "Call Employee Service Again":

The server log appears similar to the following :

15:19:44 WARN - [Thread10       ] ngBeanApplicationInstance - Failed to invoke Spring Bean method 
'addEmployee(com.sungard.jpa.test.Employee)'.
java.lang.Exception

Exception Propagation Mode - always

This scenario requires the following entry in carnot.properties

Carnot.Engine.ErrorHandling.ApplicationExceptionPropagation = always

The Stardust documentation states the following:

always - The engine is automatically rolling back the transaction. The engine attempts to log the original exception to the 
audit trail in a separate transaction. After that, the exception is propagated to the client (re-thrown). 

Scenario - TX rollback exception on first service

Since this scenario requires a TX exception to be raised we expect the process to be rolled back all the way. Enter an Emp Id that already exists in the Employee table. This will ensure that a PK violation is thrown by the database and result in a TX rollback exception in the first service invocation. As shown below, the process is rolled back to the manual activity:

The server log shows the following exception:

org.springframework.transaction.UnexpectedRollbackException: JTA transaction unexpectedly rolled back (maybe due to a 
timeout); nested exception is javax.transaction.RollbackException: Unable to commit: transaction marked for rollback

The process history shows that it is currently suspended again in the manual activity after the rollback:

The Employee database does not contain an entry for this Emp Id.

Scenario - Application Exception on first service

Since this scenario requires an application exception to be raised we expect the process to be interrupted. Enter the Emp Id 1001. Per the service definition described earlier this ought to result in an application exception. In this scenario however, we also see an exception in the UI since the engine always rolls the process regardless of whether the underlying transaction has been marked for rollback. A ServiceException is displayed on the UI as shown below:

The process history looks similar to the following and the process is suspended at the first manual activity.

The server log shows an exception similar to the following:

org.eclipse.stardust.common.error.ServiceException: BPMRT03702 - Activity thread rolled back at activity with ID 
'CallEmployeeService', reason: java.lang.Exception: java.lang.Exception

The Employee database does not contain an entry for this Emp Id.

In addition, the following entry is now visible in the Stardust log_entry table since this information has been committed by another transaction and has therefore survived the rollback.

Scenario - TX rollback exception on second service

Since this scenario requires a TX exception to be raised we expect the process to be rolled back all the way to the previous commit point. Enter an Emp Id that is greater by one than the largest current entry in the Employee table (for e.g. if the Employee table currently has Emp Ids 1 thru 7, enter the value 8). This will ensure that the first service call executes without any exception while a PK violation is thrown by the database for the second service call when it tries to insert Emp Id 7.This will result in a TX rollback exception in the second service invocation. As shown below, the process is rolled back to the previous commit point which happens to be right after the first service call. This happens due to the "Fork on Traversal" flag set on the outgoing transition from activity "Call Employee Service". This ensures that a unit of work is defined for all activities from the first manual activity to the service call. Therefore the external data store now shows a new entry for Emp Id 8.There is no exception message shown on the UI in this case since the roll back isn't upto the manual activity. Rather we would have to look through the server log. If the user desired to execute both service calls atomically they would have to turn off the "Fork on Traversal" flag. In this case the process would rollback all the way to first manual activity. The reader may also refer to the model "JTAWorkflowModel2" in the attached zip. The process history view in the portal looks similar to thee following:

As seen above, the process state is still Active and there is no interrupted activity due to the rollback. The server log reveals an exception message similar to the following indicating that there has been a  rollback and that the default Stardust mechanism has attempted to retry the service call 10 times (default).

15:05:47 WARN - [Thread8        ] MultipleTryInterceptor    - Unexpected exception : JTA transaction unexpectedly rolled 
back (maybe due to a timeout); nested exception is javax.transaction.RollbackException: Unable to commit: transaction 
marked for rollback.
15:05:47 WARN - [Thread8        ] MultipleTryInterceptor    - Retrying 1 time.

Scenario - Application exception on second service

Even though this scenario requires only an application exception to be raised we expect the process to be rolled back in this case. Enter the Emp Id 12. This PK does not currently exist in the database. So it would result in the first service call executing without any exceptions. Per the service definition described earlier this ought to result in an application exception only in the second service. As shown from the process history below, the process is in this case still in the Active state and rolled back to the last commit point "Call Employee Service". The new Emp Id with PK 12 is now seen in the external database.

The server log displays the following exception and indicates that the default retry mechanism (10 tries) attempted to execute the service several times.

16:55:33 WARN - [Thread11       ] AuditTrailLogger          - BPMRT03702 - Activity thread rolled back at activity with ID 
'CallEmployeeServiceAgain', reason: java.lang.Exception: 
java.lang.Exception (User: system_carnot_engine (Realm: system_carnot_engine) (system_carnot_engine, system_carnot_engine))
16:55:33 WARN - [Thread11       ] MultipleTryInterceptor    - Expected exception : BPMRT03702 - Activity thread rolled back 
at activity with ID 'CallEmployeeServiceAgain', 
reason: java.lang.Exception: java.lang.Exception.
16:55:33 WARN - [Thread11       ] MultipleTryInterceptor    - Retrying 1 time.

The Stardust engine logs the rollback associated with each retry into the log_entry table of the Audit Trail as shown below: