Skip to main content

Notice: This Wiki is now read only and edits are no longer possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "Stardust/Knowledge Base/Performance Tuning/Achieving high Throughput with transient Processes"

(Created page with "== Introduction == Traditionally, BPM systems have been employed in designing Human-centric or Case Management type process flows. Such processes rely on the Audit Trail capab...")
 
(Example)
Line 43: Line 43:
 
[[File:Stardust Knowledge Base Performance Tuning Transient3.png|center|Figure 3: Custom ClusterSafeObjectProvider implementation]]
 
[[File:Stardust Knowledge Base Performance Tuning Transient3.png|center|Figure 3: Custom ClusterSafeObjectProvider implementation]]
 
<br />Figure 3: Custom ClusterSafeObjectProvider implementation<br />
 
<br />Figure 3: Custom ClusterSafeObjectProvider implementation<br />
 +
Add the following line to your carnot.properties entry so that the Stardust engine can use your implementation at runtime:
 +
<source lang="text">
 +
Carnot.Engine.Tuning.SupportTransientProcesses.ClusterSafeObjectProvider = com.test.CustomObjectProvider
 +
</source>
 +
We now have a basic setup in place to support Transient Processes. Let us go ahead and deploy the process model described above. Since the process has a manual trigger we execute it from the Portal a few times and verify from the logs that the process has executed. Sample output from a process run is shown below:
 +
[[File:Stardust Knowledge Base Performance Tuning Transient4.png|800px|center|Figure 4: Sample log output for a process run transiently]]
 +
<br />Figure 4: Sample log output for a process run transiently<br />
 +
The log file indicates that Process Instance 101 was executed successfully and Activity Instances 101 and 102 were completed in this process context. However we cannot locate this process when we search for it in the Portal (Figure 5 below) or query the AuditTrail process execution tables (activity_instance, process_instance etc.).
 +
[[File:Stardust Knowledge Base Performance Tuning Transient5.png|center|Process information is not available in the Portal]]<br />Figure 5: Process Information is not available in the Portal

Revision as of 03:27, 10 December 2013

Introduction

Traditionally, BPM systems have been employed in designing Human-centric or Case Management type process flows. Such processes rely on the Audit Trail capabilities of the underlying BPM solution. While Stardust already provides good support for such processes, it is also capable of supporting integration workflows. Here, it acts as a routing and transformation engine to orchestrate a process flow. In such cases the need for low latency and high throughput takes precedence over Audit Trail data persistence.

Transient Processes provide the following advantages:

  • Very little communication with the database.
  • All runtime process data is kept in memory for the lifetime of the Process Instance.
  • Optimized for low latency
  • Significantly higher throughput

Additional information about Transient Processes is available here. All artifacts referred to in the following discussion can be downloaded from here.

Example

We use the following example to start off our discussion on Transient Processes (see Figure 1 below).

Figure 1: Example Transient Process

Figure 1: Example Transient Process


As shown above, the process consists of a couple of activities each of which invokes a POJO with a couple of In/Out parameters. Ordinarily this data would be persisted to the AuditTrail during process execution and the process history would display the execution sequence if one were to view this information in the Portal. To mark the process as transient we select the corresponding property in the Process Properties dialog box (see Figure 2 below).

Figure 2: Process Properties for Transient Process

Figure 2: Process Properties for Transient Process

To ensure that this property is used by the Stardust engine at runtime we need to set the “SupportTransientProcesses” property in the carnot.properties file. We also need to ensure that the database we use supports sequences (such as Oracle). For this discussion we will use MySQL as our AuditTrail database. To enable support for sequences in MySQL we are required to do the following:

1. Create a compatible Stardust AuditTrail schema Create a MySQL database and run the sysconsole command as follows to create an AuditTrail that supports sequences:

sysconsole -v -r com.mysql.jdbc.Driver -dbschema <schema_name> -t mysql_seq -l jdbc:mysql://<servername>:<port>/<dbname> 
-d <username>-s <password> -p sysop createschema

Note that the dbtype option is set to "mysql_seq". Additional information on creating an AuditTrail on MySQL is available here. To learn more about the sysconsole command please refer to the Stardust documentation.

2. Set the appropriate database properties in carnot.properties Add/Edit the following entries to carnot.properties:

AuditTrail.Type = MYSQL_SEQ
AuditTrail.DriverClass = com.mysql.jdbc.Driver
AuditTrail.URL = jdbc:mysql://<servername>:<port>/<dbname>
AuditTrail.Schema = <dbname>
AuditTrail.User = <username>
AuditTrail.Password = <password>

The URL, Schema, User and Password entries should match the values provided in the sysconsole command above.

3. Provide an implementation of a ClusterSafeObjectProviderHolder. This is required by the Stardust engine to share objects internally. A simple implementation is provided below:

Figure 3: Custom ClusterSafeObjectProvider implementation


Figure 3: Custom ClusterSafeObjectProvider implementation
Add the following line to your carnot.properties entry so that the Stardust engine can use your implementation at runtime:

Carnot.Engine.Tuning.SupportTransientProcesses.ClusterSafeObjectProvider = com.test.CustomObjectProvider

We now have a basic setup in place to support Transient Processes. Let us go ahead and deploy the process model described above. Since the process has a manual trigger we execute it from the Portal a few times and verify from the logs that the process has executed. Sample output from a process run is shown below:

Figure 4: Sample log output for a process run transiently


Figure 4: Sample log output for a process run transiently
The log file indicates that Process Instance 101 was executed successfully and Activity Instances 101 and 102 were completed in this process context. However we cannot locate this process when we search for it in the Portal (Figure 5 below) or query the AuditTrail process execution tables (activity_instance, process_instance etc.).

Process information is not available in the Portal

Figure 5: Process Information is not available in the Portal

Back to the top