Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Stardust/Knowledge Base/Integration/Camel/Camel Based Monitoring and Auditing Transient Processes Using SEDA

Overview

In this article we are going to explore, how we can use Stardust Camel Applications,Triggers to monitoring and auditing the details of Transient process.

Use Case:

  • There is one process which is marked as transient(having only non-interactive activities), so this process will not be persisted in audittrail database.
  • For auditing purpose user wants minimum information to be captured.
  • Monitoring of Transient process should be asynchronous and should not affect it's execution.

Why Transient Process ?

Before we jump to modeling above use case let's first discuss why do we have decided the process should not be saved in audiTrail and marked it as Transient ? see below process

FileProcessingProcess.PNG

  • As per above process if you see there is one camel trigger which is polling to a directory for some files.
  • It picks the file and passed to another camel application "Send File to JMS/FTP server" which does some processing and again sends to FTP or JMS server.
  • Since this process is processing the files and files sizes may vary for e.g. 100 MB 200 MB or bigger. And we are storing file content into Process data and this data will persist into auditrail db.
  • Storing the big files in process data will result in un-necessary growing database size.
  • To avoid above situation we decided, not to persist this process details in database by making this process as Transient:

Transient.PNG

  • So once the process is completed all the process data will be discarded.

Issue

As we discussed transient process will not be saved to auditrail database it means we will not be able to identify what all files are processed by which processes ? From an auditing purpose atleast following information should be captured:

  • File name of the processed files.
  • File processing started by transient process.
  • File Processing completed by transient process.

Approach

To solve above issue and monitoring the execution of transient process we used camel SEDA component.Because it provides asynchronous SEDA(Staged Event Driven Architecture) behavior:

  • Modified camel route in application "Send file to JMS/FTP server" as below:

SEDAInterceptors.PNG

  • Above route will send events to SEDA endpoints before and after completing the file processing.
  • Create consumer for SEDA START events

StartEventProc.PNG

MonitorStartEvents.PNG

  • Add process descriptor to this process

StartEventsDesc.PNG

  • Create consumer for SEDA END events

EndEvents.PNG

ENDEVENTSROUTE.PNG

  • Add below process descriptor

EndEventDescriptors.PNG

NOTE: If you notice the SEDA endpoints, we use "size"(size=500) parameter to prevent even if consumer stops working then it will not result into your server to go OUT OF MEMORY.

Execution

  • To execute discussed process download the attached model from here File:SEDA Model.zip
  • Deploy the above downloaded model.
  • Once the files will be processed by transient process for each processed file you could see below entries in process overview:

AuditingLogs.PNG

Back to the top