View source for Stardust/Knowledge Base/Integration/Camel/Camel File based inter-system integration - Camel

== Overview ==
In this article we are going to explore, how we can use Stardust Camel Application types to integrate 2 different systems. The medium of communication would be a Stardust generated CSV file.

The Use Case:
* User enters details in Stardust portal UI into a stardust process data.
* Stardust converts the process data into a CSV file and put it into a designated folder (e.g. outbound) on the file system and halts the process execution.
* The other application (e.g. SAP) picks up the CSV file, after working on it, the other application puts the file in a designated folder (e.g. inbound) for stardust halted process instance to pick it up and continue.
* Another stardust process instance picks up the file from the designated folder (e.g. inbound), performs some logic to identify which CSV file belongs to which halting process instance (of step 1) and sends a message to that instance to resume its operations further.
* The resuming process instance then consumes the CSV file and converts its content into a stardust process data for further use.

== Approach ==
To implement the use case we will be:
* Defining 2 processes (1 which will do the processing like, convert process data into CSV, halting, resuming and converting CSV back into process data and another which will monitor the designated folder where the other application (e.g. SAP) is going to write the processed CSV file.

* 1st process will have 3 activities (2 interactive and 1 application that will use stardust Camel application type)

* The Camel application activity will configure Camel routes to convert process data into CSV, put it to a designated folder and to go into hibernation. Camel application in the 2nd process (e.g. SAP) will consume the file from this location.

* 2nd process will use Camel application type to monitor the designated folder where the other integrating application will write the processed CSV and will apply some logic to create a message to send to hibernating process instance of the 1st process.

1st process, lets name it as Order Entry Process

2nd process, lets name it as CSV Consumer Process

== How To Do ==

For this example we are considering '''outbound''' as a folder where stardust will write converted CSV file and '''inbound''' as a folder where integrating application will write back processed CSV file and inbound is the folder that will monitored by CSV Consumer Process's camel application.

* Defining Order Entry Process

[[File:orderentryproc.JPG]]

The route of the Convert Order To CSV application explained:

[[File:GenerateCVS.jpg]]

* Producer part:

<source lang="xml">
<setHeader headerName="CamelFileName">
<simple>${header.ippActivityInstanceOid}_Order.csv</simple>
</setHeader>
<to uri="file://c:\temp\outbound"/>
</source>

In this route we are converting process data into CSV using stardust OOTB feature, Outbound Conversion -> To CSV (see the image above) and then constructing the name of the file to include an identifier (Activity Instance OID, which we are getting from the header ippActivityInstanceOid) and then putting the file to a location called outbound under folder c:\temp.

Once this route ends the process instance gets into hibernation mode, see the consumer route of it below, as its waiting for a message at the endpoint direct:csvTempQueue, where as per our design the CSV consumer process will send the message.

* Consumer part (this is responsible to make process instance getting into hibernated mode)

<source lang="xml">
<from uri="direct:csvTempQueue"/>
<to uri="ipp:authenticate:setCurrent?user=motu&password=motu" />
<setHeader headerName="CamelLanguageScript">
<constant>
function setOutHeader(key, output){
request.setHeader(key, output);
}
var line= request.headers.get("ActivityOID");
//var part = line.split("_");
setOutHeader("ActOID", parseInt(line));
</constant>
</setHeader>
<to uri="language:javascript?transform=false"/>
<to uri="ipp:activity:find?expectedResultSize=-1&activityInstanceOid=$simple{header.ActOID}&states=Application,Created,Hibernated,Interrupted,Suspended" />
<convertBodyTo type="java.lang.String"/>
<to uri="ipp:activity:complete" />
</source>

Once Consume CSV process sends a message at endpoint direct:csvTempQueue along with a header ActivityOID, thsi route is going to fetch the value out of the ActivityOID header and use it for:
<source lang="xml">
uri="ipp:activity:find?expectedResultSize=-1&'''activityInstanceOid=$simple{header.ActOID}'''&states=Application,Created,Hibernated,Interrupted,Suspended" </source>

to find the activity instance with matching OID and only completes that one. to convert CSV back into process data, we are again using stardust OOTB feature Inbound Conversion -> From CSV (see the image above)

'''Note: Ensure that out parameter that will hold the converted CSV into process data should be exactly named as the fist element of the structure data that is type of out parameter. In this example ''Orders'' first element name is ''Items'', hence out parameter name is Items'''.
 

* Defining CSV Consumer Process

[[File:csvconsumerproc.JPG]]

The route of the consumer process application "Consume CSV File":
<source lang="xml">
<from  uri="file://c:\temp\inbound?delete=false&consumer.initialDelay=2000&include=.*csv$"/>
<setHeader headerName="CamelLanguageScript">
<constant>
function setOutHeader(key, output){
request.setHeader(key, output);
}
var fileName = request.getHeader("CamelFileName");
var actOID = fileName.split("_");
setOutHeader("ActivityOID", parseInt(actOID[0]));
</constant>
</setHeader>
<to uri="language:javascript?transform=false"/>
<to uri="direct:csvTempQueue"/>
</source>

When integrating application puts the processed CSV into '''c:\temp\inbound''' folder this route picks the file, gets the file name, splits the name in parts to fetch activity instance OID (the identifier we created) and set the ActivityOID header with the activity instance OID as value. This value is going to be used at hibernating process instance.

== Execution ==

Please see attached web recording for execution experience [[File:Async-CSV-Processing_2.zip]]

== Artifacts ==

Artifacts can be found at [[File:CSVAsyncExample_good.zip]]