Skip to main content

Notice: This Wiki is now read only and edits are no longer possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "Stardust/Knowledge Base/Integration/Camel/Consuming A CSV File - at FTP or Local FileSystem"

(Created page with "== Use Case == Consuming a CSV file (delimter could be anything) arriving at either an FTP location or in a folder on local file system. E.g. a csv file arriving at: * C:\temp...")
 
Line 4: Line 4:
 
* /FtpFeed/comException folder.
 
* /FtpFeed/comException folder.
  
−
commissionRates.csv - containts commission rates. All the content except the first line would be transfered into an Stardust process data, named CommissionRates.
+
'''commissionRates.csv''' - containts commission rates. All the content except the first line would be transfered into an Stardust process data, named CommissionRates.
 
<source lang="javascript">
 
<source lang="javascript">
 
name;code;currency;vatIndicator;amount
 
name;code;currency;vatIndicator;amount
Line 14: Line 14:
 
'''Process Definition of LoadCommissionRates -'''  
 
'''Process Definition of LoadCommissionRates -'''  
  
−
[[File:Example.jpg|thumbnail]]
+
[[File:loadCommission.JPG]]
  
−
commissionRatesException.csv - per exception a seperate process instance of Stardust process will be intiated.
+
'''commissionRatesException.csv''' - per exception a seperate process instance of Stardust process will be intiated.
 
<source lang="javascript">
 
<source lang="javascript">
 
A;B;USD;Y;123;AAAAAAAAAAAAAAAAAAAAAAAAA
 
A;B;USD;Y;123;AAAAAAAAAAAAAAAAAAAAAAAAA
Line 22: Line 22:
 
</source>
 
</source>
 
Process named '''CommissionExceptionMgmt''' will be triggered on arrival of this file (as many instances of the process as the number of rows in this file).
 
Process named '''CommissionExceptionMgmt''' will be triggered on arrival of this file (as many instances of the process as the number of rows in this file).
 +
 +
'''Process Definition of CommissionExceptionMgmt -'''
 +
 +
[[File:commissionExceptionMgmnt.JPG]]
  
 
== How do we achieve this? ==
 
== How do we achieve this? ==
Line 69: Line 73:
 
</source>
 
</source>
 
* Click on Paramters Tab and map data as shown below:
 
* Click on Paramters Tab and map data as shown below:
 +
[[File:commissionRates.JPG]]
  
−
Data Definition of CommissionRates:
+
* Data Definition of CommissionRates:
−
 
+
[[File:commissionRateData.JPG]]
  
 
'''For Case 2:'''
 
'''For Case 2:'''
Line 112: Line 117:
  
 
* Click on Paramters Tab and map data as shown below:
 
* Click on Paramters Tab and map data as shown below:
−
 
+
[[File:commissionException.JPG]]
  
 
* Data Definition of ComssionException:
 
* Data Definition of ComssionException:
 +
[[File:commissionExceptionData.JPG]]
  
 
<big>''The current use case explains to trigger processes on the arrival of file(s) to a folder on local file system. Just change the first route as shown below to respond to file(s) arrival at an FTP location:''</big>
 
<big>''The current use case explains to trigger processes on the arrival of file(s) to a folder on local file system. Just change the first route as shown below to respond to file(s) arrival at an FTP location:''</big>
Line 120: Line 126:
 
'''For Case 1''' -  
 
'''For Case 1''' -  
 
<source lang="javascript">
 
<source lang="javascript">
−
<from uri="file://c:\temp\temp.csv?fileName=commissionRates.csv&initialDelay=5000&delay=5000&readLock=markerFile"/>
+
<from uri="ftp://motu@localhost/FtpFeed/comRate?password=motu&amp;stepwise=false&amp;delay=5000&amp;delete=true"/>
 
</source>
 
</source>
 
'''For Case 2''' -  
 
'''For Case 2''' -  
 
<source lang="javascript">
 
<source lang="javascript">
−
<from uri="file://c:\temp\temp9.csv?fileName=commissionRatesException.csv&initialDelay=5000&delay=5000&readLock=markerFile"/>
+
<from uri="ftp://motu@localhost/FtpFeed/comException?password=motu&amp;stepwise=false&amp;delay=5000&amp;delete=true"/>
 
</source>
 
</source>
 
== Runtime behavior: ==
 
== Runtime behavior: ==
 
'''File commissionRates.csv arrives at destination folder:'''
 
'''File commissionRates.csv arrives at destination folder:'''
  
−
 
+
[[File:commRateValidation.JPG]]
  
 
'''File commissionRatesException.csv arrives at destination folder:'''
 
'''File commissionRatesException.csv arrives at destination folder:'''
  
−
Please find the model here.
+
[[File:worklist.JPG]]
 +
 
 +
'''2 instance becasue we have 2 rows in the file'''
 +
[[File:commExcpt1.JPG]]
 +
 
 +
[[File:commExcpt2.JPG]]
 +
 
 +
Please find the model here [[File:SBBCommissionCapture.zip]].
 +
 
 +
 
 +
<big>In the example the files have ';' as delimiter. Incases you have other delimiter characters, please change the route section below accordingly:</big>
 +
 
 +
<source lang="javascript">
 +
<unmarshal>
 +
<csv skipFirstLine="true"  '''delimiter=";"'''/>
 +
</unmarshal>
 +
..
 +
..
 +
..
 +
</source>

Revision as of 05:24, 19 March 2014

Use Case

Consuming a CSV file (delimter could be anything) arriving at either an FTP location or in a folder on local file system. E.g. a csv file arriving at:

  • C:\temp\temp.csv named temp.csv, or
  • /FtpFeed/comException folder.

commissionRates.csv - containts commission rates. All the content except the first line would be transfered into an Stardust process data, named CommissionRates. 

name;code;currency;vatIndicator;amount
A;B;USD;Y;123
A1;B1;HKD;N;11123

Process named LoadCommissionRates will be trigerred on arrival of this file (only 1 process instance to load entire file into Stardust process).

Process Definition of LoadCommissionRates -

LoadCommission.JPG

commissionRatesException.csv - per exception a seperate process instance of Stardust process will be intiated. 

A;B;USD;Y;123;AAAAAAAAAAAAAAAAAAAAAAAAA
A1;B1;HKD;N;11123;BBBBBBBBBBBBBBBBBBBBBBBBB

Process named CommissionExceptionMgmt will be triggered on arrival of this file (as many instances of the process as the number of rows in this file).

Process Definition of CommissionExceptionMgmt -

CommissionExceptionMgmnt.JPG

How do we achieve this?

We will use Stardust's Generic Camel Route Event for this purpose.

Note: Please change your profile from Business Analyst to Integrator in Business Process Modelling perspective.

For Case 1:

  • Create Start Event, change its type from None to Message and then add the below route in the Configuration Tab:
<from uri="file://c:\temp\temp.csv?fileName=commissionRates.csv&amp;initialDelay=5000&amp;delay=5000&amp;readLock=markerFile"/>
<unmarshal>
<csv skipFirstLine="true"  delimiter=";"/>
</unmarshal>
<setHeader headerName="CamelLanguageScript">
<constant>
<![CDATA[
importClass(java.util.HashMap);
importClass(java.util.ArrayList);
var results=new HashMap();
var commissionRates=new ArrayList();
 
var lines=request.body;
for (var i = 0; i < lines.size(); i++) {
 
    var line = lines.get(i);
    print("----"+line );
    print("****length="+line.size());
    var commissionRate= new HashMap();
 
    commissionRate.put("name",line.get(0));
    commissionRate.put("code", line.get(1));
    commissionRate.put("currency",line.get(2));
    commissionRate.put("vatIndicator",line.get(3));
    commissionRate.put("amount",line.get(4));
 
    commissionRates.add(commissionRate);   
 
} 
results.put("commissionRates",commissionRates);
exchange.out.body=results;
]]>
</constant>
</setHeader>
<to uri="language:javascript"/>
<to uri="ipp:direct"/>
  • Click on Paramters Tab and map data as shown below:

CommissionRates.JPG

  • Data Definition of CommissionRates:

CommissionRateData.JPG

For Case 2:

  • Create Start Event, change its type from None to Message and then add the below route in the Configuration Tab:
<from uri="file://c:\temp\temp9.csv?fileName=commissionRatesException.csv&amp;initialDelay=5000&amp;delay=5000&amp;readLock=markerFile"/>
 <split>
 <tokenize token="\n" />
 <unmarshal>
<csv  delimiter=";"/>
</unmarshal>
<setHeader headerName="CamelLanguageScript">
<constant>
<![CDATA[
importClass(java.util.HashMap);
importClass(java.util.ArrayList);
var results=new HashMap();
var line=request.body.get(0);
print("----"+line );
print("****length="+line.size());
var commissionEx= new HashMap();
 
    commissionEx.put("name",line.get(0));
    commissionEx.put("code", line.get(1));
    commissionEx.put("currency",line.get(2));
    commissionEx.put("vatIndicator",line.get(3));
    commissionEx.put("amount",line.get(4));
    commissionEx.put("exception",line.get(5));
 
results.put("commissionException",commissionEx);
exchange.out.body=commissionEx;
]]>
</constant>
</setHeader>
<to uri="language:javascript"/>
<to uri="ipp:direct"/>
</split>

Note: Please mark the use of split and route ipp:direct inside it. This is the reason to start as many process instance as the number of rows in the file.

  • Click on Paramters Tab and map data as shown below:

CommissionException.JPG

  • Data Definition of ComssionException:

CommissionExceptionData.JPG

The current use case explains to trigger processes on the arrival of file(s) to a folder on local file system. Just change the first route as shown below to respond to file(s) arrival at an FTP location:

For Case 1 - 

<from  uri="ftp://motu@localhost/FtpFeed/comRate?password=motu&amp;stepwise=false&amp;delay=5000&amp;delete=true"/>

For Case 2 - 

<from  uri="ftp://motu@localhost/FtpFeed/comException?password=motu&amp;stepwise=false&amp;delay=5000&amp;delete=true"/>

Runtime behavior:

File commissionRates.csv arrives at destination folder:

CommRateValidation.JPG

File commissionRatesException.csv arrives at destination folder:

Worklist.JPG

2 instance becasue we have 2 rows in the file CommExcpt1.JPG

CommExcpt2.JPG

Please find the model here File:SBBCommissionCapture.zip.


In the example the files have ';' as delimiter. Incases you have other delimiter characters, please change the route section below accordingly: 

<unmarshal>
<csv skipFirstLine="true"  '''delimiter=";"'''/>
</unmarshal>
..
..
..
Retrieved from "https://wiki.eclipse.org/index.php?title=Stardust/Knowledge_Base/Integration/Camel/Consuming_A_CSV_File_-_at_FTP_or_Local_FileSystem&oldid=358426"

Back to the top