Notice: This Wiki is now read only and edits are no longer possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Difference between revisions of "Stardust/Knowledge Base/Integration/Camel/Consuming A CSV File - at FTP or Local FileSystem"
(Created page with "== Use Case == Consuming a CSV file (delimter could be anything) arriving at either an FTP location or in a folder on local file system. E.g. a csv file arriving at: * C:\temp...") |
|||
Line 4: | Line 4: | ||
* /FtpFeed/comException folder. | * /FtpFeed/comException folder. | ||
− | commissionRates.csv - containts commission rates. All the content except the first line would be transfered into an Stardust process data, named CommissionRates. | + | '''commissionRates.csv''' - containts commission rates. All the content except the first line would be transfered into an Stardust process data, named CommissionRates. |
<source lang="javascript"> | <source lang="javascript"> | ||
name;code;currency;vatIndicator;amount | name;code;currency;vatIndicator;amount | ||
Line 14: | Line 14: | ||
'''Process Definition of LoadCommissionRates -''' | '''Process Definition of LoadCommissionRates -''' | ||
− | [[File: | + | [[File:loadCommission.JPG]] |
− | commissionRatesException.csv - per exception a seperate process instance of Stardust process will be intiated. | + | '''commissionRatesException.csv''' - per exception a seperate process instance of Stardust process will be intiated. |
<source lang="javascript"> | <source lang="javascript"> | ||
A;B;USD;Y;123;AAAAAAAAAAAAAAAAAAAAAAAAA | A;B;USD;Y;123;AAAAAAAAAAAAAAAAAAAAAAAAA | ||
Line 22: | Line 22: | ||
</source> | </source> | ||
Process named '''CommissionExceptionMgmt''' will be triggered on arrival of this file (as many instances of the process as the number of rows in this file). | Process named '''CommissionExceptionMgmt''' will be triggered on arrival of this file (as many instances of the process as the number of rows in this file). | ||
+ | |||
+ | '''Process Definition of CommissionExceptionMgmt -''' | ||
+ | |||
+ | [[File:commissionExceptionMgmnt.JPG]] | ||
== How do we achieve this? == | == How do we achieve this? == | ||
Line 69: | Line 73: | ||
</source> | </source> | ||
* Click on Paramters Tab and map data as shown below: | * Click on Paramters Tab and map data as shown below: | ||
+ | [[File:commissionRates.JPG]] | ||
− | Data Definition of CommissionRates: | + | * Data Definition of CommissionRates: |
− | + | [[File:commissionRateData.JPG]] | |
'''For Case 2:''' | '''For Case 2:''' | ||
Line 112: | Line 117: | ||
* Click on Paramters Tab and map data as shown below: | * Click on Paramters Tab and map data as shown below: | ||
− | + | [[File:commissionException.JPG]] | |
* Data Definition of ComssionException: | * Data Definition of ComssionException: | ||
+ | [[File:commissionExceptionData.JPG]] | ||
<big>''The current use case explains to trigger processes on the arrival of file(s) to a folder on local file system. Just change the first route as shown below to respond to file(s) arrival at an FTP location:''</big> | <big>''The current use case explains to trigger processes on the arrival of file(s) to a folder on local file system. Just change the first route as shown below to respond to file(s) arrival at an FTP location:''</big> | ||
Line 120: | Line 126: | ||
'''For Case 1''' - | '''For Case 1''' - | ||
<source lang="javascript"> | <source lang="javascript"> | ||
− | <from uri=" | + | <from uri="ftp://motu@localhost/FtpFeed/comRate?password=motu&stepwise=false&delay=5000&delete=true"/> |
</source> | </source> | ||
'''For Case 2''' - | '''For Case 2''' - | ||
<source lang="javascript"> | <source lang="javascript"> | ||
− | <from uri=" | + | <from uri="ftp://motu@localhost/FtpFeed/comException?password=motu&stepwise=false&delay=5000&delete=true"/> |
</source> | </source> | ||
== Runtime behavior: == | == Runtime behavior: == | ||
'''File commissionRates.csv arrives at destination folder:''' | '''File commissionRates.csv arrives at destination folder:''' | ||
− | + | [[File:commRateValidation.JPG]] | |
'''File commissionRatesException.csv arrives at destination folder:''' | '''File commissionRatesException.csv arrives at destination folder:''' | ||
− | Please find the model here. | + | [[File:worklist.JPG]] |
+ | |||
+ | '''2 instance becasue we have 2 rows in the file''' | ||
+ | [[File:commExcpt1.JPG]] | ||
+ | |||
+ | [[File:commExcpt2.JPG]] | ||
+ | |||
+ | Please find the model here [[File:SBBCommissionCapture.zip]]. | ||
+ | |||
+ | |||
+ | <big>In the example the files have ';' as delimiter. Incases you have other delimiter characters, please change the route section below accordingly:</big> | ||
+ | |||
+ | <source lang="javascript"> | ||
+ | <unmarshal> | ||
+ | <csv skipFirstLine="true" '''delimiter=";"'''/> | ||
+ | </unmarshal> | ||
+ | .. | ||
+ | .. | ||
+ | .. | ||
+ | </source> |
Revision as of 05:24, 19 March 2014
Use Case
Consuming a CSV file (delimter could be anything) arriving at either an FTP location or in a folder on local file system. E.g. a csv file arriving at:
- C:\temp\temp.csv named temp.csv, or
- /FtpFeed/comException folder.
commissionRates.csv - containts commission rates. All the content except the first line would be transfered into an Stardust process data, named CommissionRates.
name;code;currency;vatIndicator;amount A;B;USD;Y;123 A1;B1;HKD;N;11123
Process named LoadCommissionRates will be trigerred on arrival of this file (only 1 process instance to load entire file into Stardust process).
Process Definition of LoadCommissionRates -
commissionRatesException.csv - per exception a seperate process instance of Stardust process will be intiated.
A;B;USD;Y;123;AAAAAAAAAAAAAAAAAAAAAAAAA A1;B1;HKD;N;11123;BBBBBBBBBBBBBBBBBBBBBBBBB
Process named CommissionExceptionMgmt will be triggered on arrival of this file (as many instances of the process as the number of rows in this file).
Process Definition of CommissionExceptionMgmt -
How do we achieve this?
We will use Stardust's Generic Camel Route Event for this purpose.
Note: Please change your profile from Business Analyst to Integrator in Business Process Modelling perspective.
For Case 1:
- Create Start Event, change its type from None to Message and then add the below route in the Configuration Tab:
<from uri="file://c:\temp\temp.csv?fileName=commissionRates.csv&initialDelay=5000&delay=5000&readLock=markerFile"/> <unmarshal> <csv skipFirstLine="true" delimiter=";"/> </unmarshal> <setHeader headerName="CamelLanguageScript"> <constant> <![CDATA[ importClass(java.util.HashMap); importClass(java.util.ArrayList); var results=new HashMap(); var commissionRates=new ArrayList(); var lines=request.body; for (var i = 0; i < lines.size(); i++) { var line = lines.get(i); print("----"+line ); print("****length="+line.size()); var commissionRate= new HashMap(); commissionRate.put("name",line.get(0)); commissionRate.put("code", line.get(1)); commissionRate.put("currency",line.get(2)); commissionRate.put("vatIndicator",line.get(3)); commissionRate.put("amount",line.get(4)); commissionRates.add(commissionRate); } results.put("commissionRates",commissionRates); exchange.out.body=results; ]]> </constant> </setHeader> <to uri="language:javascript"/> <to uri="ipp:direct"/>
- Click on Paramters Tab and map data as shown below:
- Data Definition of CommissionRates:
For Case 2:
- Create Start Event, change its type from None to Message and then add the below route in the Configuration Tab:
<from uri="file://c:\temp\temp9.csv?fileName=commissionRatesException.csv&initialDelay=5000&delay=5000&readLock=markerFile"/> <split> <tokenize token="\n" /> <unmarshal> <csv delimiter=";"/> </unmarshal> <setHeader headerName="CamelLanguageScript"> <constant> <![CDATA[ importClass(java.util.HashMap); importClass(java.util.ArrayList); var results=new HashMap(); var line=request.body.get(0); print("----"+line ); print("****length="+line.size()); var commissionEx= new HashMap(); commissionEx.put("name",line.get(0)); commissionEx.put("code", line.get(1)); commissionEx.put("currency",line.get(2)); commissionEx.put("vatIndicator",line.get(3)); commissionEx.put("amount",line.get(4)); commissionEx.put("exception",line.get(5)); results.put("commissionException",commissionEx); exchange.out.body=commissionEx; ]]> </constant> </setHeader> <to uri="language:javascript"/> <to uri="ipp:direct"/> </split>
Note: Please mark the use of split and route ipp:direct inside it. This is the reason to start as many process instance as the number of rows in the file.
- Click on Paramters Tab and map data as shown below:
- Data Definition of ComssionException:
The current use case explains to trigger processes on the arrival of file(s) to a folder on local file system. Just change the first route as shown below to respond to file(s) arrival at an FTP location:
For Case 1 -
<from uri="ftp://motu@localhost/FtpFeed/comRate?password=motu&stepwise=false&delay=5000&delete=true"/>
For Case 2 -
<from uri="ftp://motu@localhost/FtpFeed/comException?password=motu&stepwise=false&delay=5000&delete=true"/>
Runtime behavior:
File commissionRates.csv arrives at destination folder:
File commissionRatesException.csv arrives at destination folder:
2 instance becasue we have 2 rows in the file
Please find the model here File:SBBCommissionCapture.zip.
In the example the files have ';' as delimiter. Incases you have other delimiter characters, please change the route section below accordingly:
<unmarshal> <csv skipFirstLine="true" '''delimiter=";"'''/> </unmarshal> .. .. ..