VFS transport
feature in ESB can be used to process the files in the system
periodically and take actions as configured. Basically you can use
this to transfer file from one location to other with the mediation
in between.
Below example shows
how to process simple csv flat file in a folder and process each
record inside ESB proxy service and forward for further processing
into a separate sequence.
We will be using
smooks mediator to read csv records from the file and load those into
the proxy payload as xml records.
Step 1: Download and install WSO2 ESB
- You can get the latest installation from ( http://wso2.com/more-downloads/esb )
- Also follow the documentation from https://docs.wso2.org/dashboard.action
Step 2: Enable VFS transport on ESB and enable JMS transport.
- Uncomment the below two lines from /<WSO2_ESB>/repository/conf/axis2/axis2.xml
<transportSender
name="vfs"
class="org.apache.synapse.transport.vfs.VFSTransportSender"/>
<transportReceiver
name="vfs"
class="org.apache.synapse.transport.vfs.VFSTransportListener"/>
|
Step 3: Add local entry to refer smooks configuration file
- Go to the WSO2 ESB Management console ( https://localhost:9443/carbon )
- Select “Local Entries” from the left panel and add new “Source URL Entry” with below values
- Name : smook_config
- URL : file:repository/resources/smooks/smooks-config-mapping.xml
- Also add copy smook-config-mapping.xml file to the <WSO2_ESB>/repository/resources/smooks/ folder
smook-config-mapping.xml
<smooks-resource-list
xmlns="http://www.milyn.org/xsd/smooks-1.0.xsd">
<!--Configure
the CSVParser to parse the message into a stream of SAX events.
-->
<resource-config
selector="org.xml.sax.driver">
<resource>org.milyn.csv.CSVParser</resource>
<param
name="fields"
type="string-list">firstname,lastname,gender,age,country</param>
</resource-config>
</smooks-resource-list>
|
Step 4: Create Proxy as VFS Listener and Sequence to process each record
Add Sequence to used by proxy to forward the each record:
<?xml
version="1.0" encoding="UTF-8"?>
<sequence
xmlns="http://ws.apache.org/ns/synapse"
name="RecProcessSequence">
<log
level="full">
<property
name="==Processing record =====" value="=="/>
</log>
</sequence>
|
Add the Proxy :
This Listner proxy
will read the files from the input path and load into the proxy
payload and iterate each record and call sequence
(RecProcessSequence) to process each record further.
<?xml
version="1.0" encoding="UTF-8"?>
<proxy
xmlns="http://ws.apache.org/ns/synapse"
name="TestSmookRecVFS"
transports="vfs"
startOnLoad="true"
trace="disable">
<description/>
<target>
<inSequence>
<smooks
config-key="smook_config">
<input
type="text"/>
<output
type="xml"/>
</smooks>
<iterate
xmlns:ns2="http://org.apache.synapse/xsd"
xmlns:ns="http://org.apache.synapse/xsd"
xmlns:sec="http://secservice.samples.esb.wso2.org"
expression="//csv-set/csv-record">
<target>
<sequence>
<clone>
<target sequence="RecProcessSequence"/>
</clone>
</sequence>
</target>
</iterate>
</inSequence>
</target>
<parameter
name="transport.vfs.ActionAfterProcess">MOVE</parameter>
<parameter
name="transport.PollInterval">10</parameter>
<parameter
name="transport.vfs.MoveAfterProcess">file:///home/jayalal/Work/SmookRes/out</parameter>
<parameter
name="transport.vfs.FileURI">file:///home/jayalal/Work//SmookRes/in</parameter>
<parameter
name="transport.vfs.MoveAfterFailure">file:///home/jayalal/Work/
/SmookRes/failed</parameter>
<parameter
name="transport.vfs.FileNamePattern">.*.txt</parameter>
<parameter
name="transport.vfs.ContentType">text/plain</parameter>
<parameter
name="transport.vfs.ActionAfterFailure">MOVE</parameter>
</proxy>
|
Step 5: Testing and Verify
Copy the input.txt
file into the “transport.vfs.FileURI” folder.
name1,lastname1,Male,30,country1
name2,lastname2,Female,40,country2
name3,lastname3,Female,40,country3
name4,lastname4,Female,30,country4
name5,lastname5,Female,40,country5
|
You will see the ESB logs will print the messages process by ESB.
[2014-01-29
15:11:41,922] INFO - LogMediator To: , WSAction: urn:mediate,
SOAPAction: urn:mediate, MessageID:
urn:uuid:de01ae68-6afc-4539-9c0e-f820fa181c19, Direction: request,
==Processing record ===== = ==, Envelope: <?xml version='1.0'
encoding='utf-8'?><soapenv:Envelope
xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><csv-record
number="1"><firstname>name1</firstname><lastname>lastname1</lastname><gender>Male</gender><age>30</age><country>SriLanka</country></csv-record></soapenv:Body></soapenv:Envelope>
[2014-01-29 15:11:41,924] INFO - LogMediator
To: , WSAction: urn:mediate, SOAPAction: urn:mediate, MessageID:
urn:uuid:b436bd8d-a30b-40e1-83c2-00f109b793a5, Direction: request,
==Processing record ===== = ==, Envelope: <?xml version='1.0'
encoding='utf-8'?><soapenv:Envelope
xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><csv-record
number="2"><firstname>name2</firstname><lastname>lastname2</lastname><gender>Female</gender><age>40</age><country>SriLanka</country></csv-record></soapenv:Body></soapenv:Envelope>
|
More References:
Thank you for your post. Now I need to get further. I want to process this CSV File, and send it as a SOAP Request to a external webservice.
ReplyDeleteDo you have a starting point to guide me? Thank you.