- This topic has 9 replies, 1 voice, and was last updated 9 years, 7 months ago by
.
-
Posts
-
-
Hi,
Project BED : Biztalk 2004 ,Oracle , SQLServer 2000,VS.Net 1.1
Project Desc :
Extracting data from a oracle table from one end and ftping it to the other end.Here one more orchestration picks this up and inserts the same into another oracle db table.
The data fetching is done at a particular time which is scheduled through services.
Here extracting and creating csv is done by an .Net assembly.Biztalk just ftp’s the xml to Other end. the Other end orch just picks up and calls another external assembly to insert the rec.
We run nearly 60 instances like this.
I need a better architecture for the same. I would like to convert this to BTS 2006. Please advice on the benefits I will get by converting this to BTS 2006. Also advice on re-Architecting the same.
regards
VJ
-
VJ,
I have a pretty good idea of one possible solution, however a couple of clarification points if you will:
- Is there a reason why you need to FTP the source data to BizTalk or could that step be removed?
- You mention that the extract generates a CSV file, but then mention that Xml is FTP’d to the ‘ther end’. Can you clarify whether BizTalk receives a CSV or Xml file?
Rgds, Nick.
-
1. The data from Loc A to Loc B. Both Loc are in diff n/w with quite a large distance.
2. CSV is picked by BTS and through custom pipeline converted to xml
Eagerely waiting for a solution
Thanks
VJ
-
-
Hi – I have a problem, although I am not even sure it is a problem. I’ll provide you with inconsistent details about the project, and no explantion of what I hope to improve or gain. Can you please tell me what to do?
I’m waiting….
-
-
-
Given what you’ve explained, I don’t see a large benefit from using BizTalk2006. If you do not need the FTP or Oracle Adapters, 2004 would work.
You might see better performance and you would have a better deployment / admin story in 2006 – but conceptually you’ll still be doing the same thing.
If you wanted to remove the services and get BizTalk to do more for the work I can see larger benefit in 2006.
But I’m a fan of if it isn’t broken don’t fix it.
-
-
-
-
-
-
VJ,
I’m going to split the proposed architecture into two sections, a poll and publish:
Receive Data from Oracle Database
I see two possible options here:
1. Use your existing .Net assembly (I’m presuming a Windows Service) to poll the Oracle database, generate and FTP the CSV file to the remote BizTalk server for later processing.
or
2. Use BizTalk with the Scheduling Adapter (see http://www.codeplex.com/BizTalkScheduledTask) to poll the Oracle database at specified intervals and transform the message returned by the Oracle adapter into a CSV file and FTP to the remote BizTalk server for later processing.
I think that FTP-ing a CSV file at this point is beneficial as you will reduce the total size of the file being transferred over FTP when compared to Xml. Furthermore, if you plan on using BizTalk as the poller, I would suggest you look at either Branch or Standard Edition to reduce licencing costs (although you would lose relisiency).
Publish Data to Remote Oracle Database
Given that you are looking to have a single controller handle the 60 message types, I would suggest a simple custom ESB implementation. The CSV file can be disassembled and mapped (on the receive port) to a common (or set of common) canonical message formats that can be handled together rather than individually, allowing you to have just one orchestration (or maybe a small set of orchestrations) that handle message publishing to the destination Oracle database.
The CSV file would be received and disassembled (preferably generating a single Xml message per source database record, to ensure small units of work) and the resulting messages published to the Message Box. Using direct binding, the relevant orchestration would collect the message and promote routing information to aid subscription by the send ports (allowing the send port inserting records into table A to collect messages for table A and not table B for example) – this routing information could be based on message content and facts stored in the Business Rules Engine for example. The orchestration would then publish the message back to the Message Box via direct binding, allowing the relevant Oracle insertion send port to subscribe to the message based on its routing information. Given that the orchestration will be as generic as possible, the Send port will need to map (on the outbound stage of the port) from the orchestration canonical format to the format required by the Oracle adapter to insert into the relevant table. With Delivery Notification enabled, the orchestration will be notified of failed insertions and compensate accordingly.
I hope this helps get you started – the original architecture was almost there so only needed a little tweaking.
Rgds, Nick.
-
-
- The forum ‘BizTalk 2004 – BizTalk 2010’ is closed to new topics and replies.