Home Page › Forums › BizTalk 2004 – BizTalk 2010 › Message Splitting and Correlation Issue
- This topic has 2 replies, 1 voice, and was last updated 9 years, 10 months ago by
community-content.
-
AuthorPosts
-
-
November 27, 2007 at 11:28 AM #18504
I’m a BizTalk neophyte. This is my first real project using it and I’ve found it to have a fairly steep learning curve. I’m getting comfy with what I’ve learned so far, though.
I have a process that I need to complete. The basic jist of it is this:
1) Client drops a large XML file with multiple records on our FTP server.
2) BTS2006 picks up the file, splits it into lots of individual messages, converts those to XML for SQL2000.
3) BTS2006 sends the single-record XML files, one at a time, to SQL.
4) When the whole original file is done I need to generate a report.
This seems like it should be pretty easy. I have all the XML translation done. I can generate thousands of files from the single original file and convert those to the right XML form for my SQL stored procedure. I can even pass them to SQL. This took me a while but I did get it. It even works if I feed it a small original file.
The probelm comes when I feed it a large file. I start getting SQL locks. Once this starts BT will attempt to resend the XML to SQL a few times but if the file is big enough it just chokes up.
So I started looking into serializing the messages. Which lead me to Correlation and MSMQT. Sounds like a really good idea. Sounds like the perfect solution. I figure I’m golden and I’ll whip this up no problem.
Yeah, we all know how that goes. Here’s where I run into trouble. In order to keep the individual messages from stomping on each other I’m calling pipelines from my orchestration in a loop. This requires me to have an atomic scope. Cool. I can do that. I can’t, however, put my send shape inside the loop. This is because my correlation set would be initialized multiple times – a big no-no.
So here’s the question. While I don’t really care if the individual records are processed in any particular order, I do need them to process one at a time and wait for SQL to respond before moving to the next one. What should I be looking for? What set of tools within BTS?
I’m fairly adept at learning how to use new tools but I generally like to have an example to follow. Unfortunately I haven’t seen an example of some of the stuff I’m trying to do. All I really need is to be pointed in the right direction.
-
November 27, 2007 at 11:43 AM #18505
Just tried some other things as well. I was reading that a scope can have its own correlation sets. Which means that I could put a scope inside the loop and define my correlation set within that scope. That seemed like the answer only now I get a new error:
an atomic scope may not contain a receive with a correlation filtration initialized in the same scope
Remembering that my loop has to exist in an atomic scope, this becomes a problem.
-
November 27, 2007 at 12:02 PM #18506
Huh, I thought I was logged in. I am now. 🙂
-
-
-
AuthorPosts
- The forum ‘BizTalk 2004 – BizTalk 2010’ is closed to new topics and replies.