Home Page › Forums › BizTalk 2004 – BizTalk 2010 › Biztalk performance issue
- This topic has 3 replies, 1 voice, and was last updated 9 years, 6 months ago by
community-content.
-
AuthorPosts
-
-
February 26, 2007 at 5:01 AM #17792
Hi, I'm a newbie at Biztalk.
I have a project that requires
me to load data that is stored in an xml file to two tables on a MSSQL
server. Now I've created my orchestration and have my receive and set
ports all set. The setup works but for very small sets of files. For
example, If its processing 100 xml messages, Biztalk is able to process
all the information in no time. However, it gets slower and slower when
there are more files. For instance if I had 6000 xml messages, it would
slow down to a crawl and I estimated it would take a few days to
process all the information. My questions are:
1. Why is there such a huge drop in performance when the number of messages are increased?2.
What would be a possible solution to this? I am guaranteed to have
close to 30,000 xml messages to process, since they are generated by
the HIPAA accelerator which is processing 834 files that we get from
our client.3. Do I need to add some external program to
basically move small set of messages for my project to consume?
(Because processing of a 1000 messages takes less time than the
processing time for 6000 messages)
Oh and just to let you
know, the orchestration is a sequential convoy. There is a reason for
this because each time my sproc is called, it is inserting the data
into two tables (but one table is tied to the other table by its id
field). I know this is a slow way but it still does not explain why the
performance drops significantly when the number of messages is
increased.Thanks in advance
– George John
-
February 27, 2007 at 9:59 AM #17806
George,
Let's start with this – To answer #1, BizTalk is writing the messages to the Message Box db in BizTalk. From what I've seen (and I'm no expert in BT) it consumes the input files first before it does any writing. In my app I ran into the same problem as I had been testing with batches of 20-100 orders and everything was cool. I fed it an XML with 5800 orders and it went into a coma.
You don't indicate how much ram you have, but our test server has 4gb and that makes all the difference in the world compared to my dev system with 2gb. Even though it takes longer, on the development side I will test with small batches on the dev side, then migrate the whole mess to the test server to test the big batches. Other than pegging the CPU it does reasonably well.
Hope this helps.
-
March 2, 2007 at 8:25 AM #17843
Thanks a bunch Henry, this actually makes sense. The machine I'm doing this on had 1gb of ram, we upgraded it to 3gb and it improved it a little but not by much. It still bogs down when you have too many messages, almost feels like there is something that needs to be done to the database to optimize it.
-
March 7, 2007 at 8:49 AM #17883
Are you using tables (e.g. lookup tables) other than the ones BT installs?
-
-
-
-
AuthorPosts
- The forum ‘BizTalk 2004 – BizTalk 2010’ is closed to new topics and replies.