- This topic has 4 replies, 1 voice, and was last updated 9 years, 1 month ago by
.
-
Posts
-
-
Ok.. I am trying to find out how to determine the maximum message throughput on BizTalk 2006. I know if will vary based on machine type, message size, etc. But I am looking for a resource that will tell me in general any of the following.
1. Maximum throughput at highest level possible. Theoretical limit?
2. How to calculate how many messages can be processed by a given setup.
Any information on this would be appreciated. Right now I can not push enough records.. but I am off by days vs hours of processing time. I am trying to determine if BizTalk can even handle it.
-
Well, I don’t know any way to really calculate it. It all depends on all the things you just said.
I have seen BizTalk process over 900 messages per second in a lab with a whole ton of servers pushing messages. We got about 100 message / sec on the receive side and 150 message / sec on the send.
I’d take a look at how you can scale BizTalk to meet your needs based on any actual results you might have now. You also might want to look at how you can adjust BizTalk settings (like water marks, hosts, etc) to perform best for your solution.
Hope this helps.
-
I am still in testing stages on a lab server trying to build an application. Presently I am looking at the possibility (extremely high one at that) of pulling upto 9 Millions records. I will be pulling them from a SQL Server database, applying business rules, and then exporting them as a single file (or set of a dozen broken by a code) to CSV style format.
Is BizTalk built to even come close to pushing 5-10 million records from a SQL Server to a Flat File in say 2-3 hours?
I know there are a lot of unknowns… but in theory is it possible? Are you likely talking 1 server, 10 servers? I will not hold anyone to it… just trying to evaluate and decide if other options than BizTalk should be persued.
Thanks.
-
10 million records in 3 hours = 925 records per second.
To achieve some sort of parallel processing you will need to split the SQL message in the pipeline, process in multiple orchestrations and append the output records into a file. The receive port and send port will be single threaded, but the orchestrations will multi-thread.
If you require the order of records to be preserved, then you will need to single thread the entire process.
Personally I would look at using SSIS, Biztalk introduces a lot of overhead that you do not need for this process.
-
That is my thought. I am from the SQL Server world and BizTalk is new to me, so I immediatly think to use DTS/SSIS. But I was trying stay to one technology and take advantage of BizTalk technologies like changing output formats easily and applying business rules. But in the sample runs I have done I see a 9 million row export taking well over a 2-3 hours in time and I have not even applied business rules yet in my orch.
I am running a single high end server, and was starting to wonder if BizTalk just can't handle that kind of volume. If a 100 messages a second is normal processing then I can use that as a base calculation… and show that even in doubling the throughput it still has a performance issue.
-
-
-
-
- The forum ‘BizTalk 2004 – BizTalk 2010’ is closed to new topics and replies.