Large Messages Can Cause Large Problems in BizTalk
Recently I have seen a lot of talk about large message support in BizTalk 2004. The most interesting and depressing was on the BizTalk Core Engine WebLog. It seems that large messages do not work as well in BizTalk 2004 as I would have hoped.
I have had success with some types of large messages using the file adapter and mapping. I have gotten it to work with messages up to 500 MB but in a production environment this would not be a viable or supported solution.
During some recent testing, I discovered the follow condition that rather surprised me.
Scenario
I was returning an unpredictable amount of data out of an Oracle database using .net code. This data can be 1 MB or up to 250MB+ and needed to be processed in a single batch. This data is then converted into Xml, cast to a message, and mapped with the results written to disk using a Send Shape. Life was good until the extracted data returned was greater then 220 MB in size.
At this point, everything went downhill. BizTalk Server began to have serious problems. Receive functions were shutting down and I received the following Warning message in the Application Log:
A catastrophic failure occurred in the BizTalk service. The service will shutdown and auto-restart in 1 minute. If the database is still unavailable, this cycle will be repeated.
Error message: Exception of type System.OutOfMemoryException was thrown.
Error source:
BizTalk host name: BizTalkServerApplication Windows service name: BTSSvc{E93E7C4A-9E8B-40F6-AD7C-4F53B85F36C4}
For more information, see Help and Support Center at
All appears to be well and good since BizTalk seems to be handling the situation as expected… except for when the service actually starts again. Since the Orchestration that caused the fatal error is still Active (the Orchestration itself did not through any errors), BizTalk starts to process the Orchestration from the last persistence point. Thus, repeating the cycle in an endless loop.
Now, for the interesting behavior… If this is running in a multi server BizTalk environment at the point the 1st BizTalk Server fails the 2nd one picks up; like I would expect. I end up getting a completed Orchestration with no errors. Super! But, I only get a zero byte file as output. So, looking inside HAT everything would look like it completed successfully with no errors. Not good…
Overall, I found that an Orchestration seems to be able to handle about 440 MB of “stuff” before it crashes. Just make sure you know what makes a copy of the message (like mapping). Of course, processes with this amount of data would not be supported nor do I recommend actually doing it. But, it has been fun to play around with and see what fails.
The moral of this story is to be very mindful of your message size.