Large Messages Can Cause Large Problems in BizTalk

Recently I have seen a lot of talk about large message support in BizTalk 2004.  The most interesting and depressing was on the BizTalk Core Engine WebLog.  It seems that large messages do not work as well in BizTalk 2004 as I would have hoped.

I have had success with some types of large messages using the file adapter and mapping.  I have gotten it to work with messages up to 500 MB but in a production environment this would not be a viable or supported solution. 

During some recent testing, I discovered the follow condition that rather surprised me.


I was returning an unpredictable amount of data out of an Oracle database using .net code.  This data can be 1 MB or up to 250MB+ and needed to be processed in a single batch.  This data is then converted into Xml, cast to a message, and mapped with the results written to disk using a Send Shape.  Life was good until the extracted data returned was greater then 220 MB in size.


At this point, everything went downhill.  BizTalk Server began to have serious problems.  Receive functions were shutting down and I received the following Warning message in the Application Log:

A catastrophic failure occurred in the BizTalk service. The service will shutdown and auto-restart in 1 minute. If the database is still unavailable, this cycle will be repeated.


Error message: Exception of type System.OutOfMemoryException was thrown.

Error source:


BizTalk host name: BizTalkServerApplication Windows service name: BTSSvc{E93E7C4A-9E8B-40F6-AD7C-4F53B85F36C4}


For more information, see Help and Support Center at

All appears to be well and good since BizTalk seems to be handling the situation as expected… except for when the service actually starts again.  Since the Orchestration that caused the fatal error is still Active (the Orchestration itself did not through any errors), BizTalk starts to process the Orchestration from the last persistence point.  Thus, repeating the cycle in an endless loop.

Now, for the interesting behavior…  If this is running in a multi server BizTalk environment at the point the 1st BizTalk Server fails the 2nd one picks up; like I would expect.  I end up getting a completed Orchestration with no errors.  Super!  But, I only get a zero byte file as output.  So, looking inside HAT everything would look like it completed successfully with no errors.  Not good…

Overall, I found that an Orchestration seems to be able to handle about 440 MB of “stuff” before it crashes.  Just make sure you know what makes a copy of the message (like mapping).  Of course, processes with this amount of data would not be supported nor do I recommend actually doing it.  But, it has been fun to play around with and see what fails.

The moral of this story is to be very mindful of your message size.

Large Messages and Large Maps in BizTalk

Updated 8/8/2005:
Please note the information below is outdated and I would never recommend mapping messages of this size inside Biztalk 2004.  Biztalk 2006 will have support for larger messages by breaking up the message into chunks.  More information on large message support can be found on the Performance Blog.


Original Post:


Ok, I hear it all the time: “Can BizTalk handle large files?” 

But, really the question should be: “How large of a file do you want BizTalk to process?”

Large is a subjective term.  For the purposes of this post, large is a 150 MB XML Document.  I see lots of blogs with negative posts on big messages in BizTalk 2004.  My experiences have been quite the opposite.  But, I’ll let you make the determination yourself.  Here are my test results.

My Test Computer: P4 2.4 GHz with 1.25 GB Ram.


CRITICAL: Working with large files such as this will consume a HUGE amount of system resources.  My CPU was maxed at 100% and RAM utilization was extensive.  A better approach is to break the large file down, but that is a subject for another post coming soon…

Sample File Size: 180 MB

Schema Nodes: Around 400

All Mapping on Send Port, map size is around 1.2 MB

Test1: Straight line mapping to itself

Run Time: 7 minutes 13 seconds

Test2: Straight line mapping to itself

Functoids: Around 80 String Manipulation

Run Time: 8 minutes 26 seconds

Test3: Straight line mapping to itself

Functoids (over 250):

          Around 80 String Manipulation

          Around 50 Logical

          Around 50 Value Mapping

          Around 80 Scripting with calls to an external assembly

Run Time: 9 minutes 42 seconds

Take Away: I was surprised and impressed with the processing time of large files inside BizTalk.  But, I’ll let you make the final determination…