Home Page › Forums › BizTalk 2004 – BizTalk 2010 › Flat File Messages Processing Slow in File Adapter
- This topic has 4 replies, 1 voice, and was last updated 9 years, 9 months ago by
community-content.
-
AuthorPosts
-
-
November 12, 2009 at 11:07 AM #23761
I have a very large flat file (15MB, 125,000 records in 500K lines). My file receive adapter has a flat file disassembler pipeline to publish the messages from this file to the MsgBox. I stopped the send port that subscribes to the messages, so I can get a read on how fast the messages are being created and sent to the MsgBox.
This occurs on a test server that has BT 2006 r2 and SQL Server 05 installed. The server is Win Svr Std 03, 64 bit (SP2) and has 4 processors, Xeon 3.40 GHz with 12 GB of ram.
I’ve worked very hard with the BT Group properties (Large File threshold & fragments) and the throttling params for the Receive Host, but cannot get the throughput on this file to get above 122 messages/sec (PerfMon’s Msg publishing incoming rate and outgoing rate counters). Watching these two PerfMon counters, I will see both of these drop to zero frequently and stay there for several seconds.
Can you suggest anything else I could be looking at? I think with this kind of horsepower, there should be more throughput, despite having BT and SQL on the same server.
Am I just wrong on this assumption? And is 122 msgs/sec about all that I can expect?
-
November 12, 2009 at 12:02 PM #23762
Hi JLingenfelder,
Consult the following MSDN article on best practices to increase performance of BizTalk Server in processing large messages:
http://technet.microsoft.com/en-us/library/aa560481(BTS.20).aspx
You probably have done some of the things mentioned in the article, but it’s always good to have another look so as not to overlook any options.
Daniel.
-
November 12, 2009 at 1:28 PM #23763
Thanks, but “OH, YEAH” — I’ve been through that page several times.
The question relates more to the throughput and if there is some registry setting that may improve the performance here. There is a good discussion about throttling at http://msdn.microsoft.com/en-us/library/aa559628(BTS.10).aspx but it has not helped in this case.
-
-
November 12, 2009 at 1:39 PM #23764
One other piece of info. The performance counter “In-process message count” stays at 0 (zero) and the “In-process message count threshold” is set to 1000.
-
November 16, 2009 at 8:25 PM #23781
If you have followed on processing large messages: http://technet.microsoft.com/en-us/library/aa560481(BTS.20).aspx
then you need to fine tune your system. As you are using BizTalk server & SQL server on the same machine this can be a one performance hit.
You can try with the following performance counters what I have successfully used to fine tune the system.
- 1. Process Memory
(Performance counter located inside the BizTalk:Message Agent performance object)
The default Process memory usage setting is set to 25. Because your system has 12 GB RAM.
So using the defaults, when a BizTalk host operating system process uses more than 3 GB of
memory (25 percent of 12GB), process memory throttling is introduced, which will restrict the size of
the thread pool and/or introduce a progressive delay to message publishing. So you can increase
percentage & see the throughput. Apply only one threshold change at once & fine tune the system.
- 2. Database Session Count –
Database session count throttling indicates that too many concurrent database connections
have been opened by the BizTalk host. The number of database sessions currently open is
viewable via the Database session performance counter located in the BizTalk:Message Agent
performance object.
The default Database connections per CPU setting are set to 0, which means that this particular
type of throttling is disabled. So can have infinite database connections. You can give 6,7,8 & see
the performance . For our tune-up system it took “8” .
Do test and see what happened.
-
-
-
AuthorPosts
- The forum ‘BizTalk 2004 – BizTalk 2010’ is closed to new topics and replies.