Hi, I'm a newbie at Biztalk.
I have a project that requires me to load data that is stored in an xml file to two tables on a MSSQL server. Now I've created my orchestration and have my receive and set ports all set. The setup works but for very small sets of files. For example, If its processing 100 xml messages, Biztalk is able to process all the information in no time. However, it gets slower and slower when there are more files. For instance if I had 6000 xml messages, it would slow down to a crawl and I estimated it would take a few days to process all the information. My questions are:
1. Why is there such a huge drop in performance when the number of messages are increased?
2. What would be a possible solution to this? I am guaranteed to have close to 30,000 xml messages to process, since they are generated by the HIPAA accelerator which is processing 834 files that we get from our client.
3. Do I need to add some external program to basically move small set of messages for my project to consume? (Because processing of a 1000 messages takes less time than the processing time for 6000 messages)
Oh and just to let you know, the orchestration is a sequential convoy. There is a reason for this because each time my sproc is called, it is inserting the data into two tables (but one table is tied to the other table by its id field). I know this is a slow way but it still does not explain why the performance drops significantly when the number of messages is increased.
Thanks in advance
– George John