Biztalk performance issue

Home Page Forums BizTalk 2004 – BizTalk 2010 Biztalk performance issue

Viewing 1 reply thread
  • Author
    Posts
    • #17792

      Hi, I'm a newbie at Biztalk.

       I have a project that requires
      me to load data that is stored in an xml file to two tables on a MSSQL
      server. Now I've created my orchestration and have my receive and set
      ports all set. The setup works but for very small sets of files. For
      example, If its processing 100 xml messages, Biztalk is able to process
      all the information in no time. However, it gets slower and slower when
      there are more files. For instance if I had 6000 xml messages, it would
      slow down to a crawl and I estimated it would take a few days to
      process all the information. My questions are:

       
      1. Why is there such a huge drop in performance when the number of messages are increased?

      2.
      What would be a possible solution to this? I am guaranteed to have
      close to 30,000 xml messages to process, since they are generated by
      the HIPAA accelerator which is processing 834 files that we get from
      our client.

      3. Do I need to add some external program to
      basically move small set of messages for my project to consume?
      (Because processing of a 1000 messages takes less time than the
      processing time for 6000 messages)

       
      Oh and just to let you
      know, the orchestration is a sequential convoy. There is a reason for
      this because each time my sproc is called, it is inserting the data
      into two tables (but one table is tied to the other table by its id
      field). I know this is a slow way but it still does not explain why the
      performance drops significantly when the number of messages is
      increased.

      Thanks in advance

      – George John

    • #17806

      George,

      Let's start with this – To answer #1, BizTalk is writing the messages to the Message Box db in BizTalk.  From what I've seen (and I'm no expert in BT) it consumes the input files first before it does any writing.  In my app I ran into the same problem as I had been testing with batches of 20-100 orders and everything was cool.  I fed it an XML with 5800 orders and it went into a coma.

      You don't indicate how much ram you have, but our test server has 4gb and that makes all the difference in the world compared to my dev system with 2gb.  Even though it takes longer, on the development side I will test with small batches on the dev side, then migrate the whole mess to the test server to test the big batches.  Other than pegging the CPU it does reasonably well.

      Hope this helps.
       

       

      • #17843

        Thanks a bunch Henry, this actually makes sense. The machine I'm doing this on had 1gb of ram, we upgraded it to 3gb and it improved it a little but not by much. It still bogs down when you have too many messages, almost feels like there is something that needs to be done to the database to optimize it.

        • #17883

          Are you using tables (e.g. lookup tables) other than the ones BT installs?

Viewing 1 reply thread
  • The forum ‘BizTalk 2004 – BizTalk 2010’ is closed to new topics and replies.