Global counter from database deadlock

Home Page Forums BizTalk 2013 & BizTalk 2013 R2 Global counter from database deadlock

Viewing 3 reply threads
  • Author
    Posts
    • #27988

      Hi All,

      I’ve got a receive port that takes a single CSV file
      and uses a disassembler pipeline that generates about 18000 XML messages from
      that file. There is a send port with a subscription to this receive port and it
      has a custom pipeline component. One of the tasks for this custom pipeline
      component is to set a global incremental counter to each of the messages. This
      is done by calling the following stored procedure (where COUNTERID is a
      parameter, but I simplified it a bit).

       CREATE procedure [dbo].[sprGetNewCounterId] as  
        begin  
          begin tran work  
          declare @Result as bigint  
          update tblCounters set CounterValue = isnull(CounterValue,0) +1 where upper(CounterKey) = ‘COUNTERID’   
          set @Result =  (select CounterValue from tblCounters where upper(CounterKey) = ‘COUNTERID’)   
          commit tran work   
          select @Result as NewIdVal
        end

      Now, for a small file this works just fine. But with a
      full file the receive port wil generate 18000 messages and they all simultaneously
      request a new ID from the database, throwing deadlocks. I could rewrite the solution to use an orchestration and sequentially loop to all of the messages, but I don’t like the orchestration overhead. Would there be any other way to retreive an incremental counter? The counter has to be an integer value (we can’t use a GUID) and it has to be unique, so if a new file arrives it has to continue counting where the last one ended. So we have to store the value somewhere and be sure that every number only gets assigned once.

      Is there a way to throttle the send port so it is not activating thousands of messages at once? Or possibly a better solution for this?

       

       

    • #28032

      Update: I think the solution would be to enable ordered delivery. That way the send port wil get message after message sequentially in turn instead of a lot of messages at once. We have to try this, but it seems hopefull.

      • #28050

        Hmmm… we just checked and unfortunatly this is not working.

        • #28058

          Correction: This does work! Unless you have no mouse coordination and check "Enable routing for failed messages" instead of "Ordered Delivery".

          • #28060

            Hi,

             

              You can also check this implementation of File Splitter, as I believe it would satisfy your requirements:

             

            File Splitter

    • #28035

      I don’t know how fast it has to be, but you can push all 18000 XML documents into another database (e.g. Trigger Database) and from there BizTalk get each message after another (using SQL-Adapter; select TOP 1 …) and delete each message once the message is stored in the msgbox.

    • #28036

      When running a transactional application you have to live with deadlocks. They are not problematic as long as they do not occur too frequently. The standard SHOW ENGINE INNODB STATUS gives information on the latest deadlocks but it is not very useful when you want to know the total number of deadlocks or the number of deadlocks per unit of time.

Viewing 3 reply threads
  • The forum ‘BizTalk 2013 & BizTalk 2013 R2’ is closed to new topics and replies.