Home Page › Forums › BizTalk 2013 & BizTalk 2013 R2 › Global counter from database deadlock
- This topic has 6 replies, 1 voice, and was last updated 7 years, 3 months ago by community-content.
-
AuthorPosts
-
-
July 21, 2015 at 12:33 AM #27988
Hi All,
I’ve got a receive port that takes a single CSV file
and uses a disassembler pipeline that generates about 18000 XML messages from
that file. There is a send port with a subscription to this receive port and it
has a custom pipeline component. One of the tasks for this custom pipeline
component is to set a global incremental counter to each of the messages. This
is done by calling the following stored procedure (where COUNTERID is a
parameter, but I simplified it a bit).CREATE procedure [dbo].[sprGetNewCounterId] as
begin
begin tran work
declare @Result as bigint
update tblCounters set CounterValue = isnull(CounterValue,0) +1 where upper(CounterKey) = ‘COUNTERID’
set @Result = (select CounterValue from tblCounters where upper(CounterKey) = ‘COUNTERID’)
commit tran work
select @Result as NewIdVal
endNow, for a small file this works just fine. But with a
full file the receive port wil generate 18000 messages and they all simultaneously
request a new ID from the database, throwing deadlocks. I could rewrite the solution to use an orchestration and sequentially loop to all of the messages, but I don’t like the orchestration overhead. Would there be any other way to retreive an incremental counter? The counter has to be an integer value (we can’t use a GUID) and it has to be unique, so if a new file arrives it has to continue counting where the last one ended. So we have to store the value somewhere and be sure that every number only gets assigned once.Is there a way to throttle the send port so it is not activating thousands of messages at once? Or possibly a better solution for this?
-
July 21, 2015 at 4:29 AM #28032
Update: I think the solution would be to enable ordered delivery. That way the send port wil get message after message sequentially in turn instead of a lot of messages at once. We have to try this, but it seems hopefull.
-
July 23, 2015 at 5:59 AM #28050
Hmmm… we just checked and unfortunatly this is not working.
-
July 23, 2015 at 7:54 AM #28058
Correction: This does work! Unless you have no mouse coordination and check "Enable routing for failed messages" instead of "Ordered Delivery".
-
July 26, 2015 at 3:37 PM #28060
Hi,
You can also check this implementation of File Splitter, as I believe it would satisfy your requirements:
-
-
-
-
September 15, 2015 at 5:23 AM #28035
I don’t know how fast it has to be, but you can push all 18000 XML documents into another database (e.g. Trigger Database) and from there BizTalk get each message after another (using SQL-Adapter; select TOP 1 …) and delete each message once the message is stored in the msgbox.
-
October 20, 2015 at 11:04 PM #28036
When running a transactional application you have to live with deadlocks. They are not problematic as long as they do not occur too frequently. The standard SHOW ENGINE INNODB STATUS gives information on the latest deadlocks but it is not very useful when you want to know the total number of deadlocks or the number of deadlocks per unit of time.
-
-
AuthorPosts
- The forum ‘BizTalk 2013 & BizTalk 2013 R2’ is closed to new topics and replies.