BizTalk: Compensation Model

There are still many questions in using Compensation in BizTalk.

%u00b7 What is the proper order of the Exception and Compensating blocks (handlers), Compensate shapes?

%u00b7 Do we have to rethrow the Exceptions?

%u00b7 Do we have to use Compensate shape for the current transaction or for the internal transactions?

%u00b7 What exactly does the Compensate shape?

%u00b7 What is the difference between Atomic and Long-Running transactions from the Compensation point of view?

Let’s start from the theory.

I have to use term the Rollback in this document. It is not from the BizTalk Documentation.

Data_1–> [Transaction: Data_1to2]–> Data_2

I use the Rollback term like the same term in the database transactions. Rollback means restoring all data to the point before transaction. It is completely automatic, atomic process. We cannot control rollback. Only one operation is possible, Start Rollback. It rolls back only the local orchestration data, orchestration variables and parameters.

The local orchestration data outside the transaction could be only Data_1 or Data_2. When Transaction is Committed, the data is Data_2. When Transaction is Rollbacked, the data automatically returns to the pre-transaction state, to the Data_1. We never get the Data_1to2 outside the Transaction. We all know this; it is ABC of the Transaction model.

But BizTalk is not a data base server as SQL Server and transaction in BizTalk is not a data base transaction. For example the Send shape inside transaction scope can send message to the external system and change data in this system. In general case BizTalk cannot automatically roll back this operation and won’t do this. We have to manually create a code to roll back the external changes. Atomic transaction might change the external system in the same way as Long-Running transaction. BizTalk helps us with managing this roll back procedure. We include the custom roll back code in the Compensation block. This code is invoked automatically by BizTalk or manually by a Compensate shape and the whole process called the Compensation.

Compensation it is not a Rollback. It is another process.

Compensation is a controlled process. Here is a typical example: code inside BizTalk transaction inserts rows in the database. BizTalk does not automatically rollback this operation. But it gives us possibility to implement [maybe] complex business process of deleting inserted rows. Plus, the last uncontrolled step of the Compensation is a Rollback. We implement some custom business process of compensation and then BizTalk invoke Rollback.

All Rollbacks and Compensations are initiated by an Exception. If an Exception is fired inside transaction, BizTalk invokes Rollback, but Compensation for this transaction is usually prohibited.

BizTalk invokes Rollback, but before that BizTalk invokes the Exception block of this transaction, if there is such block and if this block catches the Exception.

Note: Compensation block and Compensation shape are different things. Compensation shape is a method to call the Compensation block.

By design Atomic transaction should be used for the fast and simple procedures. The data before Atomic transaction is not persisted on the disk. Atomic transaction is quick that’s why it does not damage the total reliability. You can mention that Atomic transaction property the Timeout automatically set up to 60 sec, but not for the Long-Running transaction. Moreover only the Atomic transaction has the Property the Retry because it makes sense to automatically retry only the short process not the long one.

The likelihood of the system crash is bigger in the durations of hours and days than in seconds. That’s why data before Long-Running transaction persisted to disk. That’s why the orchestration data used in the Long- Running transaction must be serializable. That’s why the Long-Running transaction theoretically slower than Atomic transaction.

Note: Atomic transaction in BizTalk is not atomic in strict sense. It can include requests to the external processes, calls to the .NET classes, etc. Each of these processes could change something outside the orchestration and the transaction Rollback does not automatically roll back all these changes. It would be dangerous.

The Atomic transaction is short by definition. It cannot nest other transactions inside. The Atomic transaction is processed as a single unit of work. If Exception is fired inside the Atomic transaction, all we have to do is to roll back this transaction. And BizTalk makes this automatically. We cannot change this roll back process no matter what sort of Exception was fired.

Seems this is the main difference between n the Atomic and Long-Running transactions in BizTalk, how they behave in case of the internal Exception. Atomic transaction always automatically performs Rollback. We cannot avoid this Rollback and cannot do something else. Long-Running transaction could catch the Exception in the optional Exception block and process the rollback in specific custom code. But the Exception block cannot prevent the automatic Rollback. The Rollback occurs after executing the Exception block. If a Long-Running transaction does not have the Exception blocks it processes an Exception the same way as an Atomic transaction.

Rule 1: Transaction block can get only one custom Compensation block.

Rule 2: If the Compensation block is not defined, the Transaction uses the default Compensation. Default Compensation is the Compensation for the current Transaction. That means the unwiring the Compensations for the nested Transactions and Rollback (restoring the Orchestration variables).

Rule 3: Firing an Exception is the only way to initialize the Compensation process.

Rule 4: Atomic transaction block cannot get the Exception block.

Rule 5: Long-Running transaction use the default Exception block for the “General Exception” if the custom Exception block for the “General Exception” is not defined. Default Exception block invokes the Compensation shape for current transaction and rethrow the Exception.

Rule 6: Compensation shape can be inserted only inside Exception block or Compensation blocks.

It seems obvious but anyway

We can include the Compensate shape nowhere but only in the Exception and Compensation blocks. We cannot start the compensation process arbitrary. No exception – no compensation! Sometimes we need to utilize the automatic compensation ability of the BizTalk in the ordinary business process. Firing Exception is not the ordinary situation; an exception should be always indicator of the error in process. But here we have to use exception, there are no other ways.

Rule 7: Transaction that fired the Exception processes a Rollback right after this Exception. We cannot compensate this transaction.*

*) As Charles Young mentioned here is a workaround for this rule (see the “Using Compensation Blocks on L-R Transactions” topic in his article), but you have to use undocumented features of the BizTalk. BizTalk is wisely trying to prevent compensating uncommitted transaction.

Let’s cover the standard compensation process.

Below is the simple orchestration with one LR nesting transaction (1—) and two nested transactions (11 and 12).

Second nested transaction fired the Exception.

Let see what is going on here:

1. Code generates the DivideByZeroException

2. The Exception is caught in the Exception block of the same Transaction.

3. The Exception is rethrown.

4. BizTalk rolls back the Transaction, that means it restore the orchestration variables and parameters.

5. The Exception is caught in the Exception block of the nesting, outer Transaction.

6. Calls to the Compensating of the nested Transactions in the revers order. Compensation of the 12 cannot be executed because this transaction was not Committed (completed).

7. Compensating of the 11 Transaction.

8. Exception is rethrown. Orchestration is suspended.

Why the Compensation blocks the Comp12 and Comp 1 were not executed? For different reasons.

Comp12 was not executed because Transaction the 12 was not committed but rollbacked and a try to call the Compensation in the C_Ex 1 block was skiped.

Comp1 was not executed because it could be called only from the outer, nesting transaction. We don’t have such transaction here.

The trace log was:

To me there are several confusing things in the Compensation model.

First, the Compensation blocks are called outside. Event to call the Compensation comes outside! It is pretty unusual from the developer experience. One more time: the Compensation block is called outside. We are waiting that methods start from the first operator, and the exceptions catch blocks are placed in the end of code. The Exception blocks in the Orchestration are placed as we are waiting for, in the end of the code. Compensation blocks placed in the same place but worked in opposite manner. It is unintuitive. Now the similar graphic presentation of the Exception and Compensation blocks mixed with different behaviour of the Exception and Compensation blocks. It is misleading.

Second, Rollbacks and Compensation are kind of unsynchronized. I will tell about this in the next sample.

Let’s cover the more complex compensation process.

This is the transaction tree:

Below is the orchestration picture. Nesting transaction was implemented in nested loops.

Exception was fired in the 1222 transaction.

The trace log was:

1000*** 1100=== 1110— 1111… 1112… 1120— 1121… 1122… 1200=== 1210— 1211… 1212… 1220— 1221… 1222…******* Throw exception(1)

1000*** 1100=== 1110— 1111… 1112… 1120— 1121… 1122… 1200=== 1210— 1211… 1212… 1220— 1221… >>> C_Ex 3(2)

1000*** 1100=== 1110— 1111… 1112… 1120— 1121… 1122… 1200=== 1210— 1211… 1212… 1220— 1221… >>> Comp 4(3)

1000*** 1100=== 1110— 1111… 1112… 1120— 1121… 1122… 1200=== 1210— 1211… 1212… 1220— 1221… <<< Comp 4(4)

1000*** 1100=== 1110— 1111… 1112… 1120— 1121… 1122… 1200=== 1210— 1211… 1212… 1220— 1221… <<< C_Ex 3 ******* ReThrow exception(5)

1000*** 1100=== 1110— 1111… 1112… 1120— 1121… 1122… 1200=== 1210— 1211… 1212… >>> C_Ex 2(6)

1000*** 1100=== 1110— 1111… 1112… 1120— 1121… 1122… 1200=== 1210— 1211… 1212… >>> Comp 3(7)

1000*** 1100=== 1110— 1111… 1112… 1120— 1121… 1122… 1200=== 1210— 1211… 1212… >>>Comp 4(8)

1000*** 1100=== 1110— 1111… 1112… 1120— 1121… 1122… 1200=== 1210— 1211… 1212… <<< Comp 4(9)

1000*** 1100=== 1110— 1111… 1112… 1120— 1121… 1122… 1200=== 1210— 1211… >>> Comp 4(10)

1000*** 1100=== 1110— 1111… 1112… 1120— 1121… 1122… 1200=== 1210— 1211… <<< Comp 4 (11)

1000*** 1100=== 1110— 1111… 1112… 1120— 1121… 1122… 1200=== 1210— 1211… 1212… <<<Comp 3(12)

1000*** 1100=== 1110— 1111… 1112… 1120— 1121… 1122… 1200=== 1210— 1211… 1212… <<< C_Ex 2 ******* ReThrow exception(13)

1000*** 1100=== 1110— 1111… 1112… 1120— 1121… 1122… >>> C_Ex 1(14)

1000*** 1100=== 1110— 1111… 1112… 1120— 1121… 1122… >>> Comp 2(15)

1000*** 1100=== 1110— 1111… 1112… 1120— 1121… 1122… >>> Comp 3(16)

1000*** 1100=== 1110— 1111… 1112… 1120— 1121… 1122… >>> Comp 4(17)

1000*** 1100=== 1110— 1111… 1112… 1120— 1121… 1122… <<< Comp 4(18)

1000*** 1100=== 1110— 1111… 1112… 1120— 1121… >>> Comp 4(19)

1000*** 1100=== 1110— 1111… 1112… 1120— 1121… <<< Comp 4(20)

1000*** 1100=== 1110— 1111… 1112… 1120— 1121… 1122… <<< Comp 3(21)

1000*** 1100=== 1110— 1111… 1112… >>> Comp 3(22)

1000*** 1100=== 1110— 1111… 1112… >>> Comp 4(23)

1000*** 1100=== 1110— 1111… 1112… <<< Comp 4(24)

1000*** 1100=== 1110— 1111… >>> Comp 4(25)

1000*** 1100=== 1110— 1111… <<< Comp 4(26)

1000*** 1100=== 1110— 1111… 1112… <<< Comp 3(27)

1000*** 1100=== 1110— 1111… 1112… 1120— 1121… 1122… <<< Comp 2(28)

1000*** 1100=== 1110— 1111… 1112… 1120— 1121… 1122… <<< C_Ex 1(29)

Picture of whole compensation process:

As an initial exception (1) is fired, there are several stages in the compensation process. A catch block follows after each exception, then compensations (blue arrow) and rollbacks (green arrow) for nested transaction(s). Each compensation is followed by a rollback with nested compensations and rollbacks in between.

For example, an Exception (5) in the Catch Exception block the “C_Ex 3” of the transaction 1220 is caught into the Catch Exception block (6) of the Transaction 1200, then follows the Compensation “Comp3” for the Transaction 1210 (7), the Compensation (8) and the Rollback (9) for the Transaction 1212, the Compensation (10) and the Rollback (11) for the Transaction 1211, the Rollback (12) for the Transaction 1210.

Rule 7: The orchestration data is roll backed to the initial point of the transaction where the last exception is rethrown.

Let’s summarize how to create the standard compensation process.

1. Define the Compensation blocks for all transaction you want to be compensated. This is the most chalenging task.

2. Define the Exception blocks where the Exception(s) should be cought.

3. Add the Compensate shape(s) to this Exception block. Define what transaction you want to compensate. The standard way is to define a current transaction. All nested transactions would be compensated automaticaly in the right order.

4. If you need a predictable rollback process for the orchestration data, rethrow an Exception in each Exception block where you added the Compensate shape.