BizTalk: Long Running Processes – Friend or Foe?

Something that I’ve come across in recent years and it concerns me more and morelong
running transactions.

For example let’s take an Insurance Company implementing a Claims Process.

The way it works is:

• Design Long Running Business Processes around BizTalk Orchestrations
  
  Sounds great on the surface and since BizTalk 2004, the techniques for implementing
  this were easier.
  Basically – the BizTalk Environment will look after ensuring state is maintained,
  waiting Orchestrations are managed and Correlations are in place for return messages,
  that may return seconds, minutes, weeks or months later.

  So in this case we’d implement a main claims process manager which
  is runs for the duration the claim is active in the system.

  A Claim comes in, enters the System and the Claims Process Manager initiates and we’re
  off and running.

  A common technique with long running processes is to forcibly suspend biztalk
  messages that are in error. At a later date someone looks into the BizTalk
  Admin Console (or via a WMI query) and ’deals with’ the suspended messages.

  The benefit of these suspended messages is that they potentially can be resumed right
  where they left off and these messages are stored in the MsgBoxDB awaiting attention.

The reason why I don’t think this works:

• Messages are immutable – meaning that while they’re in the MsgBoxDB they can’t be
  changed (technically we *can* changed these messages as a hack, but it’s *not supported*).
  So if the message is incorrect and in the overall process, we might fix the problem
  and resubmit that message – we can’t do this from within the MessageBox. We have to
  export the message out and provide some ’resubmit to biztalk’ port (usually a file
  port).
• BizTalk MessageBoxDB is keeping state of the system. In process Claims are part floating
  around as part of our system (we could also be a bank processing Loans etc etc). If
  we lose the MessageBoxDB this could spell even more trouble.
• Also system upgrade complexity moves up that extra notch, careful planning and various
  considerations need to be thought out. Pending Orchestrations have to be allowed to
  run through to completion; hydrated messages waiting to be sent through Ports, means
  that those ports must stay around until these messages are dealt with and many other.
• Backup – despite the recent advancements in SQL Server 2008 (mirroring) we can’t take
  advantage of it in the BizTalk world.
  The supported Technique is to use Log Shipping – The recommended
  backup interval is 15 minutes so worse case your system is out 15
  minutes in the case of a crash.

  This is not entirely true on busy systems the actual log shipping process may take
  between 15-30 mins to backup. This means that during the time while log shipping backup
  is running, the system is not being backed up. So all in all your system could be
  running for 1hr (approx.) with no covering backup.

  This essentially is the state of your solution.

What Does Work.in my opinion.

• Manage the State of your System in another area, such as SQL or SharePoint.
• Where possible keep the Orchestrations short running.
• Upgrades are simplier
• System maintenance is simplier.
• Provide a MSMQ or File Inbound Port for ’Resubmission into BizTalk’.
• Use Content Based Routing to establish mutually exclusive processes.

Food for thought folks, from what I’ve worked on and noticed out in the field.

Mick.