So, I’m looking at my blog today, and I see I haven’t posted anything in 2 months and a day. Shameful!!! Anyway, I’m climbing back in the saddle.
In what little free time I had before, I was working on a whitepaper for MSDN for the BizTalk Adapter for SQL Server. I’m glad to say the paper is complete, and should be on the MSDN site very shortly. I will post some examples from the paper here very shortly.
The question came up at work today about debatching in Orchestrations. This is a very common scenario in business processing, or at least the issue of dealing with batch style messages is. Stephen Thomas has done a great job in writing up some examples of how to deal with this. So good, that I won’t bother to recap. Anyway, there are a couple of important things to keep in mind when considering your options.
- Is it important that “good“ messages must be processesed, and “bad“ messages be dealt with?
- Are the file sizes significant?
- Does the service level agreement have specific time constraints?
- Does the trailer and header (if flat file messages) of the message have to be dealt with?
Many times you can’t let one single bad record cause an entire file not to be processed. If this is the case, you will have to deal with your messages in another fashion. One option that is common is to debatch, or “shred“ your messages in the orchestration using XPATH. You will notice from Stephen’s examples that when using XPATH to debatch messages inside your orchestration, peformance starts to degrade as the file size grows. I suspect what is happening is that as the XPATH iteration gets deeper in the nest, it is still starting at the beginning of the file, so it slows over time.
If you have larger files and have a time constraint, there are still a few options available to you. Since XPATH seems to be slow with large files due to it’s apparent scanning of the entire message each time, you may want to try Streaming XPATH. This option has all the benefits of XPATH, whithout the negative side effects. Look at this information to see if it might work for you.
There are still a couple of other options that may be available to you. I will start to post some more thoughts and examples on this topic in the next several weeks.
Cheers,
Todd