Just got the new MSDN Magazine in the mail, and now the issue is online. Check out a great read on 8 Tips And Tricks For Better BizTalk Programming co-written by my old pal Marty. Fairly useful stuff.
Technorati Tags: BizTalk
I’ve been interesting in using SSIS ever since I first heard about it. Since I specialize in doing integration work, primarily with BizTalk, I was curious about this new “Integration Services” component of SQL Server. I was a little bit worried that it would supplant BizTalk for a lot of integration scenarios (worried since I’ve invested so much time understanding BizTalk). Since SQL2005 was released I have had a chance to read a lot more about SSIS and also to use it in a couple of projects. I am happy to report that it does not make BizTalk obsolete. But it is important to understand whether to use BizTalk or SSIS for certain integration problems. After reading a few things and discussing this question with some Microsoft folks I believe that the following guidelines should apply:
Use SSIS For:
Use BizTalk for scenarios involving:
A combination of BizTalk and SSIS may be useful in certain circumstances. I have seen situations where an enterprise needed to process a large number of complex operations specified by a partner companies. The partner enterprises would send large files (by FTP for example) to the enterprise on a daily basis. These files contained a list of requests for particular services that involved many steps and several enterprise systems. However the first step is to parse and import the large file. SSIS is best suited to import the file. The information about the operations could be imported into SQL Server. For example, each operation record could be inserted into a “Pending Processing” table. BizTalk could then use the SQL adapter to pull each operation record from the table and begin the operation. This is very much like the classic splitter pattern.
I would suggest increasinge thread usage, this is documented here. My general guideline is the number of physical processors x100, 1 processor 100 threads, 4 processors 400 threads.
If those steps do not fix the problem, then a deeper dive into the performance needs to be considered. If you are using 2006, there are some performance counters that come installed. What I like to look at is the BizTalk:Message Agent, there I like to look at Message publishing throttling state. How do we get to there?
Bring up Performance from Adminstaive tools or c:\perfmon from a dos prompt click on System Monitor and the + sign
Let me not forget the link to Microsoft’s official performance information here, and please refer to the disclaimer on the side of my blog.
This should get you well on your way to finding out what is going on, and being able to improve performance of your BizTalk setup.
The first thing to do is if it is v 1.0 (2004) make sure that you have the latest hotfixes installed.
If this does not solve the problem, then to increase the thread usage, this is documented here. My general guideline is the number of physical processors x100, 1 processor 100 threads, 4 processors 400 threads.
If those steps do not fix the problem, then a deeper dive into the performance needs to be considered. If you are using 2006, there are some performance counters that come installed. What I like to look at is the BizTalk:Message Agent, there I like to look at Message publishing throttling state. How do we get to there?
Bring up Performance from Adminstaive tools or c:\perfmon from a dos prompt click on System Monitor and the + sign
Let me not forget the link to Microsoft’s official performance information here, and please refer to the disclaimer on the side of my blog.
A frequent question I get asked is how to check for the existence of a given promoted property in the message context, inside an orchestration. The answer is the “exists” operator, described briefly in the documentation, which returns a boolean. With it, you can use an %u00abif%u00bb or a Decide Shape to do the test. Example usage:
MyPropertySchema.MyProperty exists myMessage
Somewhat magic, I think.
If you have a distinguished field, instead of a promoted property, you can resort to a different technique, based on the use of the xpath function:
theStringValue = xpath(inputMessage, “string(//*[local-name()=’MyElementName’])”);
And now you can use System.String.IsNullOrEmpty(theStringValue) to check if the value is present.
Over the past couple of weeks I have been documenting the procedure I use to build new BizTalk Environments, this includes things like making sure that the backup jobs are in place and that the backup files get deleted (the default backup job shipped with BizTalk cleans up the history records in the database, but does not clean up the files created by the backup). This week I was helping a client clean up a tracking database that had gotten very large because of a miss configured purge job, while I was working on this problem I noticed that the full backups for the tracking database via the BizTalk backup job were happening at 10:00am local time (Melbourne, Australia). From my past experience as a SQL DBA, doing a full backup on a production database at 10:00am in the morning is probably not the best time to do it, I dismissed it as something up with the scheduler. Today while completing the setup of a new BizTalk server for another client, I again noticed that the full backups for all the BizTalk databases were happening at 10:00am, so I though to myself, what is up with this. The other strange thing is the file name did not match the time the file was created, here is an example:
READIFYPOC_BizTalkDTADb_Full_BTS_2007_04_12_00_00_08_497.bak – with a create date & time of –12/04/2007 10:00 AM
So something is really up with this backup job, I opened the backup job and looked at thefirst step, it calls the stored procedure sp_BackupAllFull_Schedule, so I open the stored procedure and on line 29 I find this:
select @CurrDT = getutcdate()
the picture suddenly becomes very clear, Melbourne has a UTC offset of +10 hours, now I know why the backups are running at 10:00am in the morning.
I understand why they used getutcdate(), almost all the dates in the BizTalk tracking database are stored in UTC time and when they generate reports they use something like this:
dateadd(minute, @UtcOffsetMin, [ServiceInstance/StartTime]) – from the “Most recent 100 services instances” query in HAT.
Now that I figured out why it was doing the full backups at 10:00am in the morning I started trying to find out how to fix the problem without changing the BizTalkstored procedures, I searched Google and the only thing that I found was someone that have setup a job that ran at 5 minutes before they wanted to do the full backup that executed the sp_ForceFullBackup stored procedure to force the next run of the backup job to do a full backup. This was not the solution I was looking for, since what that would produce is a backup at the time I select and then another backup when the UTC date changed at 10:00am. I then went and looked at the other 2 steps of the backup job, the second step that does the transaction log backups also uses getutcdate(), but the job that delete the backup history uses getdate(), so it deletes the backup history 10 hours ahead of schedule.
So as bad as I hate to change the shipped stored procedures, I decided to change them and see if that fixed the problem, YES, the problem is fixed, the full backup runs the 1st run after midnight local time and the file name matches the file create time of the file system.
I guess it is now time to open a support incident and see if they can explain why doing a full backup at 10:00am is a good thing. If they don’t I will have to create a set of _LocalTime stored procedure with my changes in them to prevent services packs from unfixing my fix. I have already done this with my sp_DeleteBackupHistoryAndFiles stored procedure that also deletes the backup files from the file system (see the download section of my website for your copy)
v\:* {behavior:url(#default#VML);} o\:* {behavior:url(#default#VML);} w\:* {behavior:url(#default#VML);} .shape {behavior:url(#default#VML);} By Jesus Rodriguez Overview A few weeks ago, in an SOA forum, someone inquired about which technologies to use…(read more)
Sorry for the lack of blogging – I have a bunch of good posts on WF/WCF and BizTalk
R2 planned -but I’ve been busy flying around the world running Microsoft’s jumpstart
program on BizTalk R2. I am excited to announce Pluralsight’s
Applied BizTalk R2 course (I think the first public course offering of a
BizTalk Server 2006 R2 course – although I could be wrong)/
The first offering will be in my home town of LA – http://www.pluralsight.com/courses/AppliedBizTalkServer2006R2.aspx
– on my birthday no less!. Come to LA – learn about BizTalk R2 and we’ll go
hang out in Hollywood with Paris Hilton for my birthday (oh wait – maybe not)… how
about the beach? …. sushi? (well – we’ll figure out something fun to do..)