by community-syndication | Jun 28, 2005 | BizTalk Community Blogs via Syndication
Tech-Ed 2005 took place a couple of weeks ago and I’m now finding time to post some updates. Both Brandon and I staffed the Microsoft CSI booth and presented on Implementing Real-World Integration Patterns with BizTalk. For more information on the session we presented, CSIC14 (Cabana Talk) Implementing Real-World Integration Patterns with BizTalk, please take a look at the Tech-Ed website. Our goal of the session was to take something more or less theoretical (ie patterns) and put some criteria around the patterns to determine whether application of that pattern made sense to a given business scenario. We spent the first half of our session presenting on common integration patterns that Brandon had developed for Microsoft. We also presented on the importance of patterns and contributing back to the integration community. The patterns developed are only going to grow through input from people implementing the patterns. If you’re interested in contributing to the integration patterns, feel free to look at the BEP workspace on gotdotnet. In the last half of the session we divided the group into two teams and gave each team a similiar business scenario with slightly different requirements. Based on the requirements we worked with each group and discussed which criteria to use when determining how to implement one or more integration pattern. If you weren’t able to make it to Tech-Ed or to our session you will still be able to grab our presentation, sample business scenarios, and the Pattern decision tree used to help determine if an integration pattern makes sense for the given scenario. I’ll make sure to post the information before the end of this week. I’d love any comments people might have about our presentation or materials….
by community-syndication | Jun 28, 2005 | BizTalk Community Blogs via Syndication
I’ll be working on the “Ask the Experts” stand for BizTalk Server at TechEd in Amsterdam next week, along with other MVPs and Microsoft staff. Feel free to drop in if you have any BizTalk related questions. There’s a lot of BizTalk 2006 presentations that I want to catch, I got an early build of 2006 a couple of weeks ago, and I’d be keen to see a run-through of the new features. The new admin console is great, and the setup and config is a lot easier (have not tried a multi-box install yet though…), and working in Visual Studio 2005 is very sweet.
If any “Bloggers Guide” contributors can catch me anywhere near a bar next week, you are welcome to a free beer (or drink if you are wise enough not to indulge). It would be great to say “Cheers” for all the contributions, and keeping the guide alive. (That’s not an offer I could afford to make in Stockholm J).
by community-syndication | Jun 28, 2005 | BizTalk Community Blogs via Syndication
Abstract: I’ve using BAM to measure execution time in some orchestrations. I cannot use HAT because I needed to do some calculations to rest waiting time of the external processes. It’s extremely easy. Things get funny when we needed to get more that one row per orchestration instance (because of loops which are there)
Today I’ve doing this BAM at a customer’s. I’m going back home in a high velocity train, but it’s a long trip, so I think this post is going to get long I’ll divide it in 2 parts:
Part 1- The easy sample
The scenario is quite simple: some orchestrations provide a synchronous web service fa%u00e7ade for a legacy system. The logic for integration with lecagy includes files interchange, sockets comminications and some transformations from/to Xml and HL7 flat files.
What we want: stress the solution and get detailed information about orchestration execution time. This execution measured time should not include web service communications or file interchange with the legacy system, because the intention is to measure BizTalk performance, regardless of network bandwidth or external dependencies.
Here is the sample:
In the simplified sample (actual processes are a little more complex :-), we must measure time between start (Receive 1) and Send 1, and time between Receive 2 and last send back (Send 2). Time between Send 1 and Receive 2 must be taken away.
All this information is in tracking database, and can be queried at a basic level with HAT. Probably it’s not very complex to query the tracking database to get these values. Since each shape has a start and end time, it’s just a question of making the right queries.
Anyways, as I have only one working day to prepare the environment for the tests, I don’t have time to study the tracking DB internals. I’ve done the easy way: BAM milestones. It’s easy because there’s no need to know about BizTalk internals (tracking db design) and flexible because BAM definitions can be designed and deployed independently to the orchestration development.
So, I’ll show how to do it in just 10 minutes (for the above sample):
1- Create BAM definition with the BAM Structures Mega Designer (aka the little Excel sheet :-). These definitions include 4 Activity Items, all of them as milestones. I’ll not use duration view items to catch intervals, instead I’ll do a Transact/SQL DATEDIFF later.
Time estimated: 3 minutes
2- Export the BAM definition to an Xml file, and deploy it in the server with BAM Management Utility (bm.exe).
Use the Tracking Profile Editor to assign each activity item to orchestration shapes.
Don’t forget to assign an Activity Id, tipically an Id field takes from the message schema.
Deploy the profiles to the server.
Time estimated: 3 minutes
3- Only have 4 minutes left, so I’ll assume there is some test automation :-).
Launch the tests (with NUnit in my case).
Open SQL Server Query Analyzer, connect to the DB server, switch to the BAMPrimaryImport database and open the corresponding BAM view. There you can see the milestones catched.
Now you can calculate execution times with basic date adds and rests. It you want to do it with SQL DATEDIFFs, execute something like this:
SELECT [fields],
DATEDIFF(ms, [Milestone1], [Milestone2]) As Segment1,
DATEDIFF(ms, [Milestone3], [Milestone4]) As Segment2,
(Segment1+Segment2) As TotalTime
Time estimated: 5 minutes (ouch, 1 minute late!)
well, actually the last part is not true you cannot use Segment1 and Segment2 aliases in the last line, so you’ll need to repeat the DATEDIFF statements. I’ve included in this way to improve readability.
Also, another nice way to do it is query the standard views with Excel and play with columns and formulas.
Now we are ready to launch stress tests and query orchestration execution times easily. The stress tests will be launched via Application Center Test, so we can control concurrent users, test timing, etc.
Part 2- I have a Loop and BAM Activity items cannot be assigned
The second orchestration has been more tricky. It’s a 1-to-many fa%u00e7ade, so it has a loop to send many messages to the legacy systems. Something like:
The problem: Activity items cannot be assigned to shapes inside the loop. Why? because each activity item is a row, and the Activity Id is taken from a message field that is unique for the orchestration.
The solution: use the API to add new BAM rows for each loop iteration. The Activity Id of each child-row will be the parent Activity Id plus the counter of the loop, so they can be related easily in a query.
The BAM API is very straightforward. One object, three methods: The EventStream object is used to create a new activity (BeginActivity method), update rows (UpdateActivity) of close the activity (EndActivity).
You can use a DirectEventStream object that writes the data in the BAMPrimaryImport database directly. This can impact the performance because it’s synchronous.
You can either use the BufferedEventStream object that use a asynchronous store-and-forward. It does not guarantee that the event will be inserted in BAM immediately, but it does not affect caller performance.
Here is the tricky thing: There is a property called ConnectionString that points to the database. If you use the DirectEventStream, ConnectionString points to BAMPrimaryImport database. But if you use BufferedEventStream, its ConnectionString points to the MessageBox database, because it’s used for store-and-forward.
My code between the orchestration and a helper component does more or less the following:
For each loop iteration:
– Begin an new Activity, using [ActivityId].[LoopCounter]
– Update with a milestone before sending to legacy
– Update with a milestone after receiving from legacy
– Close the Activity
So, for each orchestration instance, I have in the BAM database:
– One row with
– Orchestration Activity Id
– Milestones for BeginOrch and EndOrch
– Many “child” rows (the loop) with
– Orchestration Activity Id + Counter
– Milestones for BeginSend and EndReceive
The query to aggregate everything (courtesy of eXtreme.net, because I don’t have quick Transact/SQL skills), return a single row with the calculated fields for TotalTime and TotalTimeWithoutLegacyStuff. It uses DATEDIFF for the differences, and SUBSTRING to relate different parent and child Activity IDs
by stephen-w-thomas | Jun 27, 2005 | Stephen's BizTalk and Integration Blog
What are Promoted Properties and why does BizTalk have them?
Long and short of it: Property promotion is about getting easy and fast access to data.
Easy: Because you typically do not have to know anything about your data to get access to it. Example: Message(some property). Rather then some long and error prone xpath query.
Fast: Because you do not have to load the whole message into memory to get access to the data. It is placed inside the message context (a collection of key-value pairs that are associated with a message).
BizTalk provides two different types of promoted properties based on what you want to do with the data. The two types are Promoted Properties and Distinguished Fields.
Distinguished Fields are light weight and only accessible inside an Orchestration. They could be used inside a decision shape or for extracting data out of a message for dynamic routing. They are read/write so it provides an easy way to change values inside your message without using a map – such as updating a status field.
Promoted Properties are system wide but could also be used like
Distinguished Fields. They are “more expensive” then Distinguished Fields and typically should only be used when you need to do Context Based Routing on these values. These are also read/write (in most cases). BizTalk Server provides several context properties but you can create as many more as you like using a custom property schema.
How does something get promoted? A lot of items are promoted by BizTalk for you at various points in the process. You can also promote items yourself using Promotion inside a schema or by using a custom pipeline. It all depends on what you want to accomplish.
More information on Promoted Properties can be found inside the help guide.
by community-syndication | Jun 22, 2005 | BizTalk Community Blogs via Syndication
The product group have asked me to post this announcement to let you know about the great online support that we have available, I strongly encourage you to take a look because (1) its free J, (2) there is a lot of good traffic thru it and (3) a lot of the folks in the product group monitor and answer questions for their specific feature areas.
The general BizTalk newsgroup can be found here
Get Connected to Free Product Support and Tremendous Online Collaboration
Have you ever wanted to speak to Microsoft developers of a specific feature of BizTalk Server? I am sure your answer was “Yes let me at them”, so the Business Process Integration Division is extending an invitation to all customers to join our key feature developers, program managers, and testers in the following newsgroups:
microsoft.public.biztalk.accelerator.forsuppliers
microsoft.public.biztalk.newuser
microsoft.public.biztalk.accelerator.rosettanet
microsoft.public.biztalk.admin
microsoft.public.biztalk.appintegration
microsoft.public.biztalk.framework
microsoft.public.biztalk.general
microsoft.public.biztalk.library
microsoft.public.biztalk.nonxml
microsoft.public.biztalk.orchestration
microsoft.public.biztalk.sdk
microsoft.public.biztalk.server
microsoft.public.biztalk.setup
microsoft.public.biztalk.tools
microsoft.public.biztalk.xlangs
microsoft.public.biztalk.xsharp
We’ve been working very hard over the past year to connect with folks just like you and want to include you in our community of Most Valuable Professionals (MVP), developers, information technology professionals, chief information officers, chief executive officers, or any other role within large, medium, and small companies that hang out in our online newsgroup communities. We want to have you join in this vibrant online community to ask those questions you always wanted to ask but did not know where to go. Well, now you know where to go, we want you to come on in and join us!
If you are new to BizTalk Server, try out the NewUser newsgroup, Microsoft.public.biztalk.newuser. We are encouraging novice users to post question here.
We’re offering two levels of interaction with Microsoft Corporation employees as follows:
- Managed Newsgroup Support
- Unmanaged Newsgroup Support
Managed Newsgroup Support
MSDN managed newsgroups are available in English to MSDN Universal, Enterprise, Professional and Operating Systems subscribers to receive free technical support on select Microsoft technologies as well as to share ideas with other subscribers. MSDN managed newsgroups provide:
- Unlimited on-line technical support – keep your PSS incidents
- A commitment to respond to your post within two business days
- Over 200 newsgroups to choose from
- Spam protection for your e-mail address when posting items
Go to the following URL to sign up: http://msdn.microsoft.com/newsgroups/managed . These newsgroups are monitored by Microsoft support engineers and product group team member as described above.
Unmanaged Newsgroup Support
MSDN unmanaged newsgroups are available to all individuals.
Go to the following URL to participate: http://msdn.microsoft.com/newsgroups. These newsgroups are monitored by Microsoft product group members, other customers like you, most valuable professionals, and various other individuals.
Questions, suggestions, and direct feedback can be sent to me.
James Fort
BPI Community Lead
mailto:[email protected]
by community-syndication | Jun 22, 2005 | BizTalk Community Blogs via Syndication
The product group have asked me to post this announcement to let you know about the great online support that we have available, I strongly encourage you to take a look because (1) its free J, (2) there is a lot of good traffic thru it and (3) a lot of the folks in the product group monitor and answer questions for their specific feature areas.
The general BizTalk newsgroup can be found here
Get Connected to Free Product Support and Tremendous Online Collaboration
Have you ever wanted to speak to Microsoft developers of a specific feature of BizTalk Server? I am sure your answer was “Yes let me at them”, so the Business Process Integration Division is extending an invitation to all customers to join our key feature developers, program managers, and testers in the following newsgroups:
microsoft.public.biztalk.accelerator.forsuppliers
microsoft.public.biztalk.newuser
microsoft.public.biztalk.accelerator.rosettanet
microsoft.public.biztalk.admin
microsoft.public.biztalk.appintegration
microsoft.public.biztalk.framework
microsoft.public.biztalk.general
microsoft.public.biztalk.library
microsoft.public.biztalk.nonxml
microsoft.public.biztalk.orchestration
microsoft.public.biztalk.sdk
microsoft.public.biztalk.server
microsoft.public.biztalk.setup
microsoft.public.biztalk.tools
microsoft.public.biztalk.xlangs
microsoft.public.biztalk.xsharp
We’ve been working very hard over the past year to connect with folks just like you and want to include you in our community of Most Valuable Professionals (MVP), developers, information technology professionals, chief information officers, chief executive officers, or any other role within large, medium, and small companies that hang out in our online newsgroup communities. We want to have you join in this vibrant online community to ask those questions you always wanted to ask but did not know where to go. Well, now you know where to go, we want you to come on in and join us!
If you are new to BizTalk Server, try out the NewUser newsgroup, Microsoft.public.biztalk.newuser. We are encouraging novice users to post question here.
We’re offering two levels of interaction with Microsoft Corporation employees as follows:
- Managed Newsgroup Support
- Unmanaged Newsgroup Support
Managed Newsgroup Support
MSDN managed newsgroups are available in English to MSDN Universal, Enterprise, Professional and Operating Systems subscribers to receive free technical support on select Microsoft technologies as well as to share ideas with other subscribers. MSDN managed newsgroups provide:
- Unlimited on-line technical support – keep your PSS incidents
- A commitment to respond to your post within two business days
- Over 200 newsgroups to choose from
- Spam protection for your e-mail address when posting items
Go to the following URL to sign up: http://msdn.microsoft.com/newsgroups/managed . These newsgroups are monitored by Microsoft support engineers and product group team member as described above.
Unmanaged Newsgroup Support
MSDN unmanaged newsgroups are available to all individuals.
Go to the following URL to participate: http://msdn.microsoft.com/newsgroups. These newsgroups are monitored by Microsoft product group members, other customers like you, most valuable professionals, and various other individuals.
Questions, suggestions, and direct feedback can be sent to me.
James Fort
BPI Community Lead
mailto:[email protected]
by stephen-w-thomas | Jun 21, 2005 | Stephen's BizTalk and Integration Blog
Have you seen this error when installing BizTalk 2004?
Failed to deploy BizTalk system assembly “C:\Program Files\Microsoft BizTalk Server 2004\Microsoft.BizTalk.GlobalPropertySchemas.dll”.
Unspecified exception: An error occurred while enlisting in a distributed transaction.
I ran into this a few weeks ago when trying to install BizTalk 2004 on a Windows 2003 Server with SP1 preloaded and using a remote SQL Server.
It appears that like Windows XP SP2, Windows SP1 changes some DTC settings. This does not affect BizTalk when SQL is running locally of course.
After working with Microsoft Support for about 2 days. It turned out that the fix is rather quit easy. In our case, we had some other issues that made it seem much more complex then it was.
To correct this problem, go to Component Services, My Computer Properties, DTC tab, and then Security Configurations (bottom left).
1. Follow the standard BizTalk Installation instructions for setting up DTC.
2. Under Client and Administration, make sure “Allow Remote Clients” is checked.
3. Under Transaction Manager Communication, make sure “No Authentication Required” is checked
The end result was changes to two registry settings:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft
\MSDTC\AllowOnlySecureRpcCalls is now 0
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft
\MSDTC\TurnOffRpcSecurity is now 1
The UI now looked like this:
by community-syndication | Jun 17, 2005 | BizTalk Community Blogs via Syndication
A whole raft of whitepapers for BizTalk have been released over the last several weeks
– see here and here.
I completed a
whitepaper a short while back (though just released) on getting the most
out of the BizTalk
2004 Management Pack and Microsoft Operations
Manager 2005. The paper serves as a good reference for the management pack,
but I hope it also serves another useful purpose. Specifically, the “operational
hand-off” phase of the software development life cycle often gets short
shrift – and it can cost organizations a lot of money, downtime, and late nights.
So, much of the paper discusses the importance of having a development team accurately
communicate the “instrumentation surface area” of their completed efforts to an operations
team.
What do I mean by “instrumentation surface area”? To start, we can consider the
sum of all diagnostic logging, event logging, WMI events, performance counters (custom
or built-in), and all other mechanisms your application uses to communicate
its current operational & health state. Moreover, we need to capture “interpretations”
of this information stream that are specific to the application. (Not just “this
send port isn’t working…” but “We are currently not talking to our primary
shipping provider…”) Finally, we need to capture suggested responses
and remediation – also specific to the application.
“Communicated to the operations team”…how exactly? With a Word doc?
Well, in particular, I talk about how to do this for a BizTalk-focused solution using
a custom MOM Management Pack that “derives” from the Microsoft-supplied BizTalk 2004
Management Pack. Done right, this will provide the highest fidelity knowledge
transfer from development to operations.
See what you think – the paper is titled “Advanced
Microsoft BizTalk Server 2004 and Microsoft Operations Manager 2005 Scenarios.”
(What a mouthful…)
Comment on this post (if you like) with your thoughts on the paper or experience in
this area…
by community-syndication | Jun 17, 2005 | BizTalk Community Blogs via Syndication
A whole raft of whitepapers for BizTalk have been released over the last several weeks
– see here and here.
I completed a
whitepaper a short while back (though just released) on getting the most
out of the BizTalk
2004 Management Pack and Microsoft Operations
Manager 2005. The paper serves as a good reference for the management pack,
but I hope it also serves another useful purpose. Specifically, the “operational
hand-off” phase of the software development life cycle often gets short
shrift – and it can cost organizations a lot of money, downtime, and late nights.
So, much of the paper discusses the importance of having a development team accurately
communicate the “instrumentation surface area” of their completed efforts to an operations
team.
What do I mean by “instrumentation surface area”? To start, we can consider the
sum of all diagnostic logging, event logging, WMI events, performance counters (custom
or built-in), and all other mechanisms your application uses to communicate
its current operational & health state. Moreover, we need to capture “interpretations”
of this information stream that are specific to the application. (Not just “this
send port isn’t working…” but “We are currently not talking to our primary
shipping provider…”) Finally, we need to capture suggested responses
and remediation – also specific to the application.
“Communicated to the operations team”…how exactly? With a Word doc?
Well, in particular, I talk about how to do this for a BizTalk-focused solution using
a custom MOM Management Pack that “derives” from the Microsoft-supplied BizTalk 2004
Management Pack. Done right, this will provide the highest fidelity knowledge
transfer from development to operations.
See what you think – the paper is titled “Advanced
Microsoft BizTalk Server 2004 and Microsoft Operations Manager 2005 Scenarios.”
(What a mouthful…)
Comment on this post (if you like) with your thoughts on the paper or experience in
this area…
by community-syndication | Jun 17, 2005 | BizTalk Community Blogs via Syndication
TechEd 2005 was a great time, and I really enjoyed the extended look at BizTalk 2006
– lots of great presentations. Much more has been done in this release than
I had anticipated, and I look forward to digging into the CTP build.
Thanks to those who attended my talk on automating BizTalk application deployments
– the questions asked during the session were great! It was fun to talk to the
folks who are using the Deployment
Framework stuff…The deck I presented is here.
BizTalk 2006 is doing a lot to make deployment easier – no comparison to
2004! I expect a subset of the tools in the Deployment Framework to remain useful
– but that is another post for another day.
If you attended my DevCon 2005 talk on mobility (and even if you didn’t) you
can find the deck available here. The
talk was specifically on the .NET Compact Framework 2.0 and Sql Server 2005 Mobile
Edition.
