2006 - Page 45 of 138 - BizTalkGurus

Microsoft SOA & Business Process Conference 2006…

by community-syndication | Oct 11, 2006 | BizTalk Community Blogs via Syndication

Update: The tracking query as a TRQ file is available here,
per a comment on this post.

I just got back from a week at the “Microsoft SOA & Business Process” conference
in Redmond. Lots of discussion on BizTalk 2006 R2, the new .NET Adapter Framework,
Windows Communication Foundation (WCF), Windows Workflow (WF), Microsoft Office Sharepoint
Server 2007 (MOSS). Great stuff.

I did a session at the conference on Friday titled “Applying Maximum Sustainable Throughput
to a Management/Operations Strategy”. (OK, the title may have scared a few people
off…) The content was an extension of what I presented to the Twin Cities BizTalk
User Group in
September. The overall theme was relating what you learn during your performance
testing phase to your operations strategy, and ensuring that you have application-level
metrics that represent operational boundaries you would like to “live within”.
In particular, I discussed how to formulate your performance goals, how to measure
against them as operational parameters, and how to proactively monitor with a tool
such as Microsoft Operations Manager (MOM). We went on to discuss how to choose
the workloads (e.g. send, receive, or orchestration work) for new servers that you
place in your BizTalk group, depending on what you see within various BizTalk work
queues and throttling states. I’ve put the slides for the session here for
your perusal.

One of the demos showed a HAT query I’ve worked up which will report on how many times
each orchestration has run within a given time window, along with the percent of total
transaction volume it represents, time duration, and activity rates (per second, per
minute, and per hour). You can put the following query in a .TRQ file (like OrchestrationDurationsAndRates.trq)
and the put it within C:\Program Files\Microsoft BizTalk Server 2006\Tracking\Queries
(or the equivalent on your server.) It will then appear within the “Queries” menu
in HAT. This can be a genuinely useful query to have at your disposal, both during
performance testing and in production. You can run this in Query Analyzer if
you prefer – just uncomment the lines that define UtcOffsetMin. Note that rate
calculations are dividing across the entire time window, which assumes a fairly steady
traffic pattern.

If you’re trying to determine compliance within an SLA such as “95% of all requests
must be satisfied within 15 seconds or less”, you might add an item to the select
clause below along the lines of “average(service duration) + (2*stddev(service duration))”,
if the distribution of your transaction durations tends to be normal (per your high
school stats class.)

declare @beginTime as datetime
declare @endTime as datetime

declare @TotalCountInWindow as int

--
--  SET THE TIME WINDOW FOR YOUR QUERY HERE
--
select @beginTime =DateAdd(hour, -2, GetDate())
select @endTime = GetDate()

-- Only need to uncomment this select when inside of query analyzer,
-- where UtcOffsetMin won't be defined.
-- declare @UtcOffsetMin as int
-- select @UtcOffsetMin = 
-- -1*DateDiff("mi",CONVERT(varchar,GetDate(),0),CONVERT(varchar,GetUTCDate(),0))

select @beginTime = dateadd(minute,-1*@UtcOffsetMin,@beginTime)
select @endTime = dateadd(minute,-1*@UtcOffsetMin,@endTime)

SELECT @TotalCountInWindow = count(*)
FROM dbo.dtav_ServiceFacts sf WITH (READPAST)
where [ServiceInstance/StartTime] between
@beginTime and @endTime
and [ServiceInstance/State] = 'Completed'
and [Service/Type] = 'Orchestration'


SELECT 
[Service/Name],
count(*) as TotalCount,
(cast(count(*) as float)/cast(@TotalCountInWindow as float))*100 as PercentOfTotal,
avg([ServiceInstance/Duration]/1000.0) as AverageDuration,
min([ServiceInstance/Duration]/1000.0) as MinDuration,
max([ServiceInstance/Duration]/1000.0) as MaxDuration,
stdev([ServiceInstance/Duration]/1000.0) as StdDevDuration,
count(*)/DateDiff("hh",@beginTime,@endTime) as perHour,
count(*)/DateDiff("mi",@beginTime,@endTime) as perMinute,
cast(count(*) as float)/DateDiff("ss",@beginTime,@endTime) as perSecond

FROM dbo.dtav_ServiceFacts sf WITH (READPAST)
where [ServiceInstance/StartTime] between
@beginTime and @endTime
--and  [ServiceInstance/Duration] > 0
and [ServiceInstance/State] = 'Completed'
and [Service/Type] = 'Orchestration'
group by [Service/Name]

Got your own favorite HAT query? Post it in the comments – maybe we’ll start a new
section for them. Some shops deploy their custom HAT queries directly with their BizTalk
applications…For Deployment
Framework folks, that looks as follows (assuming a project subdirectory called
DeployedHATQueries):

	
<target name="deployHatQueries">
  <copy todir="${btsDir}\Tracking\Queries" overwrite="true">
      <fileset basedir="DeployedHATQueries">
         <include name="**\*.trq" />
      </fileset>
  </copy>
</target>

BizTalk 2006 FailedMessage Routing

by community-syndication | Oct 10, 2006 | BizTalk Community Blogs via Syndication

There is no sample for this, but if I get enough requests (say five or so) I’ll create one and post it.

I decided to whip up a quick prototype of FailedMessage subscription in BizTalk 2006 and I ended up hitting a couple of bumps in the road on my first shot. I’m the kind that does not stop and ask for directions, nor do I take the time to look at samples. I try to do it on my own, and failing that, I’ll hit the blogs. All of that with a couple of trial and error sessions produced the following notes that I will use when trying this again:

Generating a FailedMessage

Generating a failed message is easy. Just submit a schema type that you are not looking for in any of your artifacts, or just unenlist the subscribers.

Be sure to enable FailedMessageRouting on the port that is Receiving/Sending the erroreous message

When Using A SendPort subscriber to the FailedMessage

Just create a send port and Set the filter criteria to ErrorReport.ErrorType == FailedMessage (do NOT put quotes around the failedmessage string)

When Using an Orchestration subscriber to the FailedMessage

Create a receive port that is Direct Binding (the “will be defined by filter expressions“ one)

Have your activating receive shape with the ErrorReport.ErrorType == “FailedMessage” (put quotes around the failedmessage string)

Your receive message type should just be Xmldocument

Some Insights

There are a handful of other ErrorReport fields that will help you narrow your subscription criteria if you wish I like the MessageType and ReceivePortName

If your subscription to the FailedMessages is messed up, you will see TWO reports of failed message routing per message that you submit. I can only assume that this is because the attempt to route the failedMessage also fails. If you want to explore this Disable/Enable FailedMessage routing on the port in question and see if it affects the number of reports you see.

This is all about error handling so you have to assume that some pretty significant things are allready wrong. Depending on the stage at which you get the exception that causes the failedMessage, not all of the ErrorReport feilds will be helpful. For instance, if the failure occurrs during the Xml[Dis]Assembly you can expect that the ErrorReport.MessageType will not be available.

Corollaries

This errorRouting mechanism also has the usefull behaviour of suppressing suspended messaging instances. I had a Solicit-Response call to an unreliable webservice [aren’t they all], that would fail periodically. When it did fail it would suspend the orch and the message instance. When I resumed them, it would send a dup message (on from orch and one from message instance). I’m sure there’s a better way to solve this one (but I don’t stop for directions remember) so we enabled ErrorRouting on the send port. The consequence of this was that, on failures, only the orchestration was suspended. Now when I resume, I do not get duplicate messages and I do not have to remember to terminate the message instances.

By the way, I fiddled with the SuspendAsNonResumable on this and it never seemd to work the way that the help file described. There was a lot of sketchy behavior surrounding the resuming of a failed web call… but that can be another post.

<a href="http://www.pheedo.com/click.phdo?feedUrl=http://www.geekswithblogs.net/PheedoMainFeed.aspx&itemLink=http://www.geekswithblogs.net/93756&itemDate=2006-10-11 06:12:00&itemTitle=BizTalk 2006 FailedMessage

What is the difference between Microsoft.RuleEngine.RuleSetDeploymentDriver and Microsoft.BizTalk.RuleEngineExtensions.RuleSetDeploymentDriver?

by community-syndication | Oct 10, 2006 | BizTalk Community Blogs via Syndication

The Microsoft.RuleEngine assembly can be used by any .NET application. The BizTalk Server can be considered as one of the consumers of BRE. The Microsoft.BizTalk.RuleEngineExtensions is the extension for BizTalk to provide some additional functionality. The main difference is that the Microsoft.BizTalk.RuleEngineExtensions.RuleSetDeploymentDriver writes out information to the BizTalk tracking database so that rules can be tracked using HAT. If you use the Microsoft.RuleEngine.RuleSetDeploymentDriver class, deployment go a little bit faster (no need to save the rules to the tracking database, but it is not recommended.

In general, you should use the one configured. In BizTalk 2006, the static method Configuration.GetDeploymentDriver() determines what has been configured and returns an instance of that class. This method looks at the registry entries DeploymentDriverAssembly and DeploymentDriverClass under HKLM\Software\Microsoft\BusinessRules\3.0. The default values for these keys are Microsoft.BizTalk.RuleEngineExtensions and RuleSetDeploymentDriver. In other words, the recommended way is to use the one in Microsoft.BizTalk.RuleEngineExtensions namespace.

What is the difference between Microsoft.RuleEngine.RuleSetDeploymentDriver and Microsoft.BizTalk.RuleEngineExtensions.RuleSetDeploymentDriver?

by community-syndication | Oct 10, 2006 | BizTalk Community Blogs via Syndication

The Microsoft.RuleEngine assembly can be used by any .NET application. The BizTalk Server can be considered as one of the consumers of BRE. The Microsoft.BizTalk.RuleEngineExtensions is the extension for BizTalk to provide some additional functionality. The main difference is that the Microsoft.BizTalk.RuleEngineExtensions.RuleSetDeploymentDriver writes out information to the BizTalk tracking database so that rules can be tracked using HAT. If you use the Microsoft.RuleEngine.RuleSetDeploymentDriver class, deployment go a little bit faster (no need to save the rules to the tracking database, but it is not recommended.

In general, you should use the one configured. In BizTalk 2006, the static method Configuration.GetDeploymentDriver() determines what has been configured and returns an instance of that class. This method looks at the registry entries DeploymentDriverAssembly and DeploymentDriverClass under HKLM\Software\Microsoft\BusinessRules\3.0. The default values for these keys are Microsoft.BizTalk.RuleEngineExtensions and RuleSetDeploymentDriver. In other words, the recommended way is to use the one in Microsoft.BizTalk.RuleEngineExtensions namespace.

Dynamically Configuring Pipeline Component Properties in a BizTalk Orchestration

by community-syndication | Oct 9, 2006 | BizTalk Community Blogs via Syndication

This entry discusses the use of the following to programmatically alter a receive or send pipeline component(s) properties:

BTS.SendPipelineConfig
BTS.SendPipelineResponseConfig
BTS.ReceivePipelineConfig
BTS.ReceivePipelineResponseConfig

There is a link at the end of this entry to download a couple of examples.

Before discussing the above properties, two BizTalk 2006 pipeline enhancements are:

1) The execution of send and receive pipelines in orchestrations. This feature allows a receive or send pipeline to be executed without the use of a physical receive or send port.
2) Per Instance Pipeline Configuration using the BizTalk 2006 Administration Console. This allows changing the properties of receive and send pipeline components that
are set in the various stages of receive and send pipelines. The receive and send pipelines are configured to be invoked in physical receive and send ports. For example, as below the MIME/SMIME decoder pipeline component configuration of a receive pipeline can be altered on a per instance basis using the BizTalk 2006 Administration tool.

Both 1) and 2) are extremely useful features new for BizTalk 2006.

An additional feature that would be useful (in some cases) would be to alter a pipelines component property values in an orchestration.

As below the following properties may be set for a message in an orchestration.

Property Type of Port

BTS.SendPipelineConfig Send Pipeline Configuration for a Two Way or One Way Send Port
BTS.SendPipelineResponseConfig Receive Pipeline Configuration for a Two Way Send Port

BTS.ReceivePipelineConfig Receive Pipeline For a Two or One Way Receive Port
BTS.ReceivePipelineResponseConfig Send Pipeline for a Two way Receive Port

For the below receive pipeline configured with a MIME/SMIME decoder:

The below XML is used internally by BizTalk to store the above configuration:

The above XML configuration can then be modified using a statement as below in a message assignment shape in an orchestration:

When the above altered message is sent through a physical port that is set to use a receive pipeline configured with the MIME/SMIME component (in the decode stage):
The new configuration for the MIME/SMIME component (programmatically set in the orchestration) will then temporarily override the hardcoded configuration or the per-instance configuration pipelines configuration.

Ideally this is useful to cut down on the number of pipelines or physical send or receive ports that have to be created.

In order to use this method you can do the following:

1) Create receive or send pipelines set with the appropriate pipeline components. Deploy the pipelines.
Note: You do not have to configure the pipeline components properties.
Note: You can also use the out of the box XML Receive or Send pipelines.

2) Create physical ports that are set to use the pipelines from 1).

3) Create and deploy orchestrations that temporarily override the hardcoded or per-instance send or receive ports pipeline configuration as was illustrated in an above graphic.
Logical ports in the orchestration are needed to send or receive the messages from an orchestration. The logical ports are then bound to physical ports. The configuration set in the orchestration will then temporarily override the configuration of the physical ports.

Note: To get a sample of the configuration for a pipeline. Do the below:
Using a tool such as Microsoft Sql Server Management Studio, query/open and copy the following values for a port created in 2), from a row in the following table.column in the BizTalkMgmtDB database:

bts_sendport.nvcSendPipelineData
bts_sendport.nvcReceivePipelineData
adm_receivelocation.ReceivePipelineData
adm_receivelocation.SendPipelineData

Note: If the column/row for a port is null, using the BizTalk Console, edit the receive or send ports pipeline configuration so that it overrides the default configuration. The configuration for the pipeline component(s) should then be set in one of the above table.column combinations. You can then revert back to the default configuration, by choosing the passthrough pipeline and then again choosing the original pipeline.

Note: Do not modify the contents of the rows/columns directly in the BizTalkMgmtDB database.
Note. The BizTalk Object Model could also be used to retrieve the configurations.
Note: The configuration for the pipeline can be stored and then retrieved from a configuration store (i.e. config file, SSO, database etc) and then set in the orchestration.

4) Bind the logical ports in the deployed orchestrations from 3) to physical ports from 2).

Good Things
1) This is a technique to cut down on the number of pipeline components and/or physical ports.
For example if the same XML received message was to be delivered to the same file directory i.e. (C:\flatfiles) as a delimited and positional flat file,
the following send ports, pipelines and orchestrations would be required:

Executing Send Pipelines in an Orchestration
i) Send Pipeline with FlatFile Assembler component for delimited flat file.
ii) Send Pipeline with FlatFile Assembler component for positional flat file.
iii) Send Port configured with Passthrough pipeline.
iv) Orchestration

Per Instance Pipeline Configuration
i) Send Pipeline with FlatFile Assembler.
ii) Send Port with per instance configuration of above pipeline to create delimited flat file.
iii) Send Port with per instance configuration of above pipeline to create positional flat file.
iv) Optionally an orchestration.

Dynamically Configuring Pipeline Component Properties in a BizTalk Orchestration
i) Send Pipeline with FlatFile Assembler.
ii) Send Port configured to use above pipeline.
iii) Orchestration

Not So Good Things
1) To dynamically configure and execute a receive pipeline, a two way send port will have to be used. This might introduce the need for a loopback adapter. A loopback adapter also introduces messagebox hops that will degrade performance.
2) Maintaining a store of the Pipeline configuration XML. The configuration store replaces individual pipeline components or per instance configuration of pipelines in a physical send or receive port.

There are two examples included in the download :
1) Orchestration sets receive pipeline configuration and uses a loopback adapter that executes altered configuration.
2) Orchestration sets send pipeline configuration and uses send adapter to invoke altered configuration.
Note: A loopback adapter is also included in the download. Please read the ReadMe before installing

Unfortunately
I wish you could do something like the below:

OutputMessage(BTS.ReceivePipelineConfig) = “Pipeline Config XML would go here”;
OutputMessage = Microsoft.XLANGs.Pipeline.XLANGPipelineManager.ExecuteReceivePipeline(typeof(PipelinesAndSchemas.ReceivePipeline), InputMessage);

It did not produce a runtime error, but the configuration for the Receive Pipeline was not overridden.

Last of All
I have yet to use this technique in a production environment. I have not tried this technique with pipelines configured to use multiple or custom components, but it should work with these scenarios. Please consider all the pros and cons of using this technique. Again, this could be an alternative to replace an unmanageable number of receive/send ports and pipelines.

Here we go again more company aquisitions IHUG, youtube…

by community-syndication | Oct 9, 2006 | BizTalk Community Blogs via Syndication

The buy up of NZ companies to off shore investors has been happening quite a bit recently… I know IHUG had already previously sold to Perth based iiNet for $82 million in September 2003 but this week iiNet is again selling IHUG to Vodafone for half of what they paid just 3 years ago. IHUG has skipped from NZ owned to Aus owned to UK owned property. I have put a graph together to plot the trend over the last couple of years… I’m sure there are other high profile sales that I have missed but the trend is great for the kiwi entrepreneurs but is alarming for our industry as a whole.

Sales of NZ Companies to Offshore Investors

In contrast the buy-up of Internet property in the US is again rearing it’s head with the negotiations Google is in to buy youtube for ~2.3 Billion NZD.

It really feels like it is the year 2000 all over again! This time more cash is involved!

US Sales of Internet Property

Some interesting comparisons from the above sales…

Trademe.co.nz was sold for ~14 times more than what Flickr.com was rumored to have sold to Yahoo for!

The TradeMe sale was ~80% of the sale price of Myspace.com.

It will be interesting to see what plays out next in the local and global game!

The Team at IE7 has told the world we’re coming this month!

by community-syndication | Oct 8, 2006 | BizTalk Community Blogs via Syndication

If you are a web developer or the owner of a large site… now is the time to make sure it works with IE7 (if you haven’t done already).

The final release of IE7 is fast approaching and will be delivered to customers via Automatic Updates a few weeks after it’s available for download.

Windows Workflow Rules From Outside A Workflow, And Other Conference Demos

by community-syndication | Oct 7, 2006 | BizTalk Community Blogs via Syndication

Had a great time presenting at the SOA and Business Process Conference and wanted to share some of the code bits that I demonstrated during my Developing and Maintaining Business Rule Solutions session.

One thing I demonstrated was the ability to build business rules in Windows Workflow and execute those rules from either within the workflow, or completely stand-alone. The rules themselves are fairly simple …

So I can execute those rules by kicking off the workflow, you can find lots of examples of that. On the other hand I haven’t seen a whole lot of code showing how to execute these rules outside the workflow confines. In this code snippet below, I make use of the .rules file that gets created when you build rules in WF.

//load up a rule set from file; file name abbreviated here
XmlReader xreader = XmlReader.Create(@”C:\…\LoanWorkflow.rules”);

//class that allows us to load up the rules from XML
WorkflowMarkupSerializer serialize = new WorkflowMarkupSerializer();

//inflate a RuleDefinitions object from the XML in the file
RuleDefinitions r = (RuleDefinitions)serialize.Deserialize(xreader);

xreader.Close();

//set values on the loan application member variable
loanApp.ApplicantAge = Convert.ToInt32(txtAge.Text);
loanApp.LoanPurpose = txtPurpose.Text;

//create new instance of RuleEngine class; new in the RC build
RuleEngine engine = new RuleEngine(r.RuleSets[0], typeof(WorkflowCaller));

//execute
engine.Execute(this);
MessageBox.Show(“Rule Result: Loan Applicant Qualified? ” + LoanApp.IsApplicantQualified.ToString());

Neat stuff. I passed in my actual Windows Form object which had a member variable of the same name as what the rule expected. You can download all the code for this sample here.

Another demonstration I built showed how to build your own BizTalk Business Rules Engine TrackingInterceptor. Why in the world would you do this? Any rules that get executed by an orchestration can be set up to track all the execution details (facts asserted, conditions evaluated, rules that fired, etc). But, what if you call that rule outside of BizTalk? By default, nothing gets tracked. So, I inherited the IRuleSetTrackingInterceptor interface and wrote the data I wanted to a simple custom database. I also added an input parameter to the tracking interceptor’s constructor requiring the rule caller to identify themselves. This way, I’m recording who is using a particular ruleset. Finally, I built an interesting little ASP.NET site which shows the basics of each executed policy, and if you choose, you can view all the gory details of the execution cycle.

You can grab the database script, BizTalk project (which holds the schema) tracking interceptor, WinForm app that calls the rule, and the working ASP.NET site here. You’ll probably have to fiddle around to make it work, and I make no promises that any best practices were remotely used. Another option (instead of the custom database) was mentioned to me by my new buddy Mark who suggested using the BAM API from within the Tracking Interceptor and stuffing this into (related) activities. Nice idea.

I also demonstrated the usage of my Rules Authorization Manager tool which can be downloaded from here.

One of the things I enjoyed demonstrating the most at the conference was RuleBurst which nicely integrates with the BizTalk Business Rules Engine. My next post will contain a short review of that product.

Thanks to the surprising number of folks who attended the session and I hope you enjoyed yourself.

Technorati Tags: BizTalk, Technorati Tags: Windows Workflow, Technorati Tags: Business Rules