Developer’s Troubleshooting Guide (great info on adapters!)

by community-syndication | Oct 12, 2006 | BizTalk Community Blogs via Syndication

New! Contains valuable and practical information on troubleshooting all parts of BIzTalk Server.  A lot of information on troubleshooting adapters – very useful!

http://download.microsoft.com/download/3/7/6/376a6f6c-8c97-4ab5-9d5a-416c76793fbb/bts06developerstroubleshootingguide.doc

This troubleshooting guide is intended to provide guidance to developers in the detection and correction of programming issues within a BizTalk application. It may also be useful to information technology (IT) professionals who support BizTalk applications. Topics include troubleshooting adapters, schemas, BAM, API, BAS, maps, and orchestrations.

 For additional troubleshooting information specific to adapters, take a look that this location:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/bts06coredocs/html/428b127f-df1e-4c41-9cc8-486519f15fa1.asp

Increase the number of sockets allowed from the Server

by community-syndication | Oct 12, 2006 | BizTalk Community Blogs via Syndication

Very handy to have about keeping IIS6 streamlined, and outgoing socket connections
adjustable.
So when BTS is busy crunching away and creating sockets to various endpoints.

On a current project, we use a ‘helper’ class to talk to an ERP system (pronto) and
the helper class creates (& destroys upon cleanup) a socket with each instaniation.

Problem is that Windows will not immediately return the discarded socket back to the
socket pool for up to 2 mins (due to slow networks etc. and the TCP setup needs to
be fully ‘flushed’ as I understand)

The issue is – in busy times within the registry there is a value that says – the
user (aka BTS) can only create 5000 socket connections at any one time. 1024 are already
taken in the well known port space (esp. on a server) so we found we were hovering
around the 3900 active connections at once…till things went bad.

These settings below – one for IIS accepting/servicing a higher number of socket connections
and the other is for outgoing user connections.

It’s always interesting moving these bottlenecks along….to see what the next component
that presents itself as the bottleneck.

Enjoy.

MaxConnections (HTTP.SYS)

Controls the number of simultaneous HTTP connections (and hence limits number of simultaneous
connections serviceable by IIS6).

• On Windows Server 2003 RTM x86, this comes out to around 8,700

• On Windows Server 2003 SP1, the limit has been removed

• On Windows Server 2003 SP1 x64, since NPP is bound by available memory, you can increase
  concurrent connections by merely adding more RAM.

  HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HTTP\Parameters
  Type: DWORD
  Value: Range from 0 to 2^32-1

MaxUserPort (TCPIP.SYS)

Controls the max port number that TCP can assign. Every unique client making a request
to your web server will use up at least one of these ports on the server. Web applications
on the server making outbound SQL or SMB connections also use up these ports
on the server… so it highly affects the number of concurrent connections.

HKLM\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters
Type: DWORD
Value: Range from 5000 to 65536

Managing BizTalk Business Rule Solutions With RuleBurst

by community-syndication | Oct 12, 2006 | BizTalk Community Blogs via Syndication

While I’m a big fan of the Microsoft Business Rules Engine (BRE), our ruleset design and management story isn’t as robust as many other solutions in this space. This is where ISVs like RuleBurst can now step in and fill the gap. Here I walk through the RuleBurst suite by demonstrating the solution I built for the SOA and BPM Conference last week in Redmond. I am FAR from an expert with the software, but at this point, I know enough to be dangerous.

So what is the BRE missing that RuleBurst provides? Let’s start with an easy to use management interface where I can see the various rule “projects.” The main management interface, called RuleBurst Studio is shown below.

One of the first things that jumps out when reviewing this tool is the ability to author the rulesets in common Microsoft Office applications. You can build rules in Word, Excel and Visio.

After you create the Word document in RuleBurst Studio, you can open it up and start building your ruleset. RuleBurst uses a series of format styles to identify the various components which make up a rule. For instance, the top section (in the diagram below) contains data elements used in the rules (i.e. variables). Then, each rule starts off with a “conclusion” and is followed by all the conditions that make that conclusion true. These conditions can be written in regular English, and get interpreted when this Word document is compiled. What I built into the little animation below is the basic ruleset (as authored by the user) and the markup that gets added after compilation.

Once you’re happy with the ruleset, you come back into RuleBurst Studio and compile it. Now you are able to view all of the rules contained in the Word document without opening it back up. You can see below how I can browse all the rules in my project in the RuleBurst Studio. If I click the “edit” link, RuleBurst Studio opens the source document (e.g. Word file) and jumps right to the rule I wish to edit. Nice.

Now one thing I’ve always wished for in the BRE is a basic Ctrl+F (“find”) capability so that I could find a particular rule or fact. Lucky for me, this functionality sits within RuleBurst Studio. If I perform a “find”, I can type in text and it will look for both rules and facts that meet the criteria. However, it ALSO looks like the “description” text gets searched since by query for “age” pulled up a rule where the description contained the word “manager”!

Another functionality request I often hear from customers centers around dependency checking and impact analysis. One way you can get some of this capability with RuleBurst is through Visualizations. Here I can add conditions to the pallette, and see which rules influence it, and, which rules it influences. In my animation below, you can see that after dropping the “maximum term length”, I can right click and have all related nodes added to the screen as well. Now I can see that multiple values drive this condition, and the result impacts another rule higher up. This is very key functionality for even the smallest ruleset.

You’ve got a compiled ruleset, so what’s next? Something all developers avoid like the plague … testing! Unlike testing in the BRE, RuleBurst Studio’s testing interface does NOT require a degree in advanced physics. The first aspect of testing, unit testing, is shown below. Using the Interactive Test Utility, I can enter values for a specific condition/variable, and instantly see the impact on the broader ruleset. For instance, below you’ll see that when I fill in the value “purchase” for the “LoanPurpose” variable, I’m immediately shown that the value of “MaxTermLength” gets set to “50” and the “IsLendingPurposeAcceptable” gets set to “true.” Very intuitive way to see how various combinations of values affect the rulesets.

Now let’s say you have a very complex ruleset that you’d like to test here, and you can’t keep track of what values caused the final result. In that case, you can view the Decision Report window which identifies exactly which criteria led to a particular outcome. Nice.

Unit testing is all well and good, but most shops insist on using a series of automated tests to validate their logic. RuleBurst includes the ability to build Test Scripts. You can export out data values and put those into files for bulk testing. In my case, I created a few different files containing data that would result in both true and false responses.

Once I choose to Execute Script all the test cases get evaluated and I’m left with a report validating that the response I expected matches the response reached by evaluation engine. In the report below, you can see that the data for each test case matched my expected result. This is a powerful way to regression test changes to a particular ruleset using the previously defined scripts. It’s a quick and easy way to ensure that you didn’t break anything you weren’t supposed to!

Whew! So we’ve built, compiled and tested the rule, which means NOW it’s ready for BizTalk. In this latest release of RuleBurst Studio, you can select Compile Microsoft BizTalk Policy which generates the BRL that the BRE can then import in.

What’s great is that you can control the result of the export. In this window below you can see that we can provide a description, version information, and, specific data bindings. What are those? By default RuleBurst will generate an XSD schema containing all our elements used in the ruleset. But, you may already have a schema needed for the rules processing. No problem, set up a custom data bindings file which maps to your schema ( or database tables). I haven’t played around with this much, but have seen examples of it working.

Finally, you can import the BRL data into the BRE and see your BizTalk-friendly ruleset. You can’t automatically hook a vocabulary in here (I’d like to see this functionality added), but in reality, you’d be doing all your management in RuleBurst Studio, so BRE vocabularies are a bit less important.

So what do you all think? I definitely didn’t demonstrate the whole product, but hopefully gave you an idea of what this can do. This is such a wonderful, complimentary piece of the Microsoft BPM story that really adds SIGNIFICANT value whether you have 50 rules or 5000. Is there additional functionality you’d like to see? What’s on YOUR wish list?

Technorati Tags: BizTalk

Orchestrations fail to deploy due to binding errors

by community-syndication | Oct 12, 2006 | BizTalk Community Blogs via Syndication

BizTalk 2006 seems to have a deployment quirk that has been no small source of frustration for me.  I don’t know if this is widespread or if I just have a bad install but it seems that whenever I change a port (add a port or remove a port) I get an error similar to the following when I try to deploy that assembly from Visual Studio,

Error 26 Failed to add resource(s). Change requests failed for some resources. BizTalkAssemblyResourceManager failed to complete end type change request. Failed to update binding information. Could not enlist orchestration ‘Orchestration.UpdateRequestStatus,Orchestration, Version=1.0.0.0, Culture=neutral, PublicKeyToken=f96e91b6b75bb727’. Could not enlist orchestration ‘Orchestration.UpdateRequestStatus’. All orchestration ports must be bound and the host must be set.

I’ve done some reading and a lot of thinking and all I can come up with is that BizTalk must be cacheing the binding information and reapplying it when I redeploy the assemblies. The part of it that burns me is that the DEPLOYMENT of the orchestrations fails because of a binding error. If you think about that for a second you realise that you can not change the binding information for assemblies that are not deployed so you get stuck.

I have not been able to find references to the binding cache, though my research is admittedly limited.  On the hunch that this cache exists and is the source of my problem I tried incrementing the version on my orchestration assembly so BizTalk would ignore its previously cached info (its a new assembly version after all) and voila! It deployed my assemblies without complaining.

Two possible solutions to this that i know of are:

1. Change version (or key I suppose) on the orchestration assemblies
   
2. Deploy the orchestration assembly by adding it as a resource to the appropriate app in the biztalk admin tool

ITEMFIELD at Melbourne BizTalk User Group – 19th October 2006

by community-syndication | Oct 11, 2006 | BizTalk Community Blogs via Syndication

Hi All,

We’ve organised itemfield to present at the next Melbourne BizTalk User Group on 19th October at Clifton’s in the city. Check it out here. Essentially, I was interested in Itemfield for their EXCEL and PDF parsing capabilities…. but they do lots more! Don;t miss this session as nerly every customer I meet wants to parse EXCEL documents and feed them into BizTalk. I’ve seen their demo, its a neat product.

Here is the description from the Melbourne User group site: –

We will be holding the next Melbourne BizTalk User Group Meeting on Thursday 19th October 2006 at 5:30pm

The venue and sponsor for the meeting will be Cliftons at Level 1, 440 Collins Street, Melbourne Click here for a Map

The Topic for this meeting is Itemfield and the presentation is:

Amir Grimberg of Itemfield will be Presenting

• Itemfield introduction
• ContentMaster offering in the BizTalk environment (any to any transformation visual tolling)
• Parsing By Example unique technique
• Predefined transformations for industry standards (HL7, SWIFT, EDI…)
• Specification Driven transformation
• Content Master transformation capabilities – technical demo
• Itemfield integration to Microsoft BizTalk.
• Use cases and success stories with Microsoft BizTalk

Please rsvp for [email protected] if you plan on attending the meeting.

BizTalk MVP Dinner at the SOA & Business Process Conference

by community-syndication | Oct 11, 2006 | BizTalk Community Blogs via Syndication

On Thursday evening Oct 5th, BizTalk Product Team welcomed BizTalk MVPs to a special dinner.  This event was hosted at one of the premier dining locations in Bellevue, the Daniel’s Broiler.  

Marjan Kalantar

Tracking the current state in a state machine workflow

by community-syndication | Oct 11, 2006 | BizTalk Community Blogs via Syndication

1. Custom Activities – in order to use this, you either have to subclass the state activity or write a simple activity and remember to add it to every state. HUGE headache to manage and maintain in either case and the custom state comes with its own set of problems.
2. using CallExternalMethod has the same issues as the custom activity. Who wants to add an activity all the time just to track data.
3. Using the idled event can work, but then you have to use the StateMachineWorkflowInstance class and this may entail calling up your workflow from persistence just to query information from it.

.NET Framework 3.0 eLearning clinics – Free Until the Launch of .NET 3.0!

by community-syndication | Oct 11, 2006 | BizTalk Community Blogs via Syndication

This collection of 3 2-hour premium clinics teaches about the new capabilities provided by the .NET Framework 3.0. These clinics are for experienced Developers and Software Architects who are looking to adopt Microsoft’s next generation technology within their solutions.

Topics covered within the collection include:
 Windows Presentation Foundation
 Windows Workflow Foundation
 Windows Communication Foundation

Don’t procrastinate you don’t have much time to grab these for free and once you have them they are valid for 1 year!

Quick tip… log in to eLearning first and then don’t change the “Prices for the United States are shown in US Dollars.” dropdown to NZ dollars as it seemed to break it for me when the price of the course was set to *FREE*

Enjoy!

Good reason why we should not use Constants in BizTalk Server maps

by community-syndication | Oct 11, 2006 | BizTalk Community Blogs via Syndication

I’ve always advocated to other BizTalk Server developers not to use constants in BizTalk Server maps, but I could not present a valid argument.   At my current client, the existing BizTalk Server 2004 implementation makes heavy use of the SQL Server adapter.  Because of bad practices, the underlying stored procedures were renamed causing breaking changes to the XML schema files in the BizTalk Server implementation.  I made the appropriate changes, refreshed the maps, and reset missing links.

Upon testing the implementation in the test environment, the stored procedures were failing miserably due to missing required fields.  My investigation found that many of the maps set default values for some elements in the SQL Insert message using constants.  Upon refreshing the schema, the constants were lost.

To resolve the issue I viewed the old version of the BizTalk Server maps using TextPad or XmlSpy and culled out the constants from the Constants element.  I set the default values using either a String or Integer functoid.

Microsoft SOA & Business Process Conference 2006…

by community-syndication | Oct 11, 2006 | BizTalk Community Blogs via Syndication

Update: The tracking query as a TRQ file is available here,
per a comment on this post.

I just got back from a week at the “Microsoft SOA & Business Process” conference
in Redmond.  Lots of discussion on BizTalk 2006 R2, the new .NET Adapter Framework,
Windows Communication Foundation (WCF), Windows Workflow (WF), Microsoft Office Sharepoint
Server 2007 (MOSS).  Great stuff.

I did a session at the conference on Friday titled “Applying Maximum Sustainable Throughput
to a Management/Operations Strategy”.  (OK, the title may have scared a few people
off…) The content was an extension of what I presented to the Twin Cities BizTalk
User Group in
September.  The overall theme was relating what you learn during your performance
testing phase to your operations strategy, and ensuring that you have application-level
metrics that represent operational boundaries you would like to “live within”. 
In particular, I discussed how to formulate your performance goals, how to measure
against them as operational parameters, and how to proactively monitor with a tool
such as Microsoft Operations Manager (MOM).  We went on to discuss how to choose
the workloads (e.g. send, receive, or orchestration work) for new servers that you
place in your BizTalk group, depending on what you see within various BizTalk work
queues and throttling states.  I’ve put the slides for the session here for
your perusal. 

One of the demos showed a HAT query I’ve worked up which will report on how many times
each orchestration has run within a given time window, along with the percent of total
transaction volume it represents, time duration, and activity rates (per second, per
minute, and per hour). You can put the following query in a .TRQ file (like OrchestrationDurationsAndRates.trq)
and the put it within C:\Program Files\Microsoft BizTalk Server 2006\Tracking\Queries
(or the equivalent on your server.) It will then appear within the “Queries” menu
in HAT. This can be a genuinely useful query to have at your disposal, both during
performance testing and in production.  You can run this in Query Analyzer if
you prefer – just uncomment the lines that define UtcOffsetMin.  Note that rate
calculations are dividing across the entire time window, which assumes a fairly steady
traffic pattern.

If you’re trying to determine compliance within an SLA such as “95% of all requests
must be satisfied within 15 seconds or less”, you might add an item to the select
clause below along the lines of “average(service duration) + (2*stddev(service duration))”,
if the distribution of your transaction durations tends to be normal (per your high
school stats class.)

declare @beginTime as datetime
declare @endTime as datetime

declare @TotalCountInWindow as int

--
--  SET THE TIME WINDOW FOR YOUR QUERY HERE
--
select @beginTime =DateAdd(hour, -2, GetDate())
select @endTime = GetDate()

-- Only need to uncomment this select when inside of query analyzer,
-- where UtcOffsetMin won't be defined.
-- declare @UtcOffsetMin as int
-- select @UtcOffsetMin = 
-- -1*DateDiff("mi",CONVERT(varchar,GetDate(),0),CONVERT(varchar,GetUTCDate(),0))

select @beginTime = dateadd(minute,-1*@UtcOffsetMin,@beginTime)
select @endTime = dateadd(minute,-1*@UtcOffsetMin,@endTime)

SELECT @TotalCountInWindow = count(*)
FROM dbo.dtav_ServiceFacts sf WITH (READPAST)
where [ServiceInstance/StartTime] between
@beginTime and @endTime
and [ServiceInstance/State] = 'Completed'
and [Service/Type] = 'Orchestration'


SELECT 
[Service/Name],
count(*) as TotalCount,
(cast(count(*) as float)/cast(@TotalCountInWindow as float))*100 as PercentOfTotal,
avg([ServiceInstance/Duration]/1000.0) as AverageDuration,
min([ServiceInstance/Duration]/1000.0) as MinDuration,
max([ServiceInstance/Duration]/1000.0) as MaxDuration,
stdev([ServiceInstance/Duration]/1000.0) as StdDevDuration,
count(*)/DateDiff("hh",@beginTime,@endTime) as perHour,
count(*)/DateDiff("mi",@beginTime,@endTime) as perMinute,
cast(count(*) as float)/DateDiff("ss",@beginTime,@endTime) as perSecond

FROM dbo.dtav_ServiceFacts sf WITH (READPAST)
where [ServiceInstance/StartTime] between
@beginTime and @endTime
--and  [ServiceInstance/Duration] > 0
and [ServiceInstance/State] = 'Completed'
and [Service/Type] = 'Orchestration'
group by [Service/Name]

Got your own favorite HAT query? Post it in the comments – maybe we’ll start a new
section for them. Some shops deploy their custom HAT queries directly with their BizTalk
applications…For Deployment
Framework folks, that looks as follows (assuming a project subdirectory called
DeployedHATQueries):

	
<target name="deployHatQueries">
  <copy todir="${btsDir}\Tracking\Queries" overwrite="true">
      <fileset basedir="DeployedHATQueries">
         <include name="**\*.trq" />
      </fileset>
  </copy>
</target>