BizTalk Server 2006 – Enterprise Production Considerations – Part 2 – Host and Host Instances

by | Jul 31, 2007 | BizTalk Community Blogs via Syndication


I’ve seen one too many BizTalk groups with poorly configured Host/Host Instances.  So I’ve decided to create a BASIC post on the subject. This is a pretty sophisticated topic having to do with scaling, high availability, resource management etc. so please don’t assume that what I’m about to say is universally appropriate. I’ll give a few suggestions and more importantly, I’ll give the reasons behind those suggestions which should give you the background you need to create your own configuration wisely.


Suggested Starting Point




  1. Enable “Allow host tracking” on the default in-process BizTalk Application Host

    1. Do NOT enable “Allow host tracking” on any other Host
      Confirm with the BTS docs if you like, but only one host has to have this enabled

    2. Do not use this host for any ports or orchestrations, this is dedicated for host tracking

  2. Create at least 3 in-process hosts for each logical application that you have. (Use at least 2 hosts if you are not using any orchestrations)


    1. One for Send Operations      [Bind your send points to this host]
    2. One for Receive Operations  [Bind your receive ports to this host, except HTTP or SOAP]
    3. One for Processing             [Bind your orchestrations to this host, if you have any]

    4. A case by case examination can be made here for additional hosts, usually for:


      1. High volume orchestrations or those you wish to have granular control over. These get a dedicated host

      2. Out of process hosts for HTTP/SOAP/Custom protocols . These get additional Hosts/Instances

Application Isolation
Having separate hosts for your applications allows you to restart the host instances for that application (perhaps because of a re-deployment) without interrupting the other apps.


Scaling Out
Having a host for Send/Receive/Process allows you to distribute your application accross several servers in your BizTalk group in a flexible manner.  You can increase the number of host instances running to “beef up” any combination of these three types of operations as required.


Memory Utilization
As you well know, 32 bit systems have limits regarding how much memory can be used by a single host (process).  A single host on a 32GB system will really only use about 1.8GB – 2GB or so.  If you want to take full advantage of all of that juicy memory you’ll have to create more host instances and let each of them take their 2GB bites out of it. (As a rule of thumb, be sure to leave 4GB for the OS). This is not universally true (i.e. 64bit systems are different)


High Availability
Have at least two host instances running for every host that you have.  This way, any single node can fail and the other node will continue to operate. This can become more complicated for Receive nodes so check the HA configuration for whatever adapter types you have running.  For instance, FTP does not support locking so having two host instances reading from it can cause duplicate messages.  This has to have just one host instance reading from with (which I tend to call a “Singleton” adapter configuration). In this case there is one host that is clustered for high availability


Security
You may also need to create additional hosts so that you can use different credentials to access different systems.  The security context is determined by the Host/Instance which can be very helpful in scenarios that are using AD to authenticate your requested operations.  (i.e. File Adapter, SQL Adapter, and I believe the MS CRM adapter, among others)


Some Caveats



  1. Too many hosts.   


    1.  My BizTalk groups seldom have more than 22 hosts or so (7 apps + default host for tracking)  So if you begin to experience problems with Groups having more hosts than that I recommend you consider too many hosts as a possible cause. There could be RAM availability issues at these levels.
    2. Configurations in BTSNTsvc.exe.config may give you weird results because you may be assuming that they apply to the whole group, when really they apply to the host instances
    3. Again, in the BTSNTsvc.exe.config,  the changes in this file are picked up when the host instance restarts.  You may have changed the configurations, restarted only a handful of your hosts. Later down the road, you restart other host instances that then begin to have problems because of the contents of the BTSNTsvc.exe.config.  This is VERY hard to trouble shoot and I have two recommendations.

      1. Don’t change BTSNTsvc.exe.config unless you absolutely MUST.
      2. If you change BTSNTsvc.exe.config restart ALL of your host instances immediately so you can quantify the affect of the changes immediately.

  2. Single Box groups

    1. If you’re running BizTalk all on one box then it hardly helps you to be able to partition out the operations. In this case, I would just create one host per application so you can still have Application Isolation, without worrying about the scale out features or HA (which you can’t really use with just one box anyways)

Agree, Disagree, Don’t Care?  Please comment below

BizTalk Server 2006 – Enterprise Production Considerations – Part 2 – Host and Host Instances

by | Jul 31, 2007 | BizTalk Community Blogs via Syndication


I’ve seen one too many BizTalk groups with poorly configured Host/Host Instances.  So I’ve decided to create a BASIC post on the subject. This is a pretty sophisticated topic having to do with scaling, high availability, resource management etc. so please don’t assume that what I’m about to say is universally appropriate. I’ll give a few suggestions and more importantly, I’ll give the reasons behind those suggestions which should give you the background you need to create your own configuration wisely.


Suggested Starting Point




  1. Enable “Allow host tracking” on the default in-process BizTalk Application Host

    1. Do NOT enable “Allow host tracking” on any other Host
      Confirm with the BTS docs if you like, but only one host has to have this enabled

    2. Do not use this host for any ports or orchestrations, this is dedicated for host tracking

  2. Create at least 3 in-process hosts for each logical application that you have. (Use at least 2 hosts if you are not using any orchestrations)


    1. One for Send Operations      [Bind your send points to this host]
    2. One for Receive Operations  [Bind your receive ports to this host, except HTTP or SOAP]
    3. One for Processing             [Bind your orchestrations to this host, if you have any]

    4. A case by case examination can be made here for additional hosts, usually for:


      1. High volume orchestrations or those you wish to have granular control over. These get a dedicated host

      2. Out of process hosts for HTTP/SOAP/Custom protocols . These get additional Hosts/Instances

Application Isolation
Having separate hosts for your applications allows you to restart the host instances for that application (perhaps because of a re-deployment) without interrupting the other apps.


Scaling Out
Having a host for Send/Receive/Process allows you to distribute your application accross several servers in your BizTalk group in a flexible manner.  You can increase the number of host instances running to “beef up” any combination of these three types of operations as required.


Memory Utilization
As you well know, 32 bit systems have limits regarding how much memory can be used by a single host (process).  A single host on a 32GB system will really only use about 1.8GB – 2GB or so.  If you want to take full advantage of all of that juicy memory you’ll have to create more host instances and let each of them take their 2GB bites out of it. (As a rule of thumb, be sure to leave 4GB for the OS). This is not universally true (i.e. 64bit systems are different)


High Availability
Have at least two host instances running for every host that you have.  This way, any single node can fail and the other node will continue to operate. This can become more complicated for Receive nodes so check the HA configuration for whatever adapter types you have running.  For instance, FTP does not support locking so having two host instances reading from it can cause duplicate messages.  This has to have just one host instance reading from with (which I tend to call a “Singleton” adapter configuration). In this case there is one host that is clustered for high availability


Security
You may also need to create additional hosts so that you can use different credentials to access different systems.  The security context is determined by the Host/Instance which can be very helpful in scenarios that are using AD to authenticate your requested operations.  (i.e. File Adapter, SQL Adapter, and I believe the MS CRM adapter, among others)


Some Caveats



  1. Too many hosts.   


    1.  My BizTalk groups seldom have more than 22 hosts or so (7 apps + default host for tracking)  So if you begin to experience problems with Groups having more hosts than that I recommend you consider too many hosts as a possible cause. There could be RAM availability issues at these levels.
    2. Configurations in BTSNTsvc.exe.config may give you weird results because you may be assuming that they apply to the whole group, when really they apply to the host instances
    3. Again, in the BTSNTsvc.exe.config,  the changes in this file are picked up when the host instance restarts.  You may have changed the configurations, restarted only a handful of your hosts. Later down the road, you restart other host instances that then begin to have problems because of the contents of the BTSNTsvc.exe.config.  This is VERY hard to trouble shoot and I have two recommendations.

      1. Don’t change BTSNTsvc.exe.config unless you absolutely MUST.
      2. If you change BTSNTsvc.exe.config restart ALL of your host instances immediately so you can quantify the affect of the changes immediately.

  2. Single Box groups

    1. If you’re running BizTalk all on one box then it hardly helps you to be able to partition out the operations. In this case, I would just create one host per application so you can still have Application Isolation, without worrying about the scale out features or HA (which you can’t really use with just one box anyways)

Agree, Disagree, Don’t Care?  Please comment below

Microsoft Australia Partner Conference 2007

by | Jul 31, 2007 | BizTalk Community Blogs via Syndication

This year I finally get a Guernsey to the Partner Conference in Hamilton Island! I will be presenting a session called “Optimizing your SOA and Business Process Infrastructure”!. What’s that mean? Beats me 😉 Essentially it is a chance for me to shed some insight into what some of our customers / partners are doing with our SOA and BP Platform and talk about the maturity cycles within this capability area. I’ve asked a great guy – Tony Barnes – MD from FormFill to present on the day with me. Thanks Tony!


For the latest information on the Australian Partner Conference, Partners go to this site. It features information on the Pre- Conference Day, Agenda and Registration.

Developing and debugging orchestrations using DebugView and SOAPTrace tools

by | Jul 31, 2007 | BizTalk Community Blogs via Syndication

The main problem I have with developing BizTalk orchestrations is the fact that I’m so blind when it comes to follow the runtime processing. Using the debugger that is part of the HAT tool is slow and clumsy which IMHO makes the tool almost useless in everyday development. But there is hope!

DebugView

Sysinternals (Windows Sysinternal now – Microsoft bought them last year) DebugView is a wonderful little tool and is especially useful when it comes to figure out what’s actually going on inside an orchestration. Basically the tool listens to system wide debug output. From an orchestration it’s possible to write debug information using the .NET System.Diagnostics namespace and the Debug or Trace class.

Decide on how to filter

There are a couple of handy little tricks that makes DebugView a even better in BizTalk development. First one should try and have something in the debug messages that makes it possible to filter and distinct one’s own (as DebugView listens system wide debug output all running applications debug info will show up). Our team decided on “Sogeti” (our company name) for all our development and to have a method in our BaseLibrary component that outputs something like the below (the BaseLibrary is a small little .NET component with a couple of very useful classes we use company wide in our BizTalk related development).

<div><span style="color: #000000; ">System.Diagnostics.Debug.WrtieLine(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">Sogeti, your debug/trace message here</span><span style="color: #000000; ">"</span><span style="color: #000000; ">)</span></div>

This make is possible to have a filter in DebugView and to for example have it look something like this.

Trace full context of messages

Another little useful trick is to trace the full context of messages. This is done be storing the message in a XmlDocument typed variable and get the OuterXml property of that variable. The below code is and example of this.

<div><span style="color: #000000; ">tempXml </span><span style="color: #000000; ">=</span><span style="color: #000000; "> msgFindPartyRequest.parameters;
System.Diagnostics.Trace.Write(System.String.Concat(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">Sogeti, msgFindPartyRequest: </span><span style="color: #000000; ">"</span><span style="color: #000000; ">, tempXml.OuterXml));</span></div>

Example of a full message trace.

To Trace or Debug – that’s the question

As stated earlier both System.Diagnostics.Debug and System.Diagnostics.Trace has methods (Write, WriteLine and so on) for outputting debug information. However there is only one that stays in your compiled code when switching from Development to Deployment compilation mode (guess which one ;)). So make sure you choose the right class for the right information. I like to have some critical messages left using Trace and be able to trace these even on the test and production server.

DebugView on a remote desktop

When running DebugView on an other server (say a test or a production server) using Remote Desktop I’ve found that ones has to use the console user on the server. This kind of makes sense as if we’re connection “normally” we’re creating a virtual session and that’s not were the debug information is written to.

Microsoft SOAP Toolkit 3.0

This is a totally other tool than DebugView but I thought it fit here any way. It’s a handy tool when working with SOAP based messages. Without it’s very hard to actually figure out how the raw request and response message look and why your orchestration web service is acting the way it does.

The trace tool is placed as a reverse proxy between BizTalk and the Internet. It’s setup by telling the tracing tool which localhost port to listen at (for example 9091 as in the example below) then we’ll redirect to that port by changing the setting in the BizTalk send port.

Finally we’ll set up the trace tool to listen to port 9091 and redirect all traffic to our web service URL at port 80 in this case. So basically the trace tool will catch all the traffic hitting the 9091 port and forward it.

That’s it! This is probably basic stuff for most of you but hopefully it’s useful for someone!

I’ve also noticed that the SOAP Toolkit is deprecated by Microsoft and I’d like to hear if anyone used something else (like Fiddler example) for tracing SOAP messages. I’d also love some other tips, tools and methods you use for debugging BizTalk orchestrations.

Developing and debugging orchestrations using DebugView and SOAPTrace tools

by | Jul 31, 2007 | BizTalk Community Blogs via Syndication

The main problem I have with developing BizTalk orchestrations is the fact that I’m so blind when it comes to follow the runtime processing. Using the debugger that is part of the HAT tool is slow and clumsy which IMHO makes the tool almost useless in everyday development. But there is hope!
DebugView
Sysinternals (Windows Sysinternal now - Microsoft bought them last year) DebugView is a wonderful little […]

5 worst problems of home-grade routers

by | Jul 30, 2007 | BizTalk Community Blogs via Syndication

During our crusade for Home Server and Beta programs we faced a multitude of home network configurations. We learned a lot, and some of that we did not like. In fact, that’s our beta-participants who did not like that, it’s just they did not know what it is until we did investigations on the failing sytems. Now we know.


Let me share a few points on how the choice of your router can impact you, not just with Windows Home Server, but with any computers, Windows or not. Ever had a complain of a child that they need mom’s computer to print a school report, because their computer “does not print”?


Routers is the most important piece of home network infrastructure, especially because most people set it right after cable or DSL modem and allow all the home machine to be connected to it. Here are in my opinion the worst things the router may do or have:



  1. Rejection-based firewall
    Some routers allow all traffic and only allow block a few specific ports of addresses. To make this worse, they may have a limited space for rules, hence allowing to block only a few ports. Fortunately they are also too dumb to be able to route incoming traffic, which alleviates most of security pains, but still leaves the home network pretty open, compared to permission-based firewalls, where all traffic is porhibited and opened for specific ports, with most popular ports preconfigured.

  2. No name resolution or name resolution that does not include local DHCP-managed hosts
    Some routers give out IP addresses over DHCP but don’t bother to provide name resolution for them. As a result, home network machines cannot access each other. You can do a few tricks using workgroups with WINS or static IPs, but it’s so better when a reasonable local name resolution is available.

  3. Blocking some internal traffic
    That’s why UPnP may not work. Not just with WHS, but also with your Roku and other media streaming devices. Devices consuming media over network depend on UPnP discovery process. It may also interfere with file and printer sharing – a bad thing on Internet but very important inside the house.

  4. HTTP Proxy and HTTP Proxy configuration
    Looks like a cool idea, right? Especially, if you can configure parental control to restrict your child browsing with it. Parental control may be an important thing, but there are other ways to implemnet it on a router. As to the proxy, you need a real good implementation, which should be also bound with local name resolution for everything to work right. It occurs that some proxies in some routers out there are not implemented right.

  5. Limited bandwidth
    This one came as a surprise to me. Early in the game we decided that Windows Home Server will not be used as a boundary machine. Really, if you put a lot of sensitive data on it, you don’t want it be directly connected to Internet. To my surprise, on a Russian forum on WHS (yes, we have one, WHS Beta was surprisingly popular there), a lot of people were asking if they could do that. When I asked why, the truth revealed itself. It happened that Windows Server 2003, which is the base on which Windows Home Server is built, is sometimes 10 times faster as a router than off-the-shelf gigabit routers. Apparently, gigabit network cards is not the only factor that defines your Internet speed.

I probably missed some problems like UI configuration usability, but it feels to me these are the big five. What would you add t this list?

5 worst problems of home-grade routers

by | Jul 30, 2007 | BizTalk Community Blogs via Syndication

During our crusade for Home Server and Beta programs we faced a multitude of home network configurations. We learned a lot, and some of that we did not like. In fact, that’s our beta-participants who did not like that, it’s just they did not know what it is until we did investigations on the failing sytems. Now we know.


Let me share a few points on how the choice of your router can impact you, not just with Windows Home Server, but with any computers, Windows or not. Ever had a complain of a child that they need mom’s computer to print a school report, because their computer “does not print”?


Routers is the most important piece of home network infrastructure, especially because most people set it right after cable or DSL modem and allow all the home machine to be connected to it. Here are in my opinion the worst things the router may do or have:



  1. Rejection-based firewall
    Some routers allow all traffic and only allow block a few specific ports of addresses. To make this worse, they may have a limited space for rules, hence allowing to block only a few ports. Fortunately they are also too dumb to be able to route incoming traffic, which alleviates most of security pains, but still leaves the home network pretty open, compared to permission-based firewalls, where all traffic is porhibited and opened for specific ports, with most popular ports preconfigured.

  2. No name resolution or name resolution that does not include local DHCP-managed hosts
    Some routers give out IP addresses over DHCP but don’t bother to provide name resolution for them. As a result, home network machines cannot access each other. You can do a few tricks using workgroups with WINS or static IPs, but it’s so better when a reasonable local name resolution is available.

  3. Blocking some internal traffic
    That’s why UPnP may not work. Not just with WHS, but also with your Roku and other media streaming devices. Devices consuming media over network depend on UPnP discovery process. It may also interfere with file and printer sharing – a bad thing on Internet but very important inside the house.

  4. HTTP Proxy and HTTP Proxy configuration
    Looks like a cool idea, right? Especially, if you can configure parental control to restrict your child browsing with it. Parental control may be an important thing, but there are other ways to implemnet it on a router. As to the proxy, you need a real good implementation, which should be also bound with local name resolution for everything to work right. It occurs that some proxies in some routers out there are not implemented right.

  5. Limited bandwidth
    This one came as a surprise to me. Early in the game we decided that Windows Home Server will not be used as a boundary machine. Really, if you put a lot of sensitive data on it, you don’t want it be directly connected to Internet. To my surprise, on a Russian forum on WHS (yes, we have one, WHS Beta was surprisingly popular there), a lot of people were asking if they could do that. When I asked why, the truth revealed itself. It happened that Windows Server 2003, which is the base on which Windows Home Server is built, is sometimes 10 times faster as a router than off-the-shelf gigabit routers. Apparently, gigabit network cards is not the only factor that defines your Internet speed.

I probably missed some problems like UI configuration usability, but it feels to me these are the big five. What would you add t this list?