by community-syndication | Aug 1, 2007 | BizTalk Community Blogs via Syndication
It has been a while, and now that I’m recommitting to this exercise, I want to share some thoughts about the experience. I’ve learned a couple of things about writing tutorials, especially step-by-step instructions for enterprise server applications.
The BizTalk Server 2006 end-to-end tutorials were:
– Tested as code multiple times by software test engineers.
– The first content in my division to go through Threat Model Assessment.
– Challenging, frustrating, fulfilling, a nightmare, an inspiration.
The tutorials continue to be broken. The comments and mail I’ve received through this blog continue to surface problems with Tutorial 3 in particular, and there is a recurring problem with the starter file for Tutorial 4. I’m beginning to feel jinxed. I know we’ve replaced the Tutorial 4 starter file a number of times, but the file distributed continues to be the wrong one.
A couple of lessons I’ve learned from this experience:
– Never, ever, ever put documentation support files in the \SDK folder of the product CD. The \SDK folder is sacred, and it makes PMs and developers too nervous to allow the documentation team to change and update our files as needed. In the future, we will deliver the files in our own folder.
– I am not anything like perfect. 400 pages of step-by-step instructions, deployment and configuration variations, etc., and there will always be errors. Even if the number of errors are significantly reduced, there will still be issues with user experience of the tutorials, and for every ‘improvement’, there is an equal and opposite problem introduced.
– Not all steps are equal. In an effort to provide information around complex steps, I ended up with a lot of lame content around not-so-complex steps in an effort to make all steps the same. I also missed differentiating between information that users only need to be told once with information that needed to be repeated. This led to the current constantly repeated reminders in almost every step: click to see controls! click to learn more about the tools!
Next Steps:
– I have 2 bugs open now, the first to make the updated tutorial support files available from the tutorial download page, and the second to fix the Tutorial 4 starter page. Hopefully both will be resolved soon.
– When I have the time I will republish the downloadable tutorial instructions. Various build problems have made this a very time consuming task. Some of the problems are supposed to be fixed now, and I’ll find out if this is true or not. In the meanwhile, the most updated instructions are available in the BizTalk Server 2006 R2 documentation.
– I have a copy of the Tutorial 4 starter file that I manually fixed. It has not been thoroughly tested and officially checked in, but maybe it will help you today. I’ve attached a .ZIP containing the contents of C:\Tutorials\Lessons\BAS. (http://blogs.msdn.com/lizal/attachment/4173972.ashx)
by community-syndication | Aug 1, 2007 | BizTalk Community Blogs via Syndication
Hey folks, I’ve been flat chat lately burning the candle at both ends getting ready
for my sessions at TechEd/Deep Dives.
In a *spare* moment – I am asked to do a Channel9 GeekSpeak session – talking
about Workflow
and BizTalk Integration.
(I’ve had my xbox360 confiscated for 1 month to help me get through
this month – that’s a story for another time 🙂
This session is on tomorrow morning – so for those of you feeding babies, can’t sleep
etc. I’d love to have you along for moral support as this will be my very
first session in Geek Speak.
So if you’re up for a 5am – 6am start Thursday morning I’ll see you there. Put something
Aussie in your nick name!
I’ll be posting the demo code on this post as well!
by community-syndication | Aug 1, 2007 | BizTalk Community Blogs via Syndication
Failed to trigger compile-thread on the server.
(-1040) <Description for error (-1040) cannot be found. Please refer to the Trouble Shooting Guide for this description>
Resolution:
Restart the HIPAA service to clear this issue when using the xsd2edi.exe or compeif.exe
Don’t ask me why…
by community-syndication | Jul 31, 2007 | BizTalk Community Blogs via Syndication
I’ve seen one too many BizTalk groups with poorly configured Host/Host Instances. So I’ve decided to create a BASIC post on the subject. This is a pretty sophisticated topic having to do with scaling, high availability, resource management etc. so please don’t assume that what I’m about to say is universally appropriate. I’ll give a few suggestions and more importantly, I’ll give the reasons behind those suggestions which should give you the background you need to create your own configuration wisely.
Suggested Starting Point
- Enable “Allow host tracking” on the default in-process BizTalk Application Host
- Do NOT enable “Allow host tracking” on any other Host
Confirm with the BTS docs if you like, but only one host has to have this enabled
- Do not use this host for any ports or orchestrations, this is dedicated for host tracking
- Create at least 3 in-process hosts for each logical application that you have. (Use at least 2 hosts if you are not using any orchestrations)
- One for Send Operations [Bind your send points to this host]
- One for Receive Operations [Bind your receive ports to this host, except HTTP or SOAP]
- One for Processing [Bind your orchestrations to this host, if you have any]
- A case by case examination can be made here for additional hosts, usually for:
- High volume orchestrations or those you wish to have granular control over. These get a dedicated host
- Out of process hosts for HTTP/SOAP/Custom protocols . These get additional Hosts/Instances
Application Isolation
Having separate hosts for your applications allows you to restart the host instances for that application (perhaps because of a re-deployment) without interrupting the other apps.
Scaling Out
Having a host for Send/Receive/Process allows you to distribute your application accross several servers in your BizTalk group in a flexible manner. You can increase the number of host instances running to “beef up” any combination of these three types of operations as required.
Memory Utilization
As you well know, 32 bit systems have limits regarding how much memory can be used by a single host (process). A single host on a 32GB system will really only use about 1.8GB – 2GB or so. If you want to take full advantage of all of that juicy memory you’ll have to create more host instances and let each of them take their 2GB bites out of it. (As a rule of thumb, be sure to leave 4GB for the OS). This is not universally true (i.e. 64bit systems are different)
High Availability
Have at least two host instances running for every host that you have. This way, any single node can fail and the other node will continue to operate. This can become more complicated for Receive nodes so check the HA configuration for whatever adapter types you have running. For instance, FTP does not support locking so having two host instances reading from it can cause duplicate messages. This has to have just one host instance reading from with (which I tend to call a “Singleton” adapter configuration). In this case there is one host that is clustered for high availability
Security
You may also need to create additional hosts so that you can use different credentials to access different systems. The security context is determined by the Host/Instance which can be very helpful in scenarios that are using AD to authenticate your requested operations. (i.e. File Adapter, SQL Adapter, and I believe the MS CRM adapter, among others)
Some Caveats
- Too many hosts.
- My BizTalk groups seldom have more than 22 hosts or so (7 apps + default host for tracking) So if you begin to experience problems with Groups having more hosts than that I recommend you consider too many hosts as a possible cause. There could be RAM availability issues at these levels.
- Configurations in BTSNTsvc.exe.config may give you weird results because you may be assuming that they apply to the whole group, when really they apply to the host instances
- Again, in the BTSNTsvc.exe.config, the changes in this file are picked up when the host instance restarts. You may have changed the configurations, restarted only a handful of your hosts. Later down the road, you restart other host instances that then begin to have problems because of the contents of the BTSNTsvc.exe.config. This is VERY hard to trouble shoot and I have two recommendations.
- Don’t change BTSNTsvc.exe.config unless you absolutely MUST.
- If you change BTSNTsvc.exe.config restart ALL of your host instances immediately so you can quantify the affect of the changes immediately.
- Single Box groups
- If you’re running BizTalk all on one box then it hardly helps you to be able to partition out the operations. In this case, I would just create one host per application so you can still have Application Isolation, without worrying about the scale out features or HA (which you can’t really use with just one box anyways)
Agree, Disagree, Don’t Care? Please comment below
by community-syndication | Jul 31, 2007 | BizTalk Community Blogs via Syndication
I’ve seen one too many BizTalk groups with poorly configured Host/Host Instances. So I’ve decided to create a BASIC post on the subject. This is a pretty sophisticated topic having to do with scaling, high availability, resource management etc. so please don’t assume that what I’m about to say is universally appropriate. I’ll give a few suggestions and more importantly, I’ll give the reasons behind those suggestions which should give you the background you need to create your own configuration wisely.
Suggested Starting Point
- Enable “Allow host tracking” on the default in-process BizTalk Application Host
- Do NOT enable “Allow host tracking” on any other Host
Confirm with the BTS docs if you like, but only one host has to have this enabled
- Do not use this host for any ports or orchestrations, this is dedicated for host tracking
- Create at least 3 in-process hosts for each logical application that you have. (Use at least 2 hosts if you are not using any orchestrations)
- One for Send Operations [Bind your send points to this host]
- One for Receive Operations [Bind your receive ports to this host, except HTTP or SOAP]
- One for Processing [Bind your orchestrations to this host, if you have any]
- A case by case examination can be made here for additional hosts, usually for:
- High volume orchestrations or those you wish to have granular control over. These get a dedicated host
- Out of process hosts for HTTP/SOAP/Custom protocols . These get additional Hosts/Instances
Application Isolation
Having separate hosts for your applications allows you to restart the host instances for that application (perhaps because of a re-deployment) without interrupting the other apps.
Scaling Out
Having a host for Send/Receive/Process allows you to distribute your application accross several servers in your BizTalk group in a flexible manner. You can increase the number of host instances running to “beef up” any combination of these three types of operations as required.
Memory Utilization
As you well know, 32 bit systems have limits regarding how much memory can be used by a single host (process). A single host on a 32GB system will really only use about 1.8GB – 2GB or so. If you want to take full advantage of all of that juicy memory you’ll have to create more host instances and let each of them take their 2GB bites out of it. (As a rule of thumb, be sure to leave 4GB for the OS). This is not universally true (i.e. 64bit systems are different)
High Availability
Have at least two host instances running for every host that you have. This way, any single node can fail and the other node will continue to operate. This can become more complicated for Receive nodes so check the HA configuration for whatever adapter types you have running. For instance, FTP does not support locking so having two host instances reading from it can cause duplicate messages. This has to have just one host instance reading from with (which I tend to call a “Singleton” adapter configuration). In this case there is one host that is clustered for high availability
Security
You may also need to create additional hosts so that you can use different credentials to access different systems. The security context is determined by the Host/Instance which can be very helpful in scenarios that are using AD to authenticate your requested operations. (i.e. File Adapter, SQL Adapter, and I believe the MS CRM adapter, among others)
Some Caveats
- Too many hosts.
- My BizTalk groups seldom have more than 22 hosts or so (7 apps + default host for tracking) So if you begin to experience problems with Groups having more hosts than that I recommend you consider too many hosts as a possible cause. There could be RAM availability issues at these levels.
- Configurations in BTSNTsvc.exe.config may give you weird results because you may be assuming that they apply to the whole group, when really they apply to the host instances
- Again, in the BTSNTsvc.exe.config, the changes in this file are picked up when the host instance restarts. You may have changed the configurations, restarted only a handful of your hosts. Later down the road, you restart other host instances that then begin to have problems because of the contents of the BTSNTsvc.exe.config. This is VERY hard to trouble shoot and I have two recommendations.
- Don’t change BTSNTsvc.exe.config unless you absolutely MUST.
- If you change BTSNTsvc.exe.config restart ALL of your host instances immediately so you can quantify the affect of the changes immediately.
- Single Box groups
- If you’re running BizTalk all on one box then it hardly helps you to be able to partition out the operations. In this case, I would just create one host per application so you can still have Application Isolation, without worrying about the scale out features or HA (which you can’t really use with just one box anyways)
Agree, Disagree, Don’t Care? Please comment below
by community-syndication | Jul 31, 2007 | BizTalk Community Blogs via Syndication
http://www.microsoft.com/opensource
/opensource is intended for anyone interested in understanding Microsoft’s perspective on OSS. OSS technical content will continue to be hosted on Port 25, the community site for the Open Source Software Lab at Microsoft.
by community-syndication | Jul 31, 2007 | BizTalk Community Blogs via Syndication
Am I the only one who thinks this is kinda funny 😉 VIsta Gadget showing how many days to launch…..!
by community-syndication | Jul 31, 2007 | BizTalk Community Blogs via Syndication
This year I finally get a Guernsey to the Partner Conference in Hamilton Island! I will be presenting a session called “Optimizing your SOA and Business Process Infrastructure”!. What’s that mean? Beats me 😉 Essentially it is a chance for me to shed some insight into what some of our customers / partners are doing with our SOA and BP Platform and talk about the maturity cycles within this capability area. I’ve asked a great guy – Tony Barnes – MD from FormFill to present on the day with me. Thanks Tony!
For the latest information on the Australian Partner Conference, Partners go to this site. It features information on the Pre- Conference Day, Agenda and Registration.
by Richard | Jul 31, 2007 | BizTalk Community Blogs via Syndication
The main problem I have with developing BizTalk orchestrations is the fact that I’m so blind when it comes to follow the runtime processing. Using the debugger that is part of the HAT tool is slow and clumsy which IMHO makes the tool almost useless in everyday development. But there is hope!
DebugView
Sysinternals (Windows Sysinternal now – Microsoft bought them last year) DebugView is a wonderful little tool and is especially useful when it comes to figure out what’s actually going on inside an orchestration. Basically the tool listens to system wide debug output. From an orchestration it’s possible to write debug information using the .NET System.Diagnostics namespace and the Debug or Trace class.
Decide on how to filter
There are a couple of handy little tricks that makes DebugView a even better in BizTalk development. First one should try and have something in the debug messages that makes it possible to filter and distinct one’s own (as DebugView listens system wide debug output all running applications debug info will show up). Our team decided on “Sogeti” (our company name) for all our development and to have a method in our BaseLibrary component that outputs something like the below (the BaseLibrary is a small little .NET component with a couple of very useful classes we use company wide in our BizTalk related development).
<div><span style="color: #000000; ">System.Diagnostics.Debug.WrtieLine(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">Sogeti, your debug/trace message here</span><span style="color: #000000; ">"</span><span style="color: #000000; ">)</span></div>
This make is possible to have a filter in DebugView and to for example have it look something like this.
Trace full context of messages
Another little useful trick is to trace the full context of messages. This is done be storing the message in a XmlDocument typed variable and get the OuterXml property of that variable. The below code is and example of this.
<div><span style="color: #000000; ">tempXml </span><span style="color: #000000; ">=</span><span style="color: #000000; "> msgFindPartyRequest.parameters;
System.Diagnostics.Trace.Write(System.String.Concat(</span><span style="color: #000000; ">"</span><span style="color: #000000; ">Sogeti, msgFindPartyRequest: </span><span style="color: #000000; ">"</span><span style="color: #000000; ">, tempXml.OuterXml));</span></div>
Example of a full message trace.
To Trace or Debug – that’s the question
As stated earlier both System.Diagnostics.Debug and System.Diagnostics.Trace has methods (Write, WriteLine and so on) for outputting debug information. However there is only one that stays in your compiled code when switching from Development to Deployment compilation mode (guess which one ;)). So make sure you choose the right class for the right information. I like to have some critical messages left using Trace and be able to trace these even on the test and production server.
DebugView on a remote desktop
When running DebugView on an other server (say a test or a production server) using Remote Desktop I’ve found that ones has to use the console user on the server. This kind of makes sense as if we’re connection “normally” we’re creating a virtual session and that’s not were the debug information is written to.
Microsoft SOAP Toolkit 3.0
This is a totally other tool than DebugView but I thought it fit here any way. It’s a handy tool when working with SOAP based messages. Without it’s very hard to actually figure out how the raw request and response message look and why your orchestration web service is acting the way it does.
The trace tool is placed as a reverse proxy between BizTalk and the Internet. It’s setup by telling the tracing tool which localhost port to listen at (for example 9091 as in the example below) then we’ll redirect to that port by changing the setting in the BizTalk send port.
Finally we’ll set up the trace tool to listen to port 9091 and redirect all traffic to our web service URL at port 80 in this case. So basically the trace tool will catch all the traffic hitting the 9091 port and forward it.
That’s it! This is probably basic stuff for most of you but hopefully it’s useful for someone!
I’ve also noticed that the SOAP Toolkit is deprecated by Microsoft and I’d like to hear if anyone used something else (like Fiddler example) for tracing SOAP messages. I’d also love some other tips, tools and methods you use for debugging BizTalk orchestrations.
by community-syndication | Jul 31, 2007 | BizTalk Community Blogs via Syndication
The main problem I have with developing BizTalk orchestrations is the fact that I’m so blind when it comes to follow the runtime processing. Using the debugger that is part of the HAT tool is slow and clumsy which IMHO makes the tool almost useless in everyday development. But there is hope!
DebugView
Sysinternals (Windows Sysinternal now - Microsoft bought them last year) DebugView is a wonderful little […]
