by community-syndication | Mar 24, 2005 | BizTalk Community Blogs via Syndication
A short while ago, I did a presentation for the Twin Cities .NET user group on
log4net. (You might recall an earlier
blog entry where I discussed using log4net with BizTalk 2004…)
The presentation was not specific to BizTalk 2004 – instead, it attempts to describe
why I think the log4net library is so very well thought out. You can grab it here,
if you like.
The last slide references Loren
Halvorsen’s comparison of log4net and the new Microsoft
Enterprise Library Logging Block, which I would recommend taking a look at (the
comparison, that is.) Tom Hollander (among many others!) later weighed
in with this
piece.
Lastly, 1.2.9 beta of log4net has
recently become available (release notes here)….However,
my extensions
to log4net have not yet been updated to reflect this new drop. I’ll be sure
to post when I’m able to update.
by community-syndication | Mar 24, 2005 | BizTalk Community Blogs via Syndication
Are you going to Tech Ed 2005? Brennan and I (Brandon) will be presenting a cabana session during the breakouts and available to answer questions at the BizTalk booth all week. Hope to see you there! Cabana Title: Implementing Real-World Integration Patterns with BizTalk Abstract: This session will review several BizTalk 2004 patterns, including accompanying implementation pattern examples. In addition, the session will illustrate how to successfully leverage patterns within common real-world technology and business scenarios….
by stephen-w-thomas | Mar 21, 2005 | Stephen's BizTalk and Integration Blog
It is a common request. How can I debatch a message into smaller batches rather than single messages inside BizTalk 2004?
I have tried many times to get this accomplished natively inside the pipeline. One would think it would be as easy as setting the group property to 10 if you wanted a batch size of 10, but that does not seem to work. The only way I have been able to break up a message into smaller batches has been inside custom code.
I wrote a .net component that can be called using an atomic scope shape from inside an Orchestration. This component will return smaller batches of any size you specify inside the Orchestration and wrap the smaller batches inside a specified root node. This .net component uses the XmlTextReader to stream the message along and debatch it.
This .net component is called StreamHelper. It illustrates the following:
- Using XmlTextReader
- Passing an Xlang Message into a .net component
- Returning an Xlang Message from a .net component
- Setting context properties on a message returned from a .net component
Download: Debatch Streaming Sample
Setup: Download and extract the solution. Build and GAC the StreamHelper assembly then Build and Deploy the DebatchProperties and DebatchOrch assemblies. I have included a basic sample file to test with. For more detailed information on the sample see the comments throughout the solution.
A better way to do it?
I’m sure there is. I just put this together in a short amount of time. I would guess the most efficient place for this type of work would be inside a custom pipeline component. But, working with custom pipelines can be difficult.
How does this debatching approach perform compared to the others?
Performance was almost identical to that of the Xml Node List inside an atomic scope. In this sample, I used the XmlTextReader and passed Xlang messages in and out of the component to try to increase performance. That did not seem to accomplish anything. As expected, as the total number of messages inside the atomic scope increased the performance decreased.
Type |
XML Size (MB) |
# Nodes |
Batch Size |
Time (Sec) |
Msg/Sec |
Msg Size (kb) |
.net Stream |
1.6 |
500 |
1 |
5 |
100.0 |
3.0 |
.net Stream |
3.6 |
1100 |
1 |
18 |
61.1 |
3.0 |
.net Stream |
7.2 |
2200 |
1 |
44 |
50.0 |
3.0 |
.net Stream |
18.1 |
5500 |
1 |
224 |
24.6 |
3.0 |
.net Stream |
128.6 |
38500 |
1 |
— |
FAIL |
— |
.net Stream |
1.6 |
500 |
10 |
1 |
500.0 |
3.0 |
.net Stream |
3.6 |
1100 |
10 |
3 |
366.7 |
3.0 |
.net Stream |
7.2 |
2200 |
10 |
9 |
244.4 |
3.0 |
.net Stream |
18.1 |
5500 |
10 |
15 |
366.7 |
3.0 |
.net Stream |
128.6 |
38500 |
10 |
159 |
242.1 |
3.0 |
For more debatching information please see my past post:
Debatching Options and Performance Considerations in BizTalk 2004
by community-syndication | Mar 21, 2005 | BizTalk Community Blogs via Syndication
(Update: See the latest on the Deployment Framework here.)
Just when you thought automated deployments for BizTalk 2004 were safe, another update
to the “Deploy with NAnt” template comes along… 🙂
This is a very worthwhile “upgrade” to consider taking advantage of. One of
the big items is support for “configuration info within SSO”, but a complete list
of features/fixes, etc. is below.
SSO Integration: If you are currently using the SettingsFileGenerator.xls spreadsheet
(discussed in previous entries) in conjunction with the XmlPreProcess tool
to handle “environment-relative” configuration – that is, to handle variations between
your dev/QA/production environments – you will be glad to know you can get more leverage
out of that process now…
To review, with this process, you use the names that are defined in the first column
of the SettingsFileGenerator spreadsheet (shown below) as “tokens” within your binding
file, as in:
<!-- ifdef ${ xml preprocess} -->
<!-- <Address>${FileSendLocation}\%MessageID%.xml</Address> -->
<!-- else -->
<Address>C:\temp\BizTalkSample OutDir\%MessageID%.xml</Address>
<!-- endif -->
At install time, the value for FileSendLocation that is appropriate to the environment
you are deploying to is “plugged in” by XmlPreProcess (but
for developers, the file remains legal XML.) The spreadsheet might look as follows:
(click)
Now, however, you can also put in name-value pairs (whether they vary
by environment or not) into the spreadsheet that you need access to at run time (i.e.
general configuration information.)
The name-value pairs will be deployed into a newly created affiliate application within
the SSO, and can be read using the SSOSettingsFileReader class (which serves as a
cache, too – and is provided in the download.) The static methods on this
class can be used from an orchestration expression shape, or from assemblies that
orchestrations call – as in string test = SSOSettingsFileReader.ReadValue(“BizTalkSample”,”SomeAppConfigItem”);
(By the way, the “nested names” shown in the spreadsheet above are just
meant to suggest a partitioning mechanism you might want to use, rather than one flat
“namespace” of config data.)
Of course, the deployment scripts handle all the interaction with the SSO database
for deployments/un-deployments. This whole mechanism (SettingsFileGenerator.xls,
XmlPreProcess, and now SSOSettingsFileImport.exe) is worth using even if you don’t
use the rest of this script infrastructure. Note that Jon Flanders had a great
article on SSO-based configuration using a strongly-typed approach, where as this
approach is loosely-typed, but leverages a mechanism you might already be using (i.e.
the spreadsheet concept.) (Note: Jon’s code for creating/deleting affiliate
apps helped me complete the SSOSettingsFileImport.exe utility quickly…)
Other features/fixes:
-
Support for multiple schema assemblies, including the case where schema assemblies
reference each other due to schema imports. (Undeployments occur in reverse
order of deployments.)
-
Support for multiple orchestration assemblies, including the case where orchestration
assemblies reference each other due to the use of Call/Start Orchestration shapes.
(Undeployments occur in reverse order of deployments.)
-
Support for multiple orchestration and port binding files (just list them as comma
separated values in the orchBindings/portBindings properties.)
-
Fixed bug that occurred when undeploying ISAPI extensions in Windows 2003 / IIS 6.
-
Fixed bug in undeploying pipeline components (thanks to John Adams)
-
Added /NOFORCE flag to IISRESET calls in accordance with http://support.microsoft.com/kb/286196
-
New property “includeInstallUtilForComponents” was added, which can be set to ‘true’
in your project-specific nant file. This will cause your component assemblies
to be called with “installutil.exe” (with the /u flag for undeployment.) This
is useful if you have .NET Installer-derived classes that need to get called for creating
event sources, perf counters, etc. (Yes, you could certainly argue these should
be called by the WiX piece of the script infrastructure instead, but this method works
and is more in keeping with the spirit of the entire process.) Note there is
no harm if a given component assembly listed in your components property does not have
an installer class.
-
Eliminated the need for the includeCustomTarget property, and provided additional
ways to supply custom functionality. Project-specific nant file can now supply
any or all of customDeployTarget, customPostDeployTarget, customUndeployTarget, customPostUndeployTarget
— and they will be called if they are present.
-
Integrated the deployment of log4net-specific stuff, if the includelog4net property
is set to true. See the work on BizTalk/log4net
integration for more information. (Log4net sample usage is included in the
new sample.)
-
Now using NAnt version .85 – so be sure to grab the latest NAnt/NAntContrib since
the nant scripts are using .85-specific syntax to avoid “deprecated” warnings that
were occurring otherwise. Note that even once you have installed NAnt .85, you
will still be able to deploy BizTalk projects that are using old versions of the deployment
scripts. Note also that this is a pretty basic port to .85 – there are no doubt
more elegant ways of doing lots of things that I haven’t investigated yet, now that
nant has expression support.
Other notes
The scripts like to see BizTalk projects using Debug and Release for target names.
A long while back I had suggested a file-level search/replace within btproj files
to change these, but two more elegant options are available:
-
In project properties, change the output location for the “Development”
configurations to “bin\debug” and output location of “Deployment”
configurations to “bin\release”.
-
(Via Puneet Gupta): Change the template for BizTalk projects, found in BTSApp.btproj
file in %BizTalkInstallDir%\Developer Tools\BizTalkWizards\BTSApp\Templates\1033,
to reflect Debug and Release targets.
Another note: Lots of folks have noticed that XmlPreProcess can be used for a lot
more than port binding files! Log4net config files, orchestration binding files,
you name it. See the full Word documentation in the download for an example
of how to do this within customDeployTarget.
How do I upgrade?
-
With a safe copy of your BizTalk solution, expand the “Core” zip contents
right on your project directory, such that the core scripts will overwrite what you
have now (but your project-specific scripts will be preserved.)
-
Manually copy the WiX-related updates to your *.WiXSetup directory, since they will
get unzipped to BizTalkSample.WiXSetup
-
Do a deployment on a developer workstation and a server to test things out…
Downloading
-
You can always use the right-hand links on this blog.
-
Full sample here,
core scripts here
-
GotDotNet workspace here.
Because of issues with GotDotNet, I’m only going to be keeping the EXE utility
source code up to date – not the entire sample & core tree.
Enjoy – comments always welcome and appreciated.
by community-syndication | Mar 21, 2005 | BizTalk Community Blogs via Syndication
(Update: See the latest on the Deployment Framework here.)
Just when you thought automated deployments for BizTalk 2004 were safe, another update
to the “Deploy with NAnt” template comes along… 🙂
This is a very worthwhile “upgrade” to consider taking advantage of. One of
the big items is support for “configuration info within SSO”, but a complete list
of features/fixes, etc. is below.
SSO Integration: If you are currently using the SettingsFileGenerator.xls spreadsheet
(discussed in previous entries) in conjunction with the XmlPreProcess tool
to handle “environment-relative” configuration – that is, to handle variations between
your dev/QA/production environments – you will be glad to know you can get more leverage
out of that process now…
To review, with this process, you use the names that are defined in the first column
of the SettingsFileGenerator spreadsheet (shown below) as “tokens” within your binding
file, as in:
<!-- ifdef ${ xml preprocess} -->
<!-- <Address>${FileSendLocation}\%MessageID%.xml</Address> -->
<!-- else -->
<Address>C:\temp\BizTalkSample OutDir\%MessageID%.xml</Address>
<!-- endif -->
At install time, the value for FileSendLocation that is appropriate to the environment
you are deploying to is “plugged in” by XmlPreProcess (but
for developers, the file remains legal XML.) The spreadsheet might look as follows:
(click)
Now, however, you can also put in name-value pairs (whether they vary
by environment or not) into the spreadsheet that you need access to at run time (i.e.
general configuration information.)
The name-value pairs will be deployed into a newly created affiliate application within
the SSO, and can be read using the SSOSettingsFileReader class (which serves as a
cache, too – and is provided in the download.) The static methods on this
class can be used from an orchestration expression shape, or from assemblies that
orchestrations call – as in string test = SSOSettingsFileReader.ReadValue(“BizTalkSample”,”SomeAppConfigItem”);
(By the way, the “nested names” shown in the spreadsheet above are just
meant to suggest a partitioning mechanism you might want to use, rather than one flat
“namespace” of config data.)
Of course, the deployment scripts handle all the interaction with the SSO database
for deployments/un-deployments. This whole mechanism (SettingsFileGenerator.xls,
XmlPreProcess, and now SSOSettingsFileImport.exe) is worth using even if you don’t
use the rest of this script infrastructure. Note that Jon Flanders had a great
article on SSO-based configuration using a strongly-typed approach, where as this
approach is loosely-typed, but leverages a mechanism you might already be using (i.e.
the spreadsheet concept.) (Note: Jon’s code for creating/deleting affiliate
apps helped me complete the SSOSettingsFileImport.exe utility quickly…)
Other features/fixes:
-
Support for multiple schema assemblies, including the case where schema assemblies
reference each other due to schema imports. (Undeployments occur in reverse
order of deployments.)
-
Support for multiple orchestration assemblies, including the case where orchestration
assemblies reference each other due to the use of Call/Start Orchestration shapes.
(Undeployments occur in reverse order of deployments.)
-
Support for multiple orchestration and port binding files (just list them as comma
separated values in the orchBindings/portBindings properties.)
-
Fixed bug that occurred when undeploying ISAPI extensions in Windows 2003 / IIS 6.
-
Fixed bug in undeploying pipeline components (thanks to John Adams)
-
Added /NOFORCE flag to IISRESET calls in accordance with http://support.microsoft.com/kb/286196
-
New property “includeInstallUtilForComponents” was added, which can be set to ‘true’
in your project-specific nant file. This will cause your component assemblies
to be called with “installutil.exe” (with the /u flag for undeployment.) This
is useful if you have .NET Installer-derived classes that need to get called for creating
event sources, perf counters, etc. (Yes, you could certainly argue these should
be called by the WiX piece of the script infrastructure instead, but this method works
and is more in keeping with the spirit of the entire process.) Note there is
no harm if a given component assembly listed in your components property does not have
an installer class.
-
Eliminated the need for the includeCustomTarget property, and provided additional
ways to supply custom functionality. Project-specific nant file can now supply
any or all of customDeployTarget, customPostDeployTarget, customUndeployTarget, customPostUndeployTarget
— and they will be called if they are present.
-
Integrated the deployment of log4net-specific stuff, if the includelog4net property
is set to true. See the work on BizTalk/log4net
integration for more information. (Log4net sample usage is included in the
new sample.)
-
Now using NAnt version .85 – so be sure to grab the latest NAnt/NAntContrib since
the nant scripts are using .85-specific syntax to avoid “deprecated” warnings that
were occurring otherwise. Note that even once you have installed NAnt .85, you
will still be able to deploy BizTalk projects that are using old versions of the deployment
scripts. Note also that this is a pretty basic port to .85 – there are no doubt
more elegant ways of doing lots of things that I haven’t investigated yet, now that
nant has expression support.
Other notes
The scripts like to see BizTalk projects using Debug and Release for target names.
A long while back I had suggested a file-level search/replace within btproj files
to change these, but two more elegant options are available:
-
In project properties, change the output location for the “Development”
configurations to “bin\debug” and output location of “Deployment”
configurations to “bin\release”.
-
(Via Puneet Gupta): Change the template for BizTalk projects, found in BTSApp.btproj
file in %BizTalkInstallDir%\Developer Tools\BizTalkWizards\BTSApp\Templates\1033,
to reflect Debug and Release targets.
Another note: Lots of folks have noticed that XmlPreProcess can be used for a lot
more than port binding files! Log4net config files, orchestration binding files,
you name it. See the full Word documentation in the download for an example
of how to do this within customDeployTarget.
How do I upgrade?
-
With a safe copy of your BizTalk solution, expand the “Core” zip contents
right on your project directory, such that the core scripts will overwrite what you
have now (but your project-specific scripts will be preserved.)
-
Manually copy the WiX-related updates to your *.WiXSetup directory, since they will
get unzipped to BizTalkSample.WiXSetup
-
Do a deployment on a developer workstation and a server to test things out…
Downloading
-
You can always use the right-hand links on this blog.
-
Full sample here,
core scripts here
-
GotDotNet workspace here.
Because of issues with GotDotNet, I’m only going to be keeping the EXE utility
source code up to date – not the entire sample & core tree.
Enjoy – comments always welcome and appreciated.
by community-syndication | Mar 21, 2005 | BizTalk Community Blogs via Syndication
For those who are wondering why I’m not blogging very regularly: It’s not that I’m lying on my back and sun-bathing on the beach or anything (which was never been an option since it has been snowing a lot), I’m just rather busy lately. I was doing a SP1 evaluation earlier, for which MS has granted me a prestigious award for ‘Most Valuable Debugger’. It’s a kind of a joke, but nevertheless I’m very proud and amused at the same time :-). I guess I will have to change my blogging style in order to make it up in the future. So expect some more ‘publish some ideas quickly’ posts soon…
Here is something very interesting – I’ve discovered it some time ago when I was searching for general background information about convoys. This explains why it’s so hard to find anything theoretical and not directly from MS. More precisely, the ‘Processing convoy workflow scenarios’ patent has been linked to 8 different people, amongst them 1 very famous blogger named Lee.
by stephen-w-thomas | Mar 21, 2005 | Downloads
Debatching into single messages is easy with BizTalk 2004. This sample shows how messages can be groups into batches of any size inside an Orchestration using an external .net component. The external .net component uses streams to read the message from an Orchestration. The message is passed into the component as an XLang message type.
This sample should work in BizTalk 2004 and BizTalk 2006.
Get more information from the original blog post on this topic: http://www.biztalkgurus.com/biztalk_server/biztalk_blogs/b/biztalk/archive/2005/03/22/debatching-messages-into-smaller-batches-in-an-orchestration.aspx
by community-syndication | Mar 15, 2005 | BizTalk Community Blogs via Syndication
I’ve seen numerous handy WMI scripts and code samples from fellow BizTalk bloggers and from the BizTalk help. Amazingly every one of these samples use late-bound objects calls to invoke methods and access properties from the BizTalk management objects. This makes the code rather verbose, it’s very inconvenient and also very error prone. There’s a better way to write your WMI code. Therefore you have to download first and then install the ‘Management (WMI) Extensions for Visual Studio .NET 2003 Server Explorer’. This tool makes WMI programming much easier and enjoyable (at least for me). It’s no rocket science or anything but it comes in very handy for making your first steps on the WMI moon. You don’t have to take my word on it, look for yourself:
After having installed the extension you can go into your Visual Studio’s Server Explorer and select the namespace you want to add management classes from (select ‘add classes’ from the context menu).
After selecting the desired classes from your namespace (‘root \MicrosoftBizTalkServer’ I assume) the Server Explorer window will contain a list of the selected management object-types. It also shows a list of all instances for a given class when you click it. When having selected an instance, all of its properties will be shown inside the property-window (you could modify them from there too, but I wouldn’t recommend doing it this way).
Also methods can be invoked on classes and instances; this is nice for testing purposes. *No big deal* you are thinking of course; now pay attention:
When selecting ‘Generate Managed Class’ from the menu, the extension will automatically generate a wrapper class and add it to your solution’s startup project (it targets your .net project language).
The generated classes have strongly typed properties and methods, so you have access to and can invoke by simply using their name (even enum-types are created). You benefit from good old Intellisense again. Behind the covers the generated wrapper classes still use late-bound objects and calls of course, but you don’t have to care at least about this, nor write ugly late binding code. Now you can write clean high-level WMI code a lot faster.
Take a look at this little WMI sample (where I make a call to ‘GetInstances’: this overloaded method is created automatically and it requires only the where-part of the WQL query to return a typed collection of your WMI object-type).
Try to rewrite the following method (please don’t focus on the language change; it’s purely for educational purposes. I’ve never executed a single line of this code 🙂
Into this:
Which one do you prefer?
by community-syndication | Mar 14, 2005 | BizTalk Community Blogs via Syndication
Okay, peridiodically the marketing team / other teams ask me to post things to make sure the word is getting out about events. My guess is that all of you are reading scott woodgate’s blog (see my links) and if you are not you should, so these will probably be duplicates, but please read them and in this case do the feedback so we can have a strong tech ed session this year (I will be there, so please stop by my talk and we can chat :). Also, I think I have wrangled the owner of the orchestration compiler and now one of the key developers on the engine also to put some info on the blog. That would be great stuff. He is the ultimate source of information related to compiler questions and how to express certain design patterns in your orchestration. Looking forward to hearing from Paul. Okay, the Tech-Ed announcements:
Industry leaders will conduct special in-depth workshops on Sunday, June 5, from 1 P.M. to 6 P.M. These concentrated “Deep Dive” sessions will last five hours and provide an in-depth look at important technology areas and solutions.
The pre-con will start off with an overview session led by Rebecca Dias that gives an overview of SO, Microsoft solutions in the SO space and how they work together. It is then followed by a session by Yasser Shohoud on how to build web service on .Net. The session is followed by a webservices lab. This is then followed by a HIS session by Paul Larsen that shows how to extend the reach of SOA to mainframe via HIS. Following Paul’s session, the earlier web services lab is extended with HIS to expose legacy IBM systems as a web service. Mike Wood’s session picks up from there and shows cases how to orchestrate, expose and integrate with web services. Mike’s talk is followed by another hands-on-lab session where we extend the web services / HIS lab with orchestration in the middle orchestrating these services and maybe followed by exposing the resulting orchestration again as a web service.
Title: Building Simple and Complex Web Services using the .Net Framework, BizTalk Server and Host Integration Server
Are you convinced that Service Orientation is the way to go, but don’t yet have the technical skills to begin implementation or are confused by the plethora of choices Microsoft offers you to build Service Oriented systems? Do you want fundamentals on Microsoft’s integration technologies and guidance on trade offs if you use one solution vs. the other for different applicable scenarios? If so, this is a training you won’t want to miss. This training will provide you the guidance and information you need to make an informed decision on how to best construct your integration solution using Microsoft technologies that will ensure your integration solution is extensible, maintainable and inter-operable. Our experts will give you highlights of each technology by demonstrating how you can construct loosely-coupled, composite, and secure systems that bridge corporate firewalls based on core .NET Framework technologies, expose line of business applications in your IBM datacenters as web services using Host Integration Server, and orchestrate the activities of disparate web services into a composite application using Microsoft’s Business Process Management solution BizTalk Server. To solidify your learning, each session is followed by a lab that gives you hands on experience on the technologies just covered. These labs builds upon one another, so by the end of the day you will have the experience of building a real world Service Oriented solution using all the concepts and technologies you just learned.
I am pretty sure they want you to register for the class and I think the tech ed site is at www.msteched.com
This year TechEd is hosting a Birds of a Feather for both the IT Pro and Developer audiences. Topics are then posted to the website where everyone can go in and vote on what their favorite sessions are. On April 11th we’ll close down the voting, and the sessions with the most votes will be held at Tech Ed. This tool was based on feedback we received at Tech Ed last year that attendees wanted to drive this more and be able to submit and choose what BOF topics were presented.
TechEd BOF site: http://www.msteched.com/content/bof.aspx
Thanks
Lee
by community-syndication | Mar 14, 2005 | BizTalk Community Blogs via Syndication
Steve and
I wound up working on the same problem at the same time (probably for the same person…)
When working with the MSMQ adapter, keep in mind that you must reference the Microsoft.BizTalk.Adapter.MSMQ.MsmqAdapterProperties.dll
to have access to MSMQ-specific properties. (The list of properties available
is in the adapter
documentation.)
Why? The intellisense in the expression shape (when using the parentheses syntax on
messages, ports, etc.) is looking for classes derived from Microsoft.XLANGs.BaseTypes.PropertyBase
to present in the drop down list. Some of those classes are part of your “native”
BizTalk installation, some are provided by add-on adapters, and (of course) some are
provided by property schemas that you develop. A reference to the containing
assembly is necessary to find them.
Other things to note relative to the MSMQ adapter:
-
For dynamic send ports, the syntax can look like: SomePort(Microsoft.XLANGs.BaseTypes.Address)
= @”MSMQ://FORMATNAME:DIRECT=OS:SOMEMACHINE\PRIVATE$\SOMEQ”;
-
In the case of dynamic send ports, the runtime will use non-transactional sends by
default. If you want transactional sends, you will need to set the transactional
property: outboundMsg(MSMQ.Transactional) = true;
-
Transactional messages which fail to deliver to the remote queue will be found in
the local Transactional Dead Letter queue. No error will be raised by the adapter.
This may mean that your design requires acknowledgement messages to achieve what you
are looking for.