by community-syndication | Mar 3, 2006 | BizTalk Community Blogs via Syndication
Here is a list of the behaviors that I have found when using the custom pipeline component vs the adapter for receiving and sending data:
Pipeline
- Allows multiple recieve ’locations’ for a single party, where you define a receive location that has the configuration information and then multiple pick ups and drop offs, where they can be any adapter (file, ftp, http, MQ Series, etc.) can be used
- Ability to use Send Port Groups (archiving is an example of this usage)
- Does not check the interchange control number for duplication per party
- Data that is recieved or sent, is not archived in the location defined in the documentshome location in the parame table
- No information is logged to the audin and audout tables, so the HIPAA_EDI reports in HAT is not available
- Promoted properties in the WPC schemas are not invoked
Adapter
- Only a single port can be defined for receiving and sending
- Copies of both edi and xml data is stored in the documentshome location
- Interchange control number checking is done
- Information is stored in the audin and audout tables, so the HIPAA_EDI reports are accessable
- Using the DefaultXML pipelines; promoted properties can be used for correllation/filtering/orchestration usage
I have put together a really simple example of both usages, so differences can be seen.
Instructions:
- Deploy the Solution
- Using the Deployment wizard, import the Binding.xml file
- In the Management Console, in the HIPAA_EDI adapter, in the properties of the Send Handler, set the the Party to Us
- Restart the HIPAA service
- Enable the Receive Locations
- Start the Send Port
- Drop the file into the ..\input directory and see if the file is produced in the ..\output directory
- Notice the following information in HAT – HIPAA_EDI Reports
by community-syndication | Mar 1, 2006 | BizTalk Community Blogs via Syndication
A recent post in the BizTalk forum, promted this entry.
The question was asked how you can put the MessageID into a dynamic send port. I suggested that you would put a rule SendPort(Microsoft.XLANGs.BaseTypes.Address)=@”file://c:\”+DocumentMsg(BTS.MessageID)+”.xml”;
The next post was telling that you can put the macro in the Expression, which is true, you can, but in the below demo, it shows why it would be preferable not to use the macros.
I created a simple orchestration that shows both ways, here is the orchestration:
Here is the expression rule:
Here is the results of running the file through the orchestration:
In HAT, which is where I think that the real value of which method you use lies, here is the message flow, notice the filename is right in the first page of the message flow for the first outbound message, and the macro %MessageID% is found on the second message:
The message that uses the macro, you have to click on the link to see the actual message ID of what was saved.
Again, here is the output:
At a recent client site, I used the macro initially, but then moved to the BTS.MessageID to ease in troubleshooting. Saving a couple of clicks when attempting to find the file really adds up after a while.
by community-syndication | Mar 1, 2006 | BizTalk Community Blogs via Syndication
Recently I’ve been working with a system that involves an externally facing server acting as a Gateway for
another internal box, in this case a BizTalk Server. This system as such runs into
the glorious problem with .NET 1.1 limiting the maximum number of outbound HTTP connections
to 2 by default. This could impact performance but this far hasn’t been a problem.
This system has been working reasonably well for some time, but now there is a desire
to change the model of communication from one of submitted requests which receive
asynch responses to getting synch responses. This means connection lifetime will increase,
and as such the number of simultaneous connections had better rise with it or scalability
will go to hell. The place this needs to be changed is in the machine.config file
for .NET 1.1 and specifically you are looking for the maxconnection element, which
defaults to 2.
by community-syndication | Mar 1, 2006 | BizTalk Community Blogs via Syndication
In the last release of the BizTalk Deployment Framework, an additional feature was
added that I thought deserved its own post for explanation…
If you are responsible for troubleshooting BizTalk applications in production, you
are likely hungry for all the information you can get about why something is failing.
Since all BizTalk project assemblies are in the GAC, the stack traces you
get (either from your own logging, or the event logs BizTalk generates for unhandled
exceptions) do not contain file and line number information. This makes post-mortem
analysis a lot harder…
A new custom NAnt task (getgacpath) and corresponding support within BizTalkDeploymentInclude.nant
was added for placing PDB (program database) files for all BizTalk and component assemblies
into the GAC. (You can use just this NAnt task without the rest of the Deployment
Framework, of course. In addition, a standalone command line utility called
GetGacPath has been included in the “Tools” download.)
Why is this interesting? Well, because it allows you to get stack traces that
include file and line number information even when assemblies reside in the GAC.
(This is obviously useful and useable outside the domain of BizTalk…) Without
extra work, file and line number information is generally missing from a stack
trace if a PDB file is not co-located physically with the corresponding assembly.
Putting the PDB file directly into the GAC is an easy way of solving this problem.
Set the “deployPDBsToGac” NAnt property to “true” in your project-specific NAnt file
to get this functionality.
A couple of points are worth mentioning:
-
If the directory structure of the GAC were to ever change, this would need to be updated.
(See my upcoming post on using this technique with .NET 2.0.)
-
It would be great if Gacutil.exe provided this functionaity, but it doesn’t.
-
You should strongly consider having your release-mode binaries (aka ‘deployment’ configuration
in BizTalk) build PDB files – there is no reason not to. Look under Project
properties/Configuration properties/Build/Generate Debugging Information.
-
Stack traces that show up in the event log either through your own logging, or via
BizTalk (because the exception was unhandled) will now have file and line number info.
For orchestrations, the file name will correspond to the temporary C# file generated
at compile time. If you use the techniques described in this
post , you can correlate back to the actual statement that failed. For instance,
the event log might say:
[4740] ERROR BizTalkSample.Orchestrations.TopLevelOrch - An exception was caught.
[a08701ff-bf80-4940-9f9f-c2bb8597684b]
System.Exception: Something exceptional happened.
at BizTalkSample.Orchestrations.TopLevelOrch.segment2(StopConditions stopOn) in
c:\Documents and Settings\Scott\Local Settings\Temp\h0li3wbi.0.cs:line 2538
-
If you grab the file mentioned (h0li3wbi.0.cs) using BTSFileDump (again,see here)
you can cross-reference this message to a location in the generated C# code for TopLevelOrch:
You might save off “h0li3wbi.0.cs” (the temp file name chosen when TopLevelOrch compiled) and
the other generated files when you do your build, using BTSFileDump. (That would be
tough to do if you build with CruiseControl, etc.) But you wouldn’t have to
do that – if you can retrieve the source code used for a particular deployment,
then the line number alone should be sufficient to get you to the exception
site (since you can find the correct temp file by looking for comments, etc. that
are in the orchestration – just like if you were using this
technique to do live debugging.) This all gets better with BizTalk 2006
– see a future post on this topic.
-
Final point: If you set deployPDBsToGac to “true”, deployments will begin by stopping
the BizTalk services (instead of just bouncing the services at the end.) The
reason for this is that the BTSNTSvc.exe process will not “let go” of an assembly
for which it has loaded a PDB file (so you have to terminate it.) This means
you may not always want this switch on for the developer edit/run/debug cycle.
by community-syndication | Mar 1, 2006 | BizTalk Community Blogs via Syndication
In the last release of the BizTalk Deployment Framework, an additional feature was
added that I thought deserved its own post for explanation…
If you are responsible for troubleshooting BizTalk applications in production, you
are likely hungry for all the information you can get about why something is failing.
Since all BizTalk project assemblies are in the GAC, the stack traces you
get (either from your own logging, or the event logs BizTalk generates for unhandled
exceptions) do not contain file and line number information. This makes post-mortem
analysis a lot harder…
A new custom NAnt task (getgacpath) and corresponding support within BizTalkDeploymentInclude.nant
was added for placing PDB (program database) files for all BizTalk and component assemblies
into the GAC. (You can use just this NAnt task without the rest of the Deployment
Framework, of course. In addition, a standalone command line utility called
GetGacPath has been included in the “Tools” download.)
Why is this interesting? Well, because it allows you to get stack traces that
include file and line number information even when assemblies reside in the GAC.
(This is obviously useful and useable outside the domain of BizTalk…) Without
extra work, file and line number information is generally missing from a stack
trace if a PDB file is not co-located physically with the corresponding assembly.
Putting the PDB file directly into the GAC is an easy way of solving this problem.
Set the “deployPDBsToGac” NAnt property to “true” in your project-specific NAnt file
to get this functionality.
A couple of points are worth mentioning:
-
If the directory structure of the GAC were to ever change, this would need to be updated.
(See my upcoming post on using this technique with .NET 2.0.)
-
It would be great if Gacutil.exe provided this functionaity, but it doesn’t.
-
You should strongly consider having your release-mode binaries (aka ‘deployment’ configuration
in BizTalk) build PDB files – there is no reason not to. Look under Project
properties/Configuration properties/Build/Generate Debugging Information.
-
Stack traces that show up in the event log either through your own logging, or via
BizTalk (because the exception was unhandled) will now have file and line number info.
For orchestrations, the file name will correspond to the temporary C# file generated
at compile time. If you use the techniques described in this
post , you can correlate back to the actual statement that failed. For instance,
the event log might say:
[4740] ERROR BizTalkSample.Orchestrations.TopLevelOrch - An exception was caught.
[a08701ff-bf80-4940-9f9f-c2bb8597684b]
System.Exception: Something exceptional happened.
at BizTalkSample.Orchestrations.TopLevelOrch.segment2(StopConditions stopOn) in
c:\Documents and Settings\Scott\Local Settings\Temp\h0li3wbi.0.cs:line 2538
-
If you grab the file mentioned (h0li3wbi.0.cs) using BTSFileDump (again,see here)
you can cross-reference this message to a location in the generated C# code for TopLevelOrch:
You might save off “h0li3wbi.0.cs” (the temp file name chosen when TopLevelOrch compiled) and
the other generated files when you do your build, using BTSFileDump. (That would be
tough to do if you build with CruiseControl, etc.) But you wouldn’t have to
do that – if you can retrieve the source code used for a particular deployment,
then the line number alone should be sufficient to get you to the exception
site (since you can find the correct temp file by looking for comments, etc. that
are in the orchestration – just like if you were using this
technique to do live debugging.) This all gets better with BizTalk 2006
– see a future post on this topic.
-
Final point: If you set deployPDBsToGac to “true”, deployments will begin by stopping
the BizTalk services (instead of just bouncing the services at the end.) The
reason for this is that the BTSNTSvc.exe process will not “let go” of an assembly
for which it has loaded a PDB file (so you have to terminate it.) This means
you may not always want this switch on for the developer edit/run/debug cycle.
by community-syndication | Mar 1, 2006 | BizTalk Community Blogs via Syndication
There are some updates to the Deployment Framework and log4net story that I’ve been
meaning to release for quite some time. After this, I’ll focus my deployment efforts
on BizTalk 2006 (didn’t I say that last release?)
Below are a list of revisions. On this blog’s home page under “Downloads”,
you will find an updated full download (with sample), as well as updated
“Core” and “Tools” downloads. “Core” contains only the deployment framework
itself and can be used for upgrades of existing projects. “Tools” contains source
for all utilities used by the framework. Be sure to put a fresh copy
of BizTalk.NAnt.Tasks.dll in your nant\bin directory.
-
Update to the log4net.Ext.Serializable library. Note that the “Tools” download
now has variants of this library for both log4net 1.2.0 and 1.2.9, though the sample
is built against 1.2.9. The important change here is that the original version
of the library had a threading bug…as a result, the usage pattern for the library
has changed somewhat. Now, an instance of log4net.helpers.PropertiesCollectionEx
is declared (as an orchestration variable), and an expression shape at the top of
your orchestration will do something like below (notice logProps is passed to all logging
calls so as to preserve context despite thread changes across dehydrations.)
See the sample for additional detail.
logger = log4net.Ext.Serializable.SLogManager.
GetLogger(@"BizTalkSample",log4net.helpers.CallersTypeName.Name);
...
logProps.Set("InstanceId",TopLevelOrch(Microsoft.XLANGs.BaseTypes.InstanceId));
logger.Debug(logProps,"Received top level request...");
-
Update to the SSOSettingsFileReader class to include an “Update” method.
Likewise, an additional custom NAnt task called “updatessoconfigitem” that allows
you to add or update config items within an SSO affiliate application. What
is the use case for this? Suppose you have a piece of information that you a)
don’t want to store in SettingsFileGenerator.xls (perhaps because you can’t manage that
file securely) and b) you need access to at run time. (For instance, it
might be an Oracle username/password.) You could ask an operator/admin for this
information at deployment time using SetEnvUI.exe/InstallWizard.xml (from the
deployment framework) — and then in your project-specific NAnt file do this:
<target name="customSSO" if="${property::exists('serverDeploy') and serverDeploy}">
<updatessoconfigitem ssoitemname="databaseUserName" ssoitemvalue="${sys.env.databaseUserName}"/>
<updatessoconfigitem ssoitemname="databasePassword" ssoitemvalue="${sys.env.databasePassword}"/>
</target>
At runtime, you can simply use SSOSettingsFileReader to retrieve this information.
Note that this example checks for whether we are doing a server deployment – so in
this case, we would have the development environment values in SettingsFileGenerator.xls.
-
Added a small utility called SSONameValueDump to the tools download to view current
name/value pairs of an SSO affiliate application (if you have appropriate permissions.)
-
Update to the <controlbiztalkports> custom NAnt task to allow Send Port Groups
to be shared across projects. That is, Send Port Groups will only be removed
if they no longer contain Send Ports (and putting new Send Ports into an
already-existing Send Port Group will happen automatically.)
-
XmlPreprocess will be run against log4net configuration files, if you are using log4net.
This allows you to use SettingsFileGenerator.xls to control logging levels for different
physical environments.
-
SetEnvUI.exe no longer defaults to “last directory” when doing a file browse, to avoid
confusion between applications when managing multiple BizTalk application installs.
-
If you use ElementTunnel,
you will find that the xpath file will be reversed automatically for unescape operations
(thanks to Frank de Groot for that update!)
-
Fix: When deploying HTTP/SOAP infrastructure, file system permissions granted to the
specified application pool account.
-
Fix: The “quick update” target (updateOrchestrations) now respects true/false properties
for includeSchemas, includeTransforms, etc. (A menu option in Visual Studio is
created for this target when you use the MakeBizTalkExternalTools script. The target
updates orchestrations, components, schemas, and transforms (see Flanders’ post on
this) Using this as part of your normal edit/run/debug cycle is a must for
productivity…)
-
Fix: Handle multiple assemblies (of one artifact type) correctly in more cases, including
where we are not deploying to the management database (i.e. gac deployment only.)
Download: Full
Sample, Framework
Core, Tools
Source
by community-syndication | Mar 1, 2006 | BizTalk Community Blogs via Syndication
There are some updates to the Deployment Framework and log4net story that I’ve been
meaning to release for quite some time. After this, I’ll focus my deployment efforts
on BizTalk 2006 (didn’t I say that last release?)
Below are a list of revisions. On this blog’s home page under “Downloads”,
you will find an updated full download (with sample), as well as updated
“Core” and “Tools” downloads. “Core” contains only the deployment framework
itself and can be used for upgrades of existing projects. “Tools” contains source
for all utilities used by the framework. Be sure to put a fresh copy
of BizTalk.NAnt.Tasks.dll in your nant\bin directory.
-
Update to the log4net.Ext.Serializable library. Note that the “Tools” download
now has variants of this library for both log4net 1.2.0 and 1.2.9, though the sample
is built against 1.2.9. The important change here is that the original version
of the library had a threading bug…as a result, the usage pattern for the library
has changed somewhat. Now, an instance of log4net.helpers.PropertiesCollectionEx
is declared (as an orchestration variable), and an expression shape at the top of
your orchestration will do something like below (notice logProps is passed to all logging
calls so as to preserve context despite thread changes across dehydrations.)
See the sample for additional detail.
logger = log4net.Ext.Serializable.SLogManager.
GetLogger(@"BizTalkSample",log4net.helpers.CallersTypeName.Name);
...
logProps.Set("InstanceId",TopLevelOrch(Microsoft.XLANGs.BaseTypes.InstanceId));
logger.Debug(logProps,"Received top level request...");
-
Update to the SSOSettingsFileReader class to include an “Update” method.
Likewise, an additional custom NAnt task called “updatessoconfigitem” that allows
you to add or update config items within an SSO affiliate application. What
is the use case for this? Suppose you have a piece of information that you a)
don’t want to store in SettingsFileGenerator.xls (perhaps because you can’t manage that
file securely) and b) you need access to at run time. (For instance, it
might be an Oracle username/password.) You could ask an operator/admin for this
information at deployment time using SetEnvUI.exe/InstallWizard.xml (from the
deployment framework) — and then in your project-specific NAnt file do this:
<target name="customSSO" if="${property::exists('serverDeploy') and serverDeploy}">
<updatessoconfigitem ssoitemname="databaseUserName" ssoitemvalue="${sys.env.databaseUserName}"/>
<updatessoconfigitem ssoitemname="databasePassword" ssoitemvalue="${sys.env.databasePassword}"/>
</target>
At runtime, you can simply use SSOSettingsFileReader to retrieve this information.
Note that this example checks for whether we are doing a server deployment – so in
this case, we would have the development environment values in SettingsFileGenerator.xls.
-
Added a small utility called SSONameValueDump to the tools download to view current
name/value pairs of an SSO affiliate application (if you have appropriate permissions.)
-
Update to the <controlbiztalkports> custom NAnt task to allow Send Port Groups
to be shared across projects. That is, Send Port Groups will only be removed
if they no longer contain Send Ports (and putting new Send Ports into an
already-existing Send Port Group will happen automatically.)
-
XmlPreprocess will be run against log4net configuration files, if you are using log4net.
This allows you to use SettingsFileGenerator.xls to control logging levels for different
physical environments.
-
SetEnvUI.exe no longer defaults to “last directory” when doing a file browse, to avoid
confusion between applications when managing multiple BizTalk application installs.
-
If you use ElementTunnel,
you will find that the xpath file will be reversed automatically for unescape operations
(thanks to Frank de Groot for that update!)
-
Fix: When deploying HTTP/SOAP infrastructure, file system permissions granted to the
specified application pool account.
-
Fix: The “quick update” target (updateOrchestrations) now respects true/false properties
for includeSchemas, includeTransforms, etc. (A menu option in Visual Studio is
created for this target when you use the MakeBizTalkExternalTools script. The target
updates orchestrations, components, schemas, and transforms (see Flanders’ post on
this) Using this as part of your normal edit/run/debug cycle is a must for
productivity…)
-
Fix: Handle multiple assemblies (of one artifact type) correctly in more cases, including
where we are not deploying to the management database (i.e. gac deployment only.)
Download: Full
Sample, Framework
Core, Tools
Source
by community-syndication | Mar 1, 2006 | BizTalk Community Blogs via Syndication
At my current client they have been relatively slow on the uptake of Windows Server 2003 SP1, so those of us working within the Integration Services team have not had to deal with any complications that this service pack has introduced to BizTalk. Recently however we have been installing SP1 and have noticed a number of errors that have been introduced because of this.
We observed that BizTalk Host Instances were being stopped and restarted automatically. Following this up in the Windows Event Log we noticed a number of error messages relating to DBNETLIB and other entries indicating that there were issues relating to network connectivity. Some captures of these are shown below.
After some research we came across the following Knowledge Base article at Microsoft: http://support.microsoft.com/default.aspx?scid=kb;en-us;899599
Now the symptoms described in this article are exactly what we were experiencing. Apparently Windows SP1 introduces some extra security feature that reduces the size of the queue for concurrent TCP/IP connections to the server, which helps to prevent denial of service attacks.
All well and good, however the article suggests the actual issue only materialises when TCP/IP is under heavy load, sometimes incorrectly identifying valid TCP/IP connections as a denial of service attack. However, we have been receiving the errors when the BizTalk Group was under no load whatsoever.
After implementing the registry modifications in that article the problems did disappear, but I am interested why the BizTalk Group was experiencing the condition at all when there was no load. I spent some time investigating what may be the root cause and the only conclusion I could come to was that it may be related to the number of host instances that we had within the BizTalk Group; there were 40 in this case.
I have not had the time to investigate this any further, although a simple enough test would be to see whether the errors materialise with less host instances. One other thing that I did notice after installing Windows 2003 SP1 though was the number of security log entries. Is it just me or do we now see a huge increase in the number of entries? Something else I would like to investigate time permitting.
Just a final note, I have seen this occur with both BizTalk 2004 and BizTalk 2006.
by community-syndication | Mar 1, 2006 | BizTalk Community Blogs via Syndication
At my current client they have been relatively slow on the uptake of Windows Server 2003 SP1, so those of us working within the Integration Services team have not had to deal with any complications that this service pack has introduced to BizTalk. Recently however we have been installing SP1 and have noticed a number of errors that have been introduced because of this.
We observed that BizTalk Host Instances were being stopped and restarted automatically. Following this up in the Windows Event Log we noticed a number of error messages relating to DBNETLIB and other entries indicating that there were issues relating to network connectivity. Some captures of these are shown below.
After some research we came across the following Knowledge Base article at Microsoft: http://support.microsoft.com/default.aspx?scid=kb;en-us;899599
Now the symptoms described in this article are exactly what we were experiencing. Apparently Windows SP1 introduces some extra security feature that reduces the size of the queue for concurrent TCP/IP connections to the server, which helps to prevent denial of service attacks.
All well and good, however the article suggests the actual issue only materialises when TCP/IP is under heavy load, sometimes incorrectly identifying valid TCP/IP connections as a denial of service attack. However, we have been receiving the errors when the BizTalk Group was under no load whatsoever.
After implementing the registry modifications in that article the problems did disappear, but I am interested why the BizTalk Group was experiencing the condition at all when there was no load. I spent some time investigating what may be the root cause and the only conclusion I could come to was that it may be related to the number of host instances that we had within the BizTalk Group; there were 40 in this case.
I have not had the time to investigate this any further, although a simple enough test would be to see whether the errors materialise with less host instances. One other thing that I did notice after installing Windows 2003 SP1 though was the number of security log entries. Is it just me or do we now see a huge increase in the number of entries? Something else I would like to investigate time permitting.
Just a final note, I have seen this occur with both BizTalk 2004 and BizTalk 2006.
