ESB Toolkit Tip #17 Use ETW for high-speed tracing

The .NET Framework’s standard System.Diagnostics.Trace component is often being used for instrumenting .NET applications but when event capture is enabled in the DebugView tool it significantly impacts application performance.  Therefore it’s not suitable to use in a production environment.
If you want to use tracing in all your environments you can use the BizTalk CAT Instrumentation Framework. It’s is a high performance tracing/logging framework for BizTalk that builds upon the Event Tracing for Windows (ETW) infrastructure. Microsoft used essentially the same framework to instrument BizTalk itself, as well as many recently released adapters.

You can use ETW tracing in regular BizTalk artifacts but also in custom ESB Toolkit components it’s possible to implement it.
 

  • Instrumentation of Itinerary Messaging Services
    Itinerary Messaging Services can be instrumented using TraceManager.PipelineComponent which is dedicated for this type of BizTalk artifacts.

  • Instrumentation of Itinerary Orchestration Services
    As Itinerary Orchestration Service can be complex, the instrumentation is becoming a key factor in helping to diagnose and troubleshoot behavioral problems, performance-related issues and other bottlenecks that were not foreseen or manifested themselves during development.

  • Instrumentation of Custom Components
    Custom .NET components that are executed in Itinerary Services can be instrumented by invoking the trace methods provided by TraceManager.CustomComponent

  

Usage

The examples below illustrates how to use the TraceManager component inside an Itinerary Orchestration Service.

 

 

   

BizTalk CAT Instrumentation Framework Controller

The BizTalk CAT Instrumentation Framework Controller is an easy-to-use GUI for the BizTalk CAT Instrumentation Framework. The Controller lets you start and stop a trace and adjust filter options. It can easily enable real-time tracing to Microsoft SysInternals DebugView , to a log file or to both at the same time.

 

 

 

 

 

 

 

 

 

 

When you trace to file you can open the file in a Text Editor by clicking on the “Open Log in Text Editor” button.

 

 

 

 

See Also

For more information on ETW tracing see:

  • Best Practices for Instrumenting High Performance BizTalk Solutions
  • BizTalk CAT Instrumentation Framework Controller
Debatch XML using envelope

Debatch XML using envelope

We all know that debatching should be done in the receive pipeline, I’ve yet to see any really good scenarios where you shouldn’t, but I know there is some (debatching in the orchestration can give you a huge performance loss in BizTalk).

So, I am the admin, and I rarely do development (after all, drag and drop development isn’t challenging enough for me).

I sometimes use BizTalk to help me do some jobs I’d rather not do manually. Like inserting something in a database if I have a million rows in an XML message, or if just want to play drag and drop with my kids. And I came into a case where debatching an XML file was essential for me and my kids in order to have some fun (the oldest 2 and 4 years, they need a little challenge). I looked at a few articles on the internet and none of them were really “easy to understand”. So I decided to make one on my own. Let’s just put this post in the “DevOps” category. In this example I’m using VS 2013 and BizTalk 2013 R2.

  1. Create a new BizTalk Project (I called mine “SimpleDebatching“) but you can name it anything you want, it won’t impact any other code in this tutorial
  2. Right click your project in the “Solution Explorer” and choose “Add” and “New item…
  3. Click once on “Schema” and add a friendly name to it. I called mine “Orders
  4. Rename the “Root” name into “Order“.
  5. Right click the root name “Order” and choose “Insert Schema Node” and “Child Field Element
  6. Do step 5 a total of four times with the following names “orderId“, “orderDate“, “orderPrice” and “orderStatus
  7. If you now open the schema we named “Orders” and expand all the elements it will look like this
    Schema image
  8. And now a pet on your back, you’re doing great
  9. Now that we’ve come so far we need to understand the meanings of an envelope, people will trick you and tell you some technical ways of it.. Don’t mind them. its like any envelope, it has something in it, one or multiple papers/records.
  10. In the “Solution Explorer” right click the project and choose “Add” and “New Item…
  11. Again, click once on “Schema” and give it a friendly name. In this case I called it “OrdersEnvelope
  12. The “Schema Editor” pops up and we can see the root name is… “root“.
  13. Rename the root name into “OrderEnvelope
  14. Right click “OrderEnvelope” and choose “Insert Schema Node“, and choose “Child Record”
  15. Name this child record “Orders
  16. Get a cup of Coffee…
  17. Right Click the node “Orders” and choose “Insert Schema Node“, and choose “Any Element
  18. Now click once on the “<Schema>” and take a look at the “Properties” window
  19. Locate the option “Envelope” change this from “(Default)” to “Yes” (do this for the “OrdersEnvelope” schema)
  20. Now go to the root node (OrderEnvelope) in the “OrdersEnvelope” schema and look at the “Properties” window again, find the “Body XPath
  21. click the “magic/ellipse” button and drill till you see “Orders
    no need to write code
  22. Choose “Orders” and click “OK“.
  23. Pooof, and your schema “OrderEnvelope” is updated with the correct XPath
  24. Look at your self, and understand we still haven’t written one sentence of code..
  25. Go back to the “Solution Explorer” and right click the solution “SimpleDebatching“, choose “Add” and “New Item” and pick the “Receive Pipeline” name this “ReceiveAndDebatch
  26. The Pipeline Component “drag and drop windows” appears.
  27. Expand the “Toolbox” windows and drag the “XML Disassembler” and drop it on the “Disassembler in the main window
  28. Click the newly added XML Disassembler and look at the properties window and find “Document Schema” click the ellipse button and locate your document schema “Orders”
  29. When you find this, click on it and hit the “Add” button, you will see the schema moving from the left window to the right
  30. Click “OK
  31. Do the same with “Envelope Schema” but locate the “OrdersEnvelope” schema instaed
  32. Click “Add” and “OK
  33. Now in order for us to deploy this code to BizTalk we need to do a few things
  34. Look at the “Solution Explorer” and right click the “Solution” Choose “Properties
  35. In the “Properties window” that appear click “Deployment
  36. Update the “Application Name” to “SimpleDebatching
  37. Now we need to sign our assembly, so click the “Signing” and tick the box that says “Sign the assembly
  38. From the drop down choose “<new…>” and create a new key
  39. Hit “CTRL+S” to save it
  40. Now back in the “Solution Explorer” right click it and choose “Deploy
  41. Your code will now be deployed to your environment
  42. Open “BizTalk Administration Console” and locate your application “SimpleDebatching
  43. Create a new Receive Port and name it “ReceiveDebatchPort
  44. Create a receive location for this Receive Port and anme it “ReceiveDebatchLocation
  45. Update the receive pipeline to your newly created recive pipeline “ReceiveAndDebatch
  46. Create a new send port and name it “SendDebatch
  47. Update the send pipeline to “XMLTransmit
  48. Add a filter to your Receive Port by chooseing “BTS.ReceivePortName” and the value “ReceiveDebatchPort
  49. Now restart your host instance and start the application.
  50. You can create a test message by generating an instance from the schema “OrdersEnvelope
  51. Replace the “<any>” tag with a generated instance from the “Orders” schema (the more you add, the more records)
  52. Put the file in the in folder and see what happens.

So, you’ve done it, made a cool application, with drag and drop.. no coding.

Thanks for The help Erik Thue. 🙂 the best BT Dev in Bouvet

Enabling BizTalk Services to work with Service Bus 2.2 SDK

BizTalk Service currently requires that Service Bus entities that do not have partitioning enabled. A new feature was introduced late last year in Service Bus that now enables partitioning in Service bus entities by default. Once this property is set, it cannot be changed.

If one tries to deploy a project from Visual Studio which has Queue at the source with partitioning enabled, the following error can be expected:

Error 1 QueueSource1 deployment failed at ‘https://test.biztalk.windows.net/default/passSB/sources/QueueSource1’.

Failed to connect to the Service Bus using specified configuration.

Deployment of a project from Visual Studio having Queue\Topic with partitioning enabled as a destination completes successfully. However, during runtime, when a message is actually being sent to the Queue\Topic, the following error can be expected:

This client is not supported for a partitioned entity. The client version should be greater or equal than version ‘2013-10′..58ca562e-49ef-4c92-823d-1a219c3f52bd_G8

Following are some of the ways to address this issue:

1.  Customer can use a queue created from SB Explorer. Queues created this way, do not have entity partitioning enabled by default

2. Use custom create option while creating queues\topics. In the second page, user should disable partitioning

 

 

 

Blog Post by: BizTalk Blog

Announcement: MABS August 2014 Update is now live!

Microsoft Azure BizTalk Services August 2014 Update is now live!

We are pleased to announce the August 2014 update of Microsoft Azure BizTalk Services and the associated SDK.

The release provides ability to configure and manage agreements and bridges separately, new features for EDI, enhanced encryption for AS2, supports advanced XML schema constructs in transforms and is Drummond certified.

While these features are immediately available in any new BizTalk services you create, existing services will be upgraded over in next few days. There would be no impact to availability of existing services as we roll out the upgrade. There are no pricing changes with this announcement.

Key features you can now leverage with this release are:

 

Ability to Configure and Manage Agreements and Bridge Separately

You can now change settings in a bridge (transforms, transport settings …) without having to redeploy the corresponding agreement. Additionally one bridge can now cater to multiple agreements. Agreements and bridges can be configured and managed separately enabling reuse of configurations and ease of management.

Visit BizTalk Portal now to try it out…

Note: all your existing agreements and bridges continue to function as before. This feature update decouples existing agreements and its associated bridges but does not break functionality in any way

 

EDI Delimiters at a Transaction Set Level

When sending messages to a B2B partner via an agreement you could use only one pair of delimiters for that partner. With this release Azure BizTalk Services allows configuration of delimiter set per message type on the outbound side. This is applicable for both X12 and EDIFACT protocols. Acknowledgements generated during receive side processing can also choose to use the delimiter set of the incoming message for which they were generated.

 

Enhanced digest and encryption algorithms for AS2

Supported symmetric key encryptions:

Existing: RC2, 3DES

New: AES-128, AES-192, and AES-256

Supported algorithms for MIC calculation:

Existing: MD5, SHA1

New: SHA2-256, SHA2-384, and SHA2-512

 

AS2 Drummond Certified

We are happy to announce that with this release Azure BizTalk Services is now AS2 Drummond Certified. See the same being announced by the Drummond Group here and here.

 

Support for Advanced XML Schema Constructs

All advanced XML schema constructs used in complex schemes, including derived types (xs:extension and xs:restriction types), are now fully supported in Transforms. These derived types are typically used in OAGIS, Salesforce and GJXDM schemas. When you create links from elements under a “derived type” node in the Source schema, that link is only executed when the “xsi:type” value in the input XML instance matches the type value for that specific node. For “derived type” node in the Target schema, the correct “xsi:type” is stamped automatically in the output XML instance based on the incoming mapping links.

Following is the list of the XML schema constructs that are newly supported.

  • Derived complex types – i.e., <xs:extension> and <xs:restriction> being used for a complex-type definition (with a complex type as its base type)
  • <xs:choice> constructs
  • <xs:group> constructs
  • <xs:attributeGroup> constructs
  • <xs:any>
  • <xs:anyAttribute> 

 

Azure BizTalk Services Team

Blog Post by: BizTalk Blog

Solution to Common BizTalk Deployment errors in Visual Studio

There are a few common deployment errors in Microsoft Visual Studio when redeploying a previously deployed BizTalk project.
“Failed to add resource(s). Change requests failed for some resource. BizTalkAssemblyResourceManager failed to complete end type change request. Failed to update binding information. Cannot update receive port “”. Transform “” not found”

 
There are a few other similar errors […]
Blog Post by: Kevin Morillo

Getting Started with BizTalk Server 2013 R2’s Built-in Health Monitoring

Getting Started with BizTalk Server 2013 R2’s Built-in Health Monitoring

This is the second in a series of posts exploring What’s New in BizTalk Server 2013 R2. With the BizTalk Server 2013 R2 release, Microsoft has finally implemented a common request to have some level of built-in monitoring tool for a BizTalk Server installation. While this built-in option won’t replace things like the BizTalk Server […]
Blog Post by: Nick Hauenstein

Purchasing a Current Transformer

Purchasing a Current Transformer

When purchasing a current transformer or transducer, the most important considerations are: What type of input are you expecting? This may include an AC amperage input, DC amperage input, DC voltage input, etc. What type of output is the meter/monitor you are working with going to expect? In the power monitoring industry the most common […]
Blog Post by: Fehlberg Victor

Sentinet Extensibility – Custom SMS Alert Handler with Twilio

Sentinet is highly extendable through standard Microsoft .NET, WCF and WIF extensibility points, and thorough the Sentinet API interfaces. In the previous posts we saw how to build a custom access rule expression (standard .NET library) and how to leverage the WCF estensibility with a custom endpoint behavior. With this post I would like to continue the serie about the Sentinet Extensibility and demonstrate how to create a custom SMS alert handler with Twilio.

ESB Toolkit Tip #16 Use the Deployment Framework for BizTalk Server to deploy your ESB itineraries

When an itinerary is created using the Itinerary Designer you normally deploy the Itinerary with the ESB Itinerary Import Utility. This utility is located under the \bin directory of the BizTalk ESB Toolkit and is named EsbImportUtil.exe. It can be used to publish or deploy the itinerary XML into the ESBItineraryDB database. Unfortunately, the tool is not very convenient to use because you have to deploy the Itineraries manually.
Another option is to use the Deployment Framework for BizTalk. The Deployment Framework for BizTalk eliminates the pain associated with BizTalk application deployments, and goes far beyond BizTalk’s out-of-the-box deployment functionality because you can also use it to deploy ESB itineraries.

 

Steps

The Deployment Framework can automatically deploy your ESB itineraries during deployment.  Before an itinerary can be deployed, you must manually export it to an XML file using the Itinerary Designer’s XML Exporter. 
 
1. Set the IncludeEsbItineraries property to true
The property may be included in any PropertyGroup, but is commonly placed in the first PropertyGroup in the project file.

<PropertyGroup>
 ...
 <IncludeEsbItineraries>true</IncludeEsbItineraries>
 ...
</PropertyGroup>.csharpcode, .csharpcode pre
{
	font-size: small;
	color: black;
	font-family: consolas, "Courier New", courier, monospace;
	background-color: #ffffff;
	/*white-space: pre;*/
}
.csharpcode pre { margin: 0em; }
.csharpcode .rem { color: #008000; }
.csharpcode .kwrd { color: #0000ff; }
.csharpcode .str { color: #006080; }
.csharpcode .op { color: #0000c0; }
.csharpcode .preproc { color: #cc6633; }
.csharpcode .asp { background-color: #ffff00; }
.csharpcode .html { color: #800000; }
.csharpcode .attr { color: #ff0000; }
.csharpcode .alt 
{
	background-color: #f4f4f4;
	width: 100%;
	margin: 0em;
}
.csharpcode .lnum { color: #606060; }

 

2. Add an EsbItineraries ItemGroup

The following is a typical example that properly follows the common ItemGroup structure:

<ItemGroup>
 <EsbItineraries Include="MyItinerary1.xml">
   <LocationPath>..\Itineraries</LocationPath>
 </EsbItineraries>
</ItemGroup>

.csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; }

If you have more than one ESB itinerary, then repeat the <EsbItineraries> element for each file

(usually within the same ItemGroup).

Note

The Microsoft ESB Toolkit does not include a tool to undeploy itineraries, so the Deployment Framework cannot undeploy them.  Once deployed, they remain deployed.

 

See Also

For more information on using the Deployment Framework to deploy your ESB itineraries see:

Unable to create binding configuration element for editing. Check the values of the bindyingType and BindingConfiguration properties.

Unable to create binding configuration element for editing. Check the values of the bindyingType and BindingConfiguration properties.

I have a BizTalk Server 2010 Test Environment that was working properly for some time, all the adapters from BizTalk Server Adapter Pack 2010 were installed with the last cumulative updates and also working properly, in this environment we use the SQL Server Adapter. However the last time the team try to configure the receive […]
Blog Post by: Sandro Pereira