by Saravana Kumar | May 9, 2017 | BizTalk Community Blogs via Syndication
About 10-12 years ago (around 2004-2005), Service Oriented Architecture (SOA) was a big topic in the enterprise world. Massive projects were initiated to align enterprises to be more service oriented with a middleware/ESB platform like BizTalk Server in the middle.
When we talked about SOA implementation using Microsoft technologies during this period, only two technologies/platform stood out –
- Windows Communication Foundation (WCF), and
- Microsoft BizTalk Server
WCF was set of libraries built on top of the .NET Framework that helped to build SOAP based (later REST support was added) web services. WCF was more of a platform that shipped along with the .NET Framework and evolved gradually along with the .NET releases. BizTalk Server, on the other hand, was a full blown Enterprise Server Product that acted as a Middleware platform mainly addressing the challenge of not creating an integration spaghetti between those services (represented as APP in below pictures) created using WCF.
Without BizTalk Server |
With BizTalk Server |
|
|
BizTalk360 journey between 2011 – 2016
When we started BizTalk360 back in 2010, the objectives were pretty straight forward. Nearly 10,000 medium to large organisations have invested in Microsoft BizTalk Server to address their integration requirements.
We spotted a lot of gaps in BizTalk Server when it comes to BizTalk Server day-to-day administration and monitoring. It looked like a straightforward bet to build a world-class product that can help these 10,000+ customers to improve their operational efficiency.
We started off initially with BizTalk360 as a monitoring solution for BizTalk Server and the product slowly evolved into addressing various operational and analytics needs of the BizTalk Server. There are over 100 features in the product now covering various aspects of day-to-day operational needs like Security, Governance/Auditing, Monitoring, Analytics, Productivity Tools, Knowledge Base, Rich Dashboards, etc. making it a must have add-on product for any Microsoft BizTalk Server customers.
BizTalk360 currently has over 500 enterprise customers, and we are proud to say that Microsoft itself is one of our biggest customers. Microsoft internally uses BizTalk Server for all of their IT and Supply Chain requirements and they rely on BizTalk360 for their day-to-day operations. You can read the case study “Why Microsoft chose BizTalk360 instead of SCOM”
What is the state of Integration today in Microsoft Stack?
In the last 5-6 years, the innovation around technology exploded in all directions. Cloud and mobile became mainstream and companies like Microsoft, Amazon, Google and IBM pumped a lot of investment into the cloud. On the other hand, SaaS (Software as a Service) based products exploded in other direction. Today, no one really wants to buy off-the-shelf software products, run their own servers and manage upgrades and patches. They simply want to use software or services provided by the third party rather than owning them. About 10-15 years ago, SIEBEL was the leader in CRM space, however, today that segment is ruled by technologies like Salesforce and Microsoft Dynamics 365. Similar story exists in all software segments like Source Control (GIT and Visual Studio Team Service), Productivity Tools (Office365, Google Apps), Email (Office365, Google) etc.
With all these changes, it’s natural for Integration Platforms and Products to evolve as well. The role of integration has become so immense in the last 5 years due to the above mentioned reasons. Today, companies buy/use multiple software products which do one thing really well rather than living with a monolithic beast software which claims to do everything but not good at anything.
This change in attitude also means all these independent software product/services need to talk to each other, which created a new segment of integration products in the likes of IFTTT, Zapier, Azure Logic Apps, SnapLogic, Dell Boomi, etc
Microsoft responded well to this growing challenge in the integration space, addressing the requirements of: Hybrid Integration solutions (the combination of Cloud + On-Premise integrations), Quick & Easy integration solutions, Business Users (aka citizen integrators) integration solutions etc.
If you look at Microsoft Integration Stack today, it’s not just BizTalk Server and WCF anymore. There are at least a dozen products/services that are mainly geared towards Integration in Microsoft Stack.
- Microsoft BizTalk Server
- ASP.NET Web API’s
- Azure Logic Apps
- Azure App Services (Web Jobs)
- Azure Functions (Serverless Compute)
- Azure API Management
- Azure Service Bus (Queues/Topics)
- Azure Stream Analytics
- Azure Event Hubs
- Microsoft PowerApps
- Microsoft Flow
- Azure Gateway (Hybrid Connection)
BizTalk360 – 2017 and beyond
One of the key aspects of a product company like us is that we need to be on the forefront of technology all the time. We need to adapt quickly to the change and understand the fact “change is the only constant”. We cannot just sit back and relax saying BizTalk360 is doing well. We have been closely monitoring the trend what’s going on, what will happen to the existing BizTalk Server customers in the next 10-15 years.
We are fortunate enough to build up a close relationship with Microsoft and the Integration community and are fully aware of the changes that are happening around Integration.
BizTalk360 is now slowly evolving into a product that will cater for the big picture. We wanted to create a single unified tool that will help any customer who invested in Microsoft platform for their integration needs.
We can clearly see the challenges that are building up when you have so many technologies included in a solution. There is no single management tool that will cater for the day-to-day operations. Customers might end up using BizTalk Server Admin Console for some of the activities and Azure Portal for some of the other activities, which is not productive and you are never going to be confident you set up access rights correctly across multiple tools, educating support people how to operate and monitor your solutions.
There are various other challenges like, for example, for Monitoring, you will need an end-to-end monitoring solution that covers both on-prem and cloud, you will also need tracking and analytics solution that will stretch between on-prem and cloud etc. We predict customers might end up building management tools, exactly like what they did 10 years ago for BizTalk solutions. BizTalk360 was born to address the challenge faced by BizTalk Server customers.
Today we are at a juncture where we see the same challenge at a bigger level and BizTalk360 is well positioned to address that challenge since we are already halfway through covering the on-premise solution.
What available and what’s coming in BizTalk360
We realized this situation almost 2 years ago and started expanding BizTalk360 to cover both on-premise and cloud technologies that fall into the Integration bucket with a vision of creating a single unified management and monitoring tool for Microsoft Integration.
Today, BizTalk360, in addition to its full blown support for BizTalk Server, can also manage and monitor related Azure Integration Technologies like
In the upcoming 8.5 version (planned for July 2017), we are bringing in support for Azure Integration Accounts, App Services, and Azure Stream Analytics.
We are also working on a secret project with a bigger vision, which I cannot talk much at this stage (stay tuned).
We will continue our investment in this direction and make BizTalk360 a single must have tool for anyone using Microsoft Technologies for their Integration Requirements.
Author: Saravana Kumar
Saravana Kumar is the Founder and CTO of BizTalk360, an enterprise software that acts as an all-in-one solution for better administration, operation, support and monitoring of Microsoft BizTalk Server environments. View all posts by Saravana Kumar
by Saravana Kumar | May 9, 2017 | BizTalk Community Blogs via Syndication
About 10-12 years ago (around 2004-2005), Service Oriented Architecture (SOA) was a big topic in the enterprise world. Massive projects were initiated to align enterprises to be more service oriented with a middleware/ESB platform like BizTalk Server in the middle.
When we talked about SOA implementation using Microsoft technologies during this period, only two technologies/platform stood out –
- Windows Communication Foundation (WCF), and
- Microsoft BizTalk Server
WCF was set of libraries built on top of the .NET Framework that helped to build SOAP based (later REST support was added) web services. WCF was more of a platform that shipped along with the .NET Framework and evolved gradually along with the .NET releases. BizTalk Server, on the other hand, was a full blown Enterprise Server Product that acted as a Middleware platform mainly addressing the challenge of not creating an integration spaghetti between those services (represented as APP in below pictures) created using WCF.
Without BizTalk Server |
With BizTalk Server |
|
|
BizTalk360 journey between 2011 – 2016
When we started BizTalk360 back in 2010, the objectives were pretty straight forward. Nearly 10,000 medium to large organisations have invested in Microsoft BizTalk Server to address their integration requirements.
We spotted a lot of gaps in BizTalk Server when it comes to BizTalk Server day-to-day administration and monitoring. It looked like a straightforward bet to build a world-class product that can help these 10,000+ customers to improve their operational efficiency.
We started off initially with BizTalk360 as a monitoring solution for BizTalk Server and the product slowly evolved into addressing various operational and analytics needs of the BizTalk Server. There are over 100 features in the product now covering various aspects of day-to-day operational needs like Security, Governance/Auditing, Monitoring, Analytics, Productivity Tools, Knowledge Base, Rich Dashboards, etc. making it a must have add-on product for any Microsoft BizTalk Server customers.
BizTalk360 currently has over 500 enterprise customers, and we are proud to say that Microsoft itself is one of our biggest customers. Microsoft internally uses BizTalk Server for all of their IT and Supply Chain requirements and they rely on BizTalk360 for their day-to-day operations. You can read the case study “Why Microsoft chose BizTalk360 instead of SCOM”
What is the state of Integration today in Microsoft Stack?
In the last 5-6 years, the innovation around technology exploded in all directions. Cloud and mobile became mainstream and companies like Microsoft, Amazon, Google and IBM pumped a lot of investment into the cloud. On the other hand, SaaS (Software as a Service) based products exploded in other direction. Today, no one really wants to buy off-the-shelf software products, run their own servers and manage upgrades and patches. They simply want to use software or services provided by the third party rather than owning them. About 10-15 years ago, SIEBEL was the leader in CRM space, however, today that segment is ruled by technologies like Salesforce and Microsoft Dynamics 365. Similar story exists in all software segments like Source Control (GIT and Visual Studio Team Service), Productivity Tools (Office365, Google Apps), Email (Office365, Google) etc.
With all these changes, it’s natural for Integration Platforms and Products to evolve as well. The role of integration has become so immense in the last 5 years due to the above mentioned reasons. Today, companies buy/use multiple software products which do one thing really well rather than living with a monolithic beast software which claims to do everything but not good at anything.
This change in attitude also means all these independent software product/services need to talk to each other, which created a new segment of integration products in the likes of IFTTT, Zapier, Azure Logic Apps, SnapLogic, Dell Boomi, etc
Microsoft responded well to this growing challenge in the integration space, addressing the requirements of: Hybrid Integration solutions (the combination of Cloud + On-Premise integrations), Quick & Easy integration solutions, Business Users (aka citizen integrators) integration solutions etc.
If you look at Microsoft Integration Stack today, it’s not just BizTalk Server and WCF anymore. There are at least a dozen products/services that are mainly geared towards Integration in Microsoft Stack.
- Microsoft BizTalk Server
- ASP.NET Web API’s
- Azure Logic Apps
- Azure App Services (Web Jobs)
- Azure Functions (Serverless Compute)
- Azure API Management
- Azure Service Bus (Queues/Topics)
- Azure Stream Analytics
- Azure Event Hubs
- Microsoft PowerApps
- Microsoft Flow
- Azure Gateway (Hybrid Connection)
BizTalk360 – 2017 and beyond
One of the key aspects of a product company like us is that we need to be on the forefront of technology all the time. We need to adapt quickly to the change and understand the fact “change is the only constant”. We cannot just sit back and relax saying BizTalk360 is doing well. We have been closely monitoring the trend what’s going on, what will happen to the existing BizTalk Server customers in the next 10-15 years.
We are fortunate enough to build up a close relationship with Microsoft and the Integration community and are fully aware of the changes that are happening around Integration.
BizTalk360 is now slowly evolving into a product that will cater for the big picture. We wanted to create a single unified tool that will help any customer who invested in Microsoft platform for their integration needs.
We can clearly see the challenges that are building up when you have so many technologies included in a solution. There is no single management tool that will cater for the day-to-day operations. Customers might end up using BizTalk Server Admin Console for some of the activities and Azure Portal for some of the other activities, which is not productive and you are never going to be confident you set up access rights correctly across multiple tools, educating support people how to operate and monitor your solutions.
There are various other challenges like, for example, for Monitoring, you will need an end-to-end monitoring solution that covers both on-prem and cloud, you will also need tracking and analytics solution that will stretch between on-prem and cloud etc. We predict customers might end up building management tools, exactly like what they did 10 years ago for BizTalk solutions. BizTalk360 was born to address the challenge faced by BizTalk Server customers.
Today we are at a juncture where we see the same challenge at a bigger level and BizTalk360 is well positioned to address that challenge since we are already halfway through covering the on-premise solution.
What available and what’s coming in BizTalk360
We realized this situation almost 2 years ago and started expanding BizTalk360 to cover both on-premise and cloud technologies that fall into the Integration bucket with a vision of creating a single unified management and monitoring tool for Microsoft Integration.
Today, BizTalk360, in addition to its full blown support for BizTalk Server, can also manage and monitor related Azure Integration Technologies like
In the upcoming 8.5 version (planned for July 2017), we are bringing in support for Azure Integration Accounts, App Services, and Azure Stream Analytics.
We are also working on a secret project with a bigger vision, which I cannot talk much at this stage (stay tuned).
We will continue our investment in this direction and make BizTalk360 a single must have tool for anyone using Microsoft Technologies for their Integration Requirements.
Author: Saravana Kumar
Saravana Kumar is the Founder and CTO of BizTalk360, an enterprise software that acts as an all-in-one solution for better administration, operation, support and monitoring of Microsoft BizTalk Server environments. View all posts by Saravana Kumar
by Saranya Ramakrishnan | Feb 10, 2017 | BizTalk Community Blogs via Syndication
Azure Logic Apps play a vital role in an integration platform. Keeping this in mind, we introduced ‘Azure Logic Apps Monitoring’ in BizTalk360 Version 8.1. In Version 8.3, we implemented ‘Azure Logic Apps Operations’ capability to improve the ease of use for users using Logic Apps and BizTalk360. With this functionality, users can Operate, Manage and Monitor Azure Logic App(s) from a single place. When you see a threshold violation of Logic Apps, you can rectify the problem from the BizTalk360 UI itself. You can save time to log in to the Azure Portal and manage your Logic Apps.
All the user needs to do to configure Logic Apps Operations capability is to add the Azure subscription in BizTalk360 UI and enable it for monitoring and operation. BizTalk360 also provides an option to work on multiple subscriptions simultaneously. Therefore, by adding the subscription in BizTalk360, users can view the list of available Logic Apps in that subscription along with its name, Access End Point, the current Status (Enabled or Disabled), Last Run, and other details such as Resource Group, Location, Run and Trigger history details and Success/Failure Run count.
Features of Logic Apps Operations
Enable/Disable Logic Apps: From BizTalk360 UI, users can enable/disable the Logic Apps that reflects the corresponding Logic App in the Azure Subscription. You can initiate bulk operations — enable/disable multiple Logic Apps in a single click.
Run Trigger: User can also trigger the Logic App action from the BizTalk360 UI. This action also supports the bulk operations.
Delete Logic Apps: User can delete single or multiple Logic Apps through a single click from BizTalk360 UI.
Note: The Azure Portal UI allows to operate (Enable/Disable, Delete, Run Trigger) on only one Logic App at a time, whereas from BizTalk360 UI user can initiate bulk operation (select multiple logic apps) in a single click.
Run and Trigger History details
Data like History and Runs are huge when it comes to real time scenarios/production use. For each Logic App, Run and Trigger history will maintain the last 30 records. The history details will be displayed in both list and graphical View.
List View
- Start and End time of Run/Trigger. Here the time format and time zone is based on user profile configuration.
- Run/Trigger status such as succeeded/failed, running/skipped/aborted
- Duration of each Run and Trigger
Graphical View
We have simplified the UI view of your Logic Apps details and redefined Runs and Trigger history into Graphical representations. With the graphical view, it becomes easy for the users to navigate and identify the date and time tracking.
The graphical view chart shows Logic Apps Runs in the “Y” axis and Date in the “X” axis. All basic graph operations such as zoom, hover are available in the graphical view. Additionally, you can print/download the chart.
Success and Failure Run count
Based on colour coding, you can know the Success and Failure Run counts instantly within the detail window of your Logic App.
Trigger and Action Status
Users can see the actual design of the Logic App in the Details View, say when it should be trigger and which actions are to be performed.
E.g : Trigger ‘When_a File_Is_created ’ ; Action – Email_File .
Licensing
If you are an existing user of BizTalk360, and using the Platinum tier license, you just need to upgrade to BizTalk360 Version 8.3 to use this feature. For customers using other licensing tier, if you would like to use/try this feature, please send an email to support@biztalk360.com to customize your license.
If you are new to BizTalk360 and like to explore the Logic Apps Operations capability, you can get started with the 14 day free trial of BizTalk360.
by Nick Hauenstein | Jan 25, 2017 | BizTalk Community Blogs via Syndication
At the end of last week, a few of us from QuickLearn Training hosted a webinar with an overview of a few of the new features in BizTalk Server 2016. This post serves as a proper write-up of the feature that I shared and demonstrated – Shared Access Signature Support for Relay Adapters. If you missed it, we’ve made the full recording available on YouTube here. We’ve also clipped out just the section on Shared Access Signature Support for Relay Adapters over here – which might be good to watch before reading through this post.
While that feature is not the most flashy or even the most prominent on the What’s New in BizTalk Server 2016 page within the MSDN documentation, it should come as a nice relief for developers who want to host a service in BizTalk Server while exposing it to consumers in the cloud — with the least amount of overhead possible.
Shared Access Signature (SAS) Support for Relay Adapters
You can now use SAS authentication with the following adapters:
- WCF-BasicHttpRelay
- WCF-NetTcpRelay
- WCF-BasicHttp*
- WCF-WebHttp*
* = Used only for sending messages as a client
Why Use SAS Instead of ACS?
Before BizTalk Server 2016, our only security option for the BasicHttpRelay and NetTcpRelay adapters was the Microsoft Azure Access Control Service (ACS).
One of the main scenarios that the Access Control Service was designed for was Federated Identity. For simpler scenarios, wherein I don’t need claims mapping, or even the concept of a user, using ACS adds potentially unnecessary overhead to (1) the deployed resources (inasmuch as you must setup an ACS namespace alongside the resources you’re securing), and (2) the runtime communications.
Shared Access Signatures were designed more for fine-grained and time-limited authority delegation over resources. The holder of a key could sign and distribute small string-based tokens that define a resource a client could access and timeframe within which they were allowed to access the resource.
Hosting a Relay Secured by Shared Access Signatures
In order to expose a BizTalk hosted service in the cloud via Azure Relay, you must first create a namespace for the relay – a place for the cloud endpoint to be hosted. It’s at the namespace level that you can generate keys used for signing SAS tokens that allow BizTalk server to host a new relay, and tokens that allow clients to send messages to any of that namespace’s relays.
The generated keys are associated with policies that have certain associated claims / rights that each is allowed to delegate.
In the example above, using the key associated with the biztalkhost policy, I would be able to sign tokens that allow applications to listen at a relay endpoint within the namespace, but I would not be able to sign tokens allowing applications to Send messages to the same relays.
Clicking a policy reveals its keys. Each policy has 2 keys that can be independently refreshed, allowing you to roll over to new keys while giving a grace period in which the older keys are still valid.

Either one of these keys can be provided in the BizTalk Server WCF-BasicHttpRelay adapter configuration to host a new relay.
Configuring the Security Settings for the WCF-BasicHttpRelay Adapter
When configuring the WCF-BasicHttpRelay adapter, rather than providing a pre-signed token with a pre-determined expiration date, you provide the key directly. The adapter can then sign its own tokens that will be used to authorize access to the Relay namespace and listen for incoming connections. This is configured on the Security tab of the adapter properties.
If you would like to require clients to authenticate with the relay before they’re allowed to send messages, you can set the Relay client authentication type to RelayAccessToken:
From there it’s a matter of choosing your service endpoint, and then you’re on your way to a functioning Relay:
Once you Enable the Receive Location, you should be able to see a new WCF Relay with the same name appear in the Azure Portal for your Relay namespace. If not, check your configuration and try again.
Most importantly, your clients can update their endpoint addresses to call your new service in the cloud.
The Larger Picture: BizTalk Hybrid Cloud APIs

One thing to note about this setup, however, is that the WCF-BasicHttpRelay adapter is actually not running in the Isolated Host. In other words, rather than running as part of a site in IIS, it’s running in-process within the BizTalk Server Host Instance itself. While that provides far less complexity, it also sacrifices the ability to run the request through additional processing before it hits BizTalk Server (e.g., rate limiting, blacklisting, caching, URL rewriting, etc…). If I were hosting the service on-premises I would have this ability right out of the box. So what would I do in the cloud?
Using API Management with BizTalk Server
In the cloud, we have the ability to layer on other Azure services beyond just using the Azure Relay capability. One such service that might solve our dilemma described in the previous section would be Azure API Management.
Rather than having our clients call the relay directly (and thus having all message processing done by BizTalk Server), we can provide API Management itself a token to access to our BizTalk Hosted service. The end users of the service wouldn’t know the relay address directly, or have the required credentials to access it. Instead they would direct all of their calls to an endpoint in API Management.
API Management, like IIS, and like BizTalk Server, provide robust and customizable request and response pipelines. In the case of API Management, the definitions of what happens in these pipelines are called “policies.” There are both inbound policies and outbound policies. These policies can be configured for a whole service at a time, and/or only for specific operations. They enable patterns like translation, transformation, caching, and rewriting.
In my case, I’ve designed a quick and dirty policy that replaces the headers of an inbound message so that it goes from being a simple GET request to being a POST request with a SOAP message body. It enables caching, and at a base level implements rate-limiting for inbound requests. On the outbound side it translates the SOAP response to a JSON payload — effectively exposing our on-premises BizTalk Server hosted SOAP service as a cloud-accessible RESTful API.
So what does it look like in action? Below, you can see the submission of a request from the client’s perspective:
How does BizTalk Server see the input message? It sees something like this (note that the adapter has stripped away the SOAP envelope at this point in processing):

What about on the outbound side? What did BizTalk Server send back through the relay? It sent an XML message resembling the following:

If you’re really keen to dig into the technical details of the policy configuration that made this possible, they’re all here in their terrifying glory (click to open in a new window, and read slowly from top to bottom):

The token was generated with a quick and dirty purpose-built simple console app (the best kind).
Tips, Tricks, and Stumbling Blocks
Within the API Management policy shown above, you may have noticed the CDATA sections. This is mandatory where used. You’ll end up with some sad results if you don’t remember to escape any XML input you have, or the security token itself which includes unescaped XML entities.
Another interesting thing with the policy above is that the WCF-BasicHttpRelay adapter might choke while creating a BizTalk message out of the SOAP message constructed via the policy above (which includes heaps of whitespace so as to be human readable), failing with the following message The adapter WCF-BasicHttpRelay raised an error message. Details “System.InvalidOperationException: Text cannot be written outside the root element.
This can be fixed quite easily by adjusting the adapter properties so that they’re looking for the message body with the expression set to “*”.
Questions and Final Thoughts
During the webinar the following questions came up:
- Q: Is https supported?
- A: Yes, for both the relay itself and the API management endpoint.
- Q: Maximum size is 256KB; I was able to get a response about 800 KB; Is that because BizTalk and Azure apply the compression technology and after compression the 800KB response shrinks to about 56KB?
- A: The size limit mentioned applies to brokered messages within Service Bus. Azure Relay is a separate service that isn’t storing the message for any period of time – messages are streamed to the service host. Which means if BizTalk Server disconnects, the communication is terminated. There’s a nice article comparing the two communication styles over here.
I hope this has been both helpful and informative. Be sure to keep watching for more of QuickLearn Training’s coverage of New Features in BizTalk Server 2016, and our upcoming BizTalk Server 2016 training courses.