Introducing BizTalk Server Availability Monitoring (BizTalk360 v 8.5)

Introducing BizTalk Server Availability Monitoring (BizTalk360 v 8.5)

A typical highly available BizTalk Server group contains one or more BizTalk Servers.  We have witnessed some of the complex BizTalk Server environment handling high volume traffic having an infrastructure similar to the one shown below with 6 BizTalk Server and 3-4 SQL Servers.

BizTalk-Server-Availability-Monitoring

There are 2 main reason for having multiple BizTalk Server is the group

  • Scalability
  • Resilience/High Availability

Scalability: The more servers you have in the group means more processing power. You can create multiple BizTalk Server Host and Host Instances in each one of the available servers and increase the volume of messages you can process.

Resilience/High Availability:  If you wanted to make sure the environment is highly available, you need to have at least 2 BizTalk Servers in the group and also need to carefully plan how the host/host instances are configured to make sure if one server goes down it doesn’t bring the whole environment down.

Why do you need this?

As a BizTalk Administrator, you need to make sure all of your BizTalk Servers are up and running and processing messages at the expected level.  In the above 6 server configuration, there is a possibility one of the BizTalk Servers go down and no one really notices it for a long period, until the environment itself becomes a bottle neck. In smaller environments (ex: 2 BizTalk Servers), it becomes super important to make sure your BizTalk Servers are up and running all the time to avoid down time or react to down time quickly.

To address these challenges we are introducing BizTalk Server Availability monitoring in BizTalk360 version 8.5.

As always, one of the core strength of BizTalk360 for monitoring BizTalk Server environments is we wanted to make it super simple and great user experience to configure things. It will literally take less than 2 minutes to setup BizTalk Server Availability Monitoring in-spite of the complexity of your environment.

biztalk-server-availability-monitoring

The above screen shows how you can configure the availability monitoring for a 2 BizTalk Servers group. The servers are already listed, you simply need to select them and click the “Enable Monitoring” button.  (PS: You need to understand the concept of Alarms in BizTalk360)

Protocol Type: In order for us to check the availability we need to reach the servers, we support “Ping” and “Telnet” to achieve this, one of these protocols need to be enabled.

Monitor Availability (either all or one of them)

This is very important and very specific to BizTalk Server availability monitoring. You can choose the option to alert either if one of the BizTalk Server is the group has gone down or alert only when all of the servers in the group has gone down. The first one is useful if you know there are intermittent issues and the server will come back online after some time and there is no need to alert the teams unnecessarily.

biztalk-server-availability-monitoring-options

Give it a try on your own environment by downloading a 14-day free trial of BizTalk360. You can write to us at support@biztalk360.com.

Author: Saravana Kumar

Saravana Kumar is the Founder and CTO of BizTalk360, an enterprise software that acts as an all-in-one solution for better administration, operation, support and monitoring of Microsoft BizTalk Server environments. View all posts by Saravana Kumar

Introducing BizTalk Server Availability Monitoring (BizTalk360 version 8.5)

Introducing BizTalk Server Availability Monitoring (BizTalk360 version 8.5)

A typical highly available BizTalk Server group contains one or more BizTalk Servers.  We have witnessed some of the complex BizTalk Server environment handling high volume traffic having an infrastructure similar to the one shown below with 6 BizTalk Server and 3-4 SQL Servers.

BizTalk-Server-Availability-Monitoring

There are 2 main reason for having multiple BizTalk Server is the group

  • Scalability
  • Resilience/High Availability

Scalability: The more servers you have in the group means more processing power. You can create multiple BizTalk Server Host and Host Instances in each one of the available servers and increase the volume of messages you can process.

Resilience/High Availability:  If you wanted to make sure the environment is highly available, you need to have at least 2 BizTalk Servers in the group and also need to carefully plan how the host/host instances are configured to make sure if one server goes down it doesn’t bring the whole environment down.

Why do you need this?

As a BizTalk Administrator you need to make sure all of your BizTalk Servers are up and running and processing messages at expected level.  In the above 6 server configuration, there is a possibility one of the BizTalk Servers goes down and no one really notice it for a long period, until the environment itself becomes a bottle neck. In smaller environments (ex: 2 BizTalk Servers), it becomes super important to make sure your BizTalk Servers are up and running all the time to avoid down time or react to down time quickly.

To address these challenges we are introducing BizTalk Server Availability monitoring in BizTalk360 version 8.5.

As always, one of the core strength of BizTalk360 for monitoring BizTalk Server environments is we wanted to make it super simple and great user experience to configure things. It will literally take less than 2 minutes to setup BizTalk Server Availability Monitoring in-spite of the complexity of your environment.

biztalk-server-availability-monitoring

The above screen show how you can configure the availability monitoring for a 2 BizTalk Servers group. The servers are already listed, you simply need to select them and click the “Enable Monitoring” button.  (PS: You need to understand the concept of Alarms in BizTalk360)

Protocol Type: In order for us to check the availability we need to reach the servers, we support “Ping” and “Telnet” to achieve this, one of these protocols need to be enabled.

Monitor Availability (either all or one of them)

This is very important and very specific to BizTalk Server availability monitoring. You can choose the option to alert either if one of the BizTalk Server is the group has gone down or alert only when all of the servers in the group has gone down. The first one is useful if you know there are intermittent issues and the server will come back online after some time and there is no need to alert the teams unnecessarily.

biztalk-server-availability-monitoring-options

Give it a try on your own environment by downloading a 14-day free trial of BizTalk360.

Author: Saravana Kumar

Saravana Kumar is the Founder and CTO of BizTalk360, an enterprise software that acts as an all-in-one solution for better administration, operation, support and monitoring of Microsoft BizTalk Server environments. View all posts by Saravana Kumar

Introducing Advanced BizTalk Server Host Throttling Monitoring

Introducing Advanced BizTalk Server Host Throttling Monitoring

What is BizTalk Host Throttling?

BizTalk Server being a Middleware product connected to various legacy backend systems it needs to make sure the entire ecosystem can work in an optimal way. If one of the legacy system connected to BizTalk is slow for any reason, then BizTalk Server need to act sensibly not to overload that system with messages more than what it can handle. In such scenarios, BizTalk Server will throttle itself (slow down itself) and make sure the messages are delivered to the backend in an optimal rate.

BizTalk Server achieves this capability by continuously monitoring various performance counters (memory footprint, thread count, message publishing rates, database size etc.) and self tuning itself. There are over 50 performance counters related to throttling in BizTalk Server which monitors both inbound and outbound traffic.

BizTalk360 Throttling Analyser

One of the challenges for BizTalk Server Administrators when it comes to BizTalk Throttling is, there is no out of the box tooling from Microsoft to understand whether your BizTalk Environment is working efficiently or under throttling condition. You only have raw performance counters to measure throttling. Typically the BizTalk Administrators open up Windows Perfmon tool and add all the performance counters related to Throttling record and analyse throttling conditions. This requires extensive knowledge about how BizTalk Server works, various throttling counters & conditions, whether it’s running on optimum level etc.

BizTalk360 Throttling Analyser

To address this issue, about 2 years ago we introduced “Throttling Analyser” in BizTalk360. Once enabled, BizTalk360 continuously collect all the throttling related performance counter data in our database and provide an intuitive user interface with highly interactive graphs to showcase whether the BizTalk Environment is working efficiently or under throttling condition. This saves a lot of time for BizTalk Administrators to understand environment throttling condition and the biggest advantage is you do not need to have in-depth knowledge about BizTalk Server internal architecture and throttling mechanism.

BizTalk360 Throttling Monitoring

The Throttling Analyser explained in the previous section only gives you the visual representation of the BizTalk Environment throttling condition. The BizTalk Administrator need to periodically log in to the system to see if the environment is healthy. However, we can clearly see a value in alerting the BizTalk Administrators if the environment is suffering from any critical throttling conditions for a perceived time.

That’s exactly what we have done with “Throttling Monitoring” in version 8.5 of BizTalk360. We wanted to make the experience super simple and intuitive.

How does it work?

Let’s take a close look at how this functionality is designed and how it works.

Under BizTalk Environment monitoring section we introduced a new category called “Host Throttling”, which by default will list out all the BizTalk Hosts that’s currently configured in the environment as  shown in the below picture.

BizTalk360-Host-Throttling-Main

Enabling BizTalk Host Throttling monitoring in single click

You can enable default throttling for all the BizTalk Hosts in a single click, you simply select the hosts you wanted to monitor and click the button “Enable Throttling”, this will start monitoring the BizTalk Environment for any throttling violation that persist for 60 seconds.

default-biztalk360-host-throttling-1

The whole idea for us is to make it as simple as possible to monitoring BizTalk Throttling condition, hence we provide the option to just enable the default monitoring with 60 seconds persistence in a single click.

default-biztalk360-host-throttling

The BizTalk Administrator can easily tweak the default configuration by clicking on the “Edit” button and changing the parameters. As you can see from the below picture, you can choose to monitor specific throttling condition or you can have multiple conditions with different persist duration etc, you can also have different monitoring options for publish side and delivery side.

For Publish Throttling, the user can able to monitor the following metrics:

  • Any throttling
  • 2 – Rate Throttling
  • 4 – Process memory
  • 5 – System memory
  • 6 – Database size
  • 8 – Database session
  • 9 – Thread count
  • 11 – User override

For Delivery Throttling, the user can able to monitor the following metrics:

  • Any throttling
  • 1 – Rate Throttling
  • 4 – Process memory
  • 5 – System memory
  • 9 – Thread count
  • 10 – User override

default-biztalk360-host-throttling-3

Notification of Throttling violation

Once the settings are configured to look out for throttling conditions, BizTalk360 will keep monitoring the environment. If BizTalk360 detects any threshold conditions violation, it will notify the users via configured notification channels. In BizTalk360, there are various notification channels like Email, Slack, ServiceNow, SMS, Event Viewer, Web Hook etc

Email Notification Slack Notification
Host Throttling Mail Notifications slack

Summary

Threshold monitoring is one of the key features we have introduced and it address one of the important areas of BizTalk Server Operations and Monitoring. This will help customers to keep an eye on the capacity of their BizTalk Environments in near real time and take appropriate actions.

You can write to us at support@biztalk360.com. Have a try at our latest version by downloading a 14-day free trial of BizTalk360.

Author: Saravana Kumar

Saravana Kumar is the Founder and CTO of BizTalk360, an enterprise software that acts as an all-in-one solution for better administration, operation, support and monitoring of Microsoft BizTalk Server environments. View all posts by Saravana Kumar

What are the restrictions of BizTalk Server Standard edition?

What are the restrictions of BizTalk Server Standard edition?

We often get this question asked what’s are the restrictions of BizTalk Server Standard edition. I thought of highlighting the key restrictions that are applicable to BizTalk Server Standard Edition here.

highlighting the key restrictions that are applicable to BizTalk Server Standard Edition

BizTalk Applications Limit:

You cannot deploy more than 5 BizTalk Applications in a single environment (this excludes the default BizTalk.System application). This is not a great restriction since BizTalk applications are just logical containers to group artifacts together (receive locations, send ports, schemas, orchestrations, policies etc). There is nothing stopping a customer from deploying all the artifacts inside one or two applications. In fact, this restriction results in some of the bad practices in BizTalk Application deployment. In our opinion, a better restriction will be restricting the number of artifacts like “x” number of orchestrations, send ports, receive locations.

BizTalk Servers Limits:

You cannot have more than one server in a BizTalk Environment. This is the one that forces most of the customers to move to BizTalk Server Enterprise edition. This restriction basically results in not having high availability for your BizTalk environment, if the server goes down, your environment is down.

Single Message Box:

You cannot have more than one MessageBox in your environment. Scaling MessageBox is required when you have a high throughput scenario. It’s not really big impact in BizTalk Standard Edition case since you are restricted to only one BizTalk server and hence you can only handle so much load anyway.

Number of CPU cores:

You cannot have more than 8 cores on the server. Again this restricts the scalability of your BizTalk Environment to limited processing power.

In a nutshell, the BizTalk Standard edition is designed for customers who want to take advantage of the integration platform but not concerned with scalability and high throughput.

Author: Saravana Kumar

Saravana Kumar is the Founder and CTO of BizTalk360, an enterprise software that acts as an all-in-one solution for better administration, operation, support and monitoring of Microsoft BizTalk Server environments. View all posts by Saravana Kumar

Choosing between BizTalk360 and ServiceBus360 for monitoring Azure Service Bus

Choosing between BizTalk360 and ServiceBus360 for monitoring Azure Service Bus

Recently we received few support emails where people were asking about the overlap between BizTalk360 and ServiceBus360 when it comes to monitoring Azure Service Bus. Which ones should they go for? Also, the question was extended in such a way that if they are using Azure Logic Apps and Web API’s (Web Endpoints), then which is the better product to opt for.

Given both the products got the capability to monitor Azure Service Bus, it’s a valid question and let me try to clarify the positioning of both the products.

BizTalk360

When we released BizTalk360 version 8.1, we introduced a bunch of Azure Monitoring capabilities in the product like:

The Web End Points Monitoring capability was also heavily enhanced to support features like adding query strings, body payload, HTTP headers etc., in the request message and enriched validation like JSONPath, XPath, response time etc., on the response message. The changes made the feature super powerful for monitoring SOAP, REST/HTTP-based web endpoints.

The long term goal for us at BizTalk360 is to provide a consolidated single pane of glass operations, monitoring and analytics solution for customers who are using Microsoft Integration Stack for their integration needs. In the upcoming 8.5 version, we are extending Azure capability even further by bringing support for Azure Integration Accounts within BizTalk360.

If you are a Microsoft BizTalk Server customer and slowly started leveraging Azure Service Bus, Logic Apps, API apps, and Web API’s for your integration requirements, then BizTalk360 will be the ideal product both for Managing and Monitor the entire infrastructure. Typically Microsoft BizTalk Server customers who started utilizing some of the Azure Integration technology stacks like Azure Service Bus, Logics Apps, API apps will get benefitted by using BizTalk360.

When it comes to Azure Service Bus monitoring in BizTalk360, we only cover Azure Service Bus Queues. Currently, we do not cover Azure Service Bus Topics, Azure Service Bus Relay and Azure Service Bus EventHubs. Therefore, if you are using any of these technologies (that are not monitored with BizTalk360), then you’ll also need ServiceBus360.

ServiceBus360

ServiceBus360 is designed and developed to provide complete operations and monitoring capabilities for Azure Service Bus Messaging, Relay and Event Hubs. ServiceBus360 provides in-depth monitoring capabilities for:

ServiceBus360 is also not just a monitoring solution for Azure Service Bus. The idea of ServiceBus360 is to make it a world class product for complete operations, monitoring and analytics of Azure Service Bus. The product already supports a variety of productivity and advanced operational capabilities like:

The above is not the complete list of features – it just gives you the flavor of what can be accomplished with ServiceBus360. Clearly, BizTalk360 will not have this level of coverage for Azure Service Bus.

Therefore, if you are using Azure Service Bus for mission critical integration work, then ServiceBus360 is the viable option to improve productivity and avoid disaster.

Author: Saravana Kumar

Saravana Kumar is the Founder and CTO of BizTalk360, an enterprise software that acts as an all-in-one solution for better administration, operation, support and monitoring of Microsoft BizTalk Server environments. View all posts by Saravana Kumar

What is going to be the future of BizTalk360?

What is going to be the future of BizTalk360?

About 10-12 years ago (around 2004-2005), Service Oriented Architecture (SOA) was a big topic in the enterprise world. Massive projects were initiated to align enterprises to be more service oriented with a middleware/ESB platform like BizTalk Server in the middle.

When we talked about SOA implementation using Microsoft technologies during this period, only two technologies/platform stood out –

  • Windows Communication Foundation (WCF), and
  • Microsoft BizTalk Server

WCF was set of libraries built on top of the .NET Framework that helped to build SOAP based (later REST support was added) web services. WCF was more of a platform that shipped along with the .NET Framework and evolved gradually along with the .NET releases. BizTalk Server, on the other hand, was a full blown Enterprise Server Product that acted as a Middleware platform mainly addressing the challenge of not creating an integration spaghetti between those services (represented as APP in below pictures) created using WCF.

Without BizTalk Server With BizTalk Server

BizTalk360 journey between 2011 – 2016

When we started BizTalk360 back in 2010, the objectives were pretty straight forward. Nearly 10,000 medium to large organisations have invested in Microsoft BizTalk Server to address their integration requirements.

We spotted a lot of gaps in BizTalk Server when it comes to BizTalk Server day-to-day administration and monitoring. It looked like a straightforward bet to build a world-class product that can help these 10,000+ customers to improve their operational efficiency.

We started off initially with BizTalk360 as a monitoring solution for BizTalk Server and the product slowly evolved into addressing various operational and analytics needs of the BizTalk Server. There are over 100 features in the product now covering various aspects of day-to-day operational needs like Security, Governance/Auditing, Monitoring, Analytics, Productivity Tools, Knowledge Base, Rich Dashboards, etc. making it a must have add-on product for any Microsoft BizTalk Server customers.

BizTalk360 currently has over 500 enterprise customers, and we are proud to say that Microsoft itself is one of our biggest customers. Microsoft internally uses BizTalk Server for all of their IT and Supply Chain requirements and they rely on BizTalk360 for their day-to-day operations. You can read the case study “Why Microsoft chose BizTalk360 instead of SCOM

What is the state of Integration today in Microsoft Stack?

In the last 5-6 years, the innovation around technology exploded in all directions. Cloud and mobile became mainstream and companies like Microsoft, Amazon, Google and IBM pumped a lot of investment into the cloud. On the other hand, SaaS (Software as a Service) based products exploded in other direction. Today, no one really wants to buy off-the-shelf software products, run their own servers and manage upgrades and patches. They simply want to use software or services provided by the third party rather than owning them. About 10-15 years ago, SIEBEL was the leader in CRM space, however, today that segment is ruled by technologies like Salesforce and Microsoft Dynamics 365. Similar story exists in all software segments like Source Control (GIT and Visual Studio Team Service), Productivity Tools (Office365, Google Apps), Email (Office365, Google) etc.

With all these changes, it’s natural for Integration Platforms and Products to evolve as well. The role of integration has become so immense in the last 5 years due to the above mentioned reasons. Today, companies buy/use multiple software products which do one thing really well rather than living with a monolithic beast software which claims to do everything but not good at anything.

This change in attitude also means all these independent software product/services need to talk to each other, which created a new segment of integration products in the likes of IFTTT, Zapier, Azure Logic Apps, SnapLogic, Dell Boomi, etc

Microsoft responded well to this growing challenge in the integration space, addressing the requirements of: Hybrid Integration solutions (the combination of Cloud + On-Premise integrations), Quick & Easy integration solutions, Business Users (aka citizen integrators) integration solutions etc.

If you look at Microsoft Integration Stack today, it’s not just BizTalk Server and WCF anymore. There are at least a dozen products/services that are mainly geared towards Integration in Microsoft Stack.

  • Microsoft BizTalk Server
  • ASP.NET Web API’s
  • Azure Logic Apps
  • Azure App Services (Web Jobs)
  • Azure Functions (Serverless Compute)
  • Azure API Management
  • Azure Service Bus (Queues/Topics)
  • Azure Stream Analytics
  • Azure Event Hubs
  • Microsoft PowerApps
  • Microsoft Flow
  • Azure Gateway (Hybrid Connection)

BizTalk360 – 2017 and beyond

One of the key aspects of a product company like us is that we need to be on the forefront of technology all the time. We need to adapt quickly to the change and understand the fact “change is the only constant”. We cannot just sit back and relax saying BizTalk360 is doing well. We have been closely monitoring the trend what’s going on, what will happen to the existing BizTalk Server customers in the next 10-15 years.

We are fortunate enough to build up a close relationship with Microsoft and the Integration community and are fully aware of the changes that are happening around Integration.

BizTalk360 is now slowly evolving into a product that will cater for the big picture. We wanted to create a single unified tool that will help any customer who invested in Microsoft platform for their integration needs.

We can clearly see the challenges that are building up when you have so many technologies included in a solution. There is no single management tool that will cater for the day-to-day operations. Customers might end up using BizTalk Server Admin Console for some of the activities and Azure Portal for some of the other activities, which is not productive and you are never going to be confident you set up access rights correctly across multiple tools, educating support people how to operate and monitor your solutions.

There are various other challenges like, for example, for Monitoring, you will need an end-to-end monitoring solution that covers both on-prem and cloud, you will also need tracking and analytics solution that will stretch between on-prem and cloud etc. We predict customers might end up building management tools, exactly like what they did 10 years ago for BizTalk solutions. BizTalk360 was born to address the challenge faced by BizTalk Server customers.

Today we are at a juncture where we see the same challenge at a bigger level and BizTalk360 is well positioned to address that challenge since we are already halfway through covering the on-premise solution.

What available and what’s coming in BizTalk360

We realized this situation almost 2 years ago and started expanding BizTalk360 to cover both on-premise and cloud technologies that fall into the Integration bucket with a vision of creating a single unified management and monitoring tool for Microsoft Integration.

Today, BizTalk360, in addition to its full blown support for BizTalk Server, can also manage and monitor related Azure Integration Technologies like

In the upcoming 8.5 version (planned for July 2017), we are bringing in support for Azure Integration Accounts, App Services, and Azure Stream Analytics.

We are also working on a secret project with a bigger vision, which I cannot talk much at this stage (stay tuned).

We will continue our investment in this direction and make BizTalk360 a single must have tool for anyone using Microsoft Technologies for their Integration Requirements.

Author: Saravana Kumar

Saravana Kumar is the Founder and CTO of BizTalk360, an enterprise software that acts as an all-in-one solution for better administration, operation, support and monitoring of Microsoft BizTalk Server environments. View all posts by Saravana Kumar

What is going to be the future of BizTalk360?

What is going to be the future of BizTalk360?

About 10-12 years ago (around 2004-2005), Service Oriented Architecture (SOA) was a big topic in the enterprise world. Massive projects were initiated to align enterprises to be more service oriented with a middleware/ESB platform like BizTalk Server in the middle.

When we talked about SOA implementation using Microsoft technologies during this period, only two technologies/platform stood out –

  • Windows Communication Foundation (WCF), and
  • Microsoft BizTalk Server

WCF was set of libraries built on top of the .NET Framework that helped to build SOAP based (later REST support was added) web services. WCF was more of a platform that shipped along with the .NET Framework and evolved gradually along with the .NET releases. BizTalk Server, on the other hand, was a full blown Enterprise Server Product that acted as a Middleware platform mainly addressing the challenge of not creating an integration spaghetti between those services (represented as APP in below pictures) created using WCF.

Without BizTalk Server With BizTalk Server

BizTalk360 journey between 2011 – 2016

When we started BizTalk360 back in 2010, the objectives were pretty straight forward. Nearly 10,000 medium to large organisations have invested in Microsoft BizTalk Server to address their integration requirements.

We spotted a lot of gaps in BizTalk Server when it comes to BizTalk Server day-to-day administration and monitoring. It looked like a straightforward bet to build a world-class product that can help these 10,000+ customers to improve their operational efficiency.

We started off initially with BizTalk360 as a monitoring solution for BizTalk Server and the product slowly evolved into addressing various operational and analytics needs of the BizTalk Server. There are over 100 features in the product now covering various aspects of day-to-day operational needs like Security, Governance/Auditing, Monitoring, Analytics, Productivity Tools, Knowledge Base, Rich Dashboards, etc. making it a must have add-on product for any Microsoft BizTalk Server customers.

BizTalk360 currently has over 500 enterprise customers, and we are proud to say that Microsoft itself is one of our biggest customers. Microsoft internally uses BizTalk Server for all of their IT and Supply Chain requirements and they rely on BizTalk360 for their day-to-day operations. You can read the case study “Why Microsoft chose BizTalk360 instead of SCOM

What is the state of Integration today in Microsoft Stack?

In the last 5-6 years, the innovation around technology exploded in all directions. Cloud and mobile became mainstream and companies like Microsoft, Amazon, Google and IBM pumped a lot of investment into the cloud. On the other hand, SaaS (Software as a Service) based products exploded in other direction. Today, no one really wants to buy off-the-shelf software products, run their own servers and manage upgrades and patches. They simply want to use software or services provided by the third party rather than owning them. About 10-15 years ago, SIEBEL was the leader in CRM space, however, today that segment is ruled by technologies like Salesforce and Microsoft Dynamics 365. Similar story exists in all software segments like Source Control (GIT and Visual Studio Team Service), Productivity Tools (Office365, Google Apps), Email (Office365, Google) etc.

With all these changes, it’s natural for Integration Platforms and Products to evolve as well. The role of integration has become so immense in the last 5 years due to the above mentioned reasons. Today, companies buy/use multiple software products which do one thing really well rather than living with a monolithic beast software which claims to do everything but not good at anything.

This change in attitude also means all these independent software product/services need to talk to each other, which created a new segment of integration products in the likes of IFTTT, Zapier, Azure Logic Apps, SnapLogic, Dell Boomi, etc

Microsoft responded well to this growing challenge in the integration space, addressing the requirements of: Hybrid Integration solutions (the combination of Cloud + On-Premise integrations), Quick & Easy integration solutions, Business Users (aka citizen integrators) integration solutions etc.

If you look at Microsoft Integration Stack today, it’s not just BizTalk Server and WCF anymore. There are at least a dozen products/services that are mainly geared towards Integration in Microsoft Stack.

  • Microsoft BizTalk Server
  • ASP.NET Web API’s
  • Azure Logic Apps
  • Azure App Services (Web Jobs)
  • Azure Functions (Serverless Compute)
  • Azure API Management
  • Azure Service Bus (Queues/Topics)
  • Azure Stream Analytics
  • Azure Event Hubs
  • Microsoft PowerApps
  • Microsoft Flow
  • Azure Gateway (Hybrid Connection)

BizTalk360 – 2017 and beyond

One of the key aspects of a product company like us is that we need to be on the forefront of technology all the time. We need to adapt quickly to the change and understand the fact “change is the only constant”. We cannot just sit back and relax saying BizTalk360 is doing well. We have been closely monitoring the trend what’s going on, what will happen to the existing BizTalk Server customers in the next 10-15 years.

We are fortunate enough to build up a close relationship with Microsoft and the Integration community and are fully aware of the changes that are happening around Integration.

BizTalk360 is now slowly evolving into a product that will cater for the big picture. We wanted to create a single unified tool that will help any customer who invested in Microsoft platform for their integration needs.

We can clearly see the challenges that are building up when you have so many technologies included in a solution. There is no single management tool that will cater for the day-to-day operations. Customers might end up using BizTalk Server Admin Console for some of the activities and Azure Portal for some of the other activities, which is not productive and you are never going to be confident you set up access rights correctly across multiple tools, educating support people how to operate and monitor your solutions.

There are various other challenges like, for example, for Monitoring, you will need an end-to-end monitoring solution that covers both on-prem and cloud, you will also need tracking and analytics solution that will stretch between on-prem and cloud etc. We predict customers might end up building management tools, exactly like what they did 10 years ago for BizTalk solutions. BizTalk360 was born to address the challenge faced by BizTalk Server customers.

Today we are at a juncture where we see the same challenge at a bigger level and BizTalk360 is well positioned to address that challenge since we are already halfway through covering the on-premise solution.

What available and what’s coming in BizTalk360

We realized this situation almost 2 years ago and started expanding BizTalk360 to cover both on-premise and cloud technologies that fall into the Integration bucket with a vision of creating a single unified management and monitoring tool for Microsoft Integration.

Today, BizTalk360, in addition to its full blown support for BizTalk Server, can also manage and monitor related Azure Integration Technologies like

In the upcoming 8.5 version (planned for July 2017), we are bringing in support for Azure Integration Accounts, App Services, and Azure Stream Analytics.

We are also working on a secret project with a bigger vision, which I cannot talk much at this stage (stay tuned).

We will continue our investment in this direction and make BizTalk360 a single must have tool for anyone using Microsoft Technologies for their Integration Requirements.

Author: Saravana Kumar

Saravana Kumar is the Founder and CTO of BizTalk360, an enterprise software that acts as an all-in-one solution for better administration, operation, support and monitoring of Microsoft BizTalk Server environments. View all posts by Saravana Kumar

What is going to be the future of BizTalk360?

What is going to be the future of BizTalk360?

About 10-12 years ago (around 2004-2005), Service Oriented Architecture (SOA) was a big topic in the enterprise world. Massive projects were initiated to align enterprises to be more service oriented with a middleware/ESB platform like BizTalk Server in the middle.

When we talked about SOA implementation using Microsoft technologies during this period, only two technologies/platform stood out –

  • Windows Communication Foundation (WCF), and
  • Microsoft BizTalk Server

WCF was set of libraries built on top of the .NET Framework that helped to build SOAP based (later REST support was added) web services. WCF was more of a platform that shipped along with the .NET Framework and evolved gradually along with the .NET releases. BizTalk Server, on the other hand, was a full blown Enterprise Server Product that acted as a Middleware platform mainly addressing the challenge of not creating an integration spaghetti between those services (represented as APP in below pictures) created using WCF.

Without BizTalk Server With BizTalk Server

BizTalk360 journey between 2011 – 2016

When we started BizTalk360 back in 2010, the objectives were pretty straight forward. Nearly 10,000 medium to large organisations have invested in Microsoft BizTalk Server to address their integration requirements.

We spotted a lot of gaps in BizTalk Server when it comes to BizTalk Server day-to-day administration and monitoring. It looked like a straightforward bet to build a world-class product that can help these 10,000+ customers to improve their operational efficiency.

We started off initially with BizTalk360 as a monitoring solution for BizTalk Server and the product slowly evolved into addressing various operational and analytics needs of the BizTalk Server. There are over 100 features in the product now covering various aspects of day-to-day operational needs like Security, Governance/Auditing, Monitoring, Analytics, Productivity Tools, Knowledge Base, Rich Dashboards, etc. making it a must have add-on product for any Microsoft BizTalk Server customers.

BizTalk360 currently has over 500 enterprise customers, and we are proud to say that Microsoft itself is one of our biggest customers. Microsoft internally uses BizTalk Server for all of their IT and Supply Chain requirements and they rely on BizTalk360 for their day-to-day operations. You can read the case study “Why Microsoft chose BizTalk360 instead of SCOM

What is the state of Integration today in Microsoft Stack?

In the last 5-6 years, the innovation around technology exploded in all directions. Cloud and mobile became mainstream and companies like Microsoft, Amazon, Google and IBM pumped a lot of investment into the cloud. On the other hand, SaaS (Software as a Service) based products exploded in other direction. Today, no one really wants to buy off-the-shelf software products, run their own servers and manage upgrades and patches. They simply want to use software or services provided by the third party rather than owning them. About 10-15 years ago, SIEBEL was the leader in CRM space, however, today that segment is ruled by technologies like Salesforce and Microsoft Dynamics 365. Similar story exists in all software segments like Source Control (GIT and Visual Studio Team Service), Productivity Tools (Office365, Google Apps), Email (Office365, Google) etc.

With all these changes, it’s natural for Integration Platforms and Products to evolve as well. The role of integration has become so immense in the last 5 years due to the above mentioned reasons. Today, companies buy/use multiple software products which do one thing really well rather than living with a monolithic beast software which claims to do everything but not good at anything.

This change in attitude also means all these independent software product/services need to talk to each other, which created a new segment of integration products in the likes of IFTTT, Zapier, Azure Logic Apps, SnapLogic, Dell Boomi, etc

Microsoft responded well to this growing challenge in the integration space, addressing the requirements of: Hybrid Integration solutions (the combination of Cloud + On-Premise integrations), Quick & Easy integration solutions, Business Users (aka citizen integrators) integration solutions etc.

If you look at Microsoft Integration Stack today, it’s not just BizTalk Server and WCF anymore. There are at least a dozen products/services that are mainly geared towards Integration in Microsoft Stack.

  • Microsoft BizTalk Server
  • ASP.NET Web API’s
  • Azure Logic Apps
  • Azure App Services (Web Jobs)
  • Azure Functions (Serverless Compute)
  • Azure API Management
  • Azure Service Bus (Queues/Topics)
  • Azure Stream Analytics
  • Azure Event Hubs
  • Microsoft PowerApps
  • Microsoft Flow
  • Azure Gateway (Hybrid Connection)

BizTalk360 – 2017 and beyond

One of the key aspects of a product company like us is that we need to be on the forefront of technology all the time. We need to adapt quickly to the change and understand the fact “change is the only constant”. We cannot just sit back and relax saying BizTalk360 is doing well. We have been closely monitoring the trend what’s going on, what will happen to the existing BizTalk Server customers in the next 10-15 years.

We are fortunate enough to build up a close relationship with Microsoft and the Integration community and are fully aware of the changes that are happening around Integration.

BizTalk360 is now slowly evolving into a product that will cater for the big picture. We wanted to create a single unified tool that will help any customer who invested in Microsoft platform for their integration needs.

We can clearly see the challenges that are building up when you have so many technologies included in a solution. There is no single management tool that will cater for the day-to-day operations. Customers might end up using BizTalk Server Admin Console for some of the activities and Azure Portal for some of the other activities, which is not productive and you are never going to be confident you set up access rights correctly across multiple tools, educating support people how to operate and monitor your solutions.

There are various other challenges like, for example, for Monitoring, you will need an end-to-end monitoring solution that covers both on-prem and cloud, you will also need tracking and analytics solution that will stretch between on-prem and cloud etc. We predict customers might end up building management tools, exactly like what they did 10 years ago for BizTalk solutions. BizTalk360 was born to address the challenge faced by BizTalk Server customers.

Today we are at a juncture where we see the same challenge at a bigger level and BizTalk360 is well positioned to address that challenge since we are already halfway through covering the on-premise solution.

What available and what’s coming in BizTalk360

We realized this situation almost 2 years ago and started expanding BizTalk360 to cover both on-premise and cloud technologies that fall into the Integration bucket with a vision of creating a single unified management and monitoring tool for Microsoft Integration.

Today, BizTalk360, in addition to its full blown support for BizTalk Server, can also manage and monitor related Azure Integration Technologies like

In the upcoming 8.5 version (planned for July 2017), we are bringing in support for Azure Integration Accounts, App Services, and Azure Stream Analytics.

We are also working on a secret project with a bigger vision, which I cannot talk much at this stage (stay tuned).

We will continue our investment in this direction and make BizTalk360 a single must have tool for anyone using Microsoft Technologies for their Integration Requirements.

Author: Saravana Kumar

Saravana Kumar is the Founder and CTO of BizTalk360, an enterprise software that acts as an all-in-one solution for better administration, operation, support and monitoring of Microsoft BizTalk Server environments. View all posts by Saravana Kumar

Introducing BizTalk360 Version 8.4 – BizTalk Server Licensing Widget, Data Monitoring for Azure Logic Apps, Folder Monitoring, FTP/FTPS/SFTP, IBM MQ Monitoring, BizTalk Health Monitoring Integration

Introducing BizTalk360 Version 8.4 – BizTalk Server Licensing Widget, Data Monitoring for Azure Logic Apps, Folder Monitoring, FTP/FTPS/SFTP, IBM MQ Monitoring, BizTalk Health Monitoring Integration

We are continuing our tradition of one new release every 3-4 months once continuously for the past 6 years. In every release, we wanted to make sure we add 5-6 meaningful features that will help BizTalk Server administrators with Operations, Monitoring and Analytics capabilities. In BizTalk360 version 8.4, we added some exciting new features.

  • BizTalk Server Licensing Widget
  • Data Monitoring for Azure Logic Apps
  • Folder Monitoring
  • FTP/FTPS/SFTP Monitoring
  • IBM MQ Monitoring
  • BizTalk Health Monitoring (BHM) Integration
  • Manage NT Services directly in the Web Console
  • Manage SQL Jobs directly in the Web Console
  • Few enhancements/bug fixes

BizTalk Server Licensing Widget

From our experience dealing with 100’s of BizTalk Server customers, we noticed that a lot of them struggle to understand how many BizTalk Server licenses are required for their servers. About 4 years ago I wrote an article called Understanding BizTalk Server Licensing and that’s one of the popular articles on our blog. We just thought it will be a good idea to transform that knowledge into a small widget in BizTalk360 so that people will easily understand how many licenses are required for your BizTalk Server.

Out of the box we ship this new widget in BizTalk360 version 8.4 called “BizTalk Server License” as shown below. This widget will display key information like BizTalk edition, server type, processor type, number of cores, processor manufacturer, retail license cost per server, how many servers to be licensed and total cost for the environment.

Data Monitoring for Azure Logic Apps

Data monitoring in BizTalk360 is one of the key capabilities that allows you to monitor and trigger alerts based on historical events. Some of the common use cases of data monitoring include “No Event Alerting”. Ex: If you are expected to process 5 purchase orders from an FTP location every hour and you haven’t received the expected volume, then BizTalk360 can alert you with a message.

We are taking the exact same concept to Azure Logic Apps to alert you based on historical transactions within a time window.

There are various interesting use cases. For example, as shown above, you may want to get alerted if one of your Logic App has gone crazy and firing thousands of executions and costing so many $$$. We will cover in detail about this feature in a separate blog article.

Here is the detailed list of metrics you can set data monitoring on.

Folder Monitoring

Sometimes as the product matures and you look at adding more features, you tend to miss out on the basic things. Folder monitoring is one such thing we left for so long! It’s better late than never. In BizTalk360 version 8.4 we are bringing the capability of folder monitoring. The technology might have improved significantly like Micro Services, REST API’s etc, however, file based integration is there to stay. It’s very common in the integration world where you drop a file (purchase order, batch file, EDI transactions etc) and the integration kicks in. One of the common challenges in such integration is that what happens when the integration is broken and the files started to pile up in the pickup folder.

We solve the exact problem using the folder monitoring capability.

One of the core values of BizTalk360 is to make monitoring configuration seamless and that’s how we differentiate ourselves from general purpose monitoring products. In the folder monitoring case, we automatically list down all the receive locations and send ports that use BizTalk FILE adapter and allow the user to configure in pretty much 2-3 clicks, we pick up all the values like folder location from existing configuration as shown above.

FTP/FTPS/SFTP

On the similar lines of normal Folder monitoring, FTP based integrations are key in most of the enterprise integration scenarios. BizTalk Server comes with three different adapters FTP, FTPs and SFTP to tackle FTP scenarios where the variations are mainly around the security capabilities of the FTP server.

With BizTalk360 version 8.4, you can monitor FTP locations for data pile up. We put a lot of efforts to make the configuration experience as seamless as possible, the screens will automatically list all the FTP based send ports/receive locations and all the available values like location, username etc are automatically picked up from those configurations.

IBM MQ Monitoring

For the past few releases, we are slowly bringing in the capability of Queue monitoring into BizTalk360. Queues play a vital role in enterprise integration especially for robustness, store, and forward patterns. In the previous releases, we introduced MSMQ and Azure Service Bus Queue. In BizTalk360 version 8.4, we are bringing in support for IBM MQ. We support both MQSC and MQS based configurations.

In any IBM MQ queues, you can monitor for the following 4 parameters – queue depth, backout queue depth, queue usage % and backout queue usage %.

BizTalk Health Monitoring Integration

For many years, we had support for Message Box Viewer inside BizTalk360. We periodically run MBV in configured environments, parse and store the result and display it in the BizTalk360 web console. We also got the ability to monitor and alert users based on Message Box Viewer raised errors and warning. Two years ago, Microsoft made some major changes to Message Box Viewer, re-branded it as BizTalk Health Monitor with few additional functionalities and deprecated MBV.

Support for BHM is one of the top requested features on our feedback portal. In BizTalk360 version 8.4 we introduced support for BHM and deprecated MBV support.

NT Services Operation in BizTalk and SQL Servers

One of the main objective for us from the security perspective is to stop people logging on/off into production BizTalk and SQL servers during business hours. We also wanted to audit any activities performed by BizTalk support people. Even though you shouldn’t start/stop services in your environment, we noticed in some cases NT services like world wide web, enterprise single sign on, BizTalk host instances, SQL Agent, etc needs to be started/stopped for the variety of reasons. Currently, users will RDP or have remote MMC snap-in to manage NT services in BizTalk and SQL servers. With BizTalk360 version 8.4 you can manage them directly in the web console. In addition, the activities will be audited by BizTalk360.

Manage SQL Jobs Operation from web console

In a similar concept to managing NT Services in both BizTalk and SQL Servers, SQL jobs play a vital role in a BizTalk Server environment. SQL Jobs are responsible for keeping your BizTalk environment healthy. They take care of routine housekeeping activities like moving data from Message Box database to Tracking database, purging/archiving tracked data, backup/disaster log shipping etc.

In BizTalk360 version 8.4, we brought in capabilities to manage SQL jobs directly from the web console. This functionality of monitoring SQL Jobs has been in the product for a very long time.

Few Enhancements and Bug Fixes

This article mainly covers all the new and exciting features we shipped in BizTalk360 version 8.4. In every release, we also allocate time to enhance current features and to address any top priority bugs we received from our existing customers.  Here are some of the key enhancements and bug fixes

Alarm Management Improvements: Now you have the ability to bulk enable/disable monitoring alarms. You can also rename the alarms (again, one of the most requested features on our feedback portal.)

Webhook Notification Channel: Now you have the ability to specify (override) a new endpoint URL at the alarm level.

Logic Apps Monitoring Performance Improvement: Few critical updates been made to improve the performance of monitoring Azure Logic Apps

Get started today

Author: Saravana Kumar

Saravana Kumar is the Founder and CTO of BizTalk360, an enterprise software that acts as an all-in-one solution for better administration, operation, support and monitoring of Microsoft BizTalk Server environments. View all posts by Saravana Kumar

Introducing BizTalk360 Version 8.4 – BizTalk Server Licensing Widget, Data Monitoring for Azure Logic Apps, Folder Monitoring, FTP/FTPS/SFTP, IBM MQ Monitoring, BizTalk Health Monitoring Integration

Introducing BizTalk360 Version 8.4 – BizTalk Server Licensing Widget, Data Monitoring for Azure Logic Apps, Folder Monitoring, FTP/FTPS/SFTP, IBM MQ Monitoring, BizTalk Health Monitoring Integration

We are continuing our tradition of one new release every 3-4 months once continuously for the past 6 years. In every release, we wanted to make sure we add 5-6 meaningful features that will help BizTalk Server administrators with Operations, Monitoring and Analytics capabilities. In BizTalk360 version 8.4, we added some exciting new features.

  • BizTalk Server Licensing Widget
  • Data Monitoring for Azure Logic Apps
  • Folder Monitoring
  • FTP/FTPS/SFTP Monitoring
  • IBM MQ Monitoring
  • BizTalk Health Monitoring (BHM) Integration
  • Manage NT Services directly in the Web Console
  • Manage SQL Jobs directly in the Web Console
  • Few enhancements/bug fixes

BizTalk Server Licensing Widget

From our experience dealing with 100’s of BizTalk Server customers, we noticed that a lot of them struggle to understand how many BizTalk Server licenses are required for their servers. About 4 years ago I wrote an article called Understanding BizTalk Server Licensing and that’s one of the popular articles on our blog. We just thought it will be a good idea to transform that knowledge into a small widget in BizTalk360 so that people will easily understand how many licenses are required for your BizTalk Server.

Out of the box we ship this new widget in BizTalk360 version 8.4 called “BizTalk Server License” as shown below. This widget will display key information like BizTalk edition, server type, processor type, number of cores, processor manufacturer, retail license cost per server, how many servers to be licensed and total cost for the environment.

Data Monitoring for Azure Logic Apps

Data monitoring in BizTalk360 is one of the key capabilities that allows you to monitor and trigger alerts based on historical events. Some of the common use cases of data monitoring include “No Event Alerting”. Ex: If you are expected to process 5 purchase orders from an FTP location every hour and you haven’t received the expected volume, then BizTalk360 can alert you with a message.

We are taking the exact same concept to Azure Logic Apps to alert you based on historical transactions within a time window.

There are various interesting use cases. For example, as shown above, you may want to get alerted if one of your Logic App has gone crazy and firing thousands of executions and costing so many $$$. We will cover in detail about this feature in a separate blog article.

Here is the detailed list of metrics you can set data monitoring on.

Folder Monitoring

Sometimes as the product matures and you look at adding more features, you tend to miss out on the basic things. Folder monitoring is one such thing we left for so long! It’s better late than never. In BizTalk360 version 8.4 we are bringing the capability of folder monitoring. The technology might have improved significantly like Micro Services, REST API’s etc, however, file based integration is there to stay. It’s very common in the integration world where you drop a file (purchase order, batch file, EDI transactions etc) and the integration kicks in. One of the common challenges in such integration is that what happens when the integration is broken and the files started to pile up in the pickup folder.

We solve the exact problem using the folder monitoring capability.

One of the core values of BizTalk360 is to make monitoring configuration seamless and that’s how we differentiate ourselves from general purpose monitoring products. In the folder monitoring case, we automatically list down all the receive locations and send ports that use BizTalk FILE adapter and allow the user to configure in pretty much 2-3 clicks, we pick up all the values like folder location from existing configuration as shown above.

FTP/FTPS/SFTP

On the similar lines of normal Folder monitoring, FTP based integrations are key in most of the enterprise integration scenarios. BizTalk Server comes with three different adapters FTP, FTPs and SFTP to tackle FTP scenarios where the variations are mainly around the security capabilities of the FTP server.

With BizTalk360 version 8.4, you can monitor FTP locations for data pile up. We put a lot of efforts to make the configuration experience as seamless as possible, the screens will automatically list all the FTP based send ports/receive locations and all the available values like location, username etc are automatically picked up from those configurations.

IBM MQ Monitoring

For the past few releases, we are slowly bringing in the capability of Queue monitoring into BizTalk360. Queues play a vital role in enterprise integration especially for robustness, store, and forward patterns. In the previous releases, we introduced MSMQ and Azure Service Bus Queue. In BizTalk360 version 8.4, we are bringing in support for IBM MQ. We support both MQSC and MQS based configurations.

In any IBM MQ queues, you can monitor for the following 4 parameters – queue depth, backout queue depth, queue usage % and backout queue usage %.

BizTalk Health Monitoring Integration

For many years, we had support for Message Box Viewer inside BizTalk360. We periodically run MBV in configured environments, parse and store the result and display it in the BizTalk360 web console. We also got the ability to monitor and alert users based on Message Box Viewer raised errors and warning. Two years ago, Microsoft made some major changes to Message Box Viewer, re-branded it as BizTalk Health Monitor with few additional functionalities and deprecated MBV.

Support for BHM is one of the top requested features on our feedback portal. In BizTalk360 version 8.4 we introduced support for BHM and deprecated MBV support.

NT Services Operation in BizTalk and SQL Servers

One of the main objective for us from the security perspective is to stop people logging on/off into production BizTalk and SQL servers during business hours. We also wanted to audit any activities performed by BizTalk support people. Even though you shouldn’t start/stop services in your environment, we noticed in some cases NT services like world wide web, enterprise single sign on, BizTalk host instances, SQL Agent, etc needs to be started/stopped for the variety of reasons. Currently, users will RDP or have remote MMC snap-in to manage NT services in BizTalk and SQL servers. With BizTalk360 version 8.4 you can manage them directly in the web console. In addition, the activities will be audited by BizTalk360.

Manage SQL Jobs Operation from web console

In a similar concept to managing NT Services in both BizTalk and SQL Servers, SQL jobs play a vital role in a BizTalk Server environment. SQL Jobs are responsible for keeping your BizTalk environment healthy. They take care of routine housekeeping activities like moving data from Message Box database to Tracking database, purging/archiving tracked data, backup/disaster log shipping etc.

In BizTalk360 version 8.4, we brought in capabilities to manage SQL jobs directly from the web console. This functionality of monitoring SQL Jobs has been in the product for a very long time.

Few Enhancements and Bug Fixes

This article mainly covers all the new and exciting features we shipped in BizTalk360 version 8.4. In every release, we also allocate time to enhance current features and to address any top priority bugs we received from our existing customers.  Here are some of the key enhancements and bug fixes

Alarm Management Improvements: Now you have the ability to bulk enable/disable monitoring alarms. You can also rename the alarms (again, one of the most requested features on our feedback portal.)

Webhook Notification Channel: Now you have the ability to specify (override) a new endpoint URL at the alarm level.

Logic Apps Monitoring Performance Improvement: Few critical updates been made to improve the performance of monitoring Azure Logic Apps

Get started today

Author: Saravana Kumar

Saravana Kumar is the Founder and CTO of BizTalk360, an enterprise software that acts as an all-in-one solution for better administration, operation, support and monitoring of Microsoft BizTalk Server environments. View all posts by Saravana Kumar