July 15, 2019 Weekly Update on Microsoft Integration Platform & Azure iPaaS

July 15, 2019 Weekly Update on Microsoft Integration Platform & Azure iPaaS

RRDo you feel difficult to keep up to date on all the frequent updates and announcements in the Microsoft Integration platform and Azure iPaaS?

Integration weekly update can be your solution. It’s a weekly update on the topics related to Integration – enterprise integration, robust & scalable messaging capabilities and Citizen Integration capabilities empowered by Microsoft platform to deliver value to the business.

If you want to receive these updates weekly, then don’t forget to Subscribe!


Microsoft Announcements and Updates


Community Blog Posts





How get started with iPaaS design & development in Azure?


Hope this would be helpful. Please feel free to reach out to me with your feedback and questions.

Monitor the Host Instances in Clustered/High availability environment

Monitor the Host Instances in Clustered/High availability environment

Monitor Host Instances in Clustered/High availability environment

A BizTalk Server Host is a logical container that contains the BizTalk Server runtime processes in which you deploy items such as adapter handlers, receive locations (including pipelines), and orchestrations.  The host instance is the runtime process where the message processing, receiving, and transmitting occurs. Obviously, it is very important to ensure your host instances are running in your BizTalk environment. BizTalk360 already made it easy for its customers by introducing the Host Instance Monitoring functionality. In BizTalk360, you can monitor the BizTalk host instances by setting the Expected State. If the expected state doesn’t match with the current state of the host instance, you will get notified.

Out of the box, BizTalk360 allows users to monitor the Clustered Host Instances using the “AtLeastOneActive” state.

If this state is assigned as the Expected State, then the monitoring service will verify that across all the instances for the respective clustered host, at least one host instance is active, guaranteeing that the server is running, and no downtime happened for that host/server. To know more about Clustered Host Instance monitoring, follow this blog link.

In the upcoming BizTalk360 version 9.0 Phase 2, we have extended this core functionality to monitor the host instances for high availability environment, as per the customer feedback and voting as below.


BizTalk Server High availability

The following figure shows a BizTalk Server deployment that provides high availability for the receiving host by having two host instances on different computers. Note, that in this figure the processing and sending host is not highly available, because there is only one host instance processing the BizTalk items assigned to the host.


How BizTalk360 helps to monitor the Host Instances in clustered/High availability Environments?

In a business scenario, if you have two BizTalk servers and they both have a Host Instance on them called “FTP”, both Host Instances will trigger a connection to this FTP server. If a file exists on the FTP server that matches your Receive Location’s file mask, both of these host instances will attempt to retrieve a copy of this file, as shown below.


To avoid processing duplicate messages from the high availability, organizations may use the Windows Clustering. This will guarantee that only one host instance is running at a time.


In the above cases, any one of the BizTalk servers needs to be in the active state to receive the file, if both the server is down, then the file will not get received. With BizTalk360, a user can monitor both the host instances in different servers, by setting up the “AtleastOneActive” State as shown below.


In High Availability environments, users can have all the host instances to be Active. For example, if you have 3 BizTalk Host Instances, in multiple BizTalk servers, used for different purposes like receiving, processing and sending the files.


You can set up monitoring for the above scenario in BizTalk360, by setting the expected state to Started for all the three Host Instances as you can see below. When any of the host instances goes down, you will be notified.


Setting up the “AtleastOneActive” state for Host Instances

The “AtleastOneActive” state is mainly used to monitor the Active/Passive clustered and Non-clustered host instance monitoring. The user can set the “AtleastOneActive” state only for clustered host instances and non-clustered host instances in high availability environment.


Setting the “AtleastOneActive” state will monitor across all the configured host instances of the same host and check if any one of the host instances is active. If so, the monitoring state will be healthy.

If all the configured host instances are down or stopped state, then the host instance monitoring will turn to Critical state.


The monitoring status of the host instances will get reflected in the BizTalk360 Monitoring dashboard and the same will be notified in detail in the alert email.


How does AutoCorrect functionality work for the AtleastOneActive state of the host Instance monitoring?

Do you feel frustrated to manually start host instances, when all the host instances become down in clustered/High availability environment?.

BizTalk360 comes up with another solution with its Auto Healing capability.

For Example, if you have 2 BizTalk host instances running under Clustered/high availability setup, from 2 host instances any one instance needs to be in the active state for performing file transactions. If both the host instances are down, then you need to start the host instances manually. This requires continuous monitoring, which can be handled by BizTalk360 itself, by configuring the AutoCorrect functionality to the clustered or non-clustered host instances.


In the above screenshot, the auto-correct functionality is enabled for 2 host instances in different BizTalk servers BTS1 and BTS2. If both the host instances are down, from next monitoring cycle BizTalk360 will try to auto-heal any one of the host instances from these servers. On the first attempt, it will try to start the host instance in BTS1 server. If the instance is started successfully then it skips auto-healing the next server BTS2. In case BizTalk360 fails to start the instance in BTS1, then it tries to start the host instance on the server BTS2. BizTalk360 will try to start any one of the host instances until it reaches the maximum attempt count. You can also reset the auto-correct by setting the reset interval. To know more about the auto-correct feature click here.


Monitoring the Host Instances in Clustered and High availability environment is a must for a healthy BizTalk environment. BizTalk360 monitors those host instances in an easy and simple manner. It also manages the host instances and ensures they are up and running, by using the auto-correct functionality. The upcoming version of BizTalk360 will ease your way of monitoring the host instances in clustered and high availability setup.

The post Monitor the Host Instances in Clustered/High availability environment appeared first on BizTalk360.

BizTalk360 GitHub Project

BizTalk360 GitHub Project


BizTalk360 is the one-stop tool to manage operations, monitoring and analytics of your BizTalk environment.  Over 80+ features in the product bring all the BizTalk Tools like EDI, ESB, Business Rules, BAM, and the BizTalk Health Monitor. It is a matured product with these features included of which there have been 50+ iterations of product updates. User Access Policies with governance audit (Security), Operational Tool Integration, Monitoring, and Analytics are among the BizTalk360 core capabilities. BizTalk360 has brought important business scenarios into the product like:

  • Auto-healing of BizTalk Artifacts
  • Automated actions on Suspended Instances
  • Process Monitoring and Data Monitoring

 These capabilities make BizTalk360 the must-have tool to manage the BizTalk Server.

Customization in BizTalk360

BizTalk360 has provided solutions to common Integration scenarios (BizTalk, Azure). In addition to the above-mentioned features, BizTalk360 has provided the opportunity to integrate custom solutions into the product through:

  • Custom Notification Channels
  • Custom Widgets
  • Create Alarm Tool


Custom Notification Channels and Custom Widgets are bundled into the product, during the major update of v8.0.  Users are taking advantage of these features, by customizing alert notification to their preferred channels and visualize their important data in BizTalk360 Dashboard through Custom Widgets.

The source code of such customized solutions (Notification channels, Widgets and Alarm Creation Tools) is available in our GitHub Project.

Users can utilize the source code to create new customized solutions or extend their existing solution to meet their business requirements.

Users can commit their solutions into BizTalk360 GitHub which will benefit other users who follow this channel.

Custom Notification Channels

Custom Notification Channels provide a platform to send alerts to an external system, like your ticketing system, internal databases, calling a REST endpoint, executing a PowerShell script, etc. If you want to know how you can create, customize and configure new or existing custom notification channels, read this article.

 BizTalk360 has out of box solutions for the following Custom Notification Channel:

Users can create their own customized solution for their ticketing systems rather than ServiceNow or trigger activities (PowerShell) based on their business needs. We can see some of the use cases that users can take by customizing the notification channels:

  1. ServiceNow – Customers can extend the BizTalk360 ServiceNow notification channel to add more fields to meet the business process
  2. PowerShell Notification – Users are utilizing this capability to trigger actions when monitoring conditions are violated
    • Start/Stop Host Instances

    • Terminate dehydrated service Instances

    • Terminate messages after X number of days

    • Manage Maintenance mode during BizTalk Deployments

  3. Microsoft Teams – Enhance the Microsoft Teams notification channel to include a message card
  4. SMTP Notification – Users can improve on default notification by using the SMTP Notification Channel to configure Up, AutoCorrect Email separately.

The source code in the GitHub project can be pulled, customize the code to meet your business scenario and then configure it in BizTalk360 to be able to use your custom solutions.

Custom Widgets

Custom Widget is a powerful feature by which users can take advantage in BizTalk360. Customers can integrate their business process flows by using Custom widgets. This allows integrating solutions with BizTalk360 own API’s, as well as third-party API’s (like SalesForce, Power BI, Azure, etc.) and database queries by using the Secure SQL Queries.

Use Cases:

  1. Business Process – Few customers are integrating their business flows using their internal API’s and created custom widgets by which business users take advantage of BizTalk360 dashboard integration
  2. Graphical Representation – Visualize the reports in Solid gauge using High Charts, which is available in BizTalk360
  3. Consolidated View of Monitoring Dashboard – Users are able to customize their monitoring dashboard to a single view, by grouping type of monitors
  4. Secure SQL Query – This provides the opportunity to pull data from other databases into the BizTalk360 dashboard through Secure SQL Queries
  5. Monitoring Status– Many of our users are using dashboards to monitor their business flow integrations. They can create a custom widget and pin it to a dashboard to monitor the integration. You can think of showing the following:
    • Errors & Warnings of an Alarm in a Grid

    • Last Date Time of Errors & Warnings

    • Web Endpoint Monitoring status of an integration

    • Number of Suspended Instances in an alarm

    • Number of messages processed in an application

  6. Monitoring SQL JobsSQL Jobs are monitored through BizTalk360 API’s by creating a custom widget



Alarm Configuration

Another example of running third-party executables is the creation of BizTalk360 alarms. Normally, you’ll have to do this manually after each deployment of a new or changed BizTalk application. This too can be a time-consuming and error-prone task.

By using BTDF and a tool called BT360Deploy, you can automate the creation of alarms during the deployment of your BizTalk application.

In complex BizTalk environments, creating Alarms and mapping artifacts to monitor is a tedious process. To overcome this challenge, our Technical Lead (Lex) has written an article to configure BizTalk Artifacts for monitoring through the Alarm Creation Tool. In that article, he explained how the BizTalk Deployment Framework (BTDF) is integrated with BizTalk360 Monitoring.


Users can take advantage of custom solutions using Custom Notification Channel, Custom Widgets and Alarm Creation Tools.  Customers get the maximum benefits out of BizTalk360 when creating custom solutions.

The post BizTalk360 GitHub Project appeared first on BizTalk360.

July 8, 2019 Weekly Update on Microsoft Integration Platform & Azure iPaaS

July 8, 2019 Weekly Update on Microsoft Integration Platform & Azure iPaaS

Do you feel difficult to keep up to date on all the frequent updates and announcements in the Microsoft Integration platform and Azure iPaaS?

Integration weekly update can be your solution. It’s a weekly update on the topics related to Integration – enterprise integration, robust & scalable messaging capabilities and Citizen Integration capabilities empowered by Microsoft platform to deliver value to the business.

If you want to receive these updates weekly, then don’t forget to Subscribe!

Microsoft Announcements and Updates


Community Blog Posts






How get started with iPaaS design & development in Azure?


Hope this would be helpful. Please feel free to reach out to me with your feedback and questions.

Thanks! Awarded as Microsoft Azure MVP 2019-2020

Thanks! Awarded as Microsoft Azure MVP 2019-2020

On 1st January 2011, I was awarded for the first time Microsoft Most Value Professional (MVP), back them as a BizTalk Server MVP, joining an amazing worldwide group of technicians and community leaders who actively share their high quality and real-world expertise with other users and Microsoft. And since then I was re-award in:

  • 2012, again as BizTalk Server MVP;
  • From 2013 to 2015, I was rebranded as Microsoft Integration MVP;
  • In 2017, Integration become part of Azure category, so I was awarded for the first time as Azure MVP
  • And in the middle of 2017, I was Award in two categories, Azure and Visio MVP;
  • In 2018, I was rewarded Azure MVP

And today, still with all the same pride, honor and enthusiasm than the first time, I’m delighted to share with you that in July 1st I was renewed as a Microsoft Azure MVP (Microsoft Most Valuable Professional) for one more year.


It is with great pride we announce that Sandro Pereira has been awarded as a Microsoft® Most Valuable Professional (MVP) for 7/1/2019 – 7/1/2020. The Microsoft MVP Award is an annual award that recognizes exceptional technology community leaders worldwide who actively share their high quality, real world expertise with users and Microsoft. All of us at Microsoft recognize and appreciate Sandro’s extraordinary contributions and want to take this opportunity to share our appreciation with you.

With just over 3,000 awardees worldwide, Microsoft MVPs represent a highly select group of experts. MVPs share a deep commitment to community and a willingness to help others. They represent the diversity of today’s technical communities. MVPs are present in over 90 countries, in more than 40 languages, and across numerous Microsoft technologies. MVPs share a passion for technology, a willingness to help others, and a commitment to community. These are the qualities that make MVPs exceptional community leaders. MVPs’ efforts enhance people’s lives and contribute to our industry’s success in many ways. By sharing their knowledge and experiences, and providing objective feedback, they help people solve problems and discover new capabilities every day. MVPs are technology’s best and brightest, and we are honored to welcome Sandro as one of them.

This is my 9th straight year on the MVP Program, an amazing journey, and experience that, again, started in 2011, and that gave me the opportunity, and still does, to travel the world for speaking engagement, share the knowledge, and to meet the most amazing and skilled people in our industry.

As usual, I would like to thank all of you! And special to my wife Fernanda and my three kids: Leonor, Laura, and José. And to all members of my beautiful family. THANKS for all the support during these last years! You guys are my inspiration!

And a special thanks to my MVP Lead Cristina Herrero for all the support, to Microsoft Integration Team and Azure Teams, to all my fellow Microsoft Azure (Integration) MVP’s and to DevScope (my company) and all my coworkers (no names here because all of them are amazing professionals)

It’s a big honor to be in the program and I’m looking forward to another great year!

The post Thanks! Awarded as Microsoft Azure MVP 2019-2020 appeared first on SANDRO PEREIRA BIZTALK BLOG.

Consolidated monitoring with the new BizTalk Group Dashboard

Consolidated monitoring with the new BizTalk Group Dashboard


BizTalk users who use BizTalk360 can manage their BizTalk environment in a more efficient way. Monitoring and Notification is amongst the most important features in BizTalk360, The first and foremost step for monitoring is mapping the artifacts to an alarm. In which user can use this on a daily  basis to monitor their BizTalk environment and get notified if any violation occurs.

The Monitoring Dashboard is the one stop point for users to view the status of the artifacts which are mapped to an alarm. Typically a customer displays the monitoring dashboard in big screens and looks for the health of their BizTalk environment visually.  The monitoring dashboard is structured at alarm level, which lists the status of all the artifacts mapped to the respective alarm. However, you cannot see all the alarms and their mapped artifacts status in a single view, which was a usability problem persist from our side. To overcome these we are implementing the “BizTalk Group Dashboard” for the v9.0 (phase 2). With this dashboard you can view the status of all the artifacts mapped to any number of alarm in a single view and you can easily customize it using the rich filter option provided  

Let’s deep dive into BizTalk Group Dashboard and see how it solves the problem.

BizTalk Group Dashboard

The BizTalk Group Dashboard is designed to monitor all the artifacts (and view all errors) which are mapped to all the configured alarms in BizTalk360, in a single view. The BizTalk Group Dashboard can be found under the Home of Monitoring section. The “BizTalk Group Dashboard” will automatically pick up all the artifacts (which are mapped to any of the BizTalk360 alarms) and displays the status of the artifacts in a graphical manner . Also the error details of the displayed artifacts  are shown in a grid view . The BizTalk Group dashboard checks the status of the artifacts in every 60 seconds .

For better usability, BizTalk Group Dashboard splits the windows in two sections:

  1. Displays the mapped artifacts in a graph
  2. Show the Error/Warning details in an artifact level segregation

The dashboard has a draggable separator so that the user can adjust the width of the graphical window and the error/warning pane, based on their resolution. They can even minimize or maximize the error/warning pane to the leftmost section.

There is a chance of mapping the same artifact to multiple alarms. In such cases, the BizTalk Group Dashboard picks only the positive state of artifacts for monitoring. For instance, a Receive Location is mapped to an alarm1 with the expected state as Enabled and the same receive location is mapped to another alarm, say alarm2, with the expected state as Disabled; in this scenario, the BizTalk Group dashboard will only consider the positive state mapping such as expecting state as Enabled for monitoring.

Note: BizTalk group Dashboard is only designed for visual monitoring of the BizTalk environment and artifacts. It will not trigger any alert notifications.

Advantages of BizTalk Group Dashboard

Comparing to the already existing Monitoring Dashboard, the BizTalk Group dashboard has quite some advantages. Some of the main advantages of the BizTalk Group Dashboard are:

  • It consolidates the status of all the artifacts which are mapped to any alarm and there is no need to switch alarms for viewing the status of any other artifact
  • The BizTalk Group Dashboard can be split into two windows; one for the artifact graph and the other for viewing the Error/Warning pane. For usability, the split windows can be adjusted based on the monitor resolution
  • For better monitoring, the user can apply filters (at the top of the graph) based on the artifact and/or based on the event type filter option
  • The user can view the Error/Warning details along with the graph



Many organizations use different alarm configurations and patterns for their processes. Some of the commonly used alarm configuration patterns are:

  1. Integration-based alarm configuration
  2. Role-based alarm configuration
  3. Entity-based alarm configuration

Integration-based alarm configuration means configuring BizTalk artifacts, File location, Queues, Web endpoints, etc. in an alarm, which all belong to the same integration.
Role-based alarm configuration is configuring alarms for people in different roles. For example, a platform administrator will be interested in infrastructure settings like Disk, system resources, host instances in an alarm.A Database Administrator (DBA), on the other hand, will be more interested in the SQL Server instances and the jobs which have been deployed in it.
Entity-based alarm configuration is configuring similar components in the same alarm. For instance, putting all the queues like IBM Queue, Azure service bus queue, MSMQ in one alarm.

Irrespective of the alarm configuration/pattern, now the user can customise the BizTalk group dashboard using the filter capablity .Say Administrator can monitor the infrastructure components which are mapped to all the alarm .) and queues in a single dashboard window by filtering only the Infrastructure components.


With this feature, users are able to visually monitor all the artifacts and view error/warning details in a single dashboard with a much more comprehensive view. If you have any feedback or suggestions, please write to us at support@biztalk360.com. You can get started with monitoring your BizTalk environment via the BizTalk Group Dashboard by downloading the 30-day free trial of BizTalk360.

The post Consolidated monitoring with the new BizTalk Group Dashboard appeared first on BizTalk360.

SQL Collation Support in BizTalk360

SQL Collation Support in BizTalk360

BizTalk360 always values customer feedback and we focus on improving the usability by adding new features and enhancing existing features as per customer requirements. Recently we have received a few support cases from our customer end about SQL Collation. So, we have analysed and improved it.

We are happy to share you that BizTalk360 has improved its standards by supporting case-sensitive collation. This will soon be available in the upcoming version 9.0 Phase 2. In the previous version of BizTalk360, we were supporting the general collation like Latin1_General_CI_AI for BizTalk360 Database. In this blog, we will see why we started supporting case sensitive SQL collation and what the benefits are of having it.

As you all know BizTalk360 is the one-stop monitoring tool to monitor BizTalk Server. Since BizTalk Server supports case sensitive collation, it is so important that BizTalk360 also supports case sensitive collation, to make it as a compatible tool.

About SQL Collation

A collation is a configuration setting that determines how the database engine should treat character data at the server, database, or column level. SQL Server includes a large set of collations for handling the language and regional differences that come with supporting users and applications across the world.

SQL Server is, by default case insensitive; however, it is possible to create a case sensitive SQL Server database and even to make specific table columns case sensitive. The way to determine a database or database object is by checking its “COLLATION” property and look for “CI” or “CS” in the result.

  • CI refers to Case Insensitive
  • CS refers to Case Sensitive

List of collation combination in SQL setup:

  •       SQL Server collation
  •       SQL Server and Database collation
  •       SQL Server, Database and Tables collation
  •       SQL Server, Database, Tables and Column collation

SQL has 4 levels of collation: server, database, table and column. If you change the collation of the server, database or table, then we don’t want to change the setting for each column. If needed, we can change the default collations at database/table/column level.


Example: If you change the default collation of a database, each new table you create in that database will use that collation, and if you change the default collation of a table each column you create in that table will use that collation.

To change the collation of SQL Server: SQL Server collation can be changed in two ways, one is by changing the collation during the installation and the other way is changing the collation after installation.

Changing the collation during the installation: while installing SQL server, you can find the Server Configuration section. Under this section, you can see two sections Service accounts and Collation. Click the Collation tab to verify the collation. The collation will have the default supported collation. Click on the Customize… button and select the required collation. Click Next and proceed further.

Note: To have case sensitive collation use the collation which holds CS_AS.


Why BizTalk360 supports SQL Collation

BizTalk Server supports all case sensitive and case insensitive SQL Server collations except for binary collations. So, it is important that BizTalk360 should support the same collations.

Collation refers to a set of rules that determine how data is sorted and compared. When BizTalk360 calls any functions or data from BizTalk Server which is of case sensitive collation, then it may lead to an exception.

Let’s consider that the BizTalk databases hold a case sensitive collation and BizTalk360 holds a case insensitive collation. In this case, when BizTalk360 gets integrates with BizTalk Server and tries to read the data, it is necessary to ensure all the procedure and column names should be exactly similar.

From the above picture it is clearly understandable that having a collation-based database, and when trying to fetch the data of an object, it will compare the case sensitiveness and provide the data. In case of uppercase and lowercase mismatch, it will lead to an exception.

Say for example, Invalid column name ‘nAdminHostId’ (or) Invalid object name ‘dbo.b360_ST_Schedules’ (or) Must declare the scalar variable “@expiryDateTime”etc.

Most of our customers configure BizTalk360 in the SQL Server instance where BizTalk Server is configured. In such cases, the below-mentioned scenario will add value for supporting collation.

Scenario I: BizTalkMgmtDb (Latin1_General_CS_AS) & BizTalk360 (Latin1_General_CI_AI)

Let us consider the scenario of having a SQL server with case insensitive SQL collation and case sensitive collation for BizTalkMgmtDb.

In this case, the BizTalk360 installation will be successful and BizTalk360 database will be created with case insensitive SQL collation since SQL Server holds a case insensitive SQL collation. But in the BizTalk360 application, you may face some exceptions getting popped up related to “Invalid column name ‘nAdminHostId”, in few sections like BizTalk Server, SQL Server, SQL Server instance, Analytics etc.

Scenario II: SQL Server (Latin1_General_CS_AS) & BizTalk360 Database (Latin1_General_CI_AI)

When creating a database in the SQL Server, by default the database will hold the collation of the SQL Server. In this case, the customer has the CS collation for the SQL server, thus all the BizTalk related databases hold the same case sensitive collation. In this case, when you try to install the BizTalk360 Db in the SQL server which holds Latin1_General_CS_AS collation, the installation will fail!


“ExecuteSqlStrings:  Error 0x80040e14: failed to execute SQL string, error: Must declare the scalar variable “@environmentId”., SQL key: CreateTablesWin SQL string: ALTER PROCEDURE . @EnvironmentId uniqueidentifier”

The BizTalk360 database is created with a CI collation and BizTalk360 is installed by pointing the database to the CI collation DB. But, since the database was under the case sensitive collate SQL server, we had the same exception.

Query to create database with a case CI collation


COLLATE Latin1_General_CI_AI;


In BizTalk360 v9.0 phase 2, the collation issue has been fixed across the application. We have revised the collation issue across each section in the application.

Checklist prepared to test SQL collation in BizTalk360

BizTalk360 supports many features like ESB, BAM, Azure etc. When having case sensitive collation for BizTalk360 Database, it is important that we look into all the sections of BizTalk360 (Operation, Monitoring, Analytics and Settings) to make it work correctly.

Say for example, BizTalk360 manages ESB exception data (EsbExceptionDb), in which it is important that all the column names, object names and procedures are exactly the same. We have nailed the compatibility of the BizTalk360 database with BizTalk Server database, ESB, BAM, Azure etc.


We always aim to improve the product by filling gaps in customer needs. Considering the same, we have improved the collation support in our upcoming release version 9.0 phase 2.

It is possible to have a combination of collation. Say, for example, SQL Server with the collation Latin1_General_CS_AS and BizTalk360 with Latin1_General_CI_AI. By achieving this, we have taken BizTalk360 to leading-edge technology in its standards.

The post SQL Collation Support in BizTalk360 appeared first on BizTalk360.