by Lex Hegt | Aug 1, 2019 | BizTalk Community Blogs via Syndication
After the installation of BizTalk360, the implementation phase starts. One of the steps you will want to do is taking benefit of the rich automated monitoring features the product provides. For that, you must set up alarms in BizTalk360. In this article, we intend to give you some best practices around Alarm Configuration. Firstly, however, we’ll shortly explain the different monitoring concepts which are available in BizTalk360.
A short explanation of the supported monitoring types
BizTalk360 has multiple types of monitoring, which are Threshold monitoring, Health Monitoring, and Data Monitoring. Each alarm you create in BizTalk360 can support all types of monitoring, either as separate alarms or as combined alarms.
As you can see from the above screen, BizTalk360 enables you to configure an alarm for multiple purposes. So, in case you want the same people to receive the same notifications for both Thresholds as for Health Checks, you only need to create one alarm.
Threshold monitoring
With this kind of monitoring, you get notified in case a threshold occurs. For example, that can be a Receive Location which was Disabled, while it should be Enabled or any other artefact which is mapped in an alarm and is in an unexpected state.
At the Alarm creation level, you can configure the following:
- Name – each alarm needs to be given a name. The more descriptive the name, the easier it is for identifying the purpose of the alarm
- Description – optionally, you can provide a detailed description of the purpose of the alarm
- Email Ids – each alarm can send notifications to multiple recipients
- High Priority – in case you want specific notifications (for example when anything goes wrong with the BizTalk platform) to jump out in your email box, you can enable the High Priority switch
- Violation persistence – to configure after how much time and with which interval you want to be notified of exceptions
- Limit the number of notifications per exception – to prevent you from being bombarded with notifications and lose the overview as a result
- Back to Normal notification – receive a notification when the alarm no more contains artifacts in the wrong state
- Alert reset – start receiving notifications again after a specific time in case you have received the maximum number of notifications
- Alert schedule – configure when the alarm should perform monitoring. Handy when you know that specific artifacts are not available all the time, due to, for example, maintenance
Health monitoring
It can be helpful if you receive a report with the actual status of the monitored artifacts, for example at 9AM every business day. For such scenarios, you can configure alarms for Health monitoring.
BizTalk360 Health Monitoring alarms allow you to easily configure when you want to receive these status reports via an intuitive interface.
Below screenshot shows the screen where you can select when you want to receive these status reports.
This alarm has not only been configured to send a report at the beginning of each working day but also towards the end of the business day. This helps in being aware of the status of your environment before you leave home.
Data Monitoring
A detailed explanation of Data Monitoring falls outside the scope of this article, therefore we provide a somewhat better overview of its capabilities here.
The concept of Data Monitoring allows you to set up monitoring of your data flows. As an example, your BizTalk platform and applications might be in perfectly good health, but when a misconfigured firewall prevents messages from arriving in BizTalk, no processing will be taking place. This interrupts your business processes, with maybe bad consequences for your company. Data Monitoring can be used to monitor this kind of scenarios.
Data Monitoring provides monitoring at the following levels:
- Process Monitoring – monitor whether your processes run as expected
- MessageBox Monitoring – monitor the active processes and (optionally) take automated actions
- Tracking Data Monitoring – monitor completed processes
- BAM Monitoring – monitor your deployed BAM Views and Activities
- EDI Data Monitoring – monitor your EDI processes
- ESB Data Monitoring – monitor your ESB Exceptions and resubmissions
- Logic Apps Monitoring – monitor tens of metrics about the processing of your Azure Logic Apps
- Event Log Monitoring – monitor your consolidated Event Log entries of your BizTalk and/or SQL servers
After selecting which kind of Data Monitoring you want to set up, it now comes down to configure what exactly you want to monitor and when monitoring needs to be done.
Below screenshot gives an example of Process Monitoring.
Once you have set up Data Monitoring, the output of all the different monitor runs will be shown in a consolidated Data Monitoring dashboard. This dashboard helps in answering the questions from your business/functional users whether a specific process has taken place.
You can refer to the Data Monitoring section in the Documentation portal for a more detailed explanation of this way of monitoring in BizTalk360.
Some more information about BizTalk360 Alarms can be found here.
BizTalk platform versus BizTalk application monitoring
When setting up monitoring in BizTalk360, it is handy to make a difference between monitoring the actual BizTalk platform and the BizTalk applications which are deployed on it. Making this distinction gives several advantages, like:
- Notifying different people, depending on their role and needs
- Use different monitoring settings like:
-
- The monitoring interval for early notifications in case of important issues
- The number of notifications you want to receive per exception
- Receive a ‘back to normal’ notification in case everything is healthy again
- Set the High Priority flag to easily spot important notifications in your email box
- Run PowerShell scripts for immediate actions
Even within platform monitoring, you can make segregations for different administrators. For example, a SQL Server DBA has different needs than a System (Windows) Administrator, so other artefacts need to be monitored. BizTalk360 enables you to set up different Alarms for different needs.
What to setup for Platform monitoring
When you are creating an alarm for platform monitoring, you want to be notified of unexpected situations which might occur on your BizTalk platform. For this purpose, you will create a Threshold Monitoring Alarm. If you also want to receive a daily status report at say every business day at 8:00 AM, you can configure an alarm for Health Check Notification.
See below for a list of artefacts you could consider for monitoring BizTalk platform health:
- Host Instances (clustered and non-clustered)
- Host Throttling
- BizTalk Server availability
- SQL Server Jobs
- Windows NT Services
- Enterprise Single Sign-On service
- Internet Information Services
- BizTalk Health Monitor output
- Eventlog entries
- BizTalk Criticals, Errors, and Warnings
- SQL Server Criticals, Errors, and Warnings
- ESSO Criticals, Errors and Warnings
- Internet Information Services Criticals, Errors, and Warnings
- Server restarts
- Certificate expiration
- BizTalk and SQL related stuff
- Disk space
- CPU/Memory usage
- Windows NT Services
Below screen shows the Monitoring Dashboard for a Platform alarm.
What to setup for BizTalk Application monitoring
Before we move to discuss which artifacts to monitor from BizTalk Application alarms, let’s firstly discuss how to organize your BizTalk Application-oriented alarms. Basically, there are two ways you can set up your BizTalk Application alarms.
The Alarm equals Application approach
When deploying BizTalk applications, it is often handy to create an alarm per BizTalk application for Threshold Monitoring and/or Health Check Notification. The main motivation for this is, that a BizTalk Application is the unit which you will deploy, so setting up alarms around them will help in keeping the overview. For easy reference, you could give the alarms the same name as the BizTalk application it will be monitoring. This can be considered as a bit more technical approach for your monitoring.
The Alarm equals Interface approach
Another approach to set up your BizTalk Application-oriented alarms is around the interfaces which are being processed by BizTalk. Say, your BizTalk environment contains multiple applications of which several their contained artifacts together are part of a Purchase Order. To keep the overview of such an interface, it makes sense to monitor all artifacts related to that interface. This is totally fine and BizTalk360 allows you to set up your alarm in such a way. Ideally, you can give the alarm the name of the interface. This approach is a bit more business-wise way of monitoring and can be helpful if you need to notify your business/functional users in case of any issues.
However, there is a downside to monitoring your BizTalk artifacts around interfaces. Because artifacts from multiple BizTalk Applications need to be monitored, it might be a bit harder, especially when many artifacts are involved, to keep the overview and be sure that you are really monitoring all the relevant artifacts.
No ‘Always Right’ way
Summarizing, there is no ‘always right’ way to set up your BizTalk Application monitoring. Even a ‘hybrid’ setup, which follows both just mentioned practices, of your monitoring is totally fine and not uncommon. In the end, it all depends on what works the best for you and your organization. BizTalk360 gives you all the freedom you need.
Artifacts to be monitored
When setting up monitoring for your BizTalk Applications, you will set it up for your ports and orchestrations of your BizTalk Applications. However, you can also consider the following artefacts for monitoring:
- State of Services Instances
- Suspended (Resumable/Not Resumable)
- Active
- Dehydrated
- Ready to Run
- Scheduled
- Endpoints of your Receive/Send ports
- Web Endpoints
- Queues (MSMQ, IBM MQ, Azure Service Bus)
- File shares
- FTP sites
- API Apps
- Logic Apps
Tip: To be able to find out if files/messages are being picked up, you can monitor your File shares, FTP sites, and Queues. Below screen shows an example of monitoring a File share.
As a side note, BizTalk Server Best Practices suggest that you have a proper Host configuration in place, where separation takes place based on receiving/sending messages, processing of orchestrations and tracking of processed messages/service instances. However, sometimes Host configuration has been set up for a specific process or BizTalk application. In that case, it is valid to monitor your Host Instance(s) in your BizTalk application-oriented alarm, instead (or on top) of your BizTalk platform alarm.
Advanced Monitoring setup
Besides deciding on how to setup your alarms and which artifacts to monitor, there are a couple of other topics you could consider take benefit of. You can think of:
- Auto-Correct – the ability to automatically recover artifacts
- Notification Channels – inform people not just via email but also via different channels
- Customized Notification Templates – provide additional information or brand the templates
Using such features not just makes your life as an administrator easier, it will also help you in making the best of your investment in BizTalk360. Let’s have a better look at all these options.
Auto-Correct
For state-related artefacts like BizTalk Ports and Orchestrations, Host Instances and SQL Server jobs, you could consider using the Auto-Correct feature, which, after a threshold situation has been met, tries to get the artifacts back to the expected state.
Tip: You can use this feature to automatically Enable your Receive Locations, in case they failed to connect to their endpoints due to a temporary failure!
You can read more about the Auto-Correct feature in this article.
Below screen shows that BizTalk360 will try to automatically try to recover the Host Instances once they are no more in the Started state.
Notification Channels
The default way of BizTalk360 for sending notifications is via email. However, sending notifications does not end there. The product has a number of so-called Notification Channels you can use. Examples are:
- Slack – send notifications to a Slack channel
- ServiceNow – have tickets created, directly in ServiceNow
- Microsoft Teams – send notifications to a Teams channel
- PowerShell – trigger a PowerShell script to perform an action
- SMTP – have more control over the way notifications are being sent
- Webhook – connect to an already existing REST API
Besides this rich set of notification channels, there is also an easy to use API at your convenience. By implementing the API, you can, for example, connect to your ticketing system or other important backend systems.
Below screen shows the information you can provide when using the ServiceNow Notification channel.
More information about all the Notification channels can be found here.
Customized Notification Templates
BizTalk360 allows you to customize the notifications it sends. This allows you for example to provide some additional information/instructions for the people who receive the notifications. However, it can also be used for branding purposes.
Although some notification information can simply be provided via text input fields, the actual templates are based on XSLT, so some skills with editing such files will be helpful.
More information about customizing the notification templates can be found here.
Conclusion
With this article, we intended to provide you some help and guidelines with setting up your alarms. Both the Documentation portal and our Blog contain many articles on monitoring your BizTalk environment with BizTalk360. However, if you feel you need some more support with setting up your monitoring, feel free to reach out. We are here to help you!
The post Best Practices for BizTalk360 Alarm Configuration appeared first on BizTalk360.
by Sowmiya Subramanian | Jul 11, 2019 | BizTalk Community Blogs via Syndication
Monitor Host Instances in Clustered/High availability environment
A BizTalk Server Host is a logical container that contains the BizTalk Server runtime processes in which you deploy items such as adapter handlers, receive locations (including pipelines), and orchestrations. The host instance is the runtime process where the message processing, receiving, and transmitting occurs. Obviously, it is very important to ensure your host instances are running in your BizTalk environment. BizTalk360 already made it easy for its customers by introducing the Host Instance Monitoring functionality. In BizTalk360, you can monitor the BizTalk host instances by setting the Expected State. If the expected state doesn’t match with the current state of the host instance, you will get notified.
Out of the box, BizTalk360 allows users to monitor the Clustered Host Instances using the “AtLeastOneActive” state.
If this state is assigned as the Expected State, then the monitoring service will verify that across all the instances for the respective clustered host, at least one host instance is active, guaranteeing that the server is running, and no downtime happened for that host/server. To know more about Clustered Host Instance monitoring, follow this blog link.
In the upcoming BizTalk360 version 9.0 Phase 2, we have extended this core functionality to monitor the host instances for high availability environment, as per the customer feedback and voting as below.
BizTalk Server High availability
The following figure shows a BizTalk Server deployment that provides high availability for the receiving host by having two host instances on different computers. Note, that in this figure the processing and sending host is not highly available, because there is only one host instance processing the BizTalk items assigned to the host.
How BizTalk360 helps to monitor the Host Instances in clustered/High availability Environments?
In a business scenario, if you have two BizTalk servers and they both have a Host Instance on them called “FTP”, both Host Instances will trigger a connection to this FTP server. If a file exists on the FTP server that matches your Receive Location’s file mask, both of these host instances will attempt to retrieve a copy of this file, as shown below.
To avoid processing duplicate messages from the high availability, organizations may use the Windows Clustering. This will guarantee that only one host instance is running at a time.
In the above cases, any one of the BizTalk servers needs to be in the active state to receive the file, if both the server is down, then the file will not get received. With BizTalk360, a user can monitor both the host instances in different servers, by setting up the “AtleastOneActive” State as shown below.
In High Availability environments, users can have all the host instances to be Active. For example, if you have 3 BizTalk Host Instances, in multiple BizTalk servers, used for different purposes like receiving, processing and sending the files.
You can set up monitoring for the above scenario in BizTalk360, by setting the expected state to Started for all the three Host Instances as you can see below. When any of the host instances goes down, you will be notified.
Setting up the “AtleastOneActive” state for Host Instances
The “AtleastOneActive” state is mainly used to monitor the Active/Passive clustered and Non-clustered host instance monitoring. The user can set the “AtleastOneActive” state only for clustered host instances and non-clustered host instances in high availability environment.
Setting the “AtleastOneActive” state will monitor across all the configured host instances of the same host and check if any one of the host instances is active. If so, the monitoring state will be healthy.
If all the configured host instances are down or stopped state, then the host instance monitoring will turn to Critical state.
The monitoring status of the host instances will get reflected in the BizTalk360 Monitoring dashboard and the same will be notified in detail in the alert email.
How does AutoCorrect functionality work for the AtleastOneActive state of the host Instance monitoring?
Do you feel frustrated to manually start host instances, when all the host instances become down in clustered/High availability environment?.
BizTalk360 comes up with another solution with its Auto Healing capability.
For Example, if you have 2 BizTalk host instances running under Clustered/high availability setup, from 2 host instances any one instance needs to be in the active state for performing file transactions. If both the host instances are down, then you need to start the host instances manually. This requires continuous monitoring, which can be handled by BizTalk360 itself, by configuring the AutoCorrect functionality to the clustered or non-clustered host instances.
In the above screenshot, the auto-correct functionality is enabled for 2 host instances in different BizTalk servers BTS1 and BTS2. If both the host instances are down, from next monitoring cycle BizTalk360 will try to auto-heal any one of the host instances from these servers. On the first attempt, it will try to start the host instance in BTS1 server. If the instance is started successfully then it skips auto-healing the next server BTS2. In case BizTalk360 fails to start the instance in BTS1, then it tries to start the host instance on the server BTS2. BizTalk360 will try to start any one of the host instances until it reaches the maximum attempt count. You can also reset the auto-correct by setting the reset interval. To know more about the auto-correct feature click here.
Conclusion
Monitoring the Host Instances in Clustered and High availability environment is a must for a healthy BizTalk environment. BizTalk360 monitors those host instances in an easy and simple manner. It also manages the host instances and ensures they are up and running, by using the auto-correct functionality. The upcoming version of BizTalk360 will ease your way of monitoring the host instances in clustered and high availability setup.
The post Monitor the Host Instances in Clustered/High availability environment appeared first on BizTalk360.
by Raja Sreenivasan | Jul 4, 2019 | BizTalk Community Blogs via Syndication
Introduction
BizTalk users who use BizTalk360 can manage their BizTalk environment in a more efficient way. Monitoring and Notification is amongst the most important features in BizTalk360, The first and foremost step for monitoring is mapping the artifacts to an alarm. In which user can use this on a daily basis to monitor their BizTalk environment and get notified if any violation occurs.
The Monitoring Dashboard is the one stop point for users to view the status of the artifacts which are mapped to an alarm. Typically a customer displays the monitoring dashboard in big screens and looks for the health of their BizTalk environment visually. The monitoring dashboard is structured at alarm level, which lists the status of all the artifacts mapped to the respective alarm. However, you cannot see all the alarms and their mapped artifacts status in a single view, which was a usability problem persist from our side. To overcome these we are implementing the “BizTalk Group Dashboard” for the v9.0 (phase 2). With this dashboard you can view the status of all the artifacts mapped to any number of alarm in a single view and you can easily customize it using the rich filter option provided
Let’s deep dive into BizTalk Group Dashboard and see how it solves the problem.
BizTalk Group Dashboard
The BizTalk Group Dashboard is designed to monitor all the artifacts (and view all errors) which are mapped to all the configured alarms in BizTalk360, in a single view. The BizTalk Group Dashboard can be found under the Home of Monitoring section. The “BizTalk Group Dashboard” will automatically pick up all the artifacts (which are mapped to any of the BizTalk360 alarms) and displays the status of the artifacts in a graphical manner . Also the error details of the displayed artifacts are shown in a grid view . The BizTalk Group dashboard checks the status of the artifacts in every 60 seconds .
For better usability, BizTalk Group Dashboard splits the windows in two sections:
- Displays the mapped artifacts in a graph
- Show the Error/Warning details in an artifact level segregation
The dashboard has a draggable separator so that the user can adjust the width of the graphical window and the error/warning pane, based on their resolution. They can even minimize or maximize the error/warning pane to the leftmost section.
There is a chance of mapping the same artifact to multiple alarms. In such cases, the BizTalk Group Dashboard picks only the positive state of artifacts for monitoring. For instance, a Receive Location is mapped to an alarm1 with the expected state as Enabled and the same receive location is mapped to another alarm, say alarm2, with the expected state as Disabled; in this scenario, the BizTalk Group dashboard will only consider the positive state mapping such as expecting state as Enabled for monitoring.
Note: BizTalk group Dashboard is only designed for visual monitoring of the BizTalk environment and artifacts. It will not trigger any alert notifications.
Advantages of BizTalk Group Dashboard
Comparing to the already existing Monitoring Dashboard, the BizTalk Group dashboard has quite some advantages. Some of the main advantages of the BizTalk Group Dashboard are:
- It consolidates the status of all the artifacts which are mapped to any alarm and there is no need to switch alarms for viewing the status of any other artifact
- The BizTalk Group Dashboard can be split into two windows; one for the artifact graph and the other for viewing the Error/Warning pane. For usability, the split windows can be adjusted based on the monitor resolution
- For better monitoring, the user can apply filters (at the top of the graph) based on the artifact and/or based on the event type filter option
- The user can view the Error/Warning details along with the graph
Scenarios
Many organizations use different alarm configurations and patterns for their processes. Some of the commonly used alarm configuration patterns are:
- Integration-based alarm configuration
- Role-based alarm configuration
- Entity-based alarm configuration
Integration-based alarm configuration means configuring BizTalk artifacts, File location, Queues, Web endpoints, etc. in an alarm, which all belong to the same integration.
Role-based alarm configuration is configuring alarms for people in different roles. For example, a platform administrator will be interested in infrastructure settings like Disk, system resources, host instances in an alarm.A Database Administrator (DBA), on the other hand, will be more interested in the SQL Server instances and the jobs which have been deployed in it.
Entity-based alarm configuration is configuring similar components in the same alarm. For instance, putting all the queues like IBM Queue, Azure service bus queue, MSMQ in one alarm.
Irrespective of the alarm configuration/pattern, now the user can customise the BizTalk group dashboard using the filter capablity .Say Administrator can monitor the infrastructure components which are mapped to all the alarm .) and queues in a single dashboard window by filtering only the Infrastructure components.
Conclusion
With this feature, users are able to visually monitor all the artifacts and view error/warning details in a single dashboard with a much more comprehensive view. If you have any feedback or suggestions, please write to us at [email protected]. You can get started with monitoring your BizTalk environment via the BizTalk Group Dashboard by downloading the 30-day free trial of BizTalk360.
The post Consolidated monitoring with the new BizTalk Group Dashboard appeared first on BizTalk360.
by Senthil Kumar Palanisamy | May 8, 2019 | BizTalk Community Blogs via Syndication
Introduction
BizTalk360 has powerful monitoring features to manage the BizTalk Application Artifacts, Queues, Infrastructure, Health check tools, etc. Most of the customers are using BizTalk360 for its core monitoring capabilities. BizTalk360 is the single tool to manage operational activities, monitoring and analytics of the mission-critical BizTalk environment. In this transforming phase, it is a tedious process for a dedicated person to monitor all the configurations. To address this scenario, System Alert notification is implemented in BizTalk360 v9.0.
Why System Alerts?
Currently BizTalk360 use Alarms to send out the notifications of Application artifacts, queues, Infrastructure settings health of the BizTalk environment. In the same perspective there is need to know the health of BizTalk360, which will helpful to the administrator who takes care BizTalk360. The BizTalk360 Monitoring service will take care of sending system notifications to the configured admin user email boxes. These email boxes can be configured in the system settings. A new monitoring sub service called ‘System Alert’ has been introduced to send system related notifications. System Alerts can be enabled in the System Alert Settings under Monitoring and Notification tab of system settings. A user can configure multiple admin emails with semicolon separated values.
A System Alert notification can be pushed to the admin users in the two ways:
- Scheduled
- Automatically Triggered
The BizTalk360 Monitoring service has a polling interval of 60 seconds and it will check for system alerts to be pushed. Either these alerts will be pushed based on a trigger or on a schedule.
NOTE: Custom Notification channel alerts are not applicable for Systems Alerts in this version (9.0).
System Alert Schedules
System Alert Schedules will send periodic reports of unmapped application artifacts. This will be helpful to the BizTalk Operation Users/ Administrators who can take appropriate actions to configure the artifacts for monitoring.
System Alert configuration can be scheduled as follows:
- Daily – Daily schedule will push the notification at the specified time in a day for all the days of the week.
- Weekly – Weekly schedule can push the notification on the specified day and time in the week (for example, Wednesday, 12 PM)
- Monthly – Monthly schedule can push the notification on the specified day and time in the month (for example, the 15th day of the month, 10 AM)
Based on the configuration, the Monitoring Service will alert the Unmapped Application Artifacts in the BizTalk Environment. The Unmapped Application Artifacts list will be available as an attachment in the email notification.
Unmapped BizTalk Application Artifacts
The Unmapped Application Artifacts list feature can be used to manage the artifacts which are not mapped to any of the BizTalk360 Alarms for monitoring.
Scenario
It will be helpful for the administrators who are taking care of the application artifact’s health. When BizTalk Developers or a deployment team deploy new artifacts into the BizTalk Group, the BizTalk administrators might not exactly know the newly deployed artifacts. In such cases, the Unmapped Application Artifacts status will provide a warning indication to the administrators/operators.
The status of Unmapped artifacts is shown in the monitoring dashboard. Unmapped application artifacts status will be healthy when all the application artifacts in an environment are mapped to the alarm. It will show the unhealthy status, when any of the artifacts left unmapped for monitoring. “Unmapped Application Artifacts Link” will be listed with the application artifacts which are still unmapped.
Unmapped Monitor State
From BizTalk360 version 9.0 on, the new Unmapped monitoring status is introduced. This is the default state for application artifacts. Users can set the expected state and start monitoring the artifacts.
NOTE: Users can use the Do Not Monitor state, if they don’t want to monitor an artifact. In such cases, that artifact will not be listed under Unmapped application artifacts.
Automatic Triggered Alerts
When the events or conditions meet the triggering rules, then the alerts will be triggered automatically. System Alerts can be of triggered based
- BizTalk360 License Expiration
- Database Sizes (BizTalk Database & BizTalk360)
In this version of BizTalk360, BizTalk360 License Expiration is implemented. Database Sizes triggered notification will be implemented in a future version of BizTalk360.
License Expiration
When your BizTalk360 license is about to expiry, BizTalk360 System Alerts service will notify the license expiration to the admin users. Notification can be sent on the 30, 15, 7, 2 days before the license expiry date.
System Alert History
Click on the ‘System Alert History’ button to view the historical System Alerts. Alert History is maintained for both alert types(Schedule and Trigger). Users can view the system alerts email notification which has been sent to admin users in HTML Format.
Data Purging
System Alerts historical records are maintained based on the data purging policy of “Alert & Maintenance History”. BizTalk360 will maintain the system alert notification history up to the configured number of days/months.
Based on your requirement you can set the purge duration.
Conclusion
System Alerts notifications will be helpful to BizTalk Administrators to manage the BizTalk Group health and BizTalk360 environments.
Get started with the free 30 days trial. For any queries/feedback please write to us [email protected].
The post System Alerts and Unmapped Application Artifacts in BizTalk360 appeared first on BizTalk360.
by Senthil Kumar Palanisamy | Mar 7, 2019 | BizTalk Community Blogs via Syndication
BizTalk360 Monitoring
BizTalk360 comes with out of box capabilities to monitor BizTalk Artifacts, BizTalk Environment, Queues (IBM, Azure Service Bus and MSMQ), File Location (File, FTP and SFTP) and so on. The Monitoring Dashboard represents the Monitoring Status in a nice graphical chart view. Different set of users like BizTalk Administrators, Operators, Technical members and Business users are using the monitoring features for their day to day activities. Users create multiple alarms to suffice their needs to monitor the health of their BizTalk environment.
While users are monitoring the multiple alarms, they must keep an eye on multiple dashboards. It is a cumbersome task if a user takes care of more than two Alarms. Based on this requirement, few customers have requested for consolidate view of alarms. In this article, we are going through the process of how you can create a consolidated view of multiple alarms using Custom Widgets.
Scenarios
BizTalk360 users are configuring the alarms based on their business vertical or alignment to their process. There have been different patterns in alarm configurations. We can see some of the commonly used patterns;
- Integration-based alarm configuration
Configuring BizTalk Artifacts, Web Endpoints, Queues, File Location etc. of integration in an alarm.
- Role-based alarm configuration
BizTalk Administrators will look after infrastructure settings like Disk, System Resources, Host Instances and configure in an alarm BizTalk Operators can configure BizTalk Artifacts and other entities in a single alarm
- Entity-based alarm configuration
For instance, users can configure all their Queues (IBM/Azure Service Bus) in an alarm. Similarly, they can configure Web Endpoints, Infrastructure (Disks, System Resources) in separate alarms.
Based on the above patterns, Role-based and Entity-based alarms can have a single view of multiple alarms in which you can monitor all the entities. When you follow the Integration-based alarm pattern, then entities like Application Artifacts, Azure Service Queues and BizTalk Server health activities are in different alarms.
Many organizations follow the Integration pattern by grouping all the related artefacts in an alarm. In this scenario, users (BizTalk Operators/BizTalk Technician) must monitor multiple alarms. For Instance, BizTalk Operators looking after BizTalk Artifacts or Developers responsible to monitor Azure Service Bus Queues will have to monitor the multiple alarms. Let’s see how to overcome this challenge by using custom widgets to group multiple alarms.
Consolidated Dashboard
The Monitoring Dashboard is one of the most used features among the users to monitor BizTalk Environments and Artifacts in BizTalk360. An enriched monitoring capability while using Custom widgets to group the multiple alarms to get a single view based on business or team members role.
Custom Widgets are a powerful feature in BizTalk360, as such widgets will provide the opportunity to meet their business scenarios. Many customers adopted Custom Widgets to provide solutions to end users.
Following are a few illustrated scenarios of grouping alarms when it is configured based on integrations
- Azure Service Bus Queues
- BizTalk Application Artifacts
- BizTalk Environment Health
In this article we take one of these scenarios, to show how users can achieve the grouping alarms using Custom Widgets
Azure Service Bus Queues
Custom Widgets used BizTalk360 Secure SQL Query API’s to fetch all Azure Service Bus Queues Monitoring Status from BizTalk360 Database. A user can select the filter with different alarms they want to group.
By default, the user can see only the unhealthy queues in an environment. If they set the Enable Healthy Queue option, they will be able to view all queues in the tree nodes. The Azure Service Bus Queues Custom Widget will fetch the information from Secure SQL Query with a refresh interval which is configured in the script of the custom widget.
Follow the below steps to achieve the Custom Widget’s script;
1. Secure SQL Query
Create the Secure SQL Query to fetch the monitoring status of Azure Service Bus Queues in the environment. To know more about how you can execute Secure SQL Queries using custom widgets, refer to this article.
2. Custom Widget Script
Create the initial variables, Place holders and SQL Query to call the BizTalk360 API method in the Custom Widget’s script
// BEGIN User variables
azureQueueRefresh = 20;
username = "`username_812414`";
password = "`password_701422`";
environmentId = "`productionenvid_189308`";
queryId = "8eea2771-10a7-44c6-8709-a597687434cf";
queryName = "Azure ServiceBus Queues";
sqlInstance = "kovai-bts";
database = "BizTalk360";
sqlQueryForAzureGraph = "Select AA.Id,AA.[Name],AME.MonitorStatus,AME.ExecutionResult from [dbo].[b360_alert_MonitorExecution] AME Inner Join[dbo].[b360_alert_Alarm] AA ON AME.AlarmId = AA.Id and AME.MonitorGroupType = 'Queues' WHERE AME.LastExecutionTime = (SELECT MAX(LastExecutionTime) from [dbo].[b360_alert_MonitorExecution] WHERE MonitorGroupType = 'Queues' and AA.IsAlertDisabled='false' and AlarmId = AA.Id and LastExecutionTime >= DATEADD(MINUTE,-60,GETUTCDATE())) AND AME.EnvironmentId ='`productionenvid_189308`'"; // The SQL Query which needs to be executed from the custom Widget
bt360server = "biztalk360.westus.cloudapp.azure.com"; // Name of the BizTalk360 server (needed to do an API call to execute the SQL query
//Mention the created alarm details and the respective partner name to display in the graph.
AlarmDetails = [
{ AlarmName: "Threshold-1", PartnerName: "Air India" },
{ AlarmName: "Threshold-2", PartnerName: "British Airways" },
{ AlarmName: "Threshold-3", PartnerName: "DHL" },
{ AlarmName: "BizTalk360 Default Alarm1", PartnerName: "Jet Airways" }
];
// END User variables
3. GO JS Framework
BizTalk360 uses the GO JS Framework- Organization chart to represent the monitoring hierarchical view in the UI. It will display the organization chart structure as Azure Service Bus Queue node as a root node, Integration friendly name as second level node and Queue threshold violation details as a child node. You can control the nodes expand/collapse capability of through the custom widget code.
4. Filter Options
Users can able to filter the data based on the selected alarms or the option to show healthy queues. By default, users can see only unhealthy Azure Service Bus Queues. Similarly, other scenarios are implemented through custom widgets. To get the full source code of the Custom widgets you can download it from our GitHub Project.
Summary
We hope this article is useful in grouping the multiple alarms into a single view in an environment. This will bring you more control over the custom development you may want to achieve.
Get started with the free 30 days trial. For any queries/feedback please write to us [email protected].
The post Achieving a Consolidated Monitoring View Using Custom Widgets appeared first on BizTalk360.
by Senthil Kumar Palanisamy | Nov 21, 2018 | BizTalk Community Blogs via Syndication
Introduction
The process of BizTalk Application deployment is a complex, time consuming and repetitive task. Microsoft BizTalk Server comes with capabilities to create deployment packages. However, the MSI packages you create with the out-of-the-box features of BizTalk Server have some shortcomings which lead to deployments which are error-prone.
BizTalk Deployment Framework
The BizTalk Deployment Framework (BTDF) covered these deployment issue.
- Deploy complex BizTalk solutions containing BizTalk artifacts but also for example SSO settings, ESB itineraries and SQL Server scripts, with no human intervention
- Consolidate all environment-specific configuration and runtime settings into one, easy-to-use Excel spreadsheet
Maintain a single binding file that works for all deployment environments
- Make automated deployment a native part of the BizTalk development cycle, then use the same script to deploy to BizTalk servers
To eliminate the manual process, organization’s start to adopt the DevOps practice. In this article, we will see how the BizTalk360 API’s are helpful in a BizTalk environment monitoring while deploying BizTalk Applications using Azure CI/CD Pipelines.
Deploy BizTalk Applications with Azure Pipelines
Microsoft has introduced automated deployment of BizTalk Applications in BizTalk Server 2016 Feature Packs. In BizTalk Server 2016 Feature Pack 1, automatic deployment and application lifecycle management (ALM) experience was introduced. Automatic deployment process has been improved with the release of BizTalk Server 2016 Feature Pack 2.
- Within Visual Studio for each project, set the Application Name of your BizTalk application
- In addition to using an agent-based deployment, you can also use deployment groups to deploy your BizTalk applications to multiple servers
- In the release task, you can install the BizTalk application, and enter the BizTalk management computer, and the path to the deployment package
Azure Pipelines (CICD)
With Automated build/release definitions you can set up the CI or CICD Pipelines based on environment (Development/QA/Staging/Production):
- Continuous Integration (CI) : Set up build task definitions rules to the VSTS Source code base
- Continuous Integration Continuous Deployment (CICD) : Configure both build task definition and release task definition to automate the build and test process
If you are new to Azure CI/CD Pipeline go through this article Configure automatic deployment with Visual Studio Team Services in BizTalk Server
Deployment Build Definitions
In the BizTalk Application deployment process, you can create the following build/release definitions:
- Visual Studio Build
The build and release definitions are Visual Studio Team Services tasks, and should be done by a VSTS admin. The build definition builds your project within your git or VSTS repository, and the release definitions deploys it to your BizTalk Server environment
- Publish Build Artifacts
In Publish Build Artifacts stage configure the Path to publish (Application Project Folder)
- Deploy BizTalk Server Application
A BizTalk Server application deployment build definition operation has three options available
- Create new BizTalk Application: Deploys a new application, if the application already exists, it uninstalls the current applications (full stop), and installs the new application. If continuous integration is enabled, it automatically redeploys the application when it is updated in the repository
- Update an existing BizTalk Application: Appends changes, such as schemas, to an already running application. It does not require a full redeploy of the application
- Install BizTalk Server Application: Install the application, enter the BizTalk management computer name and the deployment package path
BizTalk360 API’s in CI/CD
Customer Scenario: In Build pipeline stages the customer wants to utilize the BizTalk360 API to set the Stop Alert for Maintenance in a BizTalk environment. Once after the new build is deployed to BizTalk Server and then user wants to configure newly deployed artifacts to monitor in BizTalk360. As the last step revoke the Stop Alerts in BizTalk360 Application.
We can use the following steps in Build definitions to achieve the above scenario;
- Visual Studio Build (explained above)
- Set Stop Alert for Maintenance
BizTalk360 customers can take advantage of using the BizTalk360 API’s to set the alert maintenance during the BizTalk Application deployment.
Create the PowerShell Script stage to call the SetAlertMaintenance API method to set the BizTalk environment under the maintenance. Assign the PowerShell Script full path
SetAlertMaintenance.ps1
$Request = '{
"context": {
"callerReference": "SAMPLE_REFERENCE",
"environmentSettings": {
"id": "DC45E848-E485-41D7-92E5-7C9932D43CB8"
}
},
"alertMaintenance": {
"comment": "BizTalk Deploy",
"maintenanceStartTime": "2018-11-22T15:26:37.000",
"maintenanceTimeUnit": 0,
"maintenanceTimeLength": 2,
"isActive": false,
"expiryDateTime": ""
}
}'
$Request
$body = ConvertFrom-Json -InputObject $Request
$body
$RepsonseSet = Invoke-RestMethod -Uri http://biztalk360webserver/Services.REST/AlertService.svc/SetAlertMaintenance -Method Post -ContentType "application/json" -Body $body -UseDefaultCredentials
$RepsonseSet
Set Alert Maintenance API returns the alert maintenance object with maintenance Id (out parameter). You can refer this maintenance Id, when revoking the maintenance in the below step.
- Publish Build Artifacts (explained Above)
- Deploy BizTalk Server Application
Once after BizTalk Application deployment is completed, next step you can configure the newly artifacts to monitor. For creating BizTalk360 alarms and BizTalk Application mappings during deployments with BTDF, you could consider to BT360Deploy command-line tool. Use this GitHub project to setup pipeline stage.
- Remove the Stop alert for Maintenance
After the deployment of BizTalk Applications, the maintenance mode of BizTalk environment must be revoked in BizTalk360. To achieve this step, create a PowerShell Script stage and assign the PowerShell Script path(RemoveMaintenance.ps1)
$RequestRem = '{"context":{"callerReference":"SAMPLE_REFERENCE","environmentSettings":{"id":"DC45E848-E485-41D7-92E5-7C9932D43CB8"}},"maintenanceId":"308d8b2c-b3e0-4024-b600-845113b06b80"}'
Invoke-RestMethod -Uri http://biztalk360webserver/BizTalk360/Services.REST/AlertService.svc/RemoveAlertMaintenance -Method Post -ContentType "application/json" -Body $RequestRem -UseDefaultCredentials
Summary
Automation of Build and/or Deployment will be helpful to reduce the turnaround time. Hopefully, this article has provided insight in how you can setup the BizTalk Application Deployment automation and how you can utilize BizTalk360 API’s to stop alert during deployment and how to set up monitoring of the newly deployed BizTalk Artifacts.
Get started with the free 30 days trial. For any queries/feedback please write to us [email protected].
Author: Senthil Kumar Palanisamy
Senthil Kumar Palanisamy is the Technical Lead at BizTalk360 having 14 years of experience in Microsoft Technologies. Worked various products across domains like Health Care, Energy and Retail. View all posts by Senthil Kumar Palanisamy
by Senthil Kumar Palanisamy | Nov 14, 2018 | BizTalk Community Blogs via Syndication
Introduction
BizTalk360 is a tool for operating, monitoring and getting analytical information for BizTalk environment(s). BizTalk360 comes with productivity tools like the Advanced Event Viewer, the Throttling Analyzer, the Backup/DR visualizer, the BizTalk Health Monitor etc. These tools are also designed to help support people to perform their activities in a seamless way. BizTalk Server, being middle-ware, high volumes of data passes through the Message Box. Based on the business requirements, Tracking is enabled for important BizTalk Integration transactions.
In similar fashion, BizTalk360 collects important information to perform various activities in operations, monitoring and analytics section. In this article, we show BizTalk360’s data collection features and how you can configure the settings to optimize the data collection.
Advanced Event Viewer
The Event Log is an important component which is used by BizTalk Administrators to investigate, if any configuration or transaction errors have happened in BizTalk Server. The Advanced Event Viewer is a powerful feature in BizTalk360 which in fact is a central repository for collecting the event logs from multiple BizTalk servers and SQL servers in BizTalk environment. It has the capability to filter the event logs by various parameters like Event Id, Event Type, Sources etc.
The BizTalk360 Monitoring service collects the event log information based on Event Logs and Event Sources which are configured in the BizTalk360 Advanced Event viewer settings.
BizTalk360 Monitoring IOPS
BizTalk360 uses Windows Management Instrument (WMI) queries to collect the event log details from BizTalk Servers and SQL Servers in an environment. In a multi-server BizTalk environment, few customers raised concern about BizTalk360 IOPS (input/output operations per second). In this set-up, there is change of high volume of IOPS in BizTalk360 Box (Monitoring Service).
BizTalk360 Recommendation
To avoid large volume of data collection or to improve the performance of the BizTalk360 monitoring services, following event log configuration are suggested
1. Enable the necessary servers to collect Event Logs and Event Sources
• Primary BizTalk Servers in Active – Passive scenario
• Multiple SQL server, enable the event log data collection for important nodes.
2. BizTalk360 provides a default set of Event Logs and Event Sources which are widely used in BizTalk and SQL servers. Configure the necessary Event Sources that are required as per the business requirement:
• BizTalk Event Log Sources(Default):
Bam Event Provider, BamManagementUtility,BAMWebServices,BizTalk DW Reporting, BizTalk Server, BizTalk Server Deployment, BizTalk Server EDI, BizTalk360 Monitor, BusinessRulesEngine, ENTSSO,MSDTC,MSDTC Client, RuleEngineUpdateService, System.ServiceModel 4.0.0.0, System.ServiceModel.Install 3.0.0.0,W3SVC-WP,XLANG/s
Configure required BizTalk Sources as
“BizTalk Server,BizTalk Server Deployment,BizTalk Server EDI,ENTSSO,MSDTC,MSDTC Client,RuleEngineUpdateService”
• SQL Sources(Default):
MSSQLSERVER, MSSQLSERVER$AUDIT, MSSQLServerOLAPService, SQLSERVERAGENT
Configure required SQL Sources as
“MSSQLSERVER, SQLSERVERAGENT”
Note: This configuration will reduce the BizTalk360 Input/Output operations. You can read this article to know more details about Event-Log Data collection configuration.
Analytics Performance Data
In a data-driven business setup, there will be scenarios where critical executive decisions are made based on the performance reports of various components of the listed servers in a BizTalk environment. BizTalk360 aims to offer an out of the box tool with similar capabilities as the Performance Monitor tool in Windows servers.
BizTalk360 Analytics offers visual display of the most important performance counters that are consolidated and arranged on a single screen so that the information can be monitored in a glance. Custom reports can be built in minutes with the metrics that really count for your business, and they are easy to share with other users. In addition, dashboards can give you a summary of many reports on a single page using drag-and-drop widgets for fast and easy customization.
Performance metrics data collection can be optimized by selecting the necessary metrics (Windows, SQL, BizTalk and IIS) based on the server type.
• BizTalk Server: Enable Windows, BizTalk by default. Web Application is configured in the server enable IIS and similarly to SQL.
• SQL Server: Enable Windows and SQL performance metrics
Tracking Data
Initial design of the tracking data widgets was simple; it ran the necessary queries directly against the Tracking database and displayed the results in a graphical form on the widget. A larger database degraded the performance of these queries and it had a direct impact on the user experience of the Analytics dashboard, causing slower widget loading and occasional time outs. Since Analytics widgets offered users the option to view data up to 30 days within the widget, an efficient approach was required to improve the performance and user experience.
In BizTalk360 v8.3 to overcome these challenges, we decided to create a service that runs in the background. This service collects the tracking data periodically in small chunks and thereby avoid running expensive queries on the Tracking database. Here, only the resulting metrics are collected rather than pulling all the tracking data to BizTalk360 database. This improves the overall performance of the Tracking widgets as they are not querying the Tracking database directly.
BizTalk360 Analytics needs the tracking data to represent data in custom widgets
• Failure Rate by Port and Schema
• Message Performance by Host, Orchestration, Pipeline, Adapter, Port and Schema
BizTalk360 suggests enabling the data collection for categories and counters needed in custom widgets. If you want custom widgets for Failure Rate and Message Performance of Port and schema, enable only those counters in Tracking data collection.
Data Purging
Collecting the necessary data is important to be able to represent the data in BizTalk360. However, you don’t want unlimited growth of your BizaTalk360 comes out of box with the “Data Purging” feature to able to manage the size of BizTalk360 database. If you want to know more about Data Purging please read this article.
Constantly manual monitoring the database size is a tedious process. You can utilize the Database Query monitoring when the database size is grown over the threshold limit.
SELECT total_size_mb = CAST(SUM(size) * 8. / 1024 AS INT)
FROM sys.master_files WITH(NOWAIT)
WHERE database_id = DB_ID()
Conclusion
We hope this article is useful to optimize and manage the data collection for various features in BizTalk360.
Get started with the free 30 days trial. For any queries/feedback please write to us [email protected].
Author: Senthil Kumar Palanisamy
Senthil Kumar Palanisamy is the Technical Lead at BizTalk360 having 14 years of experience in Microsoft Technologies. Worked various products across domains like Health Care, Energy and Retail. View all posts by Senthil Kumar Palanisamy
by Lex Hegt | May 9, 2018 | BizTalk Community Blogs via Syndication
Why do we need this feature?
The BizTalk Health Monitor (BHM) and its predecessor MessageBox Viewer (MBV) are important tools to learn about the overall health of BizTalk environments.
The tools are developed and maintained by Microsoft and firstly used by their Premier Field Engineers, to help them identify problems with BizTalk environments. Later, the tool became publicly available, enabling BizTalk administrators to run the tool themselves as well. Since that time, we have seen many updates of the tools, although MBV is deprecated for few years now.
The BizTalk Health Monitor can be downloaded here:
https://www.microsoft.com/en-us/download/details.aspx?id=43716
The best practice is to run BHM frequently, say once per day at a quiet time, to be aware of overall health and check for any critical issues.
MBV did not offer a built-in scheduler, so you had to schedule it yourself with Windows Scheduler. With the introduction of BHM, the tool came with a plugin, to use it from the BizTalk Server Administration console and it also came with scheduling features.
Once scheduled, you’ll receive the output in the configured mailbox. So you have the fully detailed report easily at hand.
What are the current challenges?
Although it’s good to have the output of MBV/BHM runs in your mailbox, we think that the overall experience still can be improved.
In your day to day activities as a BizTalk administrator, you might receive many notifications in your mailbox. If not setup properly, the amount of notifications might be overwhelming, which might result in starting to ignore the notifications, thereby risking to miss important notifications.
So, you will want to focus on just the notifications which indicate something needs to be taken care of. In case of the MBV/BHM output, you will only be interested in critical and non-critical errors on which some actions must be taken. In other words, if all is healthy, you might not be interested to receive that output at all.
With BHM you will always receive the output, regardless if all is healthy or some (non-)critical issues exist. No clear notification can be received, in case something serious is going on.
Besides that, MBV/BHM is another tool/console you need to be aware of to consider the overall health of a BizTalk environment. As we want to apply our One Platform philosophy as much as possible, we brought integration with MBV/BHM to BizTalk360.
In the next section of this article, we’ll show you how BizTalk360 helps you to improve the experience of BHM and how you can have monitoring on the output of MBV/BHM runs.
How BizTalk360 solves this problem?
BizTalk360 has integration with MBV/BHM for many years. This integration enables the BizTalk360 user to schedule MBV/BHM and view the output of the different runs of the tool directly from within BizTalk360.
This makes it easy to run MBV/BHM and view its output in BizTalk360, instead of delivering the output of each run to the mailbox of the user.
However, this does not yet solve the problem of just being notified in case of critical errors. To achieve that, you can simply add the BizTalk Health Monitor to a BizTalk360 alarm. In BizTalk360, you can do this by navigating to Monitoring => Manage mapping => BizTalk Environment => BizTalk Health Monitor. Here you can configure how many critical or non-critical may occur in the output of BHM, before a notification is being sent.
As you can see from above screenshot, under Error Threshold, you can configure how many critical/non-critical errors may exist, before a notification is being sent. From the Current Configuration of the screen, you can see that in the last report there were 2 critical and 20 non-critical errors. So, in above scenario, the error thresholds have been met and a notification will be send.
If you want more fine-grained monitoring of the BHM output, BHM offers you the option to store the output of the BHM runs in a database. Say, you want to monitor on specific categories like Tracking or the BizTalk jobs, you could achieve that with Database query monitoring.
When you would like to know more on how to set this up, feel free to contact us ([email protected]) and we are happy to work with you on the scenario at hand.
Conclusion
The integration of MBV/BHM with BizTalk360 adds to the One platform philosophy. By also bringing monitoring of the output of MBV/BHM, we remove some of the clutter which appears in the mailbox of the user, providing good focus on the topics which really matter. With some customization, you could have even more fine-grained monitoring on the output of BHM.
There are some more articles on MBV/BHM integration. Check them out here:
Get started with a Free Trial today!
If you would like to try BizTalk360, why not give it a try. It takes about 10 minutes to install on your BizTalk environments and you can witness and check the security and productivity on your own BizTalk Environments. Get started with the free 30 days trial.
Author: Lex Hegt
Lex Hegt works in the IT sector for more than 25 years, mainly in roles as developer and administrator. He works with BizTalk since BizTalk Server 2004. Currently he is a Technical Lead at BizTalk360. View all posts by Lex Hegt
by Lex Hegt | Apr 18, 2018 | BizTalk Community Blogs via Syndication
This blog is a part of the series of blog articles we are publishing on the topic “Why we built XYZ feature in BizTalk360“.
Why do we need this feature?
Microsoft BizTalk Server is a powerful middleware product which allows integrating all kind of information systems, independent of whether these systems are on-premise or in the cloud. BizTalk Server uses Receive and Send ports to communicate with a broad set of on-premise/cloud systems, both on the technology level (FILE, FTP, POP3, SMTP, etc.) as on the business level (ERP, CRM, etc.).
Besides just receiving and transmitting messages, BizTalk solutions might also contain Orchestrations, which can process all kind of business logic, for example, Order processing, Credit card validation, Claims processing etc.
As BizTalk Server is very scalable, a BizTalk environment can contain many deployments for all kind of different integrations. All together a BizTalk environment may contain hundreds of ports and orchestrations. The well-being of these artifacts is of vital importance to have the integrations working properly. When not running properly, these interruptions may cause serious damage to the business processes they support.
What are the current challenges?
To be well aware of the state of the BizTalk artifacts, the BizTalk administrator needs to consult the BizTalk Administration console multiple times per day. However, there are few challenges when depending on the BizTalk Administration console for checking the state of these artifacts.
Access to the Administration console – Often, the BizTalk Administration console is only installed on the BizTalk servers. Besides that there is the risk of unintended damaging the BizTalk server while accessing the server, you need to be authorized to access the servers. Not to be forgotten, is the fact that it constantly takes time to setup Remote Desktop connections to access the BizTalk server and the Administration console. Time which can easily be used for something more tangible.
Not suited for less experienced administrators – The Administration console is such a powerful tool, which is great when you are an experienced BizTalk Administrator. However, in the hands of less experienced administrators, unintentionally a lot of damage can be done. Think of for example stopping ports or terminating processes!
Not an efficient way of monitoring – Checking the state of the ports and orchestrations manually, multiple times per day, is just not efficient. It can easily be forgotten in the rush of the day, thereby putting the wellbeing of your integrations at stake.
Too many artifacts to monitor efficiently – BizTalk environments can contain hundreds of ports and orchestrations. In most scenarios, the artifacts should be in the Enabled/Started state. But there can be several reasons why certain artifacts are not in the expected state. Just to mention a few:
- After the deployment of an update of a BizTalk application the artifacts are in a wrong state
- Due to a temporary maintenance window, some of the artifacts need not be in the Enabled/Started state, to prevent suspended instances
- Artifacts might be Disabled as a result of a temporary outage
- Certain artifacts need only be Enabled/Started as a fall back scenario
By the given examples, it must be clear, that it can be very challenging to constantly be aware of the desired state of all the artifacts. This makes it hard to manually monitor the state of the artifacts in an efficient way.
Instead of having to monitor BizTalk manually, it is much more efficient to have automated, state-bound, artifact monitoring. Automated BizTalk monitoring will prevent you of constantly having to check for the state manually, enabling you to spend your time on more important activities. In the next section, we’ll show which features BizTalk360 has, to allow you to automatically monitor your BizTalk artifacts.
How BizTalk360 solves this problem?
To give you that comforting feeling that you are in control over the state of your ports and orchestrations, BizTalk360 provides multiple features.
Threshold Monitoring – While using the concept of Threshold Monitoring, BizTalk360 allows you to setup state-bound monitoring. This means that you can easily setup monitoring to have the product check for the Expected state of your ports and orchestrations.
In above screenshot, you can see how some Send Ports are being monitored against their Expected State. Once, the Current State deviates from the Expected State, a threshold violation has occurred and BizTalk360 will notify you of that fact.
Positive and Negative monitoring – In most scenarios, the Expected State of your Ports and Orchestrations will be Enabled/Started; which is called Positive Monitoring. However, there might be valid reasons that certain ports/orchestrations need NOT be Enabled/Started. Setting up that kind of monitoring is called Negative Monitoring. BizTalk360 allows to setup for both Positive and negative Monitoring.
Receiving Notifications via multiple channels – Once a threshold violation has occurred, by default, you will receive notifications by email. Moving forward, BizTalk360 also allows you to receive notifications via multiple other channels.
We are constantly considering new ways to send notifications, but currently, the following Notification Channels are provided out-of-the-box: Email, SMS, Event Log, Slack, Microsoft Teams, ServiceNow, HP Operations Manager, Webhook. In addition, you can build your own custom notification channel like writing to a database, calling an internal system etc.
Auto Correct – BizTalk360 not just monitors your artifacts, it can also try to bring the artifacts back to their Expected state, once there is a mismatch between the Current State and the Expected State. Think of for example that FTP Receive Location which goes down at night or during the weekend.
The Auto-Correct feature works for the following artifacts:
- Receive Locations
- Orchestrations
- Send Ports
- BizTalk Host Instances
- Windows NT Services
- SQL Server Agent Jobs
- Azure Logic Apps
There is an extensive article on BizTalk360 Auto correct, which also explains in which kind of scenarios this feature comes at hand. You can that article here:
Monitoring Dashboard – Although receiving notifications of any mismatches between the Current State and the Expected State is very handy, it will equally be handy to have a large screen, at for example the Support Desk, which simply shows the current health of your artifacts. For this, we bring the Monitoring Dashboard, which does exactly that.
As you can see from below screenshot, it brings you a nice Expandable/Collapsible and filterable treeview, with automatic refresh, thereby constantly showing the current state of the artifacts.
Health Check Reports – Last but not least, are the Health Check Reports you can receive at your convenient timing. These reports function as your daily health check, providing you with an overview of the state of all the artifacts from your alarm. Very handy to receive for example at the beginning of your working day!
Conclusion
In this article, we have seen why being aware of the state of your ports and orchestrations is important. We have also seen why manually monitoring that state is very inefficient and sometimes very complex. As a company which knows these challenges from our own experiences, we have brought multiple features which will help the BizTalk administrator being in control, without having to manually check the state.
Get started with a Free Trial today!
Download and try BizTalk360 on your own environments free for 30 days. The installation will not take more than 5-10 minutes.
Author: Lex Hegt
Lex Hegt works in the IT sector for more than 25 years, mainly in roles as developer and administrator. He works with BizTalk since BizTalk Server 2004. Currently he is a Technical Lead at BizTalk360. View all posts by Lex Hegt
by Lex Hegt | Apr 11, 2018 | BizTalk Community Blogs via Syndication
This blog is a part of the series of blog articles we are publishing on the topic “Why we built XYZ feature in BizTalk360”.
Why do we need this feature?
In the day to day activities of a BizTalk administrator, you might come across integrations where FTP sites are used for receiving and transmitting messages. FTP sites are often used for cross-platform integrations. For example, when you have an SAP system on Unix that has to be integrated, via BizTalk Server, with other systems, you might use FTP for receiving and transmitting of messages.
SFTP & FTPS are just the secured version of FTP with advanced transport encryption mechanisms, so your end-to-end data transmission is secure and safe.
To keep the business process going, it can be of vital importance that the FTP/SFTP sites are online and the messages are being picked up. So, when a BizTalk administrator needs to be constantly aware whether the FTP/SFTP sites are online and working properly, the administrator needs to monitor the sites and the activities which take place on these sites.
What are the current challenges?
BizTalk Server offers no monitoring capabilities, not for Receive Locations / Send Ports and also not for endpoints like FTP, SFTP and FTPS sites. So, using just the out-of-the-box features of BizTalk Server, a BizTalk administrator will have to manually check whether the FTP sites are online and whether all (appropriate) files are being picked up for further processing.
Manual monitoring
This kind of manual monitoring can be quite cumbersome and time-consuming. The administrator will probably use multiple pieces of software to be able to perform these tasks. Think of for example the BizTalk Administration console to check whether the Receive Locations/Send Ports are up and some FTP client to check whether files are being picked up.
It is obvious that this is not a very efficient scenario, which could easily be automated by setting up monitoring.
Maintaining scripts for monitoring FTP sites
To reduce their workload, we experience that BizTalk administrators are creating their own scripts to monitor FTP sites and all kind of other resources. Although this kind of scripts certainly can be of help, we still think this does not fully solve the problem.
For example, often these kinds of scripts need maintenance when FTP sites need to be added, changed or deleted from monitoring. This kind of tasks can be easily forgotten.
Also from a knowledge transfer perspective, it’s easy to forget to update new colleagues about the existence of this kind of scripts, as they will probably be installed on some (monitoring) server.
Another challenge with solving this kind of problems with scripts is that not each administrator is capable to write this kind of scripts, which makes knowledge transfer even harder.
To keep the overview, we think that it is easier to use software, like BizTalk360, to have everything in one easily accessible place, with good visibility of all the features/capabilities, fine-grained security/auditing and without the need to maintain custom scripts etc..
How BizTalk360 solves this problem?
With BizTalk360, we make monitoring of FTP/SFTP/FTPS sites a lot easier. For a very long time, the product offers monitoring of Receive Locations and Send Ports, but for some time now, BizTalk360 also offers to monitor of the physical FTP/SFTP/FTPS endpoints.
We wanted to make setting up this kind of endpoint monitoring as seamless as possible and therefore we simply show all the ports in the current BizTalk group which make use of the FTP/SFTP/FTPS adapter.
In BizTalk360, you can find FTP monitoring under Monitoring => Manage Mapping => File Locations (File, FTP, SFTP).
Next, you can set up monitoring rules based on File Count and Directory Size and have BizTalk360 send Warning or Error notifications through the notification channels which are configured on the associated alarm.
A fully monitored FTP endpoint might look like shown below.
Of course, besides the greater-than-or-equals operator, also other common operators are available.
Conclusion
As a final point, we see from time to time that administration teams maintain a administrators handbook, which contains all the tasks a (BizTalk) administrator should take care of. We think that by using software like BizTalk360, we can reduce the number of pages in such handbooks, as the kind of scripts we mentioned no more have to be described in that kind of books.
This description could be replaced by, for example, a general guideline on how FTP sites should become monitored and the monitoring rules with BizTalk360.
As a result, we hope to make the work of BizTalk administrators a bit easier so the team can focus on the more exciting parts of the job of BizTalk administrators, instead of constantly having to update their handbooks.
So, we think that we make the day to day life of a BizTalk administrator, who needs to monitor the well-being of FTP sites, a little bit easier by bringing this feature.
If you want to read more on FTP/SFTP monitoring in much more detail, you can check the following article:
FTP, FTPS, SFTP Location monitoring
BizTalk360 also offers an advanced monitoring capability called “Data Monitoring” which allows monitoring the traffic/volume of messages going through the ports for a given period, ex: expected 50 PO orders from our partner via FTP/SFTP. Please check out this article. Introducing BizTalk Server Data Monitoring in BizTalk360
Get started with a Free Trial today!
Download and try BizTalk360 on your own environments free for 30 days. Installation will not take more than 5-10 minutes.
Author: Lex Hegt
Lex Hegt works in the IT sector for more than 25 years, mainly in roles as developer and administrator. He works with BizTalk since BizTalk Server 2004. Currently he is a Technical Lead at BizTalk360. View all posts by Lex Hegt
