Event Log Data Monitoring in BizTalk360

Event Log Data Monitoring in BizTalk360

Event logs are normally used to inform about an important event in the running applications and subsystems, which plays a vital role in troubleshooting problems.

While monitoring multi-server environments, how many times in a day does your administration team log in to multiple servers to check for the root cause of a problem? Have you ever thought of a tool that could help you avoid this time-consuming process? Yes, BizTalk360’s in-built Advanced Event Viewer (AEV) helps you solve this business problem.

Set-up AEV to retrieve the event data you want from your BizTalk and SQL servers in your environment and display it all in a single screen, where you can use the rich query capabilities to search and analyze the data.

How to Set Up AEV in BizTalk360

As a first step, in BizTalk360 settings, you need to configure event logs and event sources that you want to monitor and then Enable AEV for the environment. Now the BizTalk360 Monitoring service will collect event log data for all the configured servers in that environment and store it in BizTalk360 DB.

What ‘s new in v8.6?

BizTalk360 already supports AEV in operations and monitoring section for a long time. While demonstrating BizTalk360 to customers, we asked for “How to monitor a specific event occurring in BizTalk environments on a specific frequency and get an alert based on Threshold conditions”. So, keeping that in mind we have implemented Event Log Data Monitoring in version v8.6.

Let us take this complex scenario to understand more about Event Log Data Monitoring.

Scenario1: User wants to monitor different event logs for multiple servers. Example: If an administrator wants to monitor ESB events from BizTalk server and also wants to ensure there is no problem in SQL servers and also to monitor ENTSSO events form SSO server.

Start Monitoring Event log Data in 3 Steps:

  • Enable AEV for an environment
  • Create a Data Monitoring Alarm
  • Create a schedule under event log and configure the rich filtering conditions based on your business needs as below.
Server Type : BizTalk, SQL
Server Names :BizTalk Server ,SQL Server,SSO Server
Event Type: Error
Event Sources: ESB Itinerary Selector, ENTSSO , MSSQLSERVER,

And group (All these below conditions are true)

Event ID Greater than or equal to 3010
Event ID Less than or equal to 3034
Message Contains 'ESB.ItineraryServices.Generic.WCF/ProcessItinerary.svc'

EventID IS Between 10500-10550
Message Contains ‘SSO Database’

When we looked in more detail, it would normally take us into running a filtering query against configured event sources in servers and alert them when certain conditions are met.

Scenario 2: To detect the same event occurring on different servers. For example, when a certain instance of an orchestration is firstly executed on server 1 and throwing a certain error and next to another instance of the same orchestration throws the same error, while the instance becomes executed on server 2, this will now easily be detected with event log data monitoring.

BizTalk360 brings all these data into a single console and on top of that provides a powerful capability to set alerts based on various thresholds.

You can also set how frequently you wanted to run the queries based on their business requirements such as the frequency of daily validations (ex every 15 mins, 1 hour etc), end of business day or even monthly events such as month-end processing. With these thresholds, the result from the query will be evaluated and in case of any threshold violation, you will be notified via notification channels/Email.

Event Log Details in Alerts

Event Log Details will be listed in alerts by enabling the option ‘Send Event Log details in Mail’ while creating the schedule.

Event Log data in the Data Monitoring Dashboard

Also, the information will be visible on the Data Monitoring dashboard, you can visualize the day calendar view. If you need to understand what happened for an execution, you can click on one of the entries in the day view of the dashboard and view the details as shown below.

Note :

  • Maintenance is very simple, once after scheduling event log data monitoring, when you disable AEV for the environment, it will stop collecting Event Log data.
  • And you don’t Worry about data growth, BizTalk360 purge policy will take care of it.
  • Apart from monitoring BizTalk specific SQL server, you can also monitor other SQL servers simply by adding SQL server names for monitoring in the settings section.
Enhancements on Monitoring Alerts and Notifications in BizTalk360 v8.5

Enhancements on Monitoring Alerts and Notifications in BizTalk360 v8.5

Monitoring and Alert notification is one of the core functionality in BizTalk360. To empower better usability of a user, we have improved the alert notification in the latest release v8.5. The enhancements are typically chosen from our customer feedback based on the impact and the number of requests.

Event Log description:

Event logs can give the early warning of impending system issues and save you from unexpected disasters. It is quite important to monitor the event logs regularly to get proper insights of any windows servers.

BizTalk360 has ‘Advanced Event Viewer’ functionality in which BizTalk360 Monitoring services collect the data from event logs from defined event sources of multiple servers. And the same can be monitored with respect to event log type (Application, Security, System, Windows PowerShell etc…), event sources of each event types and also with the specific event id and texts by configuring threshold conditions.

The best way to monitor event logs is to choose a solution that reports the Windows Event ID as well as the error message. Considering this ideology, we enhanced this in 8.5 release, so that event log will be alerted along with the description. This feature is also enabled with the filter on the number of event logs and size of the description (up to 5000 characters) in the system settings. Now user can get clear insights on any events occurred in the configured BizTalk or SQL server.

Event Log Alert with Description

Event log monitoring can be done in few easy steps

  • Enable Event Log Monitoring in Setting->Advance Event Viewer section
  • Define Email Event Log Description character count and List Count in system settings
  • Configure Monitoring Threshold conditions for event log monitoring.

Migration Scenario: To use this, the user needs to define the list count and description character count in system setting. By default, the values will be zero after migration.

Data Monitoring alerts in Notification channel

Data Monitoring is one of the powerful features in BizTalk360 which run queries against multiple data sources in BizTalk which include: Message Box Queries, Tracking Queries, BAM Queries, EDI Queries and ESB Queries, Logic Apps based on the configuration how the user wants to monitor data in each of these sources.

Previously Data Monitoring alert can be sent only through static notification end points like SMS, Email, Windows Event log and HP Operation Manager. From v8.5 we have extended custom notification support for data monitoring alert. Using this functionality, the users can get Data Monitoring alert from various external systems like Slack, ServiceNow, Web Hook and they can also build their own custom notification channels to any external systems.

To use this functionality user need to configure notification channel in setting side and enable the same in Data Monitoring alarm.

Migration Scenario: User need to enable Configured Notification Channel in Existing Data Monitoring Alarm to get Data Monitoring alert on custom notification channels.

Web Endpoint Monitoring Enhancement

We have web endpoint monitoring (ability to monitor any HTTP based URL’s) for very long time in BizTalk360, right from 6.x. In fact, this is one of the key monitoring component used by a lot of our customers. Web endpoint monitoring allows you to monitor the health of any HTTP based web service. You can monitor:

  • Response status codes
  • Response Keyword, XML, JSON
  • Response Time

You can configure BizTalk360 to trigger an alert notification if the response value is not matching your requirement. We have done following improvements on web endpoint monitoring alerts.

  • Many customers suggested to include the endpoint name in an alert email, alert history and also in monitoring graph to easily identify which endpoint has gone down in case of multiple web endpoint configured for monitoring.
  • Also Expected Return code and Response Time Alert details are included in alert for more insights.

Web EndPoint Monitoring

Folder Size Monitoring

In BizTalk360 version 8.4, we introduced the capability of folder monitoring which monitors the file count for File Locations (File, FTP and SFTP) configured in BizTalk artifacts. From 8.5, we are supporting directory size monitoring for file locations which actually monitors the size of entire folder irrespective of file mask. (i.e) the folder may contain various file types XML, txt, png etc but the configured file may be either one of this type so here, folder size means entire folder with all the file types.

With in 2-3 clicks, the user can configure monitoring threshold conditions for folder size and start monitoring and get notified as and when folder size increases/decreases.

Folder Size Monitoring

Conclusion

BizTalk360 enhances the complex administrative tasks into a modern easier task. This version of 8.5 is being added with more features thinking from a usability perspective of our customers. BizTalk360 continues to provide more such features considering the feedback from our customers and stay tuned for next version.

Introducing Logic Apps Operations capability in BizTalk360

Introducing Logic Apps Operations capability in BizTalk360

Azure Logic Apps play a vital role in an integration platform. Keeping this in mind, we introduced ‘Azure Logic Apps Monitoring’ in BizTalk360 Version 8.1. In Version 8.3, we implemented ‘Azure Logic Apps Operations’ capability to improve the ease of use for users using Logic Apps and BizTalk360. With this functionality, users can Operate, Manage and Monitor Azure Logic App(s) from a single place. When you see a threshold violation of Logic Apps, you can rectify the problem from the BizTalk360 UI itself. You can save time to log in to the Azure Portal and manage your Logic Apps.

All the user needs to do to configure Logic Apps Operations capability is to add the Azure subscription in BizTalk360 UI and enable it for monitoring and operation. BizTalk360 also provides an option to work on multiple subscriptions simultaneously. Therefore, by adding the subscription in BizTalk360, users can view the list of available Logic Apps in that subscription along with its name, Access End Point, the current Status (Enabled or Disabled), Last Run, and other details such as Resource Group, Location, Run and Trigger history details and Success/Failure Run count.

Features of Logic Apps Operations

Enable/Disable Logic Apps: From BizTalk360 UI, users can enable/disable the Logic Apps that reflects the corresponding Logic App in the Azure Subscription. You can initiate bulk operations — enable/disable multiple Logic Apps in a single click.

Run Trigger: User can also trigger the Logic App action from the BizTalk360 UI. This action also supports the bulk operations.

Delete Logic Apps: User can delete single or multiple Logic Apps through a single click from BizTalk360 UI.

Note: The Azure Portal UI allows to operate (Enable/Disable, Delete, Run Trigger) on only one Logic App at a time, whereas from BizTalk360 UI user can initiate bulk operation (select multiple logic apps) in a single click.

BizTalk360 Logic Apps Operations

Run and Trigger History details

Data like History and Runs are huge when it comes to real time scenarios/production use. For each Logic App, Run and Trigger history will maintain the last 30 records. The history details will be displayed in both list and graphical View.

List View

  • Start and End time of Run/Trigger. Here the time format and time zone is based on user profile configuration.
  • Run/Trigger status such as succeeded/failed, running/skipped/aborted
  • Duration of each Run and Trigger

Logic App Details - BizTalk360 Logic Apps Operations

Graphical View

We have simplified the UI view of your Logic Apps details and redefined Runs and Trigger history into Graphical representations. With the graphical view, it becomes easy for the users to navigate and identify the date and time tracking.

The graphical view chart shows Logic Apps Runs in the “Y” axis and Date in the “X” axis. All basic graph operations such as zoom, hover are available in the graphical view. Additionally, you can print/download the chart.

Logic Apps Graphical View - BizTalk360 Logic Apps Operations

Success and Failure Run count

Based on colour coding, you can know the Success and Failure Run counts instantly within the detail window of your Logic App.

Trigger and Action Status

Users can see the actual design of the Logic App in the Details View, say when it should be trigger and which actions are to be performed.

E.g : Trigger  ‘When_a File_Is_created ’ ; Action – Email_File   .

Licensing

If you are an existing user of BizTalk360, and using the Platinum tier license, you just need to upgrade to BizTalk360 Version 8.3 to use this feature. For customers using other licensing tier, if you would like to use/try this feature, please send an email to support@biztalk360.com to customize your license.

If you are new to BizTalk360 and like to explore the Logic Apps Operations capability, you can get started with the 14 day free trial of BizTalk360.