by Senthil Kumar Palanisamy | Nov 14, 2018 | BizTalk Community Blogs via Syndication
Introduction
BizTalk360 is a tool for operating, monitoring and getting analytical information for BizTalk environment(s). BizTalk360 comes with productivity tools like the Advanced Event Viewer, the Throttling Analyzer, the Backup/DR visualizer, the BizTalk Health Monitor etc. These tools are also designed to help support people to perform their activities in a seamless way. BizTalk Server, being middle-ware, high volumes of data passes through the Message Box. Based on the business requirements, Tracking is enabled for important BizTalk Integration transactions.
In similar fashion, BizTalk360 collects important information to perform various activities in operations, monitoring and analytics section. In this article, we show BizTalk360’s data collection features and how you can configure the settings to optimize the data collection.
Advanced Event Viewer
The Event Log is an important component which is used by BizTalk Administrators to investigate, if any configuration or transaction errors have happened in BizTalk Server. The Advanced Event Viewer is a powerful feature in BizTalk360 which in fact is a central repository for collecting the event logs from multiple BizTalk servers and SQL servers in BizTalk environment. It has the capability to filter the event logs by various parameters like Event Id, Event Type, Sources etc.
The BizTalk360 Monitoring service collects the event log information based on Event Logs and Event Sources which are configured in the BizTalk360 Advanced Event viewer settings.
BizTalk360 Monitoring IOPS
BizTalk360 uses Windows Management Instrument (WMI) queries to collect the event log details from BizTalk Servers and SQL Servers in an environment. In a multi-server BizTalk environment, few customers raised concern about BizTalk360 IOPS (input/output operations per second). In this set-up, there is change of high volume of IOPS in BizTalk360 Box (Monitoring Service).
BizTalk360 Recommendation
To avoid large volume of data collection or to improve the performance of the BizTalk360 monitoring services, following event log configuration are suggested
1. Enable the necessary servers to collect Event Logs and Event Sources
• Primary BizTalk Servers in Active – Passive scenario
• Multiple SQL server, enable the event log data collection for important nodes.
2. BizTalk360 provides a default set of Event Logs and Event Sources which are widely used in BizTalk and SQL servers. Configure the necessary Event Sources that are required as per the business requirement:
• BizTalk Event Log Sources(Default):
Bam Event Provider, BamManagementUtility,BAMWebServices,BizTalk DW Reporting, BizTalk Server, BizTalk Server Deployment, BizTalk Server EDI, BizTalk360 Monitor, BusinessRulesEngine, ENTSSO,MSDTC,MSDTC Client, RuleEngineUpdateService, System.ServiceModel 4.0.0.0, System.ServiceModel.Install 3.0.0.0,W3SVC-WP,XLANG/s
Configure required BizTalk Sources as
“BizTalk Server,BizTalk Server Deployment,BizTalk Server EDI,ENTSSO,MSDTC,MSDTC Client,RuleEngineUpdateService”
• SQL Sources(Default):
MSSQLSERVER, MSSQLSERVER$AUDIT, MSSQLServerOLAPService, SQLSERVERAGENT
Configure required SQL Sources as
“MSSQLSERVER, SQLSERVERAGENT”
Note: This configuration will reduce the BizTalk360 Input/Output operations. You can read this article to know more details about Event-Log Data collection configuration.
Analytics Performance Data
In a data-driven business setup, there will be scenarios where critical executive decisions are made based on the performance reports of various components of the listed servers in a BizTalk environment. BizTalk360 aims to offer an out of the box tool with similar capabilities as the Performance Monitor tool in Windows servers.
BizTalk360 Analytics offers visual display of the most important performance counters that are consolidated and arranged on a single screen so that the information can be monitored in a glance. Custom reports can be built in minutes with the metrics that really count for your business, and they are easy to share with other users. In addition, dashboards can give you a summary of many reports on a single page using drag-and-drop widgets for fast and easy customization.
Performance metrics data collection can be optimized by selecting the necessary metrics (Windows, SQL, BizTalk and IIS) based on the server type.
• BizTalk Server: Enable Windows, BizTalk by default. Web Application is configured in the server enable IIS and similarly to SQL.
• SQL Server: Enable Windows and SQL performance metrics
Tracking Data
Initial design of the tracking data widgets was simple; it ran the necessary queries directly against the Tracking database and displayed the results in a graphical form on the widget. A larger database degraded the performance of these queries and it had a direct impact on the user experience of the Analytics dashboard, causing slower widget loading and occasional time outs. Since Analytics widgets offered users the option to view data up to 30 days within the widget, an efficient approach was required to improve the performance and user experience.
In BizTalk360 v8.3 to overcome these challenges, we decided to create a service that runs in the background. This service collects the tracking data periodically in small chunks and thereby avoid running expensive queries on the Tracking database. Here, only the resulting metrics are collected rather than pulling all the tracking data to BizTalk360 database. This improves the overall performance of the Tracking widgets as they are not querying the Tracking database directly.
BizTalk360 Analytics needs the tracking data to represent data in custom widgets
• Failure Rate by Port and Schema
• Message Performance by Host, Orchestration, Pipeline, Adapter, Port and Schema
BizTalk360 suggests enabling the data collection for categories and counters needed in custom widgets. If you want custom widgets for Failure Rate and Message Performance of Port and schema, enable only those counters in Tracking data collection.
Data Purging
Collecting the necessary data is important to be able to represent the data in BizTalk360. However, you don’t want unlimited growth of your BizaTalk360 comes out of box with the “Data Purging” feature to able to manage the size of BizTalk360 database. If you want to know more about Data Purging please read this article.
Constantly manual monitoring the database size is a tedious process. You can utilize the Database Query monitoring when the database size is grown over the threshold limit.
SELECT total_size_mb = CAST(SUM(size) * 8. / 1024 AS INT)
FROM sys.master_files WITH(NOWAIT)
WHERE database_id = DB_ID()
Conclusion
We hope this article is useful to optimize and manage the data collection for various features in BizTalk360.
Get started with the free 30 days trial. For any queries/feedback please write to us support@biztalk360.com.
Author: Senthil Kumar Palanisamy
Senthil Kumar Palanisamy is the Technical Lead at BizTalk360 having 14 years of experience in Microsoft Technologies. Worked various products across domains like Health Care, Energy and Retail. View all posts by Senthil Kumar Palanisamy
by Praveena Jayanarayanan | Nov 13, 2018 | BizTalk Community Blogs via Syndication
Introduction
In a data-driven business setup, there will be scenarios where critical executive decisions are made based on the performance reports of various components of the listed servers in their environment. BizTalk360 aims to offer an out of the box tool with similar capabilities as the Performance Monitor tool in Windows servers.
BizTalk360 Analytics offers visual display of the most important performance counters that are consolidated and arranged on a single screen, so that the information can be monitored in a glance. Custom reports can be built in minutes with the metrics that really count for your business, and they’re easy to share with other users in the system. In addition, dashboards can give you a summary of many reports on a single page, using drag-and-drop widgets for fast, easy customization.
The Analytics feature is available only to the Platinum licensed BizTalk360 environments. By default, the Analytics service is installed with BizTalk360 and in non-Platinum environments, users can only view sample analytics data.
The Analytics Widgets
It is with the Dashboard widgets that all the performance related information can be seen in a nice graphical display. In the Analytics section, there is also the reporting capability added, where the reports can be sent as attachment in emails.
In this blog I am going to explain a few scenarios where the Analytics widgets are more helpful. Let’s move on.
Customer Scenarios
The Analytics Dashboard contains lot of widgets that provide you detailed information of the performance of your BizTalk environment.
For example, the BizTalk Host Performance widget will help you to get the information of the top performing hosts in the BizTalk environment. The BizTalk messaging performance gives you the picture of the message count and average execution time of the total messages transferred during the configured datetime range.
Date Range – Select the date range for chart. The options are Last 24 hrs, Last 7 days, Last 30 days and Custom:
- Last 24 Hrs – Displays the data from last 24 hours
- Last 7 days – Displays the data from last 7 days
- Last 30 days – Displays the data from last 30 days
- Custom – To add a specific number of days, use the scroller to select the day
Based on the selected date range, the performance data gets collected from the BizTalk servers.
The other basic widgets include the Message volume and the Transmission failure rate widgets.
Lot more to do with Custom widgets
Don’t just stop with the basic widgets in the Analytics section; there is lot more to do with the Custom widgets.
Once the Analytics components, for which the data needs to be collected, are enabled, the different artifacts for the Custom widgets will get enabled. With the Custom widgets you can chose different metrics for which you want to view the data.
Let’s take a few scenarios to understand what we can achieve with the Custom widgets.
Comparing two BizTalk servers’ performances
There is a scenario where you would like to compare the performance of two BizTalk servers and see which one consumes more memory and CPU. This can be done by selecting the servers in the artifact section and corresponding metrics.
Get the message count for specific port
Of course, the count of messages passing through the port can be determined through process monitoring. But it will be checked with specific time intervals either in hours or minutes. But, what if you want to see the count instantly within the specific datetime range. Here comes the Custom widget, with which you can get the count of specific ports.
But to get these data, Tracking needs to be enabled for the ports. Based on the enabled tracking data collection counters, the data gets collected by the BizTalk360 Analytics service using the performance counters.
Failure rate of a port
Also, the failure rate of a particular port, running a specific host instance, can be determined with the help of a Custom widget. This would include the failure rate for the port and the number of instances running for that particular host. With the combination of these two metrics, the failure rate of a port can be determined and displayed in a nice graphical view.
Checking the MessageBoxDb
There are also counters for checking the MessageBoxDB. The key performance indicators of the message box database which includes the spool size and the status of the pending jobs, can be visualised with the widgets.
BizTalk Host Details
We do have certain metrics like Host Queue Length, number of instances and suspended instances for the particular hosts, which can be obtained from the custom widgets.
Above mentioned are the very few scenarios that can be achieved using the widgets. With the different combinations of the metrics, all performance related information can be obtained through the Analytics widgets.
Conclusion
Do you wish to see more information on the widgets or from the product itself, please put in your suggestions and feedback in our user voice portal https://feedback.biztalk360.com/. The existing ideas can also be voted for. We at BizTalk360 aim at providing the features that fulfill the customer requirements. It is from this feedback portal that the features get picked up for every release.
For more information about how these widgets work, check our Documentation portal.
Happy monitoring with BizTalk360 😊
Author: Praveena Jayanarayanan
I am working as Senior Support Engineer at BizTalk360. I always believe in team work leading to success because “We all cannot do everything or solve every issue. ‘It’s impossible’. However, if we each simply do our part, make our own contribution, regardless of how small we may think it is…. together it adds up and great things get accomplished.” View all posts by Praveena Jayanarayanan
by Praveena Jayanarayanan | Jul 31, 2018 | BizTalk Community Blogs via Syndication
Using BizTalk360 Analytics, you can visualize a lot of interesting facts about your BizTalk environment, like number of messages processed, failure rate at message type level, BizTalk server CPU/Memory performances, BizTalk process (host instances, SSO, rules engine, EDI etc), CPU/memory utilization and lots more.
In a data-driven business setup, there will be scenarios where critical executive decisions are made based on the performance reports of various components of the listed servers in their environment. BizTalk360 aims to offer an out of the box tool with similar capabilities as the Performance Monitor tool in Windows servers. BizTalk360 Analytics offers visual display of the most important performance counters that are consolidated and arranged on a single screen so that the information can be monitored in a glance. Custom reports can be built in minutes with the metrics that really count for your business, and they’re easy to share with other users in the system. In addition, dashboards can give you a summary of many reports on a single page using drag-and-drop widgets for fast, easy customization.
In every new release of BizTalk360, we include new features and enhancements to make the product more usable for the customers and adding value for them as per the below quotes,
“Excellent organisations consistently add value for the customers by understanding, anticipating and fulfilling needs, expectations and opportunities”
Let’s see what we have brought for the Analytics module in BizTalk360 v8.8:
- Zoom In and Out option for the Message Patterns
- Renaming option for the Message Pattern title
- Restart option for the Analytics and Monitoring Windows services
In this article, we are having a look at all these improvements.
Zoom In and Out option for the message patterns
In a typical BizTalk environment, there will be different message flows happening. It could be a simple one-way messaging or more complex message pattern with an orchestration connecting different ports. The unique message patterns are captured and displayed in BizTalk360 with a graphical representation. With messaging patterns, you can get to know how many messages passed through a particular port and the average time of execution for the messages across a time period.
The difficulty here lies in viewing the complex patterns within small screen resolution. For this reason, we have included the zoom in/out feature for the message patterns in the full screen mode.
This option comes into picture when the message pattern is opened in full screen mode and you can zoom in /out using either the mouse or the icons present. In the full screen mode, the properties of the ports are not displayed. To view the properties, this mode needs to be exited.
Renaming option for the Message Pattern title
Once a message pattern is recognized by BizTalk360, the message flow is saved with a GUID as its title. This would be difficult to remember and not user friendly. Hence, we had the edit option in a separate screen, enabling the user to provide a descriptive name and detailed description for the message pattern.
Based on the few requests from the customers however, there is an extra capability added to rename the message pattern title from the details screen itself. In this case, the users can directly rename the title in the details section where the message flow is seen instead of going to a separate screen.
Restart option for Analytics and Monitoring Windows services
Sometimes an exception occurs during monitoring or in the analytics section. Once that exception is resolved, it is recommended to restart these services to reset the errors in the corresponding sub services.
In this scenario, the user would need to login to the server where the service is installed and then restart it. If the service is installed in more than one server, then they would need to login to all the servers for restarting the services.
To make life a little bit easier, we have added the restart option in BizTalk360 application. When this option is clicked, the service installed in multiple servers will get restarted.
Other improvements
There are other bug fixes as well, in the Analytics section included as part of the BizTalk360 v8.8 release, which include:
- Improving the performance by deleting the temporary tables created
- In BizTalk360 Analytics section, for displaying the data for the messaging patterns, we directly query the MessageBox database and the Tracking database and store them in the temporary table in TempDB. Depending on the volume of data, these temporary tables may cause performance hinge in BizTalk360. Based on the auto growth settings in the SQL server, the TempDb will grow and results in SQL Exception for few widgets. The code is modified to delete the temporary tables after fetching data. Hence, data growth is controlled and there will not be any performance glitch.
- The edit icon for the date time range in the Analytics widget has been removed as there is already a link present
- The custom widgets in the Analytics section is designed to display the data based on the date time range, which is last 24 hours, 7 days and 30 days. You also have the option to set the custom date time range. The datetime range in the custom widget data already has the link to edit the date. There was an additional edit link present in these widgets which we thought would be unnecessary as it may not be used. Hence this link was removed to make the widgets more user friendly.
- Displaying the data for available memory in the widget
- Widgets in the Analytics Dashboard are designed to provide the user with a graphical data. There are three different types of widgets that can be used,
- Default/Basic Widgets
- Static Widgets
- Custom Widgets
- You can create and customize the different types of widgets only under the Operations Dashboard and Analytics Dashboard. The process to perform these operations remains the same.
Sometimes there may be inconsistent occurrence of certain issues which may be difficult to be identified. One such issue was that “When the Available memory % counter for SQL server was added to the custom widget and when the widget was refreshed, the data for memory % was not getting displayed in the graph”. The root cause for this case was identified and fixed in BizTalk360 v8.8.
Please refer the release notes link below for the complete list of enhancements and bug fixes in v8.8:
https://docs.biztalk360.com/v1.0/docs/v8821162405
Conclusion
Why not give BizTalk360 a try! It takes about 10 minutes to install on your BizTalk environments and you can witness and check the security and productivity features of BizTalk360, within your own BizTalk Environments. Get started with the free 30 days trial. If you are already using BizTalk360 you can download the latest version directly from the product and upgrade, it to enjoy the new features.
Author: Praveena Jayanarayanan
I am working as Senior Support Engineer at BizTalk360. I always believe in team work leading to success because “We all cannot do everything or solve every issue. ‘It’s impossible’. However, if we each simply do our part, make our own contribution, regardless of how small we may think it is…. together it adds up and great things get accomplished.” View all posts by Praveena Jayanarayanan
by Venitta Priya | Dec 7, 2017 | BizTalk Community Blogs via Syndication
You might have seen various people use this term “reporting and analytics” interchangeably to describe the typical application and use of data — to track the ongoing health of the company and to inform decision making. Reporting helps companies to monitor their business and be alerted to when data falls outside of expected ranges. Reporting is the act of translating raw data into information. This raw data can come from a multitude of data sources such as a production database, an operational database like MySQL, Google Analytics and a CRM system like Salesforce. While the reporting that comes out of these databases are useful, they’re often standardized and summarized versions of raw data. This might be enough information for your business, but to get the most out of your data—you need analytics.
In case of BizTalk Server, your day to day monitoring analytics may cover various aspects of performance like BizTalk Servers performance, CPU performance, etc.. BizTalk360 offers out of the box capabilities that provide a graphical display of the key performance metrics of the BizTalk Server. In any business environment, these metrics will be of critical importance for the management to take business decisions. As of version 8.6, BizTalk360 provides the option for users to generate PDF documents of critical performance metrics at specific time periods depending on the requirement.
BizTalk360 Reporting
We have introduced a new section called Reporting under Analytics section. BizTalk360 enables you to create and switch between multiple reports and widgets of your choice, enabling you to visualize the BizTalk performance information in the way you want it. The Reporting section is mainly introduced to monitor the performance of server, host, IIS and disk usage of the most important BizTalk databases. These are the custom reports that you can access with the help of analytics, which can also be delivered on a recurring basis to a group of end users. The Reports provide a comprehensive, high-level view of business performance for specific audiences.
Reporting Schedule
You can create a schedule and configure it to a schedule type based on the requirement. There are three types of the schedule available:
The Daily report will be sent to the user, on the configured timing (Say 10am every day).
The Weekly report will be sent every week at the configured time for the user (say every Monday 11am).
The Monthly report will be sent once in a month on the set date and time (Say 15th of every month at 11am).
Once the schedule is created, the user can map the schedule to a report. We are able to edit schedules and to, delete schedules. Once the schedule is created, there is an option for disabling the schedule and if you don’t want the email alert for that time duration you can enable the option Disable Schedule.
Manage dependency
BizTalk360 is designed such that majority of the functionalities will work out of the box without any configuration. However, certain features will depend on Third-Party files which need to be installed separately. The BizTalk360 Reporting functionality requires certain dependency files for users, to be able to download PDF reports and/or trigger email notifications about the health of the BizTalk Server environment. This dependency file is required for BizTalk Reports to map the report for the created schedule.We have an option to download the dependency file from Settings -> Manage Dependencies section. The Dependency file will get downloaded in the Source folder (Web-> Dependencies ->Analytics). From the Source folder, the file will get copied to the Destination folder (Analytics folder).
Reporting Action
Reports can be created and mapped to the schedules which have been created and widgets can also be added to the reports based on the requirement. You can add, remove and customize the widgets based on the necessity. Widgets in the report will have the live data. Reports will be sent to the configured email address at the configured time. Reports have historic data section where you will be able to see the live data and the historical data. Historical data will contain the data of the reports which has been sent in the email alert.
Following are the widgets which are available in the reports. You can add, remove and Customize the widgets based on the necessity.
- BizTalk Hosts Performance
- BizTalk Server Performance
- SQL Server Performance
- MessageBox Database Disk Usage
- MessageBox Database Top 5 Tables
- Tracking Database Disk Usage
- Tracking Database Top 5 Tables
- IIS server Performance
- Messaging Performance
- BizTalk Messaging Performance
- Message Volume by Schema
- Transmission Failure Rate
- by Schema (Top 10)
- by Port (Top 10)
- Event Log Data Count
- BizTalk
- SSO
- SQL Server
- Internet Information Server
- BizTalk360 Monitoring Service
- BizTalk360 Analytics Service
- Message Box KPI
Report notification history
Based on the report settings, BizTalk360 will trigger the reports to the users through email with the PDF attachment of the reporting dashboard. The Report notification history shows you whether the email is delivered to the configured email address or not.
I hope this blog has given some useful information about Reporting section in Analytics. Please don’t hesitate to contact us at support@biztalk360.com, if you want to know more about this particular feature or any other feature in BizTalk360.
