by Saravana Kumar | May 3, 2017 | BizTalk Community Blogs via Syndication
We are continuing our tradition of one new release every 3-4 months once continuously for the past 6 years. In every release, we wanted to make sure we add 5-6 meaningful features that will help BizTalk Server administrators with Operations, Monitoring and Analytics capabilities. In BizTalk360 version 8.4, we added some exciting new features.
- BizTalk Server Licensing Widget
- Data Monitoring for Azure Logic Apps
- Folder Monitoring
- FTP/FTPS/SFTP Monitoring
- IBM MQ Monitoring
- BizTalk Health Monitoring (BHM) Integration
- Manage NT Services directly in the Web Console
- Manage SQL Jobs directly in the Web Console
- Few enhancements/bug fixes
BizTalk Server Licensing Widget
From our experience dealing with 100’s of BizTalk Server customers, we noticed that a lot of them struggle to understand how many BizTalk Server licenses are required for their servers. About 4 years ago I wrote an article called Understanding BizTalk Server Licensing and that’s one of the popular articles on our blog. We just thought it will be a good idea to transform that knowledge into a small widget in BizTalk360 so that people will easily understand how many licenses are required for your BizTalk Server.
Out of the box we ship this new widget in BizTalk360 version 8.4 called “BizTalk Server License” as shown below. This widget will display key information like BizTalk edition, server type, processor type, number of cores, processor manufacturer, retail license cost per server, how many servers to be licensed and total cost for the environment.
Data Monitoring for Azure Logic Apps
Data monitoring in BizTalk360 is one of the key capabilities that allows you to monitor and trigger alerts based on historical events. Some of the common use cases of data monitoring include “No Event Alerting”. Ex: If you are expected to process 5 purchase orders from an FTP location every hour and you haven’t received the expected volume, then BizTalk360 can alert you with a message.
We are taking the exact same concept to Azure Logic Apps to alert you based on historical transactions within a time window.
There are various interesting use cases. For example, as shown above, you may want to get alerted if one of your Logic App has gone crazy and firing thousands of executions and costing so many $$$. We will cover in detail about this feature in a separate blog article.
Here is the detailed list of metrics you can set data monitoring on.
Folder Monitoring
Sometimes as the product matures and you look at adding more features, you tend to miss out on the basic things. Folder monitoring is one such thing we left for so long! It’s better late than never. In BizTalk360 version 8.4 we are bringing the capability of folder monitoring. The technology might have improved significantly like Micro Services, REST API’s etc, however, file based integration is there to stay. It’s very common in the integration world where you drop a file (purchase order, batch file, EDI transactions etc) and the integration kicks in. One of the common challenges in such integration is that what happens when the integration is broken and the files started to pile up in the pickup folder.
We solve the exact problem using the folder monitoring capability.
One of the core values of BizTalk360 is to make monitoring configuration seamless and that’s how we differentiate ourselves from general purpose monitoring products. In the folder monitoring case, we automatically list down all the receive locations and send ports that use BizTalk FILE adapter and allow the user to configure in pretty much 2-3 clicks, we pick up all the values like folder location from existing configuration as shown above.
FTP/FTPS/SFTP
On the similar lines of normal Folder monitoring, FTP based integrations are key in most of the enterprise integration scenarios. BizTalk Server comes with three different adapters FTP, FTPs and SFTP to tackle FTP scenarios where the variations are mainly around the security capabilities of the FTP server.
With BizTalk360 version 8.4, you can monitor FTP locations for data pile up. We put a lot of efforts to make the configuration experience as seamless as possible, the screens will automatically list all the FTP based send ports/receive locations and all the available values like location, username etc are automatically picked up from those configurations.
IBM MQ Monitoring
For the past few releases, we are slowly bringing in the capability of Queue monitoring into BizTalk360. Queues play a vital role in enterprise integration especially for robustness, store, and forward patterns. In the previous releases, we introduced MSMQ and Azure Service Bus Queue. In BizTalk360 version 8.4, we are bringing in support for IBM MQ. We support both MQSC and MQS based configurations.
In any IBM MQ queues, you can monitor for the following 4 parameters – queue depth, backout queue depth, queue usage % and backout queue usage %.
BizTalk Health Monitoring Integration
For many years, we had support for Message Box Viewer inside BizTalk360. We periodically run MBV in configured environments, parse and store the result and display it in the BizTalk360 web console. We also got the ability to monitor and alert users based on Message Box Viewer raised errors and warning. Two years ago, Microsoft made some major changes to Message Box Viewer, re-branded it as BizTalk Health Monitor with few additional functionalities and deprecated MBV.
Support for BHM is one of the top requested features on our feedback portal. In BizTalk360 version 8.4 we introduced support for BHM and deprecated MBV support.
NT Services Operation in BizTalk and SQL Servers
One of the main objective for us from the security perspective is to stop people logging on/off into production BizTalk and SQL servers during business hours. We also wanted to audit any activities performed by BizTalk support people. Even though you shouldn’t start/stop services in your environment, we noticed in some cases NT services like world wide web, enterprise single sign on, BizTalk host instances, SQL Agent, etc needs to be started/stopped for the variety of reasons. Currently, users will RDP or have remote MMC snap-in to manage NT services in BizTalk and SQL servers. With BizTalk360 version 8.4 you can manage them directly in the web console. In addition, the activities will be audited by BizTalk360.
Manage SQL Jobs Operation from web console
In a similar concept to managing NT Services in both BizTalk and SQL Servers, SQL jobs play a vital role in a BizTalk Server environment. SQL Jobs are responsible for keeping your BizTalk environment healthy. They take care of routine housekeeping activities like moving data from Message Box database to Tracking database, purging/archiving tracked data, backup/disaster log shipping etc.
In BizTalk360 version 8.4, we brought in capabilities to manage SQL jobs directly from the web console. This functionality of monitoring SQL Jobs has been in the product for a very long time.
Few Enhancements and Bug Fixes
This article mainly covers all the new and exciting features we shipped in BizTalk360 version 8.4. In every release, we also allocate time to enhance current features and to address any top priority bugs we received from our existing customers. Here are some of the key enhancements and bug fixes
Alarm Management Improvements: Now you have the ability to bulk enable/disable monitoring alarms. You can also rename the alarms (again, one of the most requested features on our feedback portal.)
Webhook Notification Channel: Now you have the ability to specify (override) a new endpoint URL at the alarm level.
Logic Apps Monitoring Performance Improvement: Few critical updates been made to improve the performance of monitoring Azure Logic Apps
Get started today
Author: Saravana Kumar
Saravana Kumar is the Founder and CTO of BizTalk360, an enterprise software that acts as an all-in-one solution for better administration, operation, support and monitoring of Microsoft BizTalk Server environments. View all posts by Saravana Kumar
by Kishorekumar Ashok | May 2, 2017 | BizTalk Community Blogs via Syndication
Introduction
File Locations Monitoring is one of the new Monitoring capabilities introduced in BizTalk360 v8.4. From our customers, we received many requests to introduce the ability to monitor Folder Level into BizTalk360. We are happy to announce that in BizTalk360’s upcoming release 8.4, we are bringing a feature to monitor file count for File Locations (File, FTP and SFTP), configured in BizTalk receive locations and send ports.
Need for File Locations Monitoring
Messaging or data exchange between business can be done in various ways. Frequent data communication process is done through for example File and FTP with XML or EDI as the message format. From BizTalk 2013 onwards, SFTP is also included in BizTalk Server which was available as Out-of-box feature in prior versions. Even though BizTalk works seamlessly with File Adapters, it has some known issues which occur due to incorrect configurations.
The BizTalk File Location Adapters (File, FTP, SFTP) fail to perform the operations on following scenarios:
- The File receive adapter cannot access the receive location on the file system or network share because the specified path does not exist. For a network share, the File receive adapter disables the receive location after all retry attempts have been exhausted.
- The File receive adapter cannot access the receive location on the file system or network share because the account used by the associated host instance does not have read-write permission for that location. For a network share, the File receive adapter disables the receive location after all retry attempts have been exhausted
- Files with names longer than 256 characters are encountered in the receive location
To resolve the above issues, we need to ensure that the specified path or share exists and the account used as the Logon should have read-write access. Additional to this, if you configure schedule/service window for your receive locations, messages will be accepted only during that time window, all other times BizTalk won’t pick up messages. Any violation to this scenario also needs to be monitored.
We often experience that organizations facing these kinds of challenges used custom solutions for this kind of monitoring. To overcome this, BizTalk360 added the File Locations Monitoring capabilities out of the box.
File Locations (File, FTP, SFTP)
In BizTalk360 v8.4, we are introducing support to monitor the File, FTP and SFTP servers under File Location Monitoring Section. File Location Monitoring will list all the locations configured in the BizTalk Artifacts (Send Ports and Receive Locations) for the Transport Types (File, FTP, SFTP) respectively, which helps users to monitor all the File Locations mapped with Receive Locations/Send Ports.
File Monitoring
To get familiar with File Adapter configuration, kindly refer below link:
https://msdn.microsoft.com/en-us/library/aa559105.aspx
In BizTalk360 File Monitoring Configuration contains three sections: Basic Information, Authentication, and File Monitoring Configurations.
- The Basic Information Section contains Folder Location and File Mask configured in BizTalk.
- The Authentication section is Optional. By default, authentication could be processed by the BizTalk360 Monitoring Service account, when credentials are not given.
- In the File Configurations Section, we can configure the Thresholds with the metric File Count to monitor
When a File Location is in the Orphaned State, BizTalk360 would let the users know about the cause of the failure on hovering the warning icon
FTP Monitoring:
To get familiar with FTP Adapter configuration, kindly refer to below link:
https://msdn.microsoft.com/en-us/library/aa561032.aspx
The FTP Configuration UI is categorized into three sections: FTP Details, Firewall Details and FTP Monitoring Configurations
- The FTP Details Section contains the details about the FTP Location, Authentication, and SSL
- The Firewall Details contains the configurations to connect FTP Server through a Firewall
- In the FTP Monitoring Config section, we can configure the monitor with Threshold Conditions for the metric File Count
SFTP Monitoring:
To get familiar with the SFTP Adapter Configuration in BizTalk, kindly refer the below link
https://msdn.microsoft.com/en-us/library/jj684551.aspx
The SFTP Monitor Tab in BizTalk360 lists the SFTP Locations which are configured in BizTalk. It contains four sections:
- SSH Server Section has the details about the SFTP Location
- The Proxy Details Section is optional to connect SFTP Server behind a firewall
Note: In BizTalk, Proxy details are available from BizTalk 2013 R2
- Security Details Section has the authentication details
- In the SFTP Monitoring Config Section, we can configure the monitor with threshold conditions for the metric File Count
Conclusion
With this latest release 8.4, BizTalk360 brings the File Locations Monitoring with the ability to monitor the File Count. In the future, we will be adding support to monitor Folder Size and Access permissions. If you have any feedback or suggestions, please write to us at [email protected].
by Kishorekumar Ashok | May 2, 2017 | BizTalk Community Blogs via Syndication
Introduction
File Locations Monitoring is one of the new Monitoring capabilities introduced in BizTalk360 v8.4. From our customers, we received many requests to introduce the ability to monitor Folder Level into BizTalk360. We are happy to announce that in BizTalk360’s upcoming release 8.4, we are bringing a feature to monitor file count for File Locations (File, FTP and SFTP), configured in BizTalk receive locations and send ports.
Need for File Locations Monitoring
Messaging or data exchange between business can be done in various ways. Frequent data communication process is done through for example File and FTP with XML or EDI as the message format. From BizTalk 2013 onwards, SFTP is also included in BizTalk Server which was available as Out-of-box feature in prior versions. Even though BizTalk works seamlessly with File Adapters, it has some known issues which occur due to incorrect configurations.
The BizTalk File Location Adapters (File, FTP, SFTP) fail to perform the operations on following scenarios:
- The File receive adapter cannot access the receive location on the file system or network share because the specified path does not exist. For a network share, the File receive adapter disables the receive location after all retry attempts have been exhausted.
- The File receive adapter cannot access the receive location on the file system or network share because the account used by the associated host instance does not have read-write permission for that location. For a network share, the File receive adapter disables the receive location after all retry attempts have been exhausted
- Files with names longer than 256 characters are encountered in the receive location
To resolve the above issues, we need to ensure that the specified path or share exists and the account used as the Logon should have read-write access. Additional to this, if you configure schedule/service window for your receive locations, messages will be accepted only during that time window, all other times BizTalk won’t pick up messages. Any violation to this scenario also needs to be monitored.
We often experience that organizations facing these kinds of challenges used custom solutions for this kind of monitoring. To overcome this, BizTalk360 added the File Locations Monitoring capabilities out of the box.
File Locations (File, FTP, SFTP)
In BizTalk360 v8.4, we are introducing support to monitor the File, FTP and SFTP servers under File Location Monitoring Section. File Location Monitoring will list all the locations configured in the BizTalk Artifacts (Send Ports and Receive Locations) for the Transport Types (File, FTP, SFTP) respectively, which helps users to monitor all the File Locations mapped with Receive Locations/Send Ports.
File Monitoring
To get familiar with File Adapter configuration, kindly refer below link:
https://msdn.microsoft.com/en-us/library/aa559105.aspx
In BizTalk360 File Monitoring Configuration contains three sections: Basic Information, Authentication, and File Monitoring Configurations.
- The Basic Information Section contains Folder Location and File Mask configured in BizTalk.
- The Authentication section is Optional. By default, authentication could be processed by the BizTalk360 Monitoring Service account, when credentials are not given.
- In the File Configurations Section, we can configure the Thresholds with the metric File Count to monitor
When a File Location is in the Orphaned State, BizTalk360 would let the users know about the cause of the failure on hovering the warning icon
FTP Monitoring:
To get familiar with FTP Adapter configuration, kindly refer to below link:
https://msdn.microsoft.com/en-us/library/aa561032.aspx
The FTP Configuration UI is categorized into three sections: FTP Details, Firewall Details and FTP Monitoring Configurations
- The FTP Details Section contains the details about the FTP Location, Authentication, and SSL
- The Firewall Details contains the configurations to connect FTP Server through a Firewall
- In the FTP Monitoring Config section, we can configure the monitor with Threshold Conditions for the metric File Count
SFTP Monitoring:
To get familiar with the SFTP Adapter Configuration in BizTalk, kindly refer the below link
https://msdn.microsoft.com/en-us/library/jj684551.aspx
The SFTP Monitor Tab in BizTalk360 lists the SFTP Locations which are configured in BizTalk. It contains four sections:
- SSH Server Section has the details about the SFTP Location
- The Proxy Details Section is optional to connect SFTP Server behind a firewall
Note: In BizTalk, Proxy details are available from BizTalk 2013 R2
- Security Details Section has the authentication details
- In the SFTP Monitoring Config Section, we can configure the monitor with threshold conditions for the metric File Count
Conclusion
With this latest release 8.4, BizTalk360 brings the File Locations Monitoring with the ability to monitor the File Count. In the future, we will be adding support to monitor Folder Size and Access permissions. If you have any feedback or suggestions, please write to us at [email protected].
