by Sivaramakrishnan Arumugam | Nov 1, 2018 | BizTalk Community Blogs via Syndication
One of the core features of BizTalk360 is it’s monitoring and notification capability. BizTalk Server is used in mission-critical environments to integrate different applications within the enterprise. While BizTalk Server can integrate with multiple applications and provide a one-stop solution for enterprises, it lacks the ability to allow the administrators to monitor, amongst others, the health of connected applications.
BizTalk360 understands the problems faced by enterprises and offers an out-of-the-box monitoring solution that assists administrators to monitor the BizTalk environments. BizTalk360 solves the purpose of being able to monitor BizTalk Server solutions in a simple and easy manner.
So, What Can BizTalk360 Monitor?
BizTalk360 allows administrators to set up alerts on specific components of BizTalk Server, such as:
- BizTalk Applications and their Send Ports, Receive Locations, Orchestrations, Service Instances
- Disks
- Event Logs
- NT Services
- System Resources (CPU, Memory)
- SQL Jobs
- Web Endpoints
- BizTalk Health Monitor, Errors and Warnings
- Host instances (normal, clustered)
- Azure services
- Queues – MSMQ, Azure Service Bus, IBM MQ
- File Location – File, FTP and SFTP
- Database queries
I would like to explain the feature of File Location monitoring usage and the challenges we faced.
File Location Monitoring
File Locations Monitoring is one of the core monitoring capabilities which we had introduced in BizTalk360 v8.4. From our customers, we received many requests to introduce the ability to monitor Folder Level into BizTalk360.
How File Location monitoring works?
BizTalk360 has the capability of fetching the file locations which are configured in the BizTalk Admin console.
File Location Monitoring lists all the locations configured in the BizTalk Artifacts (Send Ports and Receive Locations) for the Transport Types (File, FTP, SFTP) respectively and shows them in the UI. Users can map only the file location of BizTalk Artifacts; file locations which are not mapped with the artifacts will not be monitored from BizTalk360.
Cross-domain
Just to give an overview of a domain
A domain is a group of computers and devices on a network that is administered as a unit with common rules and procedures. Within the Internet, domains are defined by the IP address. All devices sharing a common part of the IP address are said to be in the same domain.
Overview of Cross-domain
By cross-domain, we refer to the communication between different domains by creating trusts.
A trust is a relationship established between two different domains that enable users in one domain to be authenticated by a domain controller in the other domain. There are different types of trusts in the Microsoft Active Directory domain such as External, Realm, Forest and shortcut. An external trust is necessary when users of two different domains of two different business units want to utilize resources such as printers and file servers of trusted domains.
Cross-domain in BizTalk
In BizTalk, the receive locations or the send ports can be configured with the file locations present in the server. These locations may be in the same server or in a different server in the different domain. With the trust created between the domains, the file transfer would happen, and the message gets transferred.
File Location monitoring in BizTalk360
As said earlier, in BizTalk360 we have the capability to monitor file locations and trigger alerts based on the file count and directory size configuration.
The BizTalk360 monitoring service will check for the file count and directory size and trigger alert emails. So, as per the environment configuration, the service account running the monitoring service will need access permissions to the BizTalk environment to fetch the data of the BizTalk artifacts.
Often the support tickets raised by customers provide different learning experiences to the support team. One such ticket was an issue with the file monitoring.
Configure File Location monitoring
Configuring File Location monitoring is a very straightforward activity in BizTalk360. You have the option to configure with authentication mode as well as anonymous mode. BizTalk360 will automatically list down all the available file locations on the server.
Click the Gear icon to open the File Details blade and toggle on the Authentication button to enter the username and password. If the username and password columns are blank, it will be considered as anonymous authentication. Configure the warning and error rules for monitoring under the File Monitoring Configurations settings. Click Save Configurations to complete the process.
Scenario 1: Cross-domain file share
One of our customers raised an issue that a file share is getting Orphaned once after configuring with the alarm. We started with the basic troubleshooting steps.
We understood that customer configured a File path in BizTalk server which is a cross domain. In detail for the specific send port, they used a file adapter and while they provide a path they have provided a path where the folder is available in a different domain.
Troubleshooting the problem
We asked the customer to share if any exception appeared and we received that “the username and password are incorrect”.
Technically, BizTalk360 should monitor the specific file without any issues. This is the reason why we are using BizTalk360’s monitoring service account to fetch the details. If BizTalk360 monitoring service needs access across the domain, it should monitor without any problem.
To find the exact root cause of the issue we have developed a console application(with logs enabled) and provided it to the customer. Developing back and forth, we found that it’s an authentication issue and solved it in version 8.9 of the product.
Scenario 2 – Monitoring a cross-domain Linux server
It’s again an interesting case once after the release of version 8.9 (fix for cross domain file monitoring). One of our customers came with a feedback that he is facing a problem with file monitoring.
Once we engaged a call with our customer, we found that they are using a Linux machine for a folder path in the BizTalk server, which is a cross domain. In detail, for the specific send port, they used a File adapter and while they provided the path, they have provided a folder which is available in a different domain in a Linux machine.
Troubleshooting the problem
We asked the customer to share if any exception appeared and we received the same error that “username and password is incorrect”.
To connect the Linux folder location from the Windows machine, the customer used a tool called Samba share. We have enhanced our code to authenticate and connect the Linux machines with encrypt and decrypt, along with NT service mechanism.
Later, we have provided a console application to our customer which solved the problem. We have planned to include the fix in our upcoming version.
What about FTP and SFTP?
By default, FTP and SFTP are hosted in a web(domain)and we support cross-domain in FTP and SFTP file monitoring.
Conclusion
This was another support case which was interesting for me as it helped me to understand the code from a Developer perspective. I have explored a different kind of cross-domains from the Tester perspective. Working along with the customer gave satisfaction as a Support engineer. Hope I will receive “an Awesome” review from the customer side 😊
Author: Sivaramakrishnan Arumugam
Sivaramakrishnan is our Support Engineer with quite a few certifications under his belt. He has been instrumental in handling the customer support area. He believes Travelling makes happy of anyone. View all posts by Sivaramakrishnan Arumugam
by Lex Hegt | Mar 28, 2018 | BizTalk Community Blogs via Syndication
This blog is a part of the series of blog articles we are publishing on the topic “Why we built XYZ feature in BizTalk360”. Read the main article here.
Why do we need this feature?
Imagine the following scenario. It’s Monday morning, you arrive at the office and you do some extra health checks on your environment, as planned maintenance has been done over the weekend, also involving the BizTalk servers. You notice that the Host Instances are not started! This occurred because, after a reboot of the BizTalk servers, the Host Instances and other vital services like the Enterprise Single Sign On service did not start. Because of this, ever since that reboot no processing has been done by BizTalk, which results in an enormous backlog of processing for an already busy Monday morning.
There is one more very common scenario. Let’s assume you are using an FTP/SFTP receive location polling files (Purchase Orders, Invoices etc) from a remote server. There are various reasons we might encounter the problem in this setup, example network connectivity, the remote server is down, someone changed the password etc.
In these instances, the BizTalk FTP/SFTP receive location will attempt to connect few times and finally quietly shut down (without notifying anyone). This is a major problem, we need a solution where a system can notify the BizTalk Administrators and also try to recover the failure condition automatically.
Above scenario is just a couple of the example of possible outage conditions. Besides Host Ithe instances and other Windows NT services, not being in the expected state, think of Receive Locations being disabled because of network disruptions, Orchestrations being unenlisted, so no messages will be picked up by then or performance degradation because the wrong Windows NT services are being started (either manually or automatically).
All these kind of artifacts being in the wrong state, may lead to disruptions of your valuable business processes and put your business at stake.
What are the current challenges?
Off course you want to prevent your business processes from being disrupted by just a few Windows NT services, or other state-bound artifacts, being in the wrong state. Ideally, you want the expected state to be recovered as soon as the artifact hits the wrong state.
No auto-recovery support by BizTalk
Unfortunately, BizTalk Server itself provides no functionality to detect that artifacts are in the wrong state, let alone that these artifacts are brought back to the desired state. So what’s left is creating custom scripts to take care of that task. More on custom scripting, a little bit later in this article.
No auto-recovery support by SCOM
Your organization might already use SCOM for monitoring your server platform. Although SCOM has a so-called Management Pack for BizTalk server, it is quite challenging to setup SCOM for proper BizTalk monitoring and operating.
Check out below whitepaper, to find out the differences between BizTalk360 and SCOM when it comes to maintaining your BizTalk environment.
Auto-recovery is one of the reasons why SCOM is not the best fit for maintaining your BizTalk environment as SCOM offers very little support for auto-recovery of your BizTalk and other artifacts which are important for your integrations. SCOM provide event-handlers which can be used for executing custom scripts, you as an administrator, still have to create these scripts.
Custom scripting
So, whether you are using just BizTalk Server or use SCOM, in case you want to auto-recover your artifacts which are in the wrong state, you need to develop custom scripts to have something in place for auto-recovery of your state-bound artifacts. Developing such scripts can be time-consuming and often it is hard to properly maintain such scripts. Also, the visibility of such scripts is bad, as they are being run through Windows Scheduler, which is, in turn, another component you should be aware of, when you are considering the overall health of your BizTalk environment.
At BizTalk360, we have the philosophy that businesses should take care of their core businesses and not be developing scripts and tools to maintain their BizTalk environments. As we have many years of experience in the field of BizTalk Server and Microsoft integration, we understand the problems you might run into, as we have faced them ourselves as well.
The goal of BizTalk360 is to take away your challenges when it comes to operating and maintaining your BizTalk environment. Even though anything can be done using custom coding/scripts, that’s not the best use of your time + management overhead of maintaining that code base.
How BizTalk360 solves this problem?
For many releases now, BizTalk360 contains the Auto Healing feature for recovery of artifacts which have hit the wrong state. While in the beginning, mainly the BizTalk artifacts could be brought back to the expected state, the feature has evolved to support below artifacts:
- Send Ports
- Receive Location
- Orchestrations
- Host Instances
- Windows NT Services
- SQL Server Jobs
- Azure Logic Apps
The Auto Healing can be configured as part of the monitoring settings for the above-mentioned artifacts. Below picture shows a Receive Location for which monitoring has been set up. The Expected state of this Receive Location is Disabled. Besides that, also Auto Healing has been set up.
Once set up, the BizTalk360 Monitoring service will evaluate whether the Receive Location is still in the expected state. In case the Receive Location hits a state which is not the expected state, the Monitoring service will try to bring the Receive Location back to the expected state. Depending on the configured value for the Max Retry setting, this will be tried at most 10 times. If the Receive Location still is not in the expected state, the artifact will move to the Critical state.
Note: To the Auto Healing, it makes no difference whether the expected state is Enabled or Disabled, or Unenlisted or Started. Depending on what you have configured for the Expected state, BizTalk360 will always try to bring the artifact to that Expected state.
Conclusion
In this article, we have seen how valuable it can be to have your artifacts being brought back to the expected state without manual intervention. Auto Healing is easy to setup with BizTalk360 as, instead of having to develop custom scripts, it is just a matter of configuring Auto Healing on the required artifacts. This increases ease of use and minimizes downtime.
Do you want to read more about Auto Healing? Here you have few articles on this topic:
Introducing Auto Healing for BizTalk Environment
Automating BizTalk Administration Tasks via BizTalk360 Auto-Healing
Get started with a Free Trial today!
Why not give BizTalk360 a try. It takes about 10 minutes to install on your BizTalk environments and you can witness the benefits of auto-healing on your own BizTalk Environments. Get started with the free 30 days trial.
The post Why did we build Auto Healing capability in BizTalk Server Monitoring? appeared first on BizTalk360.
by Saravana Kumar | Jan 19, 2018 | BizTalk Community Blogs via Syndication
This blog is a part of the series of blog articles we are publishing on the topic “Why we built XYZ feature in BizTalk360”. Read the main article here.
Why do we need this feature?
BizTalk Server being a Middleware product connected to various legacy backend systems, it needs to make sure the entire ecosystem can work in an optimal way. If one of the legacy system connected to BizTalk is slow for any reason, then BizTalk Server needs to act sensibly and not to overload that system with messages more than what it can handle. In such scenarios, BizTalk Server will throttle itself (slow down itself) and make sure the messages are delivered to the backend in an optimal rate.
BizTalk Server achieves this capability by continuously monitoring various performance counters (memory footprint, thread count, message publishing rates, database size etc.) and self tuning itself. There are over 50 performance counters related to throttling in BizTalk Server which monitors both inbound and outbound traffic.
So it’s important to understand whether your BizTalk Environment is working correctly or under throttling condition.
What are the current challenges?
BizTalk Expert Required: If you wanted to analyse and detect a throttling condition in your BizTalk Environment, an experienced BizTalk person is required. There are no out of the box tooling from Microsoft to understand throttling conditions. The person should configure all the required Perfmon counters, collect data, analyse it and predict any throttling condition.
Time Consuming: Even if you have an experience person, setting up Perfmon counters, data collection and analysis is time consuming process. In our experience, it takes anywhere from a day to a week to fine tune Throttling in BizTalk Environment.
No Monitoring: There are no out of the box monitoring solution for BizTalk Throttling conditions.
How BizTalk360 Solves the problem?
BizTalk360 addresses this problem by introducing two important features called BizTalk360 Throttling Analyser and BizTalk Host Throttling Monitor.
Throttling Analyser periodically collects all the required Performance Counters relevant to throttling and stores the values in a SQL database, so there is no necessity for setting up Perfmon as and when required. Once the data is collected, it presents the status in a very interactive graphical dashboard. The BizTalk Administrators can easily visualize the throttling condition and take appropriate action. The throttling data is kept for last 7 days, so you can login to BizTalk360 anytime and see historic throttling condition as well.
Throttling Monitoring is the new addition to the product where the Administrator can enable BizTalk Host Monitoring in almost a single click or fine tune it according their requirements. Once configured, BizTalk360 will periodically check for any throttling condition violations and alert appropriate users via various notification channels like Email, SMS, Slack, etc.
Interested to try this feature?
Download and try BizTalk360 on your own environments free for 30 days. The installation will not take more than 5-10 minutes.
Author: Saravana Kumar
Saravana Kumar is the Founder and CTO of BizTalk360, an enterprise software that acts as an all-in-one solution for better administration, operation, support and monitoring of Microsoft BizTalk Server environments. View all posts by Saravana Kumar
by Lex Hegt | Nov 30, 2017 | BizTalk Community Blogs via Syndication
Application Health |
Application Health |
|
|
|
All products deal with core Application Health scenarios very well. Nagios loses points for less visibility when determining the health of an application. |
Platform Health |
Host Instance Availability |
|
|
|
All products deal with Host Instance health scenarios. Good visibility exists when determining the health of a Host instance. The higher rank goes to BizTalk360, because of the Auto-correct feature, for which customization is needed with Nagios and SCOM. |
Host Throttling |
|
|
|
All products provide capabilities when it comes to determining Host Throttling, although in case of Nagios, it will be monitoring of PerfMon counters. BizTalk360 however, provides a more comprehensive experience and allows you, besides Host Throttling monitoring, to historically go back to determine exactly when and why BizTalk was throttling. |
Single Sign On Service availability |
|
|
|
All products provide suitable functionality in this area. BizTalk360 gets the higher mark as it also has an Auto correct feature, which tries to bring the SSO service in the expected state, after a failure |
Monitoring SQL Agent Jobs |
|
|
|
All applications do provide this support. BizTalk360 provides a very friendly user experience, whereas SCOM provide deeper functionality that is found in the SQL Server Management Pack. Nagios takes the lower ranking as complex customization is needed. |
Core Infrastructure Monitoring |
|
|
|
Nagios and SCOM provide more Core Infrastructure Monitoring, but since we are keeping this relevant to BizTalk, BizTalk360 provides enough visibility to support a BizTalk environment and earns a top score as well. |
BizTalk Health Monitor Integration |
|
|
|
BizTalk Health Monitor (BHM) integration is part of BizTalk360’s core offering and is very easy to setup. No integration exists between BizTalk Health Monitor and Nagios and SCOM. Custom development is required in order to provide this functionality. |
Log/Database Query Monitoring |
|
|
|
All products receive partial scores for these features. Currently BizTalk360 provides no capability around parsing log files. It does provide support for SQL Server databases, but not others. Nagios and SCOM both provide support for both Log and Database Monitoring. The downside is that these features often require custom scripts to be written in order to support the requirement. SCOM loses marks on the usability aspects of implementing these functions, while Nagios loses marks as a separate product will be needed for Log monitoring. |
Analytics |
|
|
|
Nagios has no BizTalk oriented analytics. SCOM uses a data warehouse for reporting and long-term data storage. Because of the extended reporting capabilities, though not easy to use, SCOM gets the higher grade. BizTalk360 provides a customizable Analytics Dashboard and many widgets give insight in the performance and processing of messages through BizTalk. |
Operating Environment |
Process Monitoring |
|
|
|
This is a core feature in BizTalk360 that detects when something is supposed to happen but does not. An example is not receiving a file from a trading partner when you expect to. Your environment can be completely healthy, but if the trading partner does not provide the file, traditional Monitoring techniques used by SCOM will not detect this. BizTalk360 will detect this scenario and send the appropriate notifications. |
Maintenance Mode/Negative Monitoring |
|
|
|
Nagios supports Scheduling down time, but has no Negative monitoring for BizTalk Server resources. Both SCOM and BizTalk360 provide functionality in these areas. BizTalk360 provides complete coverage whereas SCOM is only providing partial coverage. SCOM is able to handle the Maintenance Mode requirements, but not the complete Negative Monitoring requirements. While a SCOM administrator can provide an override when it comes to a Host Instance or Receive Location, if someone starts or enables that BizTalk Service no notifications are generated. |
Synthetic Transactions |
|
|
|
Nagios has support for Synthetic Transactions and other types of web site monitoring. SCOM does provide additional features like running Synthetic Transactions from different servers within the environment. SCOM also provides deeper interrogation of these downstream applications by using .Net Application Performance Monitoring. BizTalk360 provides good capabilities that will satisfy most requirements. Besides checking for expected HTTP return codes, it is also possible to fire custom requests and check for certain responses/response times. |
Miscellaneous |
Integration with other popular Monitoring Platforms |
|
|
|
SCOM provides broad integration with many other Monitoring Platforms and Service Desk applications. BizTalk360 integrates with HP Operations Manager, Slack, ServiceNow, Microsoft Teams and provides a simple and elegant way of doing so. One could also develop custom Notification Channels to integrate with other Service Desk applications. Nagios has slightly less features in this area and therefore gets the lower score |
Composite Dashboards |
|
|
|
All products provide this capability. BizTalk360 gets a higher grade for the simplicity of the tool. In order to build these types of Dashboards within SCOM, you need to be an advanced user of the system. Nagios Dashboards are not BizTalk oriented |
Web based User Interfaces |
|
|
|
All products provide Web based interfaces. BizTalk360 gets the top grade because administrators can perform most of their BizTalk related activities with just BizTalk360, while with Nagios/SCOM they also need access to tools like BizTalk Admin Console, SQL Server, Event Viewer, Performance Monitor, portals, etc. |
Summary Reports |
|
|
|
Nagios has all kind of reports, but these are not BizTalk oriented SCOM does not provide a holistic report out of the box. In order to get this type of reports out of SCOM, you need to custom build a report. BizTalk360 provides a scheduled Summary Report that will give subscribers an indicator that their environment is Healthy. This gives users a confirmation that everything is operating as expected within the environment. |
Governance |
|
|
|
Nagios and SCOM provide both user management, but the no BizTalk operations can be done from these users. BizTalk360 provides a much finer grained approach to Governance and Auditing. |
Knowledgebase |
|
|
|
Nagios doesn’t have a Knowledge Base, but both SCOM and BizTalk360 include the ability to build up a Company Knowledge base. SCOM also includes a Product Knowledge base provided by the BizTalk Product Group so it gets the higher score in this area. BizTalk360 provides the capability to associate KB articles to certain events, like error codes of suspended instances etc. and therefore also deserves the highest rank. |
by Sivaramakrishnan Arumugam | Nov 1, 2017 | BizTalk Community Blogs via Syndication
BizTalk360 is the one-stop monitoring tool for monitoring BizTalk server. It is possible to configure more than one BizTalk environments and monitor them in a single instance of BizTalk360. Recently in BizTalk360 technical product support, we received quite a few tickets regarding the BizTalk Server upgrade process. Some of the customers would like to test if their newly configured BizTalk environment is working as excepted and running without any issues. Also, the customer would like to test the new server using BizTalk360 as they like to monitor in future.
The customer scenario:
Most of the customers have a dedicated BizTalk360 server for their BizTalk environments. The scenario what customer required in the support ticket was to have two different environments configured on the same BizTalk360 machine for some days and then switch them when they go live onto the new environment. They just wanted to check if it is possible to setup two environments on the same BizTalk360 installation. And if possible, they requested us to provide a temporary license for migration.
One of the common queries from the customers is that, is it possible to monitor two different BizTalk environments in a single instance of BizTalk360. For example, the customer would like to monitor the old BizTalk Server 2013R2 environment and new BizTalk Server 2016 environment within a single BizTalk360 installation.
What happens in BizTalk server?
It is not possible to monitor non-identical versions of BizTalk environments like BizTalk Server 2013 R2 and BizTalk Server 2016 within a single BizTalk360 installation. This cannot be achieved even within the BizTalk server. When you try to connect the existing database of BizTalk server 2013 R2 from the BizTalk server 2016, you may face the below exception.
In the above error message, the 5th point explains the current scenario. It says the database is not compatible with the version of BizTalk server currently running. This message clearly tells the mismatch between the different BizTalk versions installed. The database of BizTalk server 2016 would not be compatible with BizTalk server 2013 R2.
At the same time when you connect to the BizTalk server 2013R2, you will not face any problem in connecting to the existing database.
The reason behind this is, when you try to connect the existing database, it will verify the table BizTalkDBVersion in BizTalkMgmtDb database to confirm if the database version and the application version are same. If there is a difference it will throw an exception while connecting.
This is what happens in BizTalk360:
Let’s take a brief look within BizTalk360. During the installation of BizTalk360, one of the primary prerequisites is to install BizTalk Server components on BizTalk360 server. Once after the installation of these components, BizTalk360 installer allows you to proceed the installation on top of it. BizTalk360 directly calls the ExplorerOM to take any action against the BizTalk server artefacts. When you install BizTalk server components inside the BizTalk360 machine it will behave in the same way as the BizTalk server.
When you are trying to add another environment with different BizTalk Server version you will face the same exception and BizTalk360 will convey the exact problem with the below exception message.
When you have multiple environments configured in BizTalk360 and if you are upgrading the BizTalk server, you must upgrade BizTalk servers in all the environments with the same version. If any machine is missed without upgrading, the below exception will appear while launching BizTalk360 UI.
BizTalk360 works with any BizTalk version but it requires the Administration Tools from BizTalk to access the BizTalk API’s and since you cannot install multiple BizTalk version in one server you need to have separate BizTalk360 installations for each BizTalk version that you need to target.
What about the BizTalk admin components:
As mentioned earlier, BizTalk360 contacts BizTalk server through ExplorerOM with the help of the BizTalk admin components installed. Hence when we upgrade the BizTalk server in the server machines, we must also upgrade BizTalk admin components in the BizTalk360 machine. In BizTalk360 database, there are two tables in which the BizTalk version is stored, namely
- b360_admin_BizTalkEnvironment -> BizTalk Version
- b360_admin_GlobalProperties -> MS_BIZTALK_VERSION & MS_BIZTALK_INSTALL_LOCATION
So, when the BizTalk server and admin components are upgraded, the above-mentioned values must be changed manually in the BizTalk360 database for the new version to come into effect. The steps to be followed are:
1.We must deactivate the BizTalk360 license.
- Then upgrade BTS server & then admin components on the BT360 server (if it is standalone).
- Then if we activate BizTalk360 license it should pick up the new BTS version.
- Then update the BizTalk server version details in the two BizTalk360 database tables.
Conclusion:
But however, BizTalk360 can monitor multiple BizTalk environments with the same BizTalk server version. Hence, if you want to monitor different BizTalk server versions, there must be different installations of BizTalk360 for each BizTalk version with the respective BizTalk version components installed on the BizTalk360 machine. This way the database incompatibility error will be avoided. But we can monitor multiple BizTalk environments of the same version within the single instance of BizTalk360.
Author: Sivaramakrishnan Arumugam
Sivaramakrishnan is our Support Engineer with quite a few certifications under his belt. He has been instrumental in handling the customer support area. He believes Travelling makes happy of anyone. View all posts by Sivaramakrishnan Arumugam
by Venitta Priya | Sep 6, 2017 | BizTalk Community Blogs via Syndication
What is Web Endpoint?
In simple terms, a web service endpoint is a web address (URL) at which customers of a specific service can gain access to it. By referencing that URL, customers can get to operations provided by that service. The endpoint is a connection point where HTML files or active server pages are exposed. Endpoints provide information needed to address a Web service endpoint.
Why Web Endpoint:
Typically, in a BizTalk environment, an application needs to associate with external web endpoints to monitor and validate the health of the web endpoint. BizTalk360 allows the administrators to set up web endpoints monitoring for the web services in the solution and validate the actual response code against the expected return code.
Before version 8.1, web endpoint monitoring had the capability to manage the external endpoints using options like to provide authorization credentials and with a proxy. While monitoring the endpoint, if there is any mismatch between the expected return code and actual return code, the system will alert to the administrator with the information. In 8.1 version, we have improved with some more interesting options like Payload and Custom HTTP headers.
Basic Detail
In basic detail, we can give the Endpoint name which is going to be monitored. In Endpoint URL, we can give the URL of the web endpoint for ex. www.google.com
Optional details configuration
In this section, we can have a deep look at the new optional settings available for monitoring the endpoint. This is an optional section, you can skip this configuration.
Authorization credentials:
Web endpoint will require Username and Password for monitoring. In that scenario, we can utilize this option. For ex, if we need to access remote machine URL, in this case, we need to specify the domain name, user name, and password. If you can monitor the endpoint without any issue, then it will be healthy else it will return an error message.
Proxy:
Some organization will be using a proxy, in that case, you can give the proxy details to monitor the endpoint. In you enable the toggle button near the Use Gateway Proxy it will fetch the proxy details which we have given in the settings -> gateway settings. Global gateway settings can be overridden by giving the below details
- Server Name– the name of your proxy server
- Port Number– the port number used to associate through your proxy
- Proxy username– the username to associate with the proxy server
- Proxy password– the password to authenticate the credentials to associate with the proxy server
Payload:
When data is sent over the Internet, each of the transmitted unit includes both header information and the actual data being sent. The header identifies the source and destination of the packet, while the actual data is referred to as the payload. The part of the message or code that carries the data. In a key-length-value structure, the key and length are descriptive data about the value (the payload). Here we can configure the parameters of GET or POST methods. Some Endpoints cannot be monitored without giving additional parameters. We can retrieve those information’s using payload and monitor those API in web endpoints.
Custom header:
HTTP headers allow the client and the server to pass additional information with the request or the response. You can give custom headers along with the endpoint which you are going to monitor. (Ex: Accept-Charset, Accept-Language, Allow, Authorization, Content-Language)
Response alert configuration
Here comes the response alert configuration section, you can see three type of response format. We will see one by one.
Response Format:
Plain text: In this type, you can see Keyword alert where you can specify the keyword which will be available in the endpoint which you are monitoring
Ex. Let’s go with www.google.com
If you mention the keyword as Success and if it is present in the endpoint the status will be healthy based on the conditions.
XML: Here you can see _XPath Alerts, you can even monitor your endpoints using XPath.
Ex. You have an endpoint with XML data and you need some specific information in that case you can give XPath and monitor the endpoint. Say, your endpoint is having information about employees and you want to fetch the name of the 3rd employee, at that time you can give as
//employees/employee/lastName Jones
Example:
<employees>
<employee>
<firstName>John</firstName> <lastName>Doe</lastName>
</employee>
<employee>
<firstName>Anna</firstName> <lastName>Smith</lastName>
</employee>
<employee>
<firstName>Peter</firstName> <lastName>Jones</lastName>
</employee>
</employees>
If the given XPath values are true, then endpoint will be healthy else it will show error/ warning based on the configurations.
JSON: Same as XPath you can see JSON path alerts, and monitor the endpoint using JSON path. Say, for an example, an endpoint with Json data and looking for some specific data – 3rd employee:
{“employees”: [
{ “firstName”:”John”, “lastName”:”Doe” },
{ “firstName”:”Anna”, “lastName”:”Smith” },
{ “firstName”:”Peter”, “lastName”:”Jones” }
]}
$. employees [2]..firstName peter
If the given Json path values is true, then endpoint will be healthy else it will show error/ warning based on the configurations.
Return Code Alert:
Endpoints can be monitored based on the return codes. If you set return code as 200 and while monitoring if the endpoint does not have the expected return code. then it will indicate error/warning based on the configuration else it will be healthy.
Response time Alerts:
The same way we can monitor the endpoints using response time. We can monitor the delay of the endpoint using this setting. We can change the response time based on the necessity.
Based on the configuration we have provided we will get the notification in email.
Conclusion:
So I hope this blog gave you some idea about how you can monitor Web Endpoint in BizTalk360. If you have any questions, contact us at [email protected]. Also, feel free to leave your feedback in our forum.
by Saravana Kumar | Aug 2, 2017 | BizTalk Community Blogs via Syndication
What is BizTalk Host Throttling?
BizTalk Server being a Middleware product connected to various legacy backend systems it needs to make sure the entire ecosystem can work in an optimal way. If one of the legacy system connected to BizTalk is slow for any reason, then BizTalk Server need to act sensibly not to overload that system with messages more than what it can handle. In such scenarios, BizTalk Server will throttle itself (slow down itself) and make sure the messages are delivered to the backend in an optimal rate.
BizTalk Server achieves this capability by continuously monitoring various performance counters (memory footprint, thread count, message publishing rates, database size etc.) and self tuning itself. There are over 50 performance counters related to throttling in BizTalk Server which monitors both inbound and outbound traffic.
BizTalk360 Throttling Analyser
One of the challenges for BizTalk Server Administrators when it comes to BizTalk Throttling is, there is no out of the box tooling from Microsoft to understand whether your BizTalk Environment is working efficiently or under throttling condition. You only have raw performance counters to measure throttling. Typically the BizTalk Administrators open up Windows Perfmon tool and add all the performance counters related to Throttling record and analyse throttling conditions. This requires extensive knowledge about how BizTalk Server works, various throttling counters & conditions, whether it’s running on optimum level etc.
To address this issue, about 2 years ago we introduced “Throttling Analyser” in BizTalk360. Once enabled, BizTalk360 continuously collect all the throttling related performance counter data in our database and provide an intuitive user interface with highly interactive graphs to showcase whether the BizTalk Environment is working efficiently or under throttling condition. This saves a lot of time for BizTalk Administrators to understand environment throttling condition and the biggest advantage is you do not need to have in-depth knowledge about BizTalk Server internal architecture and throttling mechanism.
BizTalk360 Throttling Monitoring
The Throttling Analyser explained in the previous section only gives you the visual representation of the BizTalk Environment throttling condition. The BizTalk Administrator need to periodically log in to the system to see if the environment is healthy. However, we can clearly see a value in alerting the BizTalk Administrators if the environment is suffering from any critical throttling conditions for a perceived time.
That’s exactly what we have done with “Throttling Monitoring” in version 8.5 of BizTalk360. We wanted to make the experience super simple and intuitive.
How does it work?
Let’s take a close look at how this functionality is designed and how it works.
Under BizTalk Environment monitoring section we introduced a new category called “Host Throttling”, which by default will list out all the BizTalk Hosts that’s currently configured in the environment as shown in the below picture.
Enabling BizTalk Host Throttling monitoring in single click
You can enable default throttling for all the BizTalk Hosts in a single click, you simply select the hosts you wanted to monitor and click the button “Enable Throttling”, this will start monitoring the BizTalk Environment for any throttling violation that persist for 60 seconds.
The whole idea for us is to make it as simple as possible to monitoring BizTalk Throttling condition, hence we provide the option to just enable the default monitoring with 60 seconds persistence in a single click.
The BizTalk Administrator can easily tweak the default configuration by clicking on the “Edit” button and changing the parameters. As you can see from the below picture, you can choose to monitor specific throttling condition or you can have multiple conditions with different persist duration etc, you can also have different monitoring options for publish side and delivery side.
For Publish Throttling, the user can able to monitor the following metrics:
- Any throttling
- 2 – Rate Throttling
- 4 – Process memory
- 5 – System memory
- 6 – Database size
- 8 – Database session
- 9 – Thread count
- 11 – User override
For Delivery Throttling, the user can able to monitor the following metrics:
- Any throttling
- 1 – Rate Throttling
- 4 – Process memory
- 5 – System memory
- 9 – Thread count
- 10 – User override
Notification of Throttling violation
Once the settings are configured to look out for throttling conditions, BizTalk360 will keep monitoring the environment. If BizTalk360 detects any threshold conditions violation, it will notify the users via configured notification channels. In BizTalk360, there are various notification channels like Email, Slack, ServiceNow, SMS, Event Viewer, Web Hook etc
Email Notification |
Slack Notification |
|
|
Summary
Threshold monitoring is one of the key features we have introduced and it address one of the important areas of BizTalk Server Operations and Monitoring. This will help customers to keep an eye on the capacity of their BizTalk Environments in near real time and take appropriate actions.
You can write to us at [email protected]. Have a try at our latest version by downloading a 14-day free trial of BizTalk360.
Author: Saravana Kumar
Saravana Kumar is the Founder and CTO of BizTalk360, an enterprise software that acts as an all-in-one solution for better administration, operation, support and monitoring of Microsoft BizTalk Server environments. View all posts by Saravana Kumar
by Senthil Palanisamy | Jun 20, 2017 | BizTalk Community Blogs via Syndication
Introduction
On a day to day basis, a BizTalk administrator must perform few monotonous activities such as terminating instances, enabling receive locations, ensuring the status of SQL jobs etc. BizTalk360 has few powerful features which help you to automate such monotonous tasks. These features are hidden gems and are overlooked by many BizTalk360’s users, despite the availability of a good documentation. That prompted me to start my new blog series called “Automating BizTalk administration tasks using BizTalk360”. In this blog series, I will be explaining these automation capabilities which BizTalk360 brings to its users.
To start off with in this first blog I am focusing on “Data Monitoring Actions”.
What is Data Monitoring in BizTalk360?
As we are aware, BizTalk collects a diverse set of data into message box database, tracking database, BAM primary import and ESB databases. BizTalk360 brings all these data into a single console and on top of that provides a powerful capability to set alerts based on various thresholds. This feature is called data monitoring. Below is a screenshot that shows all different data sets which can be used in data monitoring feature.
Below table briefly explains various types of data items which could be monitored.
Data monitoring category
|
Explanation |
Process Monitoring |
With process monitoring you will be able to monitor the number of messages being processed by receive ports, send ports. This is popularly also called as “non-event monitoring”
Ex: if you want to alert when less than 50 messages received in an hourly window during business hours, then process monitoring is the best fit.
Refer the assist article Process Monitoring for more information.
|
Message Box Data monitoring |
With this you will be able to set alerts on the number of suspended, running, dehydrated messaging instance.
Refer the assist article Message Box Data Monitoring for more information.
|
Tracking Data Monitoring |
With this you can set alerts on tracked messaging events and tracked service instances.
Refer the assist article Tracking Data Monitoring for more information.
|
BAM Data Monitoring |
With this you can set alerts on the data stored in BAM tables.
Refer the assist article BAM Data Monitoring for more information.
|
EDI Data Monitoring |
With this you can set alerts on the EDI and AS2 reporting data stored in BAM tables.
Refer the assist article EDI Data Monitoring for more information.
|
ESB Data Monitoring |
With this you can set alerts on the ESB data and exceptions stored in BAM and ESB tables.
Refer the assist article ESB Data Monitoring for more information
|
Logic Apps Metrics Monitoring |
With this you can set alerts on metrics emitted by Logic apps.
Refer the assist article Logic Apps Metrics Monitoring for more information
|
Message Box Data Monitoring Actions
In Message Box Data Monitoring, the user can configure the queries to monitor service instances and messages. Monitoring service will send the notification to the users whenever the service instances/Messages count violates the threshold condition.
Message Box Data Schedule can be configured in Data Monitoring > Message Box Data. It can be scheduled at the different frequencies (Daily, Weekly, and Monthly) based on the volume and priority to take the action on service instances/messages.
Query Condition
BizTalk360 provides Highly advanced query builders for selecting the precise and expected suspended instances based data-result. While querying the suspended/All In-Progress Service Instances you can apply the filters like Error Code, Application, Service Class, Service Name etc.
Context Properties
A Message-Context based query is been provided by BizTalk360 for higher business-friendly scenarios. In Message payload, context/promoted properties can be selected to know the transactional message. In Data monitoring schedule the user can choose which context promoted properties to be in an email alert.
Action on Service Instances
The operational user must closely watch the suspended service instances to act on. It is a tedious process to look after all the time. Message Box data monitoring feature will take automatic action on service instances when the set actions are configured in our schedule. The monitoring service will Terminate/Resume the service instances based on either error or warning condition which doesn’t require any manual intervention.
Archiving & Downloading the Message Instances
Message content & context is required for auditing or other purposes of reconciliation. If you have not enabled the tracking option, it is not possible to get hold of the data again. Keeping this in mind, we have implemented archiving the message context when setting the action is taken on instances. In BizTalk360 Settings>System Settings, Archive and Download location of message instances must be configured to archive and download the message instances. Automatic actions with desired backup steps are been taken to make sure all the data are preserved before taking any action.
Note: In order take action on suspended service instances the monitoring service account has to be created as superuser in BizTalk360.
In Data Monitoring dashboard, every monitoring cycle status is shown. When the user clicks on the status tab, it will bring the details about the Query result, Task Action and Exception summary.
In Task Action tab, you can download each instance separately or by using “Click here” button you can download all the instances to the server location. Service Instances messages are download in server location as Zip file with activity ID for the monitoring run cycle.
Conclusion
Data Monitoring, an Auto-Monitoring feature of BizTalk Administration which can take corrective actions with all backup steps in the event of any threshold violations. With just a one-time setting we have our BizTalk360 to make sure all your routine tasks are addressed without a manual intervention. Also, BizTalk360 offers much more monitoring features which will enable all administrators to be pro-actively monitoring the BizTalk environment(s). Next article will see the Auto correction on BizTalk Artifacts and Logic Apps.
Author: Senthil Palanisamy
Senthil Palanisamy is the Technical Lead at BizTalk360 having 12 years of experience in Microsoft Technologies. Worked various products across domains like Health Care, Energy and Retail. View all posts by Senthil Palanisamy