After the installation of BizTalk360, the implementation phase starts. One of the steps you will want to do is taking benefit of the rich automated monitoring features the product provides. For that, you must set up alarms in BizTalk360. In this article, we intend to give you some best practices around Alarm Configuration. Firstly, however, we’ll shortly explain the different monitoring concepts which are available in BizTalk360.
A short explanation of the supported monitoring types
BizTalk360 has multiple types of monitoring, which are Threshold monitoring, Health Monitoring, and Data Monitoring. Each alarm you create in BizTalk360 can support all types of monitoring, either as separate alarms or as combined alarms.
As you can see from the above screen, BizTalk360 enables you to configure an alarm for multiple purposes. So, in case you want the same people to receive the same notifications for both Thresholds as for Health Checks, you only need to create one alarm.
With this kind of monitoring, you get notified in case a threshold occurs. For example, that can be a Receive Location which was Disabled, while it should be Enabled or any other artefact which is mapped in an alarm and is in an unexpected state.
At the Alarm creation level, you can configure the following:
- Name – each alarm needs to be given a name. The more descriptive the name, the easier it is for identifying the purpose of the alarm
- Description – optionally, you can provide a detailed description of the purpose of the alarm
- Email Ids – each alarm can send notifications to multiple recipients
- High Priority – in case you want specific notifications (for example when anything goes wrong with the BizTalk platform) to jump out in your email box, you can enable the High Priority switch
- Violation persistence – to configure after how much time and with which interval you want to be notified of exceptions
- Limit the number of notifications per exception – to prevent you from being bombarded with notifications and lose the overview as a result
- Back to Normal notification – receive a notification when the alarm no more contains artifacts in the wrong state
- Alert reset – start receiving notifications again after a specific time in case you have received the maximum number of notifications
- Alert schedule – configure when the alarm should perform monitoring. Handy when you know that specific artifacts are not available all the time, due to, for example, maintenance
It can be helpful if you receive a report with the actual status of the monitored artifacts, for example at 9AM every business day. For such scenarios, you can configure alarms for Health monitoring.
BizTalk360 Health Monitoring alarms allow you to easily configure when you want to receive these status reports via an intuitive interface.
Below screenshot shows the screen where you can select when you want to receive these status reports.
This alarm has not only been configured to send a report at the beginning of each working day but also towards the end of the business day. This helps in being aware of the status of your environment before you leave home.
A detailed explanation of Data Monitoring falls outside the scope of this article, therefore we provide a somewhat better overview of its capabilities here.
The concept of Data Monitoring allows you to set up monitoring of your data flows. As an example, your BizTalk platform and applications might be in perfectly good health, but when a misconfigured firewall prevents messages from arriving in BizTalk, no processing will be taking place. This interrupts your business processes, with maybe bad consequences for your company. Data Monitoring can be used to monitor this kind of scenarios.
Data Monitoring provides monitoring at the following levels:
- Process Monitoring – monitor whether your processes run as expected
- MessageBox Monitoring – monitor the active processes and (optionally) take automated actions
- Tracking Data Monitoring – monitor completed processes
- BAM Monitoring – monitor your deployed BAM Views and Activities
- EDI Data Monitoring – monitor your EDI processes
- ESB Data Monitoring – monitor your ESB Exceptions and resubmissions
- Logic Apps Monitoring – monitor tens of metrics about the processing of your Azure Logic Apps
- Event Log Monitoring – monitor your consolidated Event Log entries of your BizTalk and/or SQL servers
After selecting which kind of Data Monitoring you want to set up, it now comes down to configure what exactly you want to monitor and when monitoring needs to be done.
Below screenshot gives an example of Process Monitoring.
Once you have set up Data Monitoring, the output of all the different monitor runs will be shown in a consolidated Data Monitoring dashboard. This dashboard helps in answering the questions from your business/functional users whether a specific process has taken place.
You can refer to the Data Monitoring section in the Documentation portal for a more detailed explanation of this way of monitoring in BizTalk360.
Some more information about BizTalk360 Alarms can be found here.
BizTalk platform versus BizTalk application monitoring
When setting up monitoring in BizTalk360, it is handy to make a difference between monitoring the actual BizTalk platform and the BizTalk applications which are deployed on it. Making this distinction gives several advantages, like:
- Notifying different people, depending on their role and needs
- Use different monitoring settings like:
- The monitoring interval for early notifications in case of important issues
- The number of notifications you want to receive per exception
- Receive a ‘back to normal’ notification in case everything is healthy again
- Set the High Priority flag to easily spot important notifications in your email box
- Run PowerShell scripts for immediate actions
Even within platform monitoring, you can make segregations for different administrators. For example, a SQL Server DBA has different needs than a System (Windows) Administrator, so other artefacts need to be monitored. BizTalk360 enables you to set up different Alarms for different needs.
What to setup for Platform monitoring
When you are creating an alarm for platform monitoring, you want to be notified of unexpected situations which might occur on your BizTalk platform. For this purpose, you will create a Threshold Monitoring Alarm. If you also want to receive a daily status report at say every business day at 8:00 AM, you can configure an alarm for Health Check Notification.
See below for a list of artefacts you could consider for monitoring BizTalk platform health:
- Host Instances (clustered and non-clustered)
- Host Throttling
- BizTalk Server availability
- SQL Server Jobs
- Windows NT Services
- Enterprise Single Sign-On service
- Internet Information Services
- BizTalk Health Monitor output
- Eventlog entries
- BizTalk Criticals, Errors, and Warnings
- SQL Server Criticals, Errors, and Warnings
- ESSO Criticals, Errors and Warnings
- Internet Information Services Criticals, Errors, and Warnings
- Server restarts
- Certificate expiration
- BizTalk and SQL related stuff
- Disk space
- CPU/Memory usage
- Windows NT Services
Below screen shows the Monitoring Dashboard for a Platform alarm.
What to setup for BizTalk Application monitoring
Before we move to discuss which artifacts to monitor from BizTalk Application alarms, let’s firstly discuss how to organize your BizTalk Application-oriented alarms. Basically, there are two ways you can set up your BizTalk Application alarms.
The Alarm equals Application approach
When deploying BizTalk applications, it is often handy to create an alarm per BizTalk application for Threshold Monitoring and/or Health Check Notification. The main motivation for this is, that a BizTalk Application is the unit which you will deploy, so setting up alarms around them will help in keeping the overview. For easy reference, you could give the alarms the same name as the BizTalk application it will be monitoring. This can be considered as a bit more technical approach for your monitoring.
The Alarm equals Interface approach
Another approach to set up your BizTalk Application-oriented alarms is around the interfaces which are being processed by BizTalk. Say, your BizTalk environment contains multiple applications of which several their contained artifacts together are part of a Purchase Order. To keep the overview of such an interface, it makes sense to monitor all artifacts related to that interface. This is totally fine and BizTalk360 allows you to set up your alarm in such a way. Ideally, you can give the alarm the name of the interface. This approach is a bit more business-wise way of monitoring and can be helpful if you need to notify your business/functional users in case of any issues.
However, there is a downside to monitoring your BizTalk artifacts around interfaces. Because artifacts from multiple BizTalk Applications need to be monitored, it might be a bit harder, especially when many artifacts are involved, to keep the overview and be sure that you are really monitoring all the relevant artifacts.
No ‘Always Right’ way
Summarizing, there is no ‘always right’ way to set up your BizTalk Application monitoring. Even a ‘hybrid’ setup, which follows both just mentioned practices, of your monitoring is totally fine and not uncommon. In the end, it all depends on what works the best for you and your organization. BizTalk360 gives you all the freedom you need.
Artifacts to be monitored
When setting up monitoring for your BizTalk Applications, you will set it up for your ports and orchestrations of your BizTalk Applications. However, you can also consider the following artefacts for monitoring:
- State of Services Instances
- Suspended (Resumable/Not Resumable)
- Ready to Run
- Endpoints of your Receive/Send ports
- Web Endpoints
- Queues (MSMQ, IBM MQ, Azure Service Bus)
- File shares
- FTP sites
- API Apps
- Logic Apps
Tip: To be able to find out if files/messages are being picked up, you can monitor your File shares, FTP sites, and Queues. Below screen shows an example of monitoring a File share.
As a side note, BizTalk Server Best Practices suggest that you have a proper Host configuration in place, where separation takes place based on receiving/sending messages, processing of orchestrations and tracking of processed messages/service instances. However, sometimes Host configuration has been set up for a specific process or BizTalk application. In that case, it is valid to monitor your Host Instance(s) in your BizTalk application-oriented alarm, instead (or on top) of your BizTalk platform alarm.
Advanced Monitoring setup
Besides deciding on how to setup your alarms and which artifacts to monitor, there are a couple of other topics you could consider take benefit of. You can think of:
- Auto-Correct – the ability to automatically recover artifacts
- Notification Channels – inform people not just via email but also via different channels
- Customized Notification Templates – provide additional information or brand the templates
Using such features not just makes your life as an administrator easier, it will also help you in making the best of your investment in BizTalk360. Let’s have a better look at all these options.
For state-related artefacts like BizTalk Ports and Orchestrations, Host Instances and SQL Server jobs, you could consider using the Auto-Correct feature, which, after a threshold situation has been met, tries to get the artifacts back to the expected state.
Tip: You can use this feature to automatically Enable your Receive Locations, in case they failed to connect to their endpoints due to a temporary failure!
You can read more about the Auto-Correct feature in this article.
Below screen shows that BizTalk360 will try to automatically try to recover the Host Instances once they are no more in the Started state.
The default way of BizTalk360 for sending notifications is via email. However, sending notifications does not end there. The product has a number of so-called Notification Channels you can use. Examples are:
- Slack – send notifications to a Slack channel
- ServiceNow – have tickets created, directly in ServiceNow
- Microsoft Teams – send notifications to a Teams channel
- PowerShell – trigger a PowerShell script to perform an action
- SMTP – have more control over the way notifications are being sent
- Webhook – connect to an already existing REST API
Besides this rich set of notification channels, there is also an easy to use API at your convenience. By implementing the API, you can, for example, connect to your ticketing system or other important backend systems.
Below screen shows the information you can provide when using the ServiceNow Notification channel.
More information about all the Notification channels can be found here.
Customized Notification Templates
BizTalk360 allows you to customize the notifications it sends. This allows you for example to provide some additional information/instructions for the people who receive the notifications. However, it can also be used for branding purposes.
Although some notification information can simply be provided via text input fields, the actual templates are based on XSLT, so some skills with editing such files will be helpful.
More information about customizing the notification templates can be found here.
With this article, we intended to provide you some help and guidelines with setting up your alarms. Both the Documentation portal and our Blog contain many articles on monitoring your BizTalk environment with BizTalk360. However, if you feel you need some more support with setting up your monitoring, feel free to reach out. We are here to help you!
The post Best Practices for BizTalk360 Alarm Configuration appeared first on BizTalk360.
From time to time we go on a call with our customers because they raised a support ticket, or simply to explain a particular feature of the product. During such sessions, in most cases, the Operations Dashboard is the first screen which will show up after starting BizTalk360.
Unfortunately, quite often we find that dashboard in its out-of-the-box status. In this article, we will explain taking maximum benefits of the Operations Dashboard.
The default dashboard looks similarly like shown below.
Of course, it is up to our customers to leave the BizTalk360 Operations Dashboard to its defaults. However, there is a big opportunity to make the Operations Dashboard much more informative about your BizTalk environment. In this article, we will show some examples on how to improve the Operations Dashboard.
What is the Operations Dashboard?
The Operations Dashboard, the Home Page of BizTalk360, is designed to provide its users with valuable information about their BizTalk environment. The advantage is that each user has its own homepage and can shape it to his/her own requirements. For example, you can think of showing information in the following categories:
• Suspended/Running Instances
• Status of BizTalk artifacts
• EDI Interchanges and Transaction Sets
• ESB Faults, Itineraries and Resubmissions
• Customized information based on REST API’s
• Shortcuts to BizTalk360 features
All the information is shown in, what we call, widgets. For most widgets, you can decide on matters like:
• Location on the dashboard
• Size of the widget
• Refresh interval of the widget
In summary, by choosing the relevant widgets from the available set of widgets, rearranging and resizing them according to your needs, you can create a highly usable and effective dashboard which gives you a good overview of the health of the environment.
You can read more about Widgets and Widget Operations in the Documentation Portal of BizTalk360.
Before I joined Kovai.co, I was a BizTalk consultant and mainly had developer and administrator roles at many different companies. During that time, I frequently helped these customers out by installing and implementing BizTalk360. Obviously, setting up the Operations Dashboard was a part of that kind of assignments.
In many cases, I ended up with a dashboard like the one which is shown below.
Above dashboard is divided into the following 5 different sections:
1 – Shortcuts to BizTalk360 features
As BizTalk360 has grown over the years to a very rich-featured product, it can be hard to find your way through all the features. Because we want to make navigation throughout all the features a bit easier, we have added pinpoints to most of the features. By clicking on such a pinpoint, a shortcut to the feature is added to the Operations Dashboard. This first section contains several such pinpoints.
2 – Status of BizTalk artifacts
Part of providing an overview of your BizTalk environment is being aware of the status of your BizTalk artifacts. You can think of your Receive/Send Ports, Orchestrations, Host Instances and depending on your scenario, also the EDI Parties & Agreements. BizTalk360 helps you in providing the status of these artifacts in one glance, after you have added these artifacts from the widget library to the dashboard, resize them and drag them to the position you want them to show up.
3 – Some miscellaneous widgets
This section shows the following 2 widgets:
• Monitoring status – this widget reflects the status of the BizTalk360 Monitoring Service. This Windows NT service is used for amongst others performing the actual monitoring, sending notifications, etc. Because this is such an important component, it makes sense to show its status on the dashboard.
• Link to the documentation portal – to help our customers, we put a lot of effort into the maintenance of the documentation portal. Besides topics like Release Notes for all its releases, you will also find detailed information about all the features of the product. That’s why it is handy to have this portal easily accessible from the dashboard.
4 – BizTalk runtime information
Another important category of information to have at hand is the runtime information about Running and Suspended instances. BizTalk 360 contains widgets for both categories and for aggregations like Service Class, Service Name, Error Code, Host Name, BizTalk Application and few more. These different aggregations enable you to take the aggregation you prefer the most.
Besides the Runtime information, the section also shows the BizTalk Environment Properties, which you will find handy to have at hand.
5 – Alarm information
This last section shows information about the two different kinds of alarms. With BizTalk360, you can setup Alarms, being:
• Alarms – used for threshold and health monitoring. The widget shows information about the number of created alarms, how many are Enabled/Disabled, how many alarms contain mappings to (BizTalk) artifacts and how many still need to be mapped to (BizTalk) artifacts
• Data Monitoring – used for monitoring the processing of messages by BizTalk. This widget shows the number of monitors created for each type of monitor.
The purpose of these widgets is to provide the insight if, for example, alarms have been turned on/off for maintenance or few alarms are just empty containers and still need to be mapped to artifacts.
The described set of widgets is just an example. If you feel like some of those widgets are not helpful for you, it is totally fine to select different widgets from the library.
In the Documentation Portal, we have written an article about creating a Custom Widget for showing the results of a SQL query. We have seen that this is something quite useful. Read this article in the Documentation portal, or this blog post, if you want to know more about how this is done.
Global and private dashboards
The Operations Dashboard is the home page for the BizTalk360 user. However, having one dashboard might not be enough for your requirements. You might need to show a lot of different information on a dashboard, and having that information on a single dashboard would make it messy and you’ll lose the overview.
To help you out, you can create additional dashboards in BizTalk360. Additionally created dashboards show up in the Operations menu, under Dashboards.
In addition, you can choose whether a dashboard is Global or Private.
A Global dashboard can be accessed by anyone who has access to BizTalk360. A Private dashboard can only be accessed by the BizTalk360 user who created the private dashboard.
User Policy limited access to information
The data which becomes shown in the dashboards depends on the permissions which are provided to the user. This means that:
- there are no limitations for a Super User
- a Normal User will only have access to the information according to his/her User Access Policy. A few examples:
- if the user has no permissions to EDI, the EDI related widgets will not show up nor be accessible for that user
- information about BizTalk Applications (Ports, Orchestrations and Service instances) for which the user has no permissions, will not show up in the dashboards
With this article, we intend to encourage you to think about your Operations Dashboard. It is not hard to customize your dashboard and you will benefit from a nicely customized dashboard immediately!
If you want to discuss your Operations Dashboard or need some help to properly set it up, feel free to contact us at firstname.lastname@example.org.
The post Taking Maximum Benefits of the Operations Dashboard appeared first on BizTalk360.
From time to time, in conversations we have with our prospects and customers, the question pops up if additional licenses needed for BizTalk components on separate servers. You can install BizTalk360 on a BizTalk server, but also on a separate server on which no BizTalk processing takes place. Normally, we recommend to install BizTalk360 on a separate server. This has advantages like not using resources which BizTalk can use for processing and (equally important) to prevent not able to access BizTalk360 when the BizTalk server goes down.
For BizTalk360 to work, it needs the so-called BizTalk Server Administration Tools. You need to install these tools on the server which will run BizTalk360, as they contain the APIs which BizTalk360 uses to access BizTalk Server.
These APIs, being used by third party products like BizTalk360, is not the only reason why an organisation might want to install these components. These APIs are part of a set of components and tools, you might want to install on a separate server. In the BizTalk Server installer, this set can be found under the category Administration Tools and Monitoring and Additional software, as you can seen in below screenshot.
The set of Additional BizTalk Server software
The Administration Tools and Monitoring and Additional software contain the following components and tools:
- Administration and Monitoring Tools
- Development Tools
- Software Development Kit(s)
- HTTP Receive Adapter
- SOAP Receive Adapter
- Windows SharePoint Services Adapter Web Service
- Windows Communication Foundation Adapters
- Business Activity Monitoring (“BAM”) Event APIs and Interceptors & Administration Tools
- BAM Alert Provider for SQL Notification Services
- BAM Client
- BizTalk Server Related Schemas and Templates
- Business Activity Services
- Master Secret Server/Enterprise Single Sign-On
- Business Rules Component
- MQSeries Agent
Why would you need these components/tools
So, given the list with available components/tools, an organisation could have several reasons to install them, depending on the purpose of the component. You could think of BizTalk users who might want to achieve, for example, the following:
- use the BizTalk APIs as this is a requirement for third party software
- use the BizTalk Server APIs for self-developed software or scripts
- provide the Business Rules Composer to the desktop computer of (functional) users
- make the Enterprise Single Sign-On (ESSO) tools highly available by installing them on separate servers
- use the Developer tools on a build server for BizTalk
- use the BAM tools on a separate server
Hence, it makes sense to be sure whether (or not) you require additional BizTalk Server licenses for the server(s) on which such components will be installed.
The hunt for information
We have been trying to find that information somewhere on the internet. However, as Microsoft is the supplier of the product, we searched for a formal statement from their side. Therefore, we checked web sites like the BizTalk Server product web site
and the BizTalk Server Core Documentation
. However, we were not able to find that information.
As we want to give our customers clarity about this matter, we decided to reach out to the Product Group directly. After a couple of emails, the Product Group informed us where to find the information we were looking for. See below, the email we received from the Product Group.
No additional licenses required for Administration Tools etc.
The email from the Product Group mentions that:
These two highlighted items, “Administration Tools” and “Business Rules Component” are the components to which you refer. The EULA indicates these can be installed on other physical or virtual systems without incurring an additional license cost.
For people who are working with BizTalk Server for some time, it is no surprise that no additional license costs are involved in installing these tools on separate machines. Main reason for this, is that by only installing these tools, the machines are not a part of the BizTalk Group and no additional BizTalk processing is taking place (like receiving/sending messages or processing orchestrations).
However, it has always been a bit frustrating that we could not back this by a statement from Microsoft. So, we are glad that we can now refer to an official resource which gives clarity about this matter.
How to find the End User License Agreement(EULA)
Now we know where to find that information, let’s have a look on how to find the EULA, including the section where the clarity is given.
1. Download and mount the en_host_integration_server_2016_enterprise_x64_cd_9503501.iso (ISO file name varies depending on where you obtain the software—e.g., Volume License site or Visual Studio or Evaluation Center)
2. Start a Windows Explorer and navigate to the BizTalk Server folder
3. Locate and load the EULA.RTF in Microsoft Word
4. See section “2. Use Rights” titled “e. Running Instances of Additional Software”
There you have it, written in black on white! No license costs will be incurred for installing the Administration Tools etc. on a separate server!
We, as a company, are glad to be able to give clarity to our customers, prospects and partners about the matter that no additional license costs will be incurred in case of installing the Administration Tools etc. at separate servers. However, also organisations who are using BizTalk Server, resellers of Microsoft products and consultancy companies who advice their customers about BizTalk Server, are now able to safely say that no additional license costs are involved in case of installing these components and tools as they are backed by the EULA from Microsoft.
From this place, we also like to thank the BizTalk Server Product Group to help in giving the clarity.
The post Do you need additional BizTalk Server Licenses for installing BizTalk components on separate servers appeared first on BizTalk360.
One of the components of BizTalk360 is a SQL Server database. This database is used for all kind of configuration like user permissions and all the monitoring settings. When that data got corrupted or lost, you would have to do all the configuration all over from scratch. To prevent this from happening, you should frequently take backups of that database. Besides creating manual backups, you can also have these backups being created automatically.
There are two different approaches to making automated backups of this database, both are shown below:
- Create a SQL Server Maintenance Plan
- Extend the Backup BizTalk Server job
The difference between these 2 methods is, that with the Maintenance Plan approach you’ll have a backup which is not in sync with the backup of your BizTalk databases, while with the latter option your BizTalk360 backup will be in sync with the BizTalk backups. This could make restoring your databases in one go easier.
In a previous post, Rochelle has already explained how to create a Maintenance Plan to take care of the BizTalk360 database backups. In this article, we’ll explain how to add the BizTalk360 database to the Backup BizTalk Server job.
Adding custom databases to the BizTalk Server Backup job
What we basically are going to do is using a feature from BizTalk Server. As you probably are aware of, BizTalk Server contains multiple databases and to be able to restore them in sync, the backup needs to be created in sync. The only by Microsoft supported way to create such backups, is by using the Backup BizTalk Server job, which is a SQL Server Agent job. You can read more about that topic in the below articles:
BizTalk users can extend the backup job with other databases which are considered important to the integrations which are deployed in BizTalk Server. This is exactly what we will be doing with the BizTalk360 database.
The process exists of the following steps:
- Prepare the BizTalk360 database
- Add the BizTalk360 database to the BizTalk Server backup job
- Start making backups
Let’s take these steps one by one and have that database added to the BizTalk Server backup job!
Prepare the BizTalk360 database
In this first step, we’ll make sure a table and some Stored Procedures will be created in the BizTalk360 database. The table which becomes created is called MarkLog. You will find this table in all the databases which are being backed up via the BizTalk Server backup job.
Perform below steps to create that table and the needed Stored Procedures:
- Open SQL Server Management Studio and connect to the SQL Server instance which contains the BizTalk360 database
- Click Open File, navigate to folder C:Program Files (x86)Microsoft BizTalk Server 2016Schema and select query Backup_Setup_All_Tables.sql
- From the Databases dropdown, select the BizTalk360 database
- Click the Execute button or hit F5 to execute the script. If the database has been created successfully, you can proceed with the next step
- Click Open File, navigate to folder C:Program Files (x86)Microsoft BizTalk Server 2016Schema and select query Backup_Setup_All_Procs.sql
- If not yet selected, select the BizTalk360 database from the Databases dropdown
- Click the Execute button or hit F5 to execute the script
If both SQL scripts have been executed successfully, an important part of the configuration has been completed. The BizTalk360 database is ready and in the next step, it will be added to the BizTalk Server backup job!
Important: Ensure yourself that the BizTalk360 is in Full Recovery Model, otherwise the backup will fail! You can check this by:
- Right-click the database then select Properties
- Select Options
- Check if the Recovery Model is set to Full
Add the BizTalk360 database to the BizTalk Server backup job
In the previous step, we prepared the BizTalk360 database to be able to be backed up by the BizTalk Server backup job. In this step, we’ll make sure that that database becomes added to a table in BizTalk Server’s management database, which will make sure that the database will be picked up by the BizTalk Server backup job.
Follow below steps, to make sure that the BizTalk360 database will be picked up by the BizTalk Server backup job:
- In SQL Server Management Studio, connect to the SQL Server instance which contains the BizTalkMgmtDb
- In the Object Explorer, expand the Databases, BizTalkMgmtDb, Tables and find the dbo.adm_OtherBackupDatabases table
- Right-click that table and, from the menu that appears, select Edit Top 200 Rows. As you are in Edit mode, you can add a new row which will contain the information about the BizTalk360 database.
- In the last row, which now only shows NULL values, enter below values
- DefaultDatabase: BizTalk360
- DatabaseName: BizTalk360
- ServerName: <Name of the SQL Server Instance which contains the BizTalk360 database>
- BTSServerName: <Name of the SQL Server Instance which contains the BizTalk360 database>
- Hit Enter to save the record in the table
The BizTalk360 database is now part of the BizTalk Server backup job. The last step we need to do is forcing a full backup, to make sure that also incremental backups can be created.
Start making backups
We are almost there; we have seen how the BizTalk360 database has been prepared to accommodate the BizTalk Server backup job. In the previous step, we have added the BizTalk360 database to the BizTalk Server backup job. In this last step, we will force a full backup, to make sure that after that, also incremental backups can be created. By default, a full backup will be created once every 24 hours; Incremental backups will be created, by default, every 15 minutes.
Perform below steps to force a full backup:
- In SQL Server Management Studio, under the BizTalkMgmtDb, expand Programmability and expand Stored Procedures
- Scroll through the Stored Procedures until you have found sp_ForceFullBackup
- Right-click that Stored Procedure and, from the menu that appears, select Execute Stored Procedure… As the Stored Procedures doesn’t need any parameter values, just click OK
- If the Stored Procedure has been executed successfully, the next time the BizTalk Server backup job runs, it will perform a full backup
There are a couple of ways to check whether the backups are really being created. You can:
- Check the output of the BizTalk Server backup job (in SQL Server Management Studio)
- Check if the backup files have been created (in Windows Explorer)
Check the output of the BizTalk Server backup job
To perform this check, perform these steps:
- In SQL Server Management Studio, you need to navigate to the SQL Server instance which contains the BizTalk Server backup job
- Next, expand SQL Server Agent
- Right-click the Backup BizTalk Server job and select View History
Check if the backup files have been created
To check the availability of the backup files, you firstly need to check where these files are located. Follow the below steps, to find that location and then check the actual location:
- In SQL Server Management Studio, you need to navigate to the SQL Server instance which contains the BizTalk Server backup job
- Next, expand SQL Server Agent
- Double-click the Backup BizTalk Server job
- In the Job Properties dialog which appears, under Select a Page, select Steps
- Now, under Job step list, double click BackupFull
- In the Job Step Properties dialog, at Command, scroll to the right to find the backup path
- Copy the backup path and close all dialog screens
- Next, open a Windows Explorer and paste the backup path in the Address bar
Now the backup files should show. Although we only checked the backup path for the full backup files, this folder might also contain the backup files of the incremental backups.
The BizTalk360 database contains valuable information about, amongst others, your monitoring configuration and the people who have access to BizTalk360. If in case of a disaster, you need to easily restore a backup of your BizTalk360 database, you need to have a recent backup of that database. The BizTalk Server backup job creates such backups. In this article, we have seen how to extend the BizTalk Server backup job to incorporate the backup of the BizTalk360 database.
The post Backup your BizTalk360 database via the BizTalk Backup job appeared first on BizTalk360.
It is nothing new that BizTalk360 enables you to set up automated monitoring for BizTalk. This includes the deployed BizTalk application artifacts and different kind of endpoints. You can even track the processing which is taking place in your BizTalk environment.
For each unexpected situation for which you have set up monitoring, you can receive notifications via for example email, SMS/Text message or the Event Log. Besides that, BizTalk360 also provides Notification channels to connect to products like Slack, Microsoft Teams and HP Operations Manager. In case you are using ServiceNow, you can even have BizTalk360 generate incidents into that product! If that still does not meet your requirements, you can use a Web Hook, or the easy-to-use API provided by the product to integrate with your systems.
The BizTalk360 database contains information about the transmitted notifications.
Collecting Alarm statistics
Recently, we had a call with a consultancy company which manages several BizTalk environments for their clients. These clients are using BizTalk360 for operating and monitoring the BizTalk environments.
The consultancy company was interested in knowing whether BizTalk360 provides any statistics about all the alarms which were triggered over time. This kind of statistics serves different goals. On one side, this kind of information gives insight in components which frequently have outage, enabling to take counter measures. On the other side, this can also be used to provide such information to the client of the consultancy company as a kind of reporting.
In this blog post we describe how you can use the information in the BizTalk360 database, to show statistics about which alarms have been transmitting which kind of notifications over time.
Showing SQL data in BizTalk360
As already mentioned, the information about the transmitted notifications is stored in the BizTalk360 database. BizTalk360 provides different capabilities to show data which is stored in SQL Server databases. These capabilities are:
• Secure SQL Queries – see https://docs.biztalk360.com/docs/secure-sql-queries
• Custom Widgets – see https://docs.biztalk360.com/docs/creating-a-custom-widget-for-executing-secure-sql-queries
Depending on the requirements how you want to be able to view that data, you can use either or both features, but the main characteristics are that Secure SQL Queries are mainly used manually, where Custom Widgets can become shown on a dashboard and the output will be refreshed automatically.
The information we will be using in this blog post is stored in a table called b360_alert_notify_ToBeNotified. The content of that table is specific to Threshold monitoring. If you need more detailed information about Data Monitoring statistics, you should have a look at a different table. Based on the b360_alert_notify_ToBeNotified table, you can write a SQL query which looks like below:
SELECT aa.Name AS 'Alarm Name',
an.NotificationRequiredType AS 'Notification Type',
COUNT(an.AlarmId) AS Count,
MAX(CreatedDateTime) AS 'Most recent'
FROM b360_alert_notify_ToBeNotified an
RIGHT OUTER JOIN b360_alert_Alarm aa ON an.AlarmId = aa.Id
WHERE (an.NotificationRequiredType <> 'Regular')
GROUP BY aa.Name, an.NotificationRequiredType
order by [Alarm Name]
In above query, notifications of type ‘Regular’ have been left out. Regular notifications refer to the Health alarms which can be received at a specified time, to show the state of all the mapped artifacts of an alarm. For this scenario, that information is considered as not relevant.
It is good to know, that without any filters, the data will be shown according the Purging policy for Alert & Maintenance History. This policy can be viewed and reconfigured in the Settings area, under BizTalk360 Health, in the Purging Policy screen.
Obviously, in case you do want to see a bit more limited data set, you can add WHERE conditions based on the CreatedDateTime field.
Showing Alarm statistics as a Secure SQL Query
When you create a Secure SQL Query for this, the output of the query can look like below.
As you can see, the query shows information about the name of the alarm, which kind of notification is involved, the count of notifications and the most recent date/time a notification has been transmitted, for that alarm/notification type combination.
If you want to know more about how to create Secure SQL Queries, you can check out the Documentation portal about this feature.
Having this kind of information as a Secure SQL Query can be very handy. You have the information easily at hand by accessing that query and run it. The results can also be exported to a .CSV file.
Showing Alarm statistics as a Custom Widget
If you want to have an even better experience, you can consider creating a Custom Widget for this. Doing so, enables you to have this information on a dashboard and because of the behavior of these dashboards, the widget will be automatically refreshed. Besides that, by a simple click, you can export the entire dashboard, which could contain more statistical widgets, to a PDF file and provide it to your clients.
Based on the query we have seen earlier in this article such a widget will look like below.
Once the widget has been added to a dashboard, the query will automatically be executed, and the data will be refreshed each minute. If needed, you can change the refresh interval in the script of the widget.
If you want to read more about creating such Custom Widgets, check the Documentation portal for it. It contains some detailed articles about this topic.
We have shown another way on how BizTalk360 can be useful. So, if you have any questions about this capability, or anything else related to BizTalk360 feel free to contact us. Why not give BizTalk360 a try! You can download a free trial which contains all the features for a limited time.
From time to time we hear that organisations are using DynaTrace for operating and monitoring their BizTalk environments. When we talk to such organisations, they ask us about the benefit of using BizTalk360 over Dynatrace for administering their BizTalk environments. In this blog post we try to give a fair comparison between both products.
BizTalk360 and Dynatrace in short
BizTalk360 is a comprehensive product for operations, monitoring and application performance management (APM) of BizTalk Server environments. Unlike other APM products, BizTalk360 is an agent less, non-intrusive software which requires zero configuration on BizTalk servers. This makes it an easy to use and operations-friendly tool. Customers have a lot of influence in the roadmap and the vendor is very receptive to feedback via http://feedback.biztalk360.com. Furthermore, Support is very responsive. This means the product is always up to date to support the latest releases of BizTalk Server.
BizTalk360 is a market leader in BizTalk Server space, has a large customer base and healthy partner ecosystem. The company understands that supporting BizTalk Server applications for business continuity and efficient operations requires a tool which can go beyond being a traditional application performance management tool. Therefore, the product provides BizTalk specific capabilities such as managing BizTalk Applications, proactive monitoring of throttling conditions, role-based user access policies, audit logs, data monitoring etc. In some instances, BizTalk360 has helped customers to meet SOX compliance.
DynaTrace, on the other hand, is a typical Application Performance Management tool with generic capabilities to monitor operating systems, services, network protocols, system metrics, network infrastructure, services, applications etc. There is no out of the box DynaTrace plugin or agent for BizTalk Server. Either the customer needs to create one of their own or use an open source plugin Fastpack. This plugin must be installed on all the BizTalk Servers and some configuration changes must be made to the BizTalk host instance’s configuration file (btsntsvc.exe.config). With an agent it brings the ability to analyse and monitor the performance counters for BizTalk host instances, databases, orchestrations, pipelines etc. The Fastpack plugin seems to have been updated a couple of years back and might not be up to date with the latest BizTalk releases; this is a typical challenge with using open source plugins from individual contributors.
In this section we will compare the 2 products, BizTalk360 and DynaTrace, for capabilities in achieving BizTalk Server monitoring requirements.
Getting a proactive alarm to warn before a problem occurs
In the BizTalk environment problems can occur in the following situations:
- Important artefacts such as receive locations, send ports, orchestrations, host instances etc get disabled or stopped, either due to errors or due to unintentionally performed tasks. This leads to outages and stops the flow of mission critical messages.
- If the SQL Server jobs are stopped, the size of databases increases over time and impacts the messaging performance.
- Due to several reasons BizTalk can start throttling messages leading to an impact on performance.
- If suspended messages are not cleaned up, the size of the message box increases resulting in performance issues.
BizTalk360 has features to monitor state-based artefacts, which ensures that all BizTalk application artefacts such as receive locations and orchestrations are up and running and generate notifications in case of aberrations. State based monitoring also ensures that essential SQL jobs and Windows services are running as expected. Features like suspended queue monitoring and throttling monitoring ensure that the BizTalk Server is running in a healthy state.
DynaTrace has no out-of-the box support for BizTalk Server. The Open Source plug-in does not have features such as state-based monitoring, throttling monitoring and suspended queue monitoring. However, it does have monitoring capabilities based on some BizTalk and SQL performance counters. In conclusion, it lags behind BizTalk360 in proactively warning of potential issues.
In some situations, issues do occur and the operations team needs easy to use tooling to correct the faults. For example, if receive locations get disabled, they will have to be re-enabled. If messages get suspended, they will have to be terminated or resumed. If we have a software which auto-corrects these faults and automates the operational tasks then it is a big bonus as it brings the down time to its minimum levels and ensures operational efficiency.
BizTalk360 has a powerful feature called Auto Healing, which allows the users to define the expected state of an artefact and in case there is an aberration, arestores the expected state without any human intervention. With another feature Operational Automation, the day to day monotonous tasks such as resuming/terminating suspended messages can be automated.
DynaTrace on the other hand has no operational capabilities. Auto-healing and Operational Automation are not features you can expect from a typical Application Performance Management system. Hence, DynaTrace will not be the right tool to be used for BizTalk related problem solving actions.
Access to the BizTalk servers
BizTalk Server has a very limited number of user groups. The BizTalk Administrators group has the highest level of privileges and the BizTalk Application Operators group has highly restricted access. There are no access levels to give intermediate level access to BizTalk applications and artefacts. Many customers have ended up providing their users with the unrestricted access to production boxes. It is typical of organisations to provide all the users the access to production servers for business continuity.
One of the widely used features of BizTalk360 is Granular User Access Control. With this feature, user access can be controlled at a very granular level. Users do not need to RDP to production boxes to support BizTalk applications. BizTalk360’s web portal allows them access to all the features without the need for them to log on to production boxes unless they are doing deployments.
Again, access control is the feature which can be expected from an Application Performance Management tool. Hence, DynaTrace clearly has no capabilities which to help to restrict access to BizTalk Servers.
One tool for operations, monitoring and analytics
If a support professional needs to use multiple tools daily, operational efficiency takes a big hit due to all the context switching he must do between the tools. This is one of the main challenges in supporting BizTalk Server applications. Every operations team needs a single tool to perform operations, monitoring and analytics.
BizTalk360 brings all the capabilities of operations, monitoring and analytics into a single web portal. Apart from this, it allows the users to access BAM data, ESB data, EDI reports from the same web portal. This means users do not have to switch to different tools.
DynaTrace being an Application Performance tool, does have the ability to monitor performance counters and detect some anomalies. However, the user will be required to switch to different tools such as admin console, event log, ESB portal, BAM portal etc. to perform various other tasks. In a true sense, it adds another tool to the list.
Depending on the performance needs, organisations enable or disable BizTalk tracking. Some organisations enable only event tracking and disable message body and property tracking. Hence each organisation has different tracking needs. Regardless, it is important to have good control to manage the tracking options at various artefacts.
BizTalk360 has a feature called Tracking Manager, which brings great control to manage tracking options at various artefacts. This helps customers to ensure they are following the tracking standards set at the organisation. BizTalk360 also has a feature called Graphical Flow, which is an analytics feature providing a visualisation of message flow across various artefacts such as receive ports, orchestrations and send ports. Enabling pipeline tracking is a minimum requirement for the feature. Not many of our customers use it due to performance requirements. This is the only feature which is dependent on tracking. Most of the earlier discussed features of BizTalk360 are not dependent on the tracking.
DynaTrace, like BizTalk360, will not be able to provide analytics related to ports and orchestrations if the tracking is not enabled. It surely is not a tracking management tool either.
Comparing the capabilities of BizTalk360 and Dynatrace against the day-to-day requirements necessary to work with BizTalk Server, BizTalk360 comes out as a clear winner. Dynatrace is a generic APM tool and will not match most of the capabilities of BizTalk360. The features such as State-Based Monitoring, Auto Healing, Operational Automation, Throttling Monitoring etc. are essential to bring the operational efficiency and ensure the health of BizTalk environments and applications.
Why not give BizTalk360 a try! A trial version of BizTalk360 can be requested here.
So, you have installed BizTalk360 and you are eager to start using it! But, BizTalk360 is such a feature-rich product, that you might find it hard to get started and implement BizTalk360 in a proper way.
In this blog post, we will give you some tips and guidelines, which will be of help when you are setting up BizTalk360. You can also use the article if you have already set up some of the features in BizTalk360 and want to make sure that you did not miss anything important.
: BizTalk360 has a To Do list
which contains a number of tasks, you could do to make full use of the product. You will find this To Do list under Settings => To Do List
. See also, this article
This blog post is separated in the following sections:
- Optimizing the BizTalk360 environment
- Benefit from automated monitoring
Optimizing the BizTalk360 environment
Once you have installed the product, there are a few things you must do, before you are able to use the product. Furthermore, you can also perform some tasks to optimize the environment. Let’s have a look at these tasks:
- Activate the license (required)
- Adding BizTalk environments to BizTalk360 (required)
- Remove unused BizTalk features from the BizTalk360 UI
- Create User Access Policies
- Create efficient dashboards
We will have a somewhat more detailed look at these tasks.
Activate the license
After finishing the installation of BizTalk360, the installer starts the product in your default browser. Once started, BizTalk360 will ask you to provide the license. If you have purchased a license, you now can provide the details of this license. Depending if the server on which BizTalk360 has been installed has internet connectivity, the process of applying the license will be a bit different.
Adding BizTalk environments to BizTalk360
After applying the license, you will be able to access the BizTalk environment, for which you purchased the license.
In case you have purchased BizTalk360 licenses for multiple BizTalk environments, there are two options on how to access these BizTalk environments from BizTalk360:
- A separate installation of BizTalk360 for each BizTalk environment
- Using one BizTalk360 instance for multiple BizTalk environments
It is totally up to you how you want to access your BizTalk environments from BizTalk360, although the two mentioned approaches both have their pros and cons. To sum up a few:
- it is handy to have multiple BizTalk environments in the same instance of BizTalk360. But, there is the risk that you think that you are stopping artefacts in your Test environment, while you were actually doing it in the Production environment
- with separate BizTalk360 instances for your BizTalk environments, it is less likely that you stop artefacts or terminate instances in the wrong environment. But, it might be hard to memorize the URL’s of the different BizTalk360 installations
If you plan to have multiple BizTalk environments in the same instance of BizTalk360, it is good to know that, to prevent confusion, you can achieve the following:
- give each environment a friendly name
- colour code each environment
Read this article
in our Documentation portal on how to maintain environments in BizTalk360. The article mentions, amongst others, how to provide a friendly name and a colour code.
Remove unused BizTalk features from the BizTalk360 UI
One of the primary design goals of BizTalk360 is to consolidate all the tools and portals you might need to operate your BizTalk environment. Just to mention a few of such portals, the product contains features to access BAM views, the ESB Exception Framework, the Business Rules Composer, EDI, etc.
However, within your organization, you might not use all these BizTalk features. Still, they show up in the BizTalk360 user interface.
Of course, you can use User Access Policies to prevent these features to show up, but then the Super User (for which no fine-grained user access policy needs to be set up) will still view these unused features.
To prevent these unused features from showing up, you can simply disable them for the environment. This way, the BizTalk360 user interface is more focused on your BizTalk environment.
Check this article
in our Documentation portal and under Allowed Features turn on/off the appropriate features.
Create User Access Policies
After the installation of the product, an User Access Policy has been created for the user who installed the product. However, it is likely that more people will use the product. To provide each user, which the access to BizTalk360 they need, it is best to create User Access Policies for each user.
You can start with giving your fellow BizTalk administrators access to BizTalk360. But when you feel confident, you can also give people outside the BizTalk Admin team access to the product.
With the fine-grained capabilities of the User Access policies, you can, for example, can give your:
- Support Engineers read-only access to BizTalk360
- Business users access to portals like BAM, EDI, ESB, BRE, etc.
Access is always in a secure and audited way. In this blog post
we describe how you can approach setting up access with BizTalk360.
Create efficient dashboards
The Operations Dashboard is the entry point of BizTalk360. This Dashboard can contain a different kind of widgets of which the goal is to immediately show useful information about the well-being of your BizTalk environment.
The widgets on the Operations Dashboard are fully customizable in terms of:
- number of widgets
- size of the widgets
- location of the widgets
- refresh interval of the widgets
In case the default collection of widgets does not completely fulfil your requirements, you can create Custom Widgets, enabling you for example to show the output of (BizTalk360) API calls or SQL queries.
The Documentation portal
contains more information on how to set up and use the Operations Dashboards. It also contains articles about creating Custom Widgets.
Benefit from automated monitoring
The main reason you purchased BizTalk360 might very well be the monitoring capabilities of the product, so you are probably interested in setting this up as soon as possible.
Setting up monitoring in BizTalk360, consists of two steps, which are:
- create Alarms – to configure HOW you want to monitor
- manage Mappings – to configure WHAT you want to monitor
Create Alarms to configure HOW you want to monitor
The first step is to create an alarm. With BizTalk360, alarms come in a few different flavours:
- Threshold alarms – for monitoring whether artefacts are in the expected state
- Health alarms – for receiving Daily check reports at configurable days/times
- Data Monitoring alarms – for monitoring the processing of messages through BizTalk
- Consolidated alarms – any combination of the 3 above mentioned alarm types
Besides stuff like giving an alarm a name and configuring how you want to be notified, depending on the alarm type, you can configure multiple other settings. So, again, with configuring these properties you are only configuring how you want to monitor.
Read the Understanding Alarms article
in the Documentation portal for more information about the different alarm types and on how to create an alarm.
: you probably want to receive notifications via email. For BizTalk360, to be able to send email notifications, you need to configure an SMTP server. Read this article
on how to set this up.
Manage Mappings to configure WHAT you want to monitor
After creating an alarm, you will add mappings to it, to start monitoring the required artefacts. With BizTalk360, there is a huge set of artefacts you can monitor. Think of for example (this is not the full list):
- BizTalk Applications – Send Ports, Receive Locations, Orchestrations, Service instances
- Host instances – in/out process, clustered
- Host Instance Throttling – generic or detailed
- BizTalk Servers/SQL Servers – Disks, Event Logs, NT Services, CPU, Memory
- BizTalk Server Availability – via Ping or Telnet
- SQL Server Instances – SQL Server Agent Jobs, SQL Queries
- HTTP Endpoints – REST, SOAP
- FILE Endpoints – FILE, FTP, SFTP, FTPS
- Queues – MSMQ, IBM MQ, Azure ServiceBus Queues
- BizTalk Health Monitor – Critical and Non-critical Errors and Warnings
Depending on the type of the artefact, you can monitor for its state (like Stopped/Started/etc.) or Warning/Error thresholds.
Although you are completely free in how you set up monitoring, we normally recommend making a distinction between monitoring the BizTalk platform and the BizTalk applications.
- Platform alarms – these contain mappings to stuff like the Host Instances, the SQL Server Agent jobs, Host Throttling, Server resources etc.
- BizTalk Application alarms – these alarms contain mappings to BizTalk application artefacts (Receive/Send Ports, Orchestrations and Service Instances) and relevant endpoints (HTTP/HTTPS, FILE, FTP/SFTP/FTPS, etc.)
Note: Did you know that BizTalk360 has a feature which is called Auto-Correct? This feature enables you to automatically (try) to bring artefacts back to the expected state, after entering the wrong state. This feature, which can be a real time-saver, works for the following components:
- BizTalk Applications – Receive Locations, Orchestrations and Send Ports
- BizTalk Platform – Host Instances
- Windows Server – Windows NT Services
- SQL Server – SQL Server Jobs
- Microsoft Azure – Logic Apps
Read more about this feature in the Documentation portal.
Besides artefact monitoring, BizTalk360 also allows you to monitor the processing which is taking place through BizTalk. The concept of Data Monitoring allows you to monitor activity around sources like the MessageBox, Tracking data, BAM data, EDI data, ESB data, Logic Apps and Even Log entries. Data Monitoring comes with the Data Monitoring Dashboard. This calendar-like dashboard shows the historic runs of all the configured Data Monitors. This dashboard will make it easy to address questions about whether certain processing did take place.
Without going in too much detail, with Data Monitoring you can cover scenarios like:
- monitoring whether batches you expect at a particular time, really arrive in BizTalk and become processed
- you may want to automatically terminate service instances, like Routing Failures, which you frequently receive
- you may want to automatically resume service instances because the messages could not be transmitted due to a temporary network failure
These are only a few of the scenarios you can think of. If you want to read more about the concept of Data Monitoring, consult the Documentation portal
Implementing BizTalk360 can be a multi-step process
You must have noticed that, once you have purchased and installed BizTalk360, there is a lot to go for. In this blog post, we only mentioned a number of features which are considered the most important to implement after installing the product.
However, there are way too many features in the product to cover in a single blog post. Given the number of features in the product, it makes sense to make implementation of BizTalk360 a multi-step process. This way, your BizTalk360 implementation can grow with your requirements and the requirements of your organization.
If you need help with implementing BizTalk360 or want to receive some more tips, feel free to contact us at email@example.com
We love to hear from you and we want you to take the full benefit of your purchase!
In this blog post, we will explain how different stakeholders within your organization can be involved in operating the BizTalk environment and managing BizTalk interfaces, by smartly, but safely deploy BizTalk360.
The BizTalk Administrators – the primary users of BizTalk360
Primarily, BizTalk360 will mainly be used by BizTalk administrators. They will use the product in their day to day operations and will probably be the ones who firstly receive notifications of threshold violations and daily health check reports.
These administrators are responsible for the day to day operations of the BizTalk group(s) and will be the main group of users of BizTalk360. Therefore, they need all the authorizations so they will have Super User accounts in BizTalk360. They will also be responsible for creating and maintaining User Access Policies.
After installation of the product, they will set up alarms for the BizTalk platform and the BizTalk applications it contains and receive notifications which are generated by the alarms.
The way they configure monitoring of their BizTalk environments with BizTalk360, might evolve like this:
- Basic Threshold/Health Check monitoring with email notification
- BizTalk Platform – Monitor platform components like Host Instances, SQL Server jobs, NT Services, BizTalk Health Monitor
- BizTalk Applications – Monitor application artifacts like Receive Locations, Orchestrations, Send Ports, Instance states
- Advanced Environment Monitoring
- Endpoints – Monitor web services, queues, file shares, FTP sites, Azure services
- Data monitoring – Monitor interface processing, automate resuming and terminating of service instances
- Enterprise notification channels – Receive notifications via HP Operations Manager, Slack, ServiceNow, Microsoft Teams, Webhook, PowerShell
In case of any issues, the (team of) BizTalk administrators will fix these issues themselves, or they contact other stakeholders to discuss how the issues need to be fixed.
Improve your business processes by extending the reach of BizTalk360
The power of BizTalk360 lies in the fact that it provides a rich user interface with many different dashboards for many different purposes, both from a technical and a functional perspective. In contrast with the tools which come out-of-the-box with BizTalk Server, all the capabilities in BizTalk360 are protected by User Access Policies and Auditing. This enables you to give any person exactly that set of permissions that they need to be able to do their job.
When the BizTalk administrators feel comfortable with the product and with BizTalk administration in general, or when the organization requires it, BizTalk360 can be deployed to other parts of the IT department or business departments.
Involve stakeholders by sending notifications and providing access to BizTalk360
By sending alerts to stakeholders directly, you improve information management as your BizTalk Administrators don’t need to send separate emails or contact these stakeholders based on issues that have occurred.
Although in many cases, email will be the primary means of sending notifications from BizTalk360, the product also contains the following Notification channels:
• Microsoft Teams
These channels can be configured on each BizTalk360 alarm and enable you to receive the notifications where it is most convenient for you. You can also use a simple to use SDK to develop your own custom Notification channel. Read more about Notification channels in our Documentation Portal.
But you can go even further, BizTalk360 allows you to give stakeholders secure and limited access to BizTalk360. This kind of access can reach from read-only access to particular parts of the BizTalk platform to the capability to act on certain issues.
Deployment of BizTalk360 to your organization can be done in the pace you and your organization feel comfortable with. By sending notifications to the stakeholders and provide access to BizTalk360, you can keep the stakeholders informed of any issues, improve the availability of your business processes and meanwhile spreading the workload between all the stakeholders.
Think of the following scenarios which could be achieved:
- Sending alerts to the help desk – This enables the help desk engineers to analyze any issues at hand and take countermeasures
- Informing your administrators directly of any issues – Think of your System Administrators or DBA’s who receive alerts about server or database issues
- Automated creation of support tickets in your ticketing system – This takes away the need to have an administrator to do it manually
- Inform business users of issues – Think of batches which are not processed (non-events) or faulty processing of their messages
- Provide access to dashboards/portals – Give your stakeholders access to portals like the BAM portal, ESB portal, Business Rules Composer, (Data) Monitoring/Analytics dashboard, etc. etc.
When it comes to deploying BizTalk360 throughout an organization, you can identify the following roles:
- IT Support personnel
- SQL Server DBA’s, System Administrators and BizTalk developers
- Business Users
Let’s have a look at the roles mentioned above and their potential responsibilities with regards to BizTalk Server/BizTalk360.
IT Support personnel
To have a good eye watching on your BizTalk environment, you could involve the IT Support staff of your organization. Depending on your organization, they might be available 24/7. So it makes sense to send notifications of unexpected behavior happening on the platform level, as they might be able to act before BizTalk Administrators can. Besides sending them notifications, you could give the IT Support staff read-only access to BizTalk, so they can explore any issues and maybe help you fix them, while you might be at home, having weekend.
SQL Server DBA’s, System Administrators and BizTalk developers
Since operating BizTalk involves more than just the BizTalk Server product itself, but also components like SQL Server databases and all kind of Windows Server components, you might consider involving System Administrators and SQL Server DBA’s in BizTalk operations by sending them notifications which might be relevant for them.
A few examples are:
- SQL Server Administrators can receive notifications in case the BizTalk related SQL Server jobs fail
- System Administrators can receive notifications when the BizTalk servers are running out of disk space
- BizTalk Developers can receive notifications in case issues arise with upgraded or newly deployed BizTalk applications
As a next step, you might give these administrators and developers access to BizTalk360. This ranges from providing read-only access to particular features to giving them full operational access to the parts of their interest.
If you give any of the stakeholders access to BizTalk360, it is also helpful to use the BizTalk360 Knowledge Base. When you properly maintain the Knowledge Base, your support people will have the help they need at their fingertips and be able to solve known issues quickly.
The Knowledge Base associates Knowledge Base articles to Service Instances, EventLog entries, ESB Exceptions, and Throttling data. Read more about its capabilities in our Documentation portal.
You might consider providing System Administrators, SQL Server DBAs and BizTalk developers with the following authorizations:
- System Administrators
- Advanced Event Viewer
- BizTalk Health Monitor
- Host/Host Instances
- BizTalk/SQL Servers
- Manage BizTalk/SQL NT Services
- Tracking Manager
- Backup/DR Visualizer
- SQL Server DBA’s
- Secure SQL Queries
- Advanced Event Viewer
- BizTalk Health Monitor
- Message Boxes
- SQL Servers
- Manage SQL NT Services
- SQL Server Instances
- Manage SQL Jobs
- Backup/DR Visualizer
- BizTalk developers
- MessageBox queries (with/without access to content/context)
- Tracking Queries
- Advanced Event Viewer
- Tracking Manager
- Secure SQL Queries
- BAM portal
- ESB Exception portal
- EDI Reports
- Messaging Patterns
This category of users might exist both inside as outside your organization. Depending on that, it will differ how they are involved in managing the interfaces. Normally, they will not take part in managing the BizTalk platform itself.
Internal business users can be informed about the processing, by providing them with notifications of disruptions in the processing of their interfaces, i.e., inform them of suspended instances, transmission failures, and failing process monitoring.
When you want to give business users access to BizTalk360, you can think of the following features:
– Specific BizTalk applications
– Message Box (Queries)
– Graphical Flow (Tracking)
– Business Rules Composer
– EDI Reports, parties and agreements
– ESB Portal
– Business Activity Monitoring
– Messages Content/Context
– Secure SQL Queries
In case external business users are involved in certain interfaces, you might send them the same notifications as internal business users. As the external business users will be outside your organization, you normally will not give them access to the BizTalk360 User Interface.
We often see, that BizTalk Server is considered as a black box and deep BizTalk knowledge is needed to be able to find out what’s all happening inside that box. By using BizTalk360, we make it easy to gain that insight, even with little BizTalk expertise. Furthermore, by deploying BizTalk360 outside the BizTalk administrators team, you can give your middleware a face and achieve much more transparency about all the processing taking place in your BizTalk environment.
By using BizTalk360 outside the admin team, it is easier to notify other stakeholders by sending them notifications directly from BizTalk360. Even further, besides sending notifications to these stakeholders within (or outside) your organization, you can give people (limited) access to BizTalk360. This way they can view for themselves how all the processing is taking place or check the wellbeing of the environment, without the need of contacting the BizTalk Administrators team.
All in all, it must be clear, that by extending the use of BizTalk360 outside the admin team, you will have a better ROI of the product. If you would like to know more about how BizTalk360 can help your organization to manage your BizTalk Server middleware platform, feel free to contact us.
In this blog post we will update you on which new capabilities came with BizTalk Server 2016 Feature Pack 3. Earlier, we have also written about Feature Pack 1 and 2. You can find these articles here:
Introduction to BizTalk Server 2016 Feature Packs
In 2017, Microsoft started releasing so-called Feature Packs for BizTalk Server 2016. The concept of releasing these Feature Packs is that Microsoft doesn’t want their BizTalk Server customers to wait for new features, until a new release of BizTalk Server arrives. Instead, they want their customers to enjoy these new features, as soon as possible once they have been developed and are ready to go to market. As these Feature Packs contain non-breaking features, there is little risk that a BizTalk customer runs into issues as a result of installing such Feature Packs. These Feature Packs are only available with the Enterprise and Developer edition of BizTalk Server 2016.
Since Microsoft started with this strategy, they have released 3 Feature Packs:
The BizTalk Server 2016 Feature Packs are cumulative. This means that once you installed, for example, Feature Pack 3, you can also enjoy the features which were brought in Feature Pack 1 and 2.
You might have also noticed, that each Feature Update also contains a Cumulative Update. It is also possible to install these Cumulative Updates without the new capabilities which are provided by the Feature Packs. To check the most recent releases of these Cumulative Updates, please check this web site:
What’s new with BizTalk Server 2016 Feature Pack 3
This Feature Pack brings improvements in below mentioned areas:
- Compliance with US government accessibility standard
- Privacy – Compliance with FIPS and GDPR
- SQL Server 2016 SP 2 – Deploy multiple databases per instance of an Availability Group
- Office365 Outlook Email – Send and Receive messages using Office365 e-mail
- Office365 Outlook Calendar – Create appointments using Office365 schedules
- Office365 Outlook Contact – Create Office365 contacts
- Web Authentication – Authenticate with Azure AD and OAuth using Microsoft Enterprise Single Sign-On
- Advanced Scheduling – Set up recurrence on Receive Locations with greater precision
Compliance with FIPS and GDPR
As of this Feature pack, BizTalk Server is compatible with FIPS (Federal Information Processing Standard) and GDPR (General Data Protection Regulation).
FIPS is a standard developed by the US federal government for use in computer systems by non-military government and agencies. The specification comes with standards for among others:
- Codes – country, region and state codes, weather conditions, emergency indications
- Encryption – Data Encryption Standard, Advanced Encryption Standard
In case of BizTalk Server, this relates to how data becomes encrypted and decrypted in SQL Server. FIPS-compliance is enabled in the Operating System, under Local Policies; refer to the screenshot below. Once enabled, SQL Server will enter the FIPS compliant mode, thereby using cryptographic standards as defined in FIPS 140-2.
GDPR is a European law on data protection which intends to regulate the privacy of individuals in the European Union. This new law supersedes Data Protection Directive 95/46/EC from 1995.
If you want to read more about BizTalk and GDPR, there are few resources written by Sandro Pereira on this topic:
Multiple databases in Availability Groups SQL instances
Also SQL Server 2016 SP 2 is supported from Feature Pack 3. This is especially good news when you are running SQL Server Availability Groups. Because this allows you to have multiple BizTalk databases in the same SQL instance. As this was not possible earlier, this made setting up BizTalk Server in Availability Groups complex and expensive. The reason for that, is because you needed to have multiple SQL Server instances than without Availability Groups and more expensive as you need to license each SQL Server instance in the Availability Groups.
New Adapters for Office365 connectivity
Few other interesting features of this Feature Pack are the adapters for Office365 Email, Calendar and Contacts. These adapters allow you to use Office365 accounts for receiving and transmitting emails, creating and updating calendar items and creating contacts.
The Office365 Email adapter
This adapter can be uses both for receiving as for transmitting messages. In BizTalk terms, you can use this adapter both on Receive Locations as on Send Ports.
On the receive side the adapter enables you to:
Select a folder from which to retrieve email
Select a timestamp from which you want to receive emails
Retrieve unread emails only
Select an action after the email has been read, like marking the email as read or deleting the email
When transmitting emails via a Send Port, you can set the following properties:
To – separate email addresses with a semicolon (‘;’), maximum 256 characters
CC – separate email addresses with a semicolon (‘;’), maximum 256 characters
Subject – enter maximum 256 characters
Importance – select Low, Normal or High from the drop down in the Send Port
Also, it is important to realize that you can only send plain text messages.
Once you receive an email on a Receive Location, the Receive pipeline adds promoted properties, which you can use for routing the emails. These promoted properties are:
The Office365 Calendar adapter
You can use this adapter both for receiving events as for transmitting events. To be able to receive/send events you need to have the XSD schemas for both operations. You will find these schemas here:
Program Files (x86)Microsoft BizTalk Server 2016SDKSchemas
Related to calendars, this folder contains the following schemas:
The advantage of having these schemas is, that you can determine yourself which elements you will promote for routing purposes.
When you want to receive calendar items, you can select a calendar from an Office365 Outlook account. Next, you can configure that you want to receive events which are starting within a particular time frame.
You can also use the adapter to create events from BizTalk Server. Therefore, you can, populate a message according the above mentioned Send schema. Yet, some event properties can also be set on the Send Port. The properties you can set on a Send Port are:
The Office365 Contact adapter
You can use the Contact Adapter for creating contacts in Office365. For this adapter too, Microsoft has provided a schema. You will find this schema in the same location as the Calendar schemas and has the following name:
Again, because you can add this schema to your BizTalk solution, you can determine yourself which fields you would like to promote. The Send Port configuration only allows you to sign in with an Office365 account. You cannot configure any contact properties on the Send Port.
Note: To be able to use the Office365 adapters from BizTalk, besides Feature Pack 3, you need to install TMS. This service refreshes the Office OAuth tokens which BizTalk uses. After installation of the Feature Pack, navigate to the installation folder of BizTalk Server (Program Files (x86)Microsoft BizTalk Server 2016). There you will find BizTalkTMS.msi, which you must install.
Read this article for more details about the Office365 adapters for BizTalk Server 2016.
Advanced Scheduling of Receive Locations
Scheduling Receive Locations has always been hard with the out-of-the-box capabilities of BizTalk Server. Often people used the open source Scheduled Task Adapter which exists since BizTalk Server 2004. Currently, Sandro Pereira maintains this adapter on GitHub.
The open source Scheduled Task Adapter does a good job. But, organisations might prefer not to go for open source software, but still they might have a need to schedule Receive Location(s). Microsoft has listened to their requests and introduced Advanced Scheduling of Receive Locations with Feature Pack 3!
Below screenshots show how the scheduling capabilities of a Receive Location look like, before and after installation of Feature Pack 3.
In case of the Feature Pack 3 scenario, the upper part of the screen is extended with capabilities to select a time zone and to configure automatic adjustment for Daylight Saving Time (DST).
The ability of setting the time zone can help in case your integration partner(s) live in different regions of the world. Using the Time Zone drop down will make it easier to configure when Receive Locations will pick up messages according the partner’s time zone.
Daylight Saving Time, also mentioned Summer Time, is a practice which is done in a number of countries in the Northern and Southern hemisphere. During DST, the clock becomes advanced one hour close to Spring and adjusted backwards in Autumn. This way daylight lasts longer in the evening. These adjustments might effect the proper working of your interfaces. That’s why Microsoft introduced the ability to automatically adjust the schedule of your Receive Locations, according DST.
By the way: did you know that the Data Monitoring features in BizTalk360 are also compliant to Daylight Savings? Our colleague Mekala wrote an article about it. Here you have the link to that article:
Especially the Service Window capability of Receive Locations are improved hugely. The original ability to configure just a Start and a Stop time has been extended with the following recurrence features:
- Daily – configure the number of recurring days and from which date the recurrence will be active
- Weekly – configure the number of recurring days, from when the recurrence will be active and on which weekdays the recurrence must be active
- Monthly – configure which months and which days the recurrence should take place
With BizTalk Server 2016 Feature Pack 3, Microsoft has released many useful features for BizTalk Server 2016. The Office365 adapters, the improved scheduling capabilities and the support of SQL Server 2016 SP2 (because of simplification of Availability Groups) can be considered as the most useful ones.
Meanwhile, the community is also waiting for the release of BizTalk Server vNext which has been announced recently. This version of BizTalk underlines the on-going commitment of Microsoft in the platform. The new version of BizTalk will, amongst others, contain all the release Feature Packs for BizTalk Server 2016.
You can receive this kind of announcements and many other interesting articles in the Microsoft Integration space directly in your email box, by subscribing to our monthly Integration Newsletter. You can find the August edition of this newsletter, which provides more information about BizTalk vNext, here.
Why do we need this feature?
One of the tools which come with BizTalk Server is the Enterprise Service Bus (ESB) Toolkit. Although the ESB Toolkit is a collection of tools that extend the service-oriented capabilities of BizTalk, the Exception Management capability is one of the most widely used features of the ESB Toolkit. This feature allows you to perform Exception Management in a centralized manner, which can be a great benefit.
What are the current challenges?
See below for a list of challenges while using the out-of-the-box ESB Toolkit portal, which comes with BizTalk Server:
- Bad quality portal
- Lack of statistic information
- No security/auditing
- No monitoring
Bad quality portal
Even though the Exception Management framework is very robust and strong, the Exception Management Portal that comes out of BizTalk Server is not that easy to configure (takes about 1 or 2 days to configure the default portal successfully).
Besides the quality of the portal, the ESB portal is yet another portal, users need to be aware of in their day to day operations with BizTalk Server.
Lack of statistic information
Further, the ESB Toolkit gives no overview of for example the amount of ESB Exceptions that occurred, the number of itineraries processed or the number of resubmissions. This kind of information can be of vital value to know if the processes are all in a healthy state.
Also, no security and auditing is in place for this portal. Anybody with access to the portal can view the exceptions, edit messages and resubmit them, which exposes its own risks.
To be able to keep an eye on the ESB processes, it is handy to monitor these processes. Unfortunately, with the ESB Toolkit, no such capability exists.
How BizTalk360 solves this problem?
BizTalk360 addresses these challenges in a number of ways. Firstly, the portal from the ESB Toolkit is replaced with a portal within BizTalk360. As with all features within BizTalk360, this portal is protected with security and auditing. The following policies are available:
- Using the portal
- Being able to resubmit messages
To be able to repair ESB Faults, there is an Edit and Resubmit feature. To make repair even easier, you can write and associate Knowledge Base articles to the ESB Faults. So in case of recurring problems, you can simply document the solution in such a KB article and associate it with the ESB Fault, thereby making repairs in the future easier.
For having a good overview of the ESB processes, BizTalk360 provides a customizable ESB Dashboard. You can create dashboards based on different categories of widgets, which are:
- ESB Fault count
- ESB Resubmission
- ESB Itineraries
Besides the ESB Dashboard, there is also the Data Monitoring feature in BizTalk360. In case of ESB, this will help you in making sure all the ESB processes run like expected.
With the ESB Exception Portal and all the other ESB focussed features, we think that we have brought another good feature to the product, taking away the need to use the out-of-the box features, while empowering the users with relevant other features.