Why did we built EDI Dashboard?

Why did we built EDI Dashboard?

Why do we need this feature?

Many customers use their BizTalk platform to exchange EDI (Electronic Data Interchange) messages with their partners. BizTalk Server contains a number of features for EDI like Interchange processing, Batch processing and Trading Partner Management.

From an operational viewpoint, all kind of features exist in BizTalk Server and many have found their way to BizTalk360 as well. However, besides querying for running processes, there is no feature which shows the EDI processes from a statistical viewpoint.

What are the current challenges?

Often, BizTalk Server is considered as a black box and organizations with little knowledge of the product, sometimes prefer not to touch it. This behaviour can be quite dangerous as the (EDI) processes in BizTalk Server might not run as expected. So, insight into these processes is required to be able to guarantee everything is okay.
Information like the number of AS2 messages or the transaction count by transaction type or by partner id over a certain period is very useful to determine the well-being of the processes, but cannot be revealed from BizTalk Server.

How BizTalk360 solves this problem?

BizTalk360 tries to breach this black box behaviour by bringing a platform which makes BizTalk Server environments easier to understand and safe to operate on.

Related to EDI, we bring a couple of features:

  • EDI Dashboards
  • EDI Status Reports
  • Reporting Manager
  • Parties and Agreements

EDI Dashboards

The EDI Dashboard is one of the features which enables the BizTalk user to get insight in what’s going in BizTalk. As the other dashboards in BizTalk360, also the EDI Dashboard is customizable and consists of widgets from different categories. In case of the EDI Dashboard, the following categories exist:

  • EDI Interchanges – widgets exist for
    • Interchange count by Partner Id/Name
    • Agreement
    • Encoding (X12/EDIFACT)
  • EDI Transaction Sets – widgets exist for
    • Transaction count by Partner Id/Name
    • Transaction type (filtered by Partner Id)
    • ACK Status (filtered by Partner Id)
  • EDI AS – widgets exist for
    • Number of messages by Partner Id
    • Number of messages by Partner Id and MDN status
    • Number of messages by MDN status

Below you can view some examples of EDI Dashboards you can create in BizTalk360.

The EDI Dashboard

The EDI Dashboard

The EDI Dashboard

EDI Reports

Comparable to the EDI query features in the BizTalk Administration Console, BizTalk360 brings a Reporting feature. On top of the set of queries you might know from the Admin console, BizTalk360 also provides a Functional ACK Status report.

EDI Reporting Manager

For easy configuration of your EDI reports, BizTalk360 brings the EDI Reporting Manager. See below for a screen print of that feature.

Parties and Agreements

Also the parties and agreements can be viewed in BizTalk360. See below for a screen print of that feature.

By bringing all these features, BizTalk360 gives good visibility of the EDI processes.

Author: Lex Hegt

Lex Hegt works in the IT sector for more than 25 years, mainly in roles as developer and administrator. He works with BizTalk since BizTalk Server 2004. Currently he is a Technical Lead at BizTalk360.

Why did we build EDI Dashboard?

Why did we build EDI Dashboard?

Why do we need this feature?

Many customers use their BizTalk platform to exchange EDI (Electronic Data Interchange) messages with their partners. BizTalk Server contains a number of features for EDI like Interchange processing, Batch processing and Trading Partner Management.

From an operational viewpoint, all kind of features exist in BizTalk Server and many have found their way to BizTalk360 as well. However, besides querying for running processes, there is no feature which shows the EDI processes from a statistical viewpoint.

What are the current challenges?

Often, BizTalk Server is considered as a black box and organizations with little knowledge of the product, sometimes prefer not to touch it. This behaviour can be quite dangerous as the (EDI) processes in BizTalk Server might not run as expected. So, insight into these processes is required to be able to guarantee everything is okay.
Information like the number of AS2 messages or the transaction count by transaction type or by partner id over a certain period is very useful to determine the well-being of the processes, but cannot be revealed from BizTalk Server.

How BizTalk360 solves this problem?

BizTalk360 tries to breach this black box behaviour by bringing a platform which makes BizTalk Server environments easier to understand and safe to operate on.

Related to EDI, we bring a couple of features:

  • EDI Dashboards
  • EDI Status Reports
  • Reporting Manager
  • Parties and Agreements

EDI Dashboards

The EDI Dashboard is one of the features which enables the BizTalk user to get insight in what’s going in BizTalk. As the other dashboards in BizTalk360, also the EDI Dashboard is customizable and consists of widgets from different categories. In case of the EDI Dashboard, the following categories exist:

  • EDI Interchanges – widgets exist for
    • Interchange count by Partner Id/Name
    • Agreement
    • Encoding (X12/EDIFACT)
  • EDI Transaction Sets – widgets exist for
    • Transaction count by Partner Id/Name
    • Transaction type (filtered by Partner Id)
    • ACK Status (filtered by Partner Id)
  • EDI AS – widgets exist for
    • Number of messages by Partner Id
    • Number of messages by Partner Id and MDN status
    • Number of messages by MDN status

Below you can view some examples of EDI Dashboards you can create in BizTalk360.

The EDI Dashboard

The EDI Dashboard

The EDI Dashboard

EDI Reports

Comparable to the EDI query features in the BizTalk Administration Console, BizTalk360 brings a Reporting feature. On top of the set of queries you might know from the Admin console, BizTalk360 also provides a Functional ACK Status report.

EDI Reporting Manager

For easy configuration of your EDI reports, BizTalk360 brings the EDI Reporting Manager. See below for a screen print of that feature.

Parties and Agreements

Also the parties and agreements can be viewed in BizTalk360. See below for a screen print of that feature.

By bringing all these features, BizTalk360 gives good visibility of the EDI processes.

Author: Lex Hegt

Lex Hegt works in the IT sector for more than 25 years, mainly in roles as developer and administrator. He works with BizTalk since BizTalk Server 2004. Currently he is a Technical Lead at BizTalk360.

Why did we build Business Rules Composer?

Why did we build Business Rules Composer?

Business Rules Composer BizTalk360

This blog is a part of the series of blog articles we are publishing on the topic “Why we built XYZ feature in BizTalk360”. Read the main article here.

Why do we need this feature?

Although in our experience many BizTalk Server customers are using the BizTalk Rules Engine, many others are not using it. This is a pity because the BizTalk Rules Engine (BRE) can be very beneficial and can reduce the costs of BizTalk development.

The concept behind the BRE, is to empower business users to create and maintain business rules, which can be accessed by their BizTalk processes.
In case such business rules need to be changed, this can be done via the so-called Business Rules Composer, by the business user.

A simple example on how to use the power of the Business Rules Engine, is a business user who works in retail and would like to, temporarily, apply some discounts to certain product groups.

Without the BRE, the user would have to implement a change in his BizTalk solution, which can be time-consuming and rather costly. Especially, as the discounts will have to be reverted after some time.
With the BRE, the user can use the Business Rules Composer to make that change by himself.

The Business Rules Composer tool comes with BizTalk Server and as demonstrated, using this tool is far more cost efficient than having to line up a development team to make the needed change in the BizTalk process, while maybe having to revert the change after short period of time, like we have seen in case of temporarily discounts.

Besides being able to modify business rules with the Business Rules Composer, it is obvious that it is also possible to just view the business rules to gain insight how the business rules are set up.

What are the current challenges?

As mentioned, the BizTalk Server Business Rules Composer enables to view, modify and deploy business rules. There are however a few challenges with the Business Rules Composer:

  • It is too technical
  • It has no security and auditing

Business Rules Composer is too technical

In most cases the Business Rules Composer is just installed on the BizTalk server(s). Although, from a technical perspective, it is possible to install the Business Rules Composer on a normal desktop computer.
But, even when the business user does have access to the Business Rules Composer, we’ll explain that the Business Rules Composer is too technical in the hands of a business user.

Let’s briefly describe a bit about Business Rules. These rules consist of the following parts:

  • Policies – a policy is a collection of one or more business rules. Policies can be created, edited, deleted, versioned, published and deployed. Once a policy is deployed, no more changes can be made to the policy. If changes are needed, a new version of the policy will have to be created.
  • Business rules – a business rule can be constructed based on conditions, which will be evaluated by the Business Rules Engine. To construct these business rules, elements of the below mentioned vocabularies can be used.
  • Vocabularies – vocabularies contain the components which can be used in the policies. These components can be elements/attributes in a XML schema, fields in a database table or elements in a .NET class.  For ease of use, these components can be provided with a friendly (business) name. Just like policies, vocabularies can be versioned and published.

After vocabularies and policies have been created, they can be saved, published and deployed from the Business Rules Composer as well.

It must be quite clear now, that the Business Rules Composer is a powerful tool, but is quite complex at the same time. The tool is suited better for technical people who are well aware of the technical insides. As business users don’t have that knowledge, it will make it hard for business users to use the original Business Rules Composer.

Business Rules Composer has no security and auditing

In the Business Rules Composer which comes with BizTalk Server, there are no security and auditing features. So, once somebody has access to the tool, they have access to all the business rules of the BizTalk environment and can make any kind of changes, without leaving a trace. We think that this is an unacceptable situation.

How BizTalk360 solves this problem?

BizTalk360 offers a web-based counterpart of the Business Rules Composer. As BizTalk360 is web-based, the Business Rules Composer is easier to access than the software which comes with BizTalk Server, as it is quite often just installed on the BizTalk Server.

BizTalk360 Business Rules ComposerFrom the BizTalk360 Rules Composer, users can for example:

  • Add, edit and remove policies
  • Add, edit, test and delete rules
  • Publish, deploy and undeploy policies

Security

Once a user has access to BizTalk360, the user can have different authorizations for the Rules Composer of BizTalk360. These authorizations are:

  • View rules only
  • Create/Save rules
  • Deploy/Publish rules

Besides security settings, there is also auditing of activity around business rules in BizTalk360.

By hiding irrelevant technical stuff in the BizTalk360 Business Rules Composer, this tool is better suited for business users than the one from BizTalk Server. Therefore, we only show the Policies and Vocabularies.

Additional features

We also brought some features to the BizTalk360 Business Rules Composer which do not exist in the original tool. Think of for example:

  • a Where function – this can be used to query database tables and retrieve the information
  • If, Else, Else If – these statements can be applied to the Actions part of the business rule, to be able to create more complex rules
  • Date Time Picker – to minimize errors while creating rules, we provide a date time picker for any function that involves picking a date/time
  • Download Test Result – besides testing of policies, you can also download XML files, which are modified as result of testing these policies

Conclusion

All in all, the BizTalk360 counterpart of the Business Rules Composer provides similar features as the one from BizTalk Server, while providing easier access, usage and security. By bringing few additional features, we make the feature even more powerful.

Do you want to know more about the Business Rules Composer in BizTalk360? You can read more on our documentation portal:

https://docs.biztalk360.com/docs/web-based-bre-composer-in-biztalk360

Get started with a Free Trial today!

Download and try BizTalk360 on your own environments free for 30 days. Installation will not take more than 5-10 minutes.

BizTalk360 30 Days Free Trial

Author: Lex Hegt

Lex Hegt works in the IT sector for more than 25 years, mainly in roles as developer and administrator. He works with BizTalk since BizTalk Server 2004. Currently he is a Technical Lead at BizTalk360.

Why did we built dedicated dashboard pages for BizTalk and SQL Servers?

Why did we built dedicated dashboard pages for BizTalk and SQL Servers?

Why do we need this feature?

The BizTalk and SQL servers are the heart and the bones of a BizTalk environment. Where the SQL Servers take care of all the database handling for BizTalk, the BizTalk servers take care of the actual message processing.
When something is wrong with one of these servers, changes are that this will affect the overall health of your BizTalk environment. So, it’s obvious that the well-being of the servers has to be considered for the overall well-being of the BizTalk environment. For that reason, administrators need to login to the servers to check the health of the distinct servers and come into action when needed.

What are the current challenges?

To be able to login to all the servers, the administrator needs to set up Remote Desktop connections to all these servers. Logging in, checking all the needed consoles and actioning when needed, can be time-consuming. Besides that, in highly secured environments, the administrator might not have access to the servers, from remote locations.

How BizTalk360 solves this problem?

Given the One platform philosophy after BizTalk360, we made access to the BizTalk and SQL servers from a BizTalk environment easy and secure. As the product retrieves the BizTalk servers of BizTalk environments from their Management database, it can provide easy access to these servers via the UI of BizTalk360. Also the SQL servers can be accessed and actioned from BizTalk360.

Overview BizTalk serverOverview SQL server

For each server, a number of the key components are shown, so you immediately have an idea on topics like CPU/Memory configuration and usage, installed server products and BizTalk Host configuration.
When you click on the orange strip, you get more detailed information of the server. In case of BizTalk servers you will have access to:

  • CPU Usage
  • Memory usage
  • Server products
  • Host Instances
  • Event Viewer (Application Log)
  • NT Services

You can Stop/Start and Enable/Disable Host Instances. NT Services can be stopped, started and restarted.

For the SQL servers you’ll have access to:

  • CPU Usage
  • Memory usage
  • Server products
  • NT Services

NT Services can be stopped, started and restarted.

Detailed view BizTalk server

By providing dedicated dashboards for the BizTalk and SQL servers of your BizTalk environment, you have easy access to important server aspects, without the hassle of having to use Remote Desktop connections.

Author: Lex Hegt

Lex Hegt works in the IT sector for more than 25 years, mainly in roles as developer and administrator. He works with BizTalk since BizTalk Server 2004. Currently he is a Technical Lead at BizTalk360.

Why did we built Message Box Viewer (MBV) / BizTalk Health Monitor (BHM) Monitoring?

Why did we built Message Box Viewer (MBV) / BizTalk Health Monitor (BHM) Monitoring?

Why do we need this feature?

The BizTalk Health Monitor (BHM) and its predecessor MessageBox Viewer (MBV) are important tools to learn about the overall health of BizTalk environments.
The tools are developed and maintained by Microsoft and firstly used by their Premier Field Engineers, to help them identify problems with BizTalk environments. Later, the tool became publicly available, enabling BizTalk administrators to run the tool themselves as well. Since that time, we have seen many updates of the tools, although MBV is deprecated for few years now.

The BizTalk Health Monitor can be downloaded here:

https://www.microsoft.com/en-us/download/details.aspx?id=43716

The best practice is to run BHM frequently, say once per day at a quiet time, to be aware of overall health and check for any critical issues.

MBV did not offer a built-in scheduler, so you had to schedule it yourself with Windows Scheduler. With the introduction of BHM, the tool came with a plugin, to use it from the BizTalk Server Administration console and it also came with scheduling features.

Once scheduled, you’ll receive the output in the configured mailbox. So you have the fully detailed report easily at hand.

What are the current challenges?

Although it’s good to have the output of MBV/BHM runs in your mailbox, we think that the overall experience still can be improved.

In your day to day activities as a BizTalk administrator, you might receive many notifications in your mailbox. If not setup properly, the amount of notifications might be overwhelming, which might result in starting to ignore the notifications, thereby risking to miss important notifications.

So, you will want to focus on just the notifications which indicate something needs to be taken care of. In case of the MBV/BHM output, you will only be interested in critical and non-critical errors on which some actions must be taken. In other words, if all is healthy, you might not be interested to receive that output at all.

With BHM you will always receive the output, regardless if all is healthy or some (non-)critical issues exist. No clear notification can be received, in case something serious is going on.

Besides that, MBV/BHM is another tool/console you need to be aware of to consider the overall health of a BizTalk environment. As we want to apply our One Platform philosophy as much as possible, we brought integration with MBV/BHM to BizTalk360.

In the next section of this article, we’ll show you how BizTalk360 helps you to improve the experience of BHM and how you can have monitoring on the output of MBV/BHM runs.

How BizTalk360 solves this problem?

BizTalk360 has integration with MBV/BHM for many years. This integration enables the BizTalk360 user to schedule MBV/BHM and view the output of the different runs of the tool directly from within BizTalk360.

Schedule BHM runs

This makes it easy to run MBV/BHM and view its output in BizTalk360, instead of delivering the output of each run to the mailbox of the user.

However, this does not yet solve the problem of just being notified in case of critical errors. To achieve that, you can simply add the BizTalk Health Monitor to a BizTalk360 alarm. In BizTalk360, you can do this by navigating to Monitoring => Manage mapping => BizTalk Environment => BizTalk Health Monitor. Here you can configure how many critical or non-critical may occur in the output of BHM, before a notification is being sent.

Monitor BHM output

As you can see from above screenshot, under Error Threshold, you can configure how many critical/non-critical errors may exist, before a notification is being sent. From the Current Configuration of the screen, you can see that in the last report there were 2 critical and 20 non-critical errors. So, in above scenario, the error thresholds have been met and a notification will be send.

If you want more fine-grained monitoring of the BHM output, BHM offers you the option to store the output of the BHM runs in a database. Say, you want to monitor on specific categories like Tracking or the BizTalk jobs, you could achieve that with Database query monitoring.
When you would like to know more on how to set this up, feel free to contact us (support@biztalk360.com) and we are happy to work with you on the scenario at hand.

Conclusion

The integration of MBV/BHM with BizTalk360 adds to the One platform philosophy. By also bringing monitoring of the output of MBV/BHM, we remove some of the clutter which appears in the mailbox of the user, providing good focus on the topics which really matter. With some customization, you could have even more fine-grained monitoring on the output of BHM.

There are some more articles on MBV/BHM integration. Check them out here:

Get started with a Free Trial today!

If you would like to try BizTalk360, why not give it a try. It takes about 10 minutes to install on your BizTalk environments and you can witness and check the security and productivity on your own BizTalk Environments. Get started with the free 30 days trial.

Author: Lex Hegt

Lex Hegt works in the IT sector for more than 25 years, mainly in roles as developer and administrator. He works with BizTalk since BizTalk Server 2004. Currently he is a Technical Lead at BizTalk360.

Why did we built Message Box Viewer (MBV)/ BizTalk Health Monitor (BHM) Integration?

Why did we built Message Box Viewer (MBV)/ BizTalk Health Monitor (BHM) Integration?

This blog is a part of the series of blog articles we are publishing on the topic “Why we built XYZ feature in BizTalk360”. Read the main article here.

Why do we need this feature?

The BizTalk Health monitor (BHM) is the successor of the Message Box Viewer (MBV). Both tools have been developed by Microsoft for health analysis of BizTalk environments. Despite its (former) name Message Box Viewer, the tool checks many more parts of BizTalk Server than just the Message Box database(s).
For example, the tool checks whether:

  • the latest Cumulative Updates are installed
  • relevant hotfixes are installed
  • all kind of configuration (Windows, BizTalk, SQL) is conform best practices, think of
    • Correct Host Configuration
    • Auto-growth settings of your BizTalk databases
    • Unnecessary Tracking switched on
  • your BizTalk databases are in good health
  • the well-being of your BizTalk related SQL jobs
  • there are (relevant) Event Log errors

After Microsoft deprecated the Message Box Viewer software few years ago, they introduced the BizTalk Health Monitor (or BHM). BHM uses the same engine as the Message Box Viewer, but also comes with some improvements. These improvements are mainly related to:

  • Visibility – the BHM software comes with a MMC snap-in, which allows you to have both the BizTalk Server Administration console and the BizTalk Health Monitor software in a single MMC snap-in
  • Scheduling – you can now setup runs of BHM directly in the tool itself. In the background BHM still uses the command line version of the tool and the Windows scheduler to perform the runs. By creating multiple profiles, you can even perform runs for different BizTalk Server environments, from the same instance of BHM.

Just like with the Message Box Viewer, it is advised to run BizTalk Health Monitor on a regular basis. The tool comes with most recent versions of BizTalk Server and needs to be installed on the BizTalk Server. The tool can also be downloaded here:

https://www.microsoft.com/en-us/download/details.aspx?id=43716

Once you have installed BHM, you can setup scheduling so you will automatically retrieve health reports in your mailbox.

What are the current challenges?

Although BHM contains important information on the health of your BizTalk environment, it is yet another console you must be aware of. Especially organizations with less knowledge of all the consoles needed to support BizTalk Server, this one can easily be overseen or forgotten, with potentially, catastrophic results.

To be able to access BHM, be it for configuration of the runs of BHM or viewing (historic) reports, you need Remote Desktop access to the BizTalk server on which the BizTalk Health Monitor software is installed. Logging in to the BizTalk server repeatedly is time-consuming and can prevented by using the MBV/BHM integration which is discussed in the next paragraph.

How BizTalk360 solves this problem?

As you probably know, the vision behind BizTalk360 is to provide One platform for BizTalk Server. By providing One platform, aka one user interface, we bring all the different consoles, a BizTalk administrator might need to be able to do his job, under one hood.
For that reason, we already brought integration with Message Box Viewer years ago and updated the integration to support the BizTalk Health Monitor. This integration exists of the following features:

  • Schedule runs of BHM
  • Run BHM profiles
  • Manually run BHM
  • View reports of BHM
  • Monitor the results of BHM runs

Schedule runs of BHM

For many years, BizTalk360 already had integration with the predecessor of BizTalk Health Monitor, being Message Box Viewer. This integration included the scheduling capability, which was brought by BHM, when the tool became available, as well.
So, when we replaced MBV integration with BHM integration, of course we made sure that you would be able to setup BHM schedules from BizTalk360 as well.

Schedule BHM runs

Run BHM Monitoring profiles

BHM allows users to create specific so-called Monitoring profiles, which can be used to execute a subset of all the available queries and rules. This can be used, for example, to monitor just specific components or artifacts.
BizTalk360 allows you to run or monitor on such a profile, by creating such a profile in BHM and manually copying it to the BizTalk360 installation folder. If you want to read more about this capability, read below article:

Getting BizTalk360 to run a BHM Monitoring profile

Manually run BHM

Besides being able to run BHM based on a schedule, as shown in the previous paragraph, you can also run the tool immediately from BizTalk360. By clicking the ‘Run BHM Now’ button (see the picture below), BizTalk360 schedules a run immediately and the results can be checked after the run has been processed.

View reports of BHM

The reports of all the different runs of BHM can be viewed directly in BizTalk360. The screenshot below shows the dropdown from where you can select, which report you would like to see. The most recent report is shown on top.

Storing the output of BHM runs
The BHM reports are stored in the BizTalk360 database and can reside there for a configurable time frame. By default, this time frame is 5 days, but you can choose values between 1 day until any number of days/weeks/months you prefer. Of course, it is wise to not to keep the date for too much time, as this will affect the size of the BizTalk360 database.
After the time frame expired, BizTalk360 will automatically delete the expired reports.

View BHM output

Monitor the results of BHM runs

Besides running and viewing the output of BHM, you can also setup monitoring on the output. This way you can get notified easily of critical and non-critical errors which might be found by BHM.

Monitor on BHM output

In another article of the `Why we built XYZ feature in BizTalk360` series, we`ll explain monitoring of BHM output in more detail.

Conclusion

By executing BizTalk Health Monitor runs, the tool reports about important topics on all kind of health and best practice related stuff. Having MBV/BHM integration in BizTalk360, the product makes it easier to schedule runs of BHM and provides you with easy access to the output of the BHM.

Get started with a Free Trial today!

Do you prefer to setup your BHM schedules and view BHM output in BizTalk360, instead of setting up RDP connections to your BizTalk server? Why not give BizTalk360 a try! Installation takes about 10 minutes on your BizTalk environment. Get started with the free 30 days trial.

Author: Lex Hegt

Lex Hegt works in the IT sector for more than 25 years, mainly in roles as developer and administrator. He works with BizTalk since BizTalk Server 2004. Currently he is a Technical Lead at BizTalk360.

Why did we built Advanced Tracking Manager for BizTalk Server?

Why did we built Advanced Tracking Manager for BizTalk Server?

This blog is a part of the series of blog articles we are publishing on the topic “Why we built XYZ feature in BizTalk360”.

Advanced Tracking Manager for BizTalk Server

Why do we need this feature?

Being able to track messages and processes in middleware software like BizTalk Server can be used for few reasons, amongst them are:

  • be sure all processes run like expected
  • analysis of issues
  • debugging during development/test phase

However, especially for live environments, the best practice is that only limited tracking should be turned on, to prevent a performance penalty. Therefore, there needs to be a balance between the amount of tracking which is turned on and the ability for the BizTalk administrator to be able to do his job when it comes to analysis of issues.

In general, when no message context/content tracking is turned on and only the default tracking is switched on, you should be good.

To prevent performance penalties, it is obvious that it helps to have a good overview of all the tracking settings of the BizTalk environment.

What are the current challenges?

All BizTalk Server related tracking settings can be maintained in the BizTalk Server Administration console. However, the tracking settings are found in different parts of the console. For example:

  • the Group Level Tracking property is found  in the BizTalk Group Settings Dashboard
  • the BizTalk artifact settings are found at application/artifact level
  • Each artifact like (receive ports, send ports, orchestrations, pipelines etc) all come with their own UI to manage tracking.

This results in not having a good overview and easy maintenance of the tracking settings with the standard BizTalk Server Administration console.

How BizTalk360 solves this problem?

At BizTalk360 we decided to design one screen, which can be used for all tracking settings, enabling to have a good overview of all the tracking settings, meanwhile working as a central hub to configure all these settings.

We came up with the Tracking Manager screen, which can be found in BizTalk360 under Operations / Infrastructure setting.

As you can see from the screen, it contains an overview of all tracking settings. On a high level, the screen is divided into two parts:

  • Quick actions
  • Applications and Rules

The Quick actions allow you to easily turn on/off tracking on a global level for the BizTalk artifact types, but also to turn on/off group level tracking.

The second part of the screen allows you to view/configure tracking until the application level. If a more fine-grained configuration is needed, you can simply click on the application and set the desired tracking for one or multiple artifacts at once. The overall result of the settings will be reflected in above screen.

Also, maybe less familiar, Business Rules can be tracked. BizTalk360 enables you to easily configure these settings from the Rules tab page.

All in all, we can conclude that the Tracking Manager in BizTalk360 gives easy access and maintenance to all the tracking settings for BizTalk Administrators.

Get started with a Free Trial today!

Download and try BizTalk360 on your own environments free for 30 days. The installation will not take more than 5-10 minutes.

Author: Lex Hegt

Lex Hegt works in the IT sector for more than 25 years, mainly in roles as developer and administrator. He works with BizTalk since BizTalk Server 2004. Currently he is a Technical Lead at BizTalk360.

Why did we build Receive Locations, Send Ports and Orchestrations Monitoring for BizTalk?

Why did we build Receive Locations, Send Ports and Orchestrations Monitoring for BizTalk?

This blog is a part of the series of blog articles we are publishing on the topic “Why we built XYZ feature in BizTalk360“.

BizTalk Server Receive Locations, Send Ports and Orchestrations Monitoring

Why do we need this feature?

Microsoft BizTalk Server is a powerful middleware product which allows integrating all kind of information systems, independent of whether these systems are on-premise or in the cloud. BizTalk Server uses Receive and Send ports to communicate with a broad set of on-premise/cloud systems, both on the technology level (FILE, FTP, POP3, SMTP, etc.) as on the business level (ERP, CRM, etc.).

Besides just receiving and transmitting messages, BizTalk solutions might also contain Orchestrations, which can process all kind of business logic,  for example, Order processing, Credit card validation, Claims processing etc.

As BizTalk Server is very scalable, a BizTalk environment can contain many deployments for all kind of different integrations. All together a BizTalk environment may contain hundreds of ports and orchestrations. The well-being of these artifacts is of vital importance to have the integrations working properly. When not running properly, these interruptions may cause serious damage to the business processes they support.

What are the current challenges?

To be well aware of the state of the BizTalk artifacts, the BizTalk administrator needs to consult the BizTalk Administration console multiple times per day. However, there are few challenges when depending on the BizTalk Administration console for checking the state of these artifacts.

Access to the Administration console – Often, the BizTalk Administration console is only installed on the BizTalk servers. Besides that there is the risk of unintended damaging the BizTalk server while accessing the server, you need to be authorized to access the servers. Not to be forgotten, is the fact that it constantly takes time to setup Remote Desktop connections to access the BizTalk server and the Administration console. Time which can easily be used for something more tangible.

Not suited for less experienced administrators – The Administration console is such a powerful tool, which is great when you are an experienced BizTalk Administrator. However, in the hands of less experienced administrators, unintentionally a lot of damage can be done. Think of for example stopping ports or terminating processes!

Not an efficient way of monitoring – Checking the state of the ports and orchestrations manually, multiple times per day, is just not efficient. It can easily be forgotten in the rush of the day, thereby putting the wellbeing of your integrations at stake.

Too many artifacts to monitor efficiently – BizTalk environments can contain hundreds of ports and orchestrations. In most scenarios, the artifacts should be in the Enabled/Started state. But there can be several reasons why certain artifacts are not in the expected state. Just to mention a few:

  • After the deployment of an update of a BizTalk application the artifacts are in a wrong state
  • Due to a temporary maintenance window, some of the artifacts need not be in the Enabled/Started state, to prevent suspended instances
  • Artifacts might be Disabled as a result of a temporary outage
  • Certain artifacts need only be Enabled/Started as a fall back scenario

By the given examples, it must be clear, that it can be very challenging to constantly be aware of the desired state of all the artifacts. This makes it hard to manually monitor the state of the artifacts in an efficient way.

Instead of having to monitor BizTalk manually, it is much more efficient to have automated, state-bound, artifact monitoring. Automated BizTalk monitoring will prevent you of constantly having to check for the state manually, enabling you to spend your time on more important activities. In the next section, we’ll show which features BizTalk360 has, to allow you to automatically monitor your BizTalk artifacts.

How BizTalk360 solves this problem?

To give you that comforting feeling that you are in control over the state of your ports and orchestrations, BizTalk360 provides multiple features.

Threshold Monitoring – While using the concept of Threshold Monitoring, BizTalk360 allows you to setup state-bound monitoring. This means that you can easily setup monitoring to have the product check for the Expected state of your ports and orchestrations.

In above screenshot, you can see how some Send Ports are being monitored against their Expected State. Once, the Current State deviates from the Expected State, a threshold violation has occurred and BizTalk360 will notify you of that fact.

Positive and Negative monitoring – In most scenarios, the Expected State of your Ports and Orchestrations will be Enabled/Started; which is called Positive Monitoring. However, there might be valid reasons that certain ports/orchestrations need NOT be Enabled/Started. Setting up that kind of monitoring is called Negative Monitoring. BizTalk360 allows to setup for both Positive and negative Monitoring.

Receiving Notifications via multiple channels – Once a threshold violation has occurred, by default, you will receive notifications by email. Moving forward, BizTalk360 also allows you to receive notifications via multiple other channels.

We are constantly considering new ways to send notifications, but currently, the following Notification Channels are provided out-of-the-box: Email, SMS, Event Log, Slack, Microsoft Teams, ServiceNow, HP Operations Manager, Webhook. In addition, you can build your own custom notification channel like writing to a database, calling an internal system etc.

Auto Correct – BizTalk360 not just monitors your artifacts, it can also try to bring the artifacts back to their Expected state, once there is a mismatch between the Current State and the Expected State. Think of for example that FTP Receive Location which goes down at night or during the weekend.

The Auto-Correct feature works for the following artifacts:

  • Receive Locations
  • Orchestrations
  • Send Ports
  • BizTalk Host Instances
  • Windows NT Services
  • SQL Server Agent Jobs
  • Azure Logic Apps

There is an extensive article on BizTalk360 Auto correct, which also explains in which kind of scenarios this feature comes at hand. You can that article here:

Monitoring Dashboard – Although receiving notifications of any mismatches between the Current State and the Expected State is very handy, it will equally be handy to have a large screen, at for example the Support Desk, which simply shows the current health of your artifacts. For this, we bring the Monitoring Dashboard, which does exactly that.

As you can see from below screenshot, it brings you a nice Expandable/Collapsible and filterable treeview, with automatic refresh, thereby constantly showing the current state of the artifacts.

Health Check Reports – Last but not least, are the Health Check Reports you can receive at your convenient timing. These reports function as your daily health check, providing you with an overview of the state of all the artifacts from your alarm. Very handy to receive for example at the beginning of your working day!

Conclusion

In this article, we have seen why being aware of the state of your ports and orchestrations is important. We have also seen why manually monitoring that state is very inefficient and sometimes very complex. As a company which knows these challenges from our own experiences, we have brought multiple features which will help the BizTalk administrator being in control, without having to manually check the state.

Get started with a Free Trial today!

Download and try BizTalk360 on your own environments free for 30 days. The installation will not take more than 5-10 minutes.

Author: Lex Hegt

Lex Hegt works in the IT sector for more than 25 years, mainly in roles as developer and administrator. He works with BizTalk since BizTalk Server 2004. Currently he is a Technical Lead at BizTalk360.

Why did we build FTP/SFTP Monitoring for BizTalk Server?

Why did we build FTP/SFTP Monitoring for BizTalk Server?

This blog is a part of the series of blog articles we are publishing on the topic “Why we built XYZ feature in BizTalk360”.

BizTalk Server FTPS-FTP-SFTP Monitoring

Why do we need this feature?

In the day to day activities of a BizTalk administrator, you might come across integrations where FTP sites are used for receiving and transmitting messages. FTP sites are often used for cross-platform integrations. For example, when you have an SAP system on Unix that has to be integrated, via BizTalk Server, with other systems, you might use FTP for receiving and transmitting of messages.

SFTP & FTPS are just the secured version of FTP with advanced transport encryption mechanisms, so your end-to-end data transmission is secure and safe.

To keep the business process going, it can be of vital importance that the FTP/SFTP sites are online and the messages are being picked up. So, when a BizTalk administrator needs to be constantly aware whether the FTP/SFTP sites are online and working properly, the administrator needs to monitor the sites and the activities which take place on these sites.

What are the current challenges?

BizTalk Server offers no monitoring capabilities, not for Receive Locations / Send Ports and also not for endpoints like FTP, SFTP and FTPS sites. So, using just the out-of-the-box features of BizTalk Server, a BizTalk administrator will have to manually check whether the FTP sites are online and whether all (appropriate) files are being picked up for further processing.

Manual monitoring

This kind of manual monitoring can be quite cumbersome and time-consuming. The administrator will probably use multiple pieces of software to be able to perform these tasks. Think of for example the BizTalk Administration console to check whether the Receive Locations/Send Ports are up and some FTP client to check whether files are being picked up.
It is obvious that this is not a very efficient scenario, which could easily be automated by setting up monitoring.

Maintaining scripts for monitoring FTP sites

To reduce their workload, we experience that BizTalk administrators are creating their own scripts to monitor FTP sites and all kind of other resources. Although this kind of scripts certainly can be of help, we still think this does not fully solve the problem.

For example, often these kinds of scripts need maintenance when FTP sites need to be added, changed or deleted from monitoring. This kind of tasks can be easily forgotten.

Also from a knowledge transfer perspective, it’s easy to forget to update new colleagues about the existence of this kind of scripts, as they will probably be installed on some (monitoring) server.

Another challenge with solving this kind of problems with scripts is that not each administrator is capable to write this kind of scripts, which makes knowledge transfer even harder.

To keep the overview, we think that it is easier to use software, like BizTalk360, to have everything in one easily accessible place, with good visibility of all the features/capabilities, fine-grained security/auditing and without the need to maintain custom scripts etc..

How BizTalk360 solves this problem?

With BizTalk360, we make monitoring of FTP/SFTP/FTPS sites a lot easier. For a very long time, the product offers monitoring of Receive Locations and Send Ports, but for some time now, BizTalk360 also offers to monitor of the physical FTP/SFTP/FTPS endpoints.

We wanted to make setting up this kind of endpoint monitoring as seamless as possible and therefore we simply show all the ports in the current BizTalk group which make use of the FTP/SFTP/FTPS adapter.

In BizTalk360, you can find FTP monitoring under Monitoring => Manage Mapping => File Locations (File, FTP, SFTP).

Next, you can set up monitoring rules based on File Count and Directory Size and have BizTalk360 send Warning or Error notifications through the notification channels which are configured on the associated alarm.

A fully monitored FTP endpoint might look like shown below.

Of course, besides the greater-than-or-equals operator, also other common operators are available.

Conclusion

As a final point, we see from time to time that administration teams maintain a administrators handbook, which contains all the tasks a (BizTalk) administrator should take care of. We think that by using software like BizTalk360, we can reduce the number of pages in such handbooks, as the kind of scripts we mentioned no more have to be described in that kind of books.

This description could be replaced by, for example, a general guideline on how FTP sites should become monitored and the monitoring rules with BizTalk360.

As a result, we hope to make the work of BizTalk administrators a bit easier so the team can focus on the more exciting parts of the job of BizTalk administrators, instead of constantly having to update their handbooks.

So, we think that we make the day to day life of a BizTalk administrator, who needs to monitor the well-being of FTP sites, a little bit easier by bringing this feature.

If you want to read more on FTP/SFTP monitoring in much more detail, you can check the following article:

FTP, FTPS, SFTP Location monitoring

BizTalk360 also offers an advanced monitoring capability called “Data Monitoring” which allows monitoring the traffic/volume of messages going through the ports for a given period, ex: expected 50 PO orders from our partner via FTP/SFTP. Please check out this article. Introducing BizTalk Server Data Monitoring in BizTalk360

Get started with a Free Trial today!

Download and try BizTalk360 on your own environments free for 30 days. Installation will not take more than 5-10 minutes.

Author: Lex Hegt

Lex Hegt works in the IT sector for more than 25 years, mainly in roles as developer and administrator. He works with BizTalk since BizTalk Server 2004. Currently he is a Technical Lead at BizTalk360.

Why did we expose all of our BizTalk Operations and Management REST API’s?

Why did we expose all of our BizTalk Operations and Management REST API’s?

This blog is a part of the series of blog articles we are publishing on the topic “Why we built XYZ feature in BizTalk360”. Read the main article here.

Why do we need this feature?

On a daily base, our customers use BizTalk360 for a lot of their activities related to BizTalk operations and monitoring. However, in a number of scenarios, BizTalk administrators are faced with activities which need to be automated, for example: during the deployment of BizTalk Applications. It is easy to forget some tasks and doing them manually can be error-prone. So, it may be better to address such challenges in an automated way.

Here are some example automation scenarios in that can happen in your BizTalk Server environment

BizTalk Server Operations

  • You might want to automatically stop certain Ports over the weekend, because of regular or temporary maintenance of internal or external systems
  • External systems, like portals, might need BizTalk data/metrics on a regular basis, like information around EDI transactions, BizTalk usage or BizTalk performance metrics
  • You want particular users to have easy access to download links for the messages of suspended instances, without having to give them access to BizTalk Server and/or BizTalk360
  • You may want to Deploy/Publish a Business Rule Policy at a particular moment (additional discounts for the holiday season or laws to which should be complied as of a particular moment)

BizTalk Applications Maintenance and Deployment

  • Before or after deployment of BizTalk Applications, you may want to automatically stop and start Host Instances
  • After a deployment, you may want to set certain ports/orchestrations to a particular state
  • You want to bring BizTalk down at a given date and time, due to system maintenance

Identity and Access Management

  • Your organisation uses tooling for Identity and Access Management and you want to automatically create User Access Policies for accessing BizTalk360 and BizTalk
  • For auditing purposes, you need to provide user lists and profiles and audit information on a regular bases

Because of the nature of this kind of scenarios, the administrators would need programmatically access to BizTalk server and rely on command files or PowerShell scripts to be able to fulfil the requirements.

What are the current options and challenges?

BizTalk Server comes with different tooling for programmatic access to BizTalk Server. This tooling is described below.

WMI and ExplorerOM

Let’s briefly discuss the purpose of both WMI and ExplorerOM:

  • WMI (Windows Management Instrumentation) – an infrastructure, which comes with Windows, and which allows you to automate administrative tasks on (remote) computers. BizTalk Server comes with its own WMI management pack to do preliminary management activities like starting/stopping host instances, restarting ports etc.
  • ExplorerOM – a DLL used to perform all kind of BizTalk Application related tasks like exploring the schemas, maps, orchestrations, operating on them like starting/stopping etc.

BizTalk PowerShell Provider

There is also a community initiative to provide a BizTalk PowerShell provider so you can execute some of the management tasks using PowerShell (since PS has become the core of any automated administrative tasks). However, the project is not actively maintained anymore and it’s not advisable to rely on this for any critical usage.

BizTalk Server 2016 Feature Packs

Starting with BizTalk Server 2016, Microsoft started to release Feature Packs. With the first Feature Pack (BizTalk Server 2016 Feature Pack 1). a unified API became available for BizTalk Server 2016 (not for earlier versions).

This Management API enables you to perform some of the scenarios which we mentioned in the first paragraph but still lacks API’s to do extensive automation. For example, stop and start host instances.

So, although BizTalk Server comes with multiple capabilities for programmatic access to BizTalk Server, writing and testing all kind of scripts is time-consuming and not all scenarios can be covered.
Even when these custom scripts seem to work nicely, non-functional requirements, like security, auditing and logging, are not taken into account in these kinds of scripts. This would leave you in the dark when something unexpected happens. Definitely, not a scenario where you want to be.

How BizTalk360 solves this problem?

BizTalk360 consists of multiple components, a responsive user interface, few backend services and a rich API service layer.

The API service layer consists of over 400 REST API operations. The user interface of BizTalk360 uses these extremely rich API’s to perform all the tasks to manage the BizTalk environment.

A couple of years ago, we thought, exposing our API’s will hugely help BizTalk Server Administrators to automate most of their routine activities we mentioned in the beginning.  Hence we documented all the BizTalk360 API’s (which are of course the management and monitoring API’s for your BizTalk Environment) using swagger and live test within the BizTalk360 application.

The API currently contains 14 services with more than 400 API operations, covering a breath of management activities you can do on your BizTalk Environment. To give you a better understanding of the capabilities, these services are:

  • ActivityMonitoringService – BAM related services
  • BizTalkQueryService – Execute all kind of MessageBox and Tracking oriented queries
  • AlertService – All kind of Monitoring related API’s
  • BizTalkGroupService – BizTalk Platform oriented API’s
  • SchedulerService – Data Monitoring Scheduler related API’s
  • ESBManagementService – Exception Management related API’s
  • AdvancedEventViewerService  – API’s to query the Eventlog entries which are collected by the Advanced Event Viewer feature
  • BizTalkApplicationService – Search for and action on BizTalk Application artifacts
  • AdminService – Get and maintain all kind of settings within BizTalk360
  • EDIManagementService – Get/Set EDI configuration and get EDI statistics
  • EnvironmentMgmtService – API’s mainly for Knowledge Base and Dashboard management
  • RulesEngineService – API’s for maintenance of Business Rules
  • AnalyticsDataService – Analytics, Message Patterns and Throttling related API’s
  • AzureService – Get/Operate on Logic Apps and get Integration Account artifacts

With the API you cannot just retrieve information from BizTalk via GET operations, but you can also action on BizTalk Server, BizTalk360 and even certain Azure services via POST operations.

Each API can be tried from the API documentation, which comes with a Swagger definition file, by using the Try it out! feature. This helps you testing with the required parameters. All you need to do is to choose the needed API, provide the needed parameters for the API, click the Try it out! button and view the results.

Let’s have a look at few examples of how you could make use of the API.

  • Stop and start Host instances before and after deployment of BizTalk applications
  • Disable/Enable Receive Locations and/or Stop/Start Orchestrations/Send Ports
  • Get a list of currently deployed BizTalk Applications for Support purposes
  • Get Event Log data of all the BizTalk servers in one go
  • Get Throttling data from the BizTalk Hosts
  • Get all kind of Performance metrics of your BizTalk system
  • Test, Deploy, Undeploy and Publish Business Rules
  • Disable/Enable BizTalk360 alarms during deployments
  • Show data in an Operations Dashboard widget, for example, the state of your SQL jobs
  • Automatically create BizTalk360 alarms during deployment of BizTalk applications

Conclusion

By using the BizTalk360 API and its documentation, the complexity from WMI and ExplorerOM is shielded from the user. Besides being able to act on BizTalk, the API also allows you to action on BizTalk360. You can even show the output of API calls in the Operations Dashboard!

Few examples of how the BizTalk360 API`s can be used in real-world scenarios: Automate Monitoring Alarm creation in BizTalk360

If you are using BTDF (BizTalk Deployment Framework) we have a wrapper console application called BT360Deploy that can be used to automate certain tasks like creating monitoring alerts during deployment. The project can be accessed here.

Get started with a Free Trial today

Why not give BizTalk360 a try. It takes about 10 minutes to install on your BizTalk environments and you can witness the benefits of auto-healing on your own BizTalk Environments. Get started with the free 30 days trial.

The post Why did we expose all of our BizTalk Operations and Management REST API’s? appeared first on BizTalk360.