Extended Import/Export Configuration – Part 2

Extended Import/Export Configuration – Part 2

This is a sequel blog to my post about the Export/Import Configuration feature. Along with BizTalk Reports and Knowledge Base, we also brought import/export capabilities for Dashboards and Custom Widgets.

A BizTalk administrator needs to perform many tasks every day. When the time comes to analyse or visualize BizTalk data, it can be difficult to break it down and discover exactly what you need to know. By building a BizTalk360 dashboard, the administrator can get a better look at exactly the information of a specific environment. BizTalk admins can create such insightful dashboard for four different sections in BizTalk360:

  • Operations
  • Analytics
  • EDI
  • ESB

The Dashboard options are great and they help to monitor and troubleshoot your environments proactively. However, the downside is that, it can take some time to craft the different dashboards you want and subject matter expertise to display the relevant metrics needed to troubleshoot a specific issue. The Dashboard Import/Export feature allows you to export your fine-tuned, purpose-built Dashboard screens, and import the ones that have been shared with you.

Exporting Dashboards

BizTalk360 admins can export all the available dashboards under each environment in BizTalk360.  Using this capability, users can import all the configured dashboards or any specific ones.

Scenario 1: For their business needs, BizTalk administrators can manage multiple environments. If the admin wants to migrate the Dashboard configuration from one environment to another, the Admin can export the configuration in one environment and import it into another one. Even the globally created dashboards can be imported and exported.


Export Dashboard

Importing Dashboards

The BizTalk admins can import the exported Dashboard configurations from another environment. The admin can create new dashboards and can also overwrite existing dashboards in the target environment. 

Import Dashboard

Having said already, the Dashboards are user specific. This means that users, for which dashboards will be exported and imported between environments, need to exist in the target environment. So, while importing dashboards from one environment to another, make sure the user is existing in the User Access Policy screen. If, by any chance, that specific user does not have a profile in the destination environment, it can be created by using the User Access Policy import capability.

Import User Access Policy

If the user does not have the access for the particular environment in the User Access Policy screen, then the admin cannot import the dashboard for the specific user. Hence, the user must be available in the destination environment.

User not exist in dashboard

In case, the user may exist in a Windows NT Group, then the admin can import the dashboard by providing the NT Group name.

NTGroup user

The result of the imported Dashboards is shown in a nicely formatted grid view, with an explanation of what happened with each Dashboard during the import process.Result summary of Dashboard

Exporting Custom Widgets

The import and export custom widgets implementation is mainly to handle the custom widgets in dashboards. Because, when a dashboard contains any custom widgets, then it will not get exported along with the dashboard functionality.  The user can export the needed custom widgets along with the dashboard from one environment to another environment.

Export Custom Widget

Note: The widgets are not environment specific. All the available widgets will be accessible for all the environments.

Importing Custom Widgets

Importing custom widgets is done by uploading an export file. Using the import functionality, the widget can be imported into other environment or used to restore widgets to the same environment. The Import process takes into consideration, if the widget already exists in the destination environment (widgets are not duplicated). If so, it will be overwritten. 

Import selection Widget - OverwriteIn case the user still wants to import that widget into the destination environment, the user can edit the original name of the widget, thereby making sure that the widget will become created in the destination environment.

Import Selection widget - CreateThe results of an import are shown in a nicely formatted grid view with an explanation of what happened with each widget during the import process.result summary custom widget

Scenario 2:  While Exporting and importing a dashboard into another environment, the dashboard can contain custom widgets for any kind of data visualization or analysis. Those custom widgets won’t be imported with the Dashboard because, in BizTalk360, the custom widgets are handled separately. In such case, the user can Export the Dashboard along with the custom widgets and import then in the required environment. This will reduce the time on manual creation of each custom widget.


dashboard and custom widget

Conclusion

The Import/Export functionality in BizTalk360, tremendously reduces the time of migrating the data configuration form one environment to another. For the business user, it helps to create dashboards in multiple environments and the user can create the dashboards into collaborative group easily.

Extended Import/Export Configurations

Extended Import/Export Configurations

Introduction

Here is another exciting capability we have added to our existing import and export capability. Curious to know? In this blog post, I would like to highlight what are all the new enhancements in Import/Export Configurations we made for our upcoming version V8.9. These enhancements would ease the BizTalk administrator’s day to day operations, particularly migrating data from one environment to another.

All the newly created BizTalk Server applications are being tested first in the preproduction or UAT and shadow environments to avoid business consequences. Likewise, customers would prefer to test all the functionalities of the BizTalk360 in their non-production environments. Further, they populate necessary data to operate, monitor and find analytical information of their environment.

Admins don’t  need to recreate the data again in their production environment as in such scenarios, our import and export capability plays a major role. Already we have support to export and import,

  1. Alarms
  2. User Access Policies
  3. Saved Queries
  4. Secure SQL queries

New Capabilities

In the version 8.9, we added support for import and export of the following sections:

  1. Knowledge Base
    • Service Instances
    • ESB Exceptions
    • Event Logs
    • Throttling Data
  2. BizTalk Reports
  3. Dashboards
    • Operation (Default & Custom Dashboards)
    • Analytics (Default & Custom Dashboards)
    • EDI Dashboards
    • ESB Dashboards
  4. Custom Widgets

Using this rich capability, one can easily export and import the configuration between environments, thereby reducing the migration effort. This is especially useful in scenarios, where the user tries to carry out testing in a test environment, and upon achieving the desired result, they can import the configuration from the production environment.

What has enhanced in Import/Export Grid Pagination

Previously, BizTalk 360 has a static pagination which is available for the grid by default. In Import/Export, the user can customize the pagination using the “Max Matches” on System Settings sections in BizTalk360. Pagination will be visible for the Import/Export grid once the row count exceeds the max match size.

Set max Match for pagination

For Instance: If the Max Matches size is set to 10 means, then the pagination for each grid in import/export will be visible when the row count exceeds 10.

Exporting configurations

Exporting your data is a great way to make the information more digestible or find patterns that can give your team better insight between different environments. The user can make use of export functionality in two ways:

  • To migrate configuration data between different Environment
  • To archive the configurations in the local system

Exporting the Knowledge Base

The BizTalk360 knowledge base is useful to document the BizTalk issues which administrators encounter on a day-to-day basis. To migrate the BizTalk360 knowledge base it is possible to use the Export functionality.Exporting the knowledge base

The user can export the articles based on the sections Service instances, Event Log, Throttling Data and ESB Exception. The knowledge base section is preselected, the user can deselect and reselect the knowledge base according to their needs. Once the selection has been made, the user can Export the knowledge base articles.

Exporting BizTalk Reports

The BizTalk Reports help organizations to visualize/monitor their business data. Using the Export functionality, the user can archive or export the BizTalk Reports.

Exporting BizTalk Reports

The BizTalk Reports and their associated schedules are preselected. The user can view the associated reports by clicking the arrow on each schedule and they can edit the selection according to their needs.

Importing Configuration

The Import functionality is mainly useful for importing large amounts of data quickly. By importing the configuration, you can replicate the required configuration on the target machine. To import the configurations from a certain environment, you should have already saved and exported those configurations in a Zip file. Import can be used in two ways in BizTalk360

  • To create a new configuration from the source environment to the destination environment
  • To update the existing configuration on the destination environment

Importing Knowledge Base

In BizTalk360, the customer can create KB articles as and when they fix issues in the environment. Over a period of time, they can have a wealth of information ready to fix those repeated issues. Those articles can be imported to another environment using the import knowledge base feature.

For Instance: The organization has Production Environment and they are planning to build a separate QA environment for testing. After creating the new environment, it is a lot of work to manually create the knowledge base article for frequent/repeated issues. Here, the import knowledge base comes into play. The user can easily import the knowledge base articles from the Production environment to QA environment.

Importing knowledge base

The existing articles can be overwritten or else can be ignored. If the article doesn’t exist, it will create a new article in the target environment.

In BizTalk360, the knowledge base articles can be created for a specific environment or globally (without selecting the environment) accessible articles. The BizTalk360 user can create or overwrite the articles.

For specific Environments: The knowledge base articles will be imported to the target environment with the configuration tags (like Host Name, Application name etc.., which is specific for each error code they encounter) for specific articles. If the article is overwritten, then the import process will overwrite the configuration tags on the existing article.

Configuration tags for specific environments

When the configuration tags are not available in the target environment, the article will not be imported and it will throw an exception on result summary.

Result summary for knowledge base

For Global articles: When the global knowledge base is imported, the environment selection will not happen. The articles are accessible to all the available environments.

Configuration tags for global article

Importing BizTalk Reports

In a business environment, the BizTalk reports provide a comprehensive, high-level overview of the performance of their BizTalk environment. The configuration of these reports, can be easily managed using the Import BizTalk Reports feature.

Importing BizTalk Reports

For instance: An organization has deployed BizTalk360 on one of the Development/QA/UAT environment(s) and they want to move all configurations to the Production environment. The BizTalk360 administrator must do the usual practice to set up the configurations manually in a production environment. It is not feasible to configure all the configuration manually, instead BizTalk360 allows you to easily export and import all the BizTalk Reports.

While importing the schedules and the associated reports to another environment, the user can edit the schedule name as per their requirement and the user can enable the schedule after importing by selecting the ON/OFF option available on each schedule. By default, the status of schedule during import, is disabled. The user can also change the Email ID after importing. This will help the user to create/overwrite the schedule and the reports as per their business needs.

Note: The user cannot import the reports without the selecting the associated schedule.

If the user sets the option to “Error or Duplicate”, the import process would not allow duplicate schedules and associated reports to be imported.

Result summary - Error on Duplicate in BizTalk Reports

If the user sets the option to “Overwrite”, the import process will overwrite the existing schedule with the same name. In case the associated reports are not available, the import process will create the reports and associate to the schedule.

Result summary - Overwrite option in BizTalk Reports

If the associated report already exists in another schedule means, it will ignore the report while importing.Result summary - Already existing schedule functionality

Conclusion

By adding the above feature, the Import/Export Configuration has become more versatile. This provides a great option for the administrators who find the migration process intimidating and would like expert help to assure an effortless transition between multiple environments. No matter where your configuration needs to go, BizTalk360 lets you Import/Export data with ease.

Apart from knowledge base and BizTalk reports we have included the Dashboards and Custom widgets for v8.9 release. I will cover those two functionalities on my next blog “Stay tuned!”.

Control and get better insight of your Event Log data using BizTalk360 Advanced Event Viewer

Control and get better insight of your Event Log data using BizTalk360 Advanced Event Viewer

Are you using the Event Log for diagnosing problems and challenges faced while performing the operation in BizTalk server administration console? Then you might face the difficulty, on having multiple BizTalk / SQL Server users who need to look Event Logs on each server when there is a problem. To access the Event Log, every user should have administration permission to access the physical server, which is not the ideal solution for the business needs.

What is the purpose of the Event Log on a BizTalk Server?

The Event Log holds information related to different aspects of, amongst others, BizTalk Server. The Windows Event Logs are used by the BizTalk admins, mostly for diagnosing problems. When they face any issue, the first step they try to diagnose the problem or faults in the BizTalk Admin console itself. If they could not achieve a solution, then they will view the Event Viewer for detailed information on such cases like errors, messed up environments and failure of message processing.

Filter for log noise reduction during Event Log collection

The BizTalk360 Advanced Event Viewer which collects the events that are related to BizTalk and SQL sources. These sources can be configured in Settings–>Advanced Event Viewer, as shown below.

Insights and control your BizTalk Environment using BizTalk360 Event Log viewer- Source configuration screen

The Event Logs are collected based on the Event Log Categories and the administrators can also add categories as per their need. The data collection can be performed based on the environment selection as below.

Insights and control your BizTalk Environment using BizTalk360 Event Log viewer - Server configuration screen

Once, after enabling the “Enable Advanced Event Viewer data collection”, that will display all the available BizTalk and SQL server of the specific environment. The BizTalk admins or the people using BizTalk server can select the servers based on their needs. Based on the configurations, the subservice “Eventlog Collection” of the BizTalk360 Monitoring NT service will start to collect Event Log data and store the information in the BizTalk360 database.

This approach will seamlessly solve the problems as we highlighted in this article. The BizTalk administrators no more need physical access to each BizTalk or SQL server to diagnose the issue. This approach will avoid data collision, improves the performance and makes the user interface clear.

Let’s have a look at few scenarios where using the Advanced Event Viewer is beneficial over the Windows Event Viewer.

Scenario 1: If there is any failure of file transaction at an application level, the user can view the relevant event log information in the single user interface. This will avoid context switching and reduces the time to diagnose the issue.

Diagnosing issue made easier than Windows Event Viewer

In real time business transactions using BizTalk can cause any kind of failure.

Scenario 2: When you try to run a BizTalk application after deploying an orchestration with custom components. In case, the BizTalk orchestration engine cannot locate the custom component, that means you will get below error.

Insights and control your BizTalk Environment using BizTalk360 Event Log viewer - A sample error in windows Event viewer

It is easy to query the Event Log details if BizTalk server is installed in a standalone environment. However, in case of multiple BizTalk servers, the user needs to login to each server to find the Event Log information and it is a tedious process to search the exact issue on ‘n’ number of results. So, it will eat up lots of time on finding the exact result.

In the BizTalk360 Advanced Event Viewer, diagnosing the problem or issue, in a single user interface related to the event “application” of BizTalk and SQL search, is made much easier than with the Windows Event Viewer.

Insights and control your BizTalk Environment using BizTalk360 Event Log viewer- Operation capability

With the Advanced Event Viewer, the BizTalk admins (and other people) can search the results using rich query builder tools, based on the different parameters such as the Event Log, Type, Source, Event, Category, User, Computer, Timestamp, and Message. One or more filters can be added, which will be useful to get an exact result and the user can view the related information the same as in the Event Viewer.

This approach will fasten the search of the Event Log information from the different servers within a stipulated time which obviously saves time.

Archiving distributed events on Centralized Location

BizTalk360 help to get the needed Event Log data from the Event Logs on different servers and shows the information in a simple, intuitive user interface. However, it is practically impossible to manually keep an eye on the Event Log data throughout the day, as they are constantly updated with the latest log data.

In BizTalk360, we solve this problem by downloading the data into Excel Format. The BizTalk admins can download the file by using the Export to Excel feature, that will download the grid data in Excel format and they can store the Event Log Data for further investigation, as shown below.

Insights and control your BizTalk Environment using BizTalk360 Event Log viewer - Download screen

It is possible to monitor Event Log Data!

The Windows Event Logs hold all the important log information from sources like SQL Server, IIS, BizTalk infrastructure and run time problems. As the Windows Event Log contains that much data, it is nearly impossible to monitor the Event Log manually. Luckily, through BizTalk360, the BizTalk admins can easily monitor the BizTalk and SQL server logs by configuring the Event Log monitoring.

Scenario 3: The BizTalk admins can customize the alert in such a way that they must be notified when there has been a sequence of start/stops of the host instances in the past 30 minutes. Without being alerted, you may not know the frequent start/stops of the host instances and this could impact the performance of the environment.

Insights and control your BizTalk Environment using BizTalk360 Event Log viewer - Advanced data monitoring screen

By mapping the Event Log to the Alarm, the administrator will get notified when the Event meets the condition. The alert can be notified through the required sources like Email and notification Channels.

The administrators can also log this Event Log alert in the Event Viewer itself, this is possible by enabling the Event Log option in alarm creation as below

Insights and control your BizTalk Environment using BizTalk360 Event Log viewer- General Monitoring event log flag

This will log the Event log alert in the Event Viewer when the alert exceeds the Host Start/Stop error/warning condition as below

Insights and control your BizTalk Environment using BizTalk360 Event Log viewer - Event log data in Operation screen

Create Log rules with an intuitive interface using the Event Log Data Monitoring capability

The primary purpose of auditing Event Log for BizTalk administrator is to discover the events of interest, whether they provide general information or call for further investigation. BizTalk360 will help to get the needed log data from multiple servers and shows the information in a simple, intuitive user interface. However, it is practically impossible to manually keep up the data throughout the day, as they are constantly updated with the latest log data. BizTalk360 also solved this by giving the real-time alert features on Event Log Data Monitoring.

Scenario 4: BizTalk Administrator wants to monitor different Events from multiple servers. For example, they might want to monitor the ESB from BizTalk Server at the same time they want to ensure the performance of the SQL Server & ENTSSO events from a separate SSO server and they also need to monitor the IIS from multiple BizTalk servers. In real-time, there is no option to monitor it with a single tool. It is hard to predict any issues and it is time-consuming for BizTalk admins to constantly monitor the Windows Event Log manually.

BizTalk solves this problem by bringing the heterogeneous data in the single intuitive user interface. On the top of that, it provides a powerful capability of getting real-time notifications through email or SMS when an event occurs on the specific frequency by setting additional threshold conditions. The admins can correlate two or more event calls for the further investigation by setting the conditions as shown below.

Insights and control your BizTalk Environment using BizTalk360 Event Log viewer- Advanced Data Monitoring configurations screen

It will be useful to query against different Event Sources of multiple servers of a single BizTalk Environment. The admins can include or exclude the events that pertain to your environment. Also, the admins can get the detailed notification on Event Logs by selecting the ‘Send Event Log in notification’.

Insights and control your BizTalk Environment using BizTalk360 Event Log viewer- Data Monitoring Filter screen

This will send the detailed information on every event source in Email which will look like below.

Insights and control your BizTalk Environment using BizTalk360 Event Log viewer - A sample Email alert

BizTalk Admins can choose the Notification Channel on the alarm creation and they can configure the Event Log as previously mentioned to get notified when any violation happened on a specific frequency. On the Data Monitoring Dashboard, the admins can visualize the triggered alert results based on the day view as shown below.

Insights and control your BizTalk Environment using BizTalk360 Event Log viewer- Dashboard View

It also easy to maintain, once after scheduling Event Log Data Monitoring; when you disable AEV for the environment, it will stop collecting Event Log data. So, the admins can enable collecting whenever they need it.

Is there any option to manage the Event Log data?

Managing the Windows Event Log is not simple. Administrators can manage or clean up the data through PowerShell script or by creating a custom log to log application errors. The BizTalk360 Monitoring Service collects the data based on the Event Log and Source configuration and stores in a separate database table in the BizTalk360 database.

In real-time business, BizTalk server will collect the log data for each event which happens during BizTalk operation. This can hugely increase the size of the database. Luckily, the administrator can set the purging for Event Log data in the BizTalk360 database as shown below.

Insights and control your BizTalk Environment using BizTalk360 Event Log viewer- Purge settings screen

Once after configuring the Purge Policy for Event Log data, it will clean up the data for every specific days/month/hour.

This approach will save the time and clean the old data easily, which will improve the overall performance of the BizTalk360 database. This also solves the problem of searching the current data with messed up old data.

Can we visualize the performance of Event Log?

 In BizTalk360 the BizTalk Administrators can indeed visualize the performance of Event Log data at a specific time.

Scenario 5: In any business environment, the performance metrics at a specific time will be critically important to take business decisions. To visualize the Event Log data the administrators can add the Event Log Data Count widget in the Reporting Dashboard on BizTalk Analytics.

Insights and control your BizTalk Environment using BizTalk360 Event Log viewer- Analytics Event Log widget screen

The Event Log reporting widget will show the exact count of Event Log data based on error, warning and critical errors for the available sources. The Event Log data widget will collect the data based on the scheduled time.

Leveraging the API Documentation Capability for Event Log

In BizTalk360, Event Log data in the API Documentation provides you with a very powerful way to integrate data and actions into your own tools or programs. Many of the BizTalk Administrators do not come from a development background and perceive an API as something for the exclusive use of BizTalk admins. This is not the case, and a mature and well-designed security product API can be of great value to other people, regardless of their development background.

Within BizTalk360, you can use the Event Log API Documentation to demonstrate broad functionality, primarily in the areas of:

  • user and organization account management
  • sample management
  • sample analysis data collection
  • threat intelligence harvesting

Insights and control your BizTalk Environment using BizTalk360 Event Log viewer - API Documentation screen

Of course, an easy way to use an API is to use a product that leverages that API.  Any RESTful API can be operated by anyone who can construct a web URI. the Event Log API documentation is also useful to create a custom widget using the JSON Response as shown below.

Insights and control your BizTalk Environment using BizTalk360 Event Log viewer - A Sample JSON Response screen

In an organization, you could easily tailor that to a list of IPs (or domains, or URLs, file hashes, etc) that had been seen in the analysis of only your own organization’s submitted samples with the help of this API’s.

Conclusion

To collect and process log data in real time, regardless of the volume of log data and the number of devices in the network, organizations need a robust log collection mechanism. BizTalk360 solves this problem by providing the Advanced Event Viewer functionality, which enables BizTalk Administrator to decipher and analyse any log data regard of its source configured. The collected log data is centrally aggregated and presented in a single console for log sources across locations. To summarise here are the Key features of BizTalk360 Event Log

Event Log Maintenance: Maintenance is also very simple when you disable AEV for the environment, it will stop collecting Event Log data. Manages data growth by Providing BizTalk360 purge policy.

Privileged Multiple Server Monitoring: The BizTalk Admins can monitor multiple BizTalk and SQL servers on a single user interface.

Server event trend analysis: It also protects security threats by collecting, analysing, reviewing and archiving event log. Through this, the BizTalk admins can get a comprehensive view of all the actions that is happening in the BizTalk server, SQL server, ENTSSO server etc..,

Real-time alerts: Receive instant alerts on unauthorized access to confidential data or even to the database server itself.

Intuitive dashboard: Drill down to the raw log level and view collected log content to facilitate simplified root cause analysis.

Out-of-the-box reports: Perform audits with pre-packaged reports that provide detailed information on Event Log Data through Event Log Count widgets.

Exploring data just got easier in BizTalk360

Exploring data just got easier in BizTalk360

Do you have a BizTalk environment which handles huge volume of transactions? With the BizTalk admin console, It is nearly impossible to go through those data. In the past, BizTalk360 solved this problem by allowing to filter various data based on parameters associated with the data. But for some cases, even if you filtered the data, you might end up having lot of rows in the grid which may not be user friendly. It seems to be slow, while visualizing large set of data in a single grid.

While searching the result with large dataset, the browser seems to jump around while scrolling, providing a not smooth and unnatural experience. This has led that, from a user experience standpoint, scrolling is no longer considered as functional.

Apart from that, even the most modern browsers will eat up lot of memory when rendering all those data in the UI. To overcome this, it’s absolutely necessary to split the results of a query into subsequence’s to achieve constant response times.
At the splitting process, it is very important to keep paging transitions fluently. That means, rows can’t appear twice or miss completely in the result. For this purpose, the so-called Pagination comes into play in BizTalk360.

What pagination really does?

Pagination is an excellent way to navigate through large amounts of content. E.g., when we search for something that returns many records, which cannot be displayed on a single web page, those records are split into number of pages that can be accessed through links via the Pagination mechanism. It enables the user to navigate to the next page one after the other and even jump forward to several pages. Pagination allows you to make large amounts of content easy to find and breaks up several entries or web content into multiple pages, allowing you to toggle through content with ease.

What is improved due to pagination in BizTalk360?

In this version of BizTalk360, we have optimized the grids by providing you the power of pagination. We have implemented pagination for each grid available in the BizTalk360. It is an easy and standard practice, that prevents you from having to load the entire result set all at once.

The grid supports both preloaded and dynamically loaded data. By using pagination, the rows in the grid gets dynamically separated into pages. The grid will automatically display 25 rows in each page, by default. The user can increase the maximum number of result rows by selecting the Max Match size in the drop down.

Basic navigation

Pagination can further be customized for different circumstances, like when a user approaches to an end or the beginning, or indicating the current page number to the user. Pagination dynamically separates the rows on each page, as shown below. By listing out the page number, users are allowed to skip ahead, one page at time.

The page numbers which are shown in grey are disabled (or inactive) and the other page numbers are shown in blue, since they are enabled (or active). The user can use the previous and next page buttons, as these can be considered a more intuitive interface for showing the previous/next records.

Sometimes you may simply require Last page and First page links, which provide simple and quick navigation to the beginning and the end of the result set, to the user. These are implemented, as shown below, to further improve the usability.

The First, Previous, Next and Last page links are appended at the bottom of each grid.

Customizing row count in System Settings

By default, the grid will automatically display 25 rows per page, the user can customize the row display by adding the desired row count in Setting->System Settings->Max Matches.

The added Max Matches are displayed in a dropdown at the bottom of each grid. According to the selection of the dropdown, the row count will be changed dynamically on the resulting grid.

Through this, we can reduce the load time on each execution. For instance, it is used by executing the result, where the user can customize grid for displaying a limited number of results on each search, or showing a limited number of rows per page. This way, the overall performance to display the result, will highly be improved.

Row count display

The total grid row count of the current page compared to the overall row count will be displayed at the bottom end of the grid.

At BizTalk360, we believe that a beautiful grid and listing data interactively to user, will make our product stand out and keep customers engaged. Keeping our customer feedback and understanding the consequence of users on execution, the amount of load time increases proportionally to the growth of data. To overcome this issue, we have provided a better solution for our customers, by implementing this type of content grids, which will highly optimize the performance and reduce the customer effort to visualize the result. A major advantage of this approach, is that now an execution can be used to happen in the lower time limit.

Setting the time zone in the BizTalk360 BAM Portal and other BAM improvements

Setting the time zone in the BizTalk360 BAM Portal and other BAM improvements

The BAM Portal in BizTalk360, provides the option for querying BAM views, perform activity searches, check user permissions and to view the query results of Related Activities and Documents the same as in the out of the box BAM Portal of BizTalk Server. However, the user couldn’t view the query results in their respective time zone in BizTalk360, like in the original BAM portal. We brought a number of improvements to the BAM Portal in BizTalk360; setting the time zone is just one of them.

Time Zone in BizTalk BAM portal


BizTalk360 Feedback portal

Some of our customers requested the functionality to view the query result on their respective time zone. In BizTalk360 v8.8, we came up with the solution, by implementing “BAM Time Zone”. With the help of this, the user can choose the time zone specifically to their requirement.

In the BizTalk BAM Portal, the query result is displayed according to the local Time Zone, while the actual data in the BAMPrimary database is stored in the UTC time zone. BAM captures milestones such that they are automatically recorded in Coordinated Universal Time (UTC) format (this is also referred to as Greenwich Mean Time). When you send date/times to BAM using the APIs, they are received in the format sent without  conversion to UTC format.

Time Zones in BizTalk360 BAM portal

Earlier versions of BizTalk360 automatically picked up those data in the BAMPrimaryImport database and display it. BizTalk360 won’t convert any time zone as might be user required; the product will display UTC time zone the same as in BAMPrimaryImport database. From this version on, BizTalk360 will pick up the UTC time in the BAMPrimaryImport database and then convert that into the user required time zone. The user can set the required time zone using the BAM Time Zone.

Configure time zone in BizTalk360 BAM portal

Usability of BAM Time Zones

The available time zones are displayed in the BAM Time Zone dropdown and the user can choose the specific time zone according to the system Timezone/ local time zone. Through this, the user can view the date time as in the original BAM portal.

Configure time zone: time zones drop down

The BAM Time Zone also includes Day Light savings, so query result is displayed according to the Day Light Savings.

Can Other Users set the BAM Time Zone?

Only the Super User will have the permission to set the BAM Time Zone. If other users also have that permission, that could create confusion, so to avoid this we have given permission to set the BAM time Zone only to Super Users.BAM Portal time zones: read-only for Normal users

Other users can view which time zone has set, but they have not access to change the set BAM Time Zone.

Related Documents Improvements

In version v8.6, we have implemented the Related Activities and Related Documents Functionality. After this release, we came to know some new workaround from our customers, who noticed an issue in Related Activities view and Related Documents Download Functionality, like

BizTalk360 Feedback portal: download Related documents

The above issue is because, when the “Activity Id” has any special characters or it is custom created, the user could not view the detailed information. This has been solved in the current version of BAM. The user can now view the detailed information and Related activities without any constraints.

BAM portal: Detailed information

Handling the XML Message in BAMPrimaryImport Database

We came across another new workaround with Related Documents Download functionality. The Related Document is not getting downloaded when the XML message is stored in BAMPrimaryImport Database.

In the BizTalk BAM portal, the user can capture the whole XML Data in LongReferenceData column in the BAMPrimaryImport Database. This can be done by using the AddReference method of the BAM Orchestration Event Stream to log the complete xml message.BAM data in BAMPrimaryImport database

BAM can store up to 1 MB of data through the AddReference method (using longReferenceData field with type NTEXT) on the event stream, a great well-performing solution for message-bodies which enables the storage of message bodies alongside the data that describes how it was processed.

Handling the XMLMessage in BizTalk360 BAM Portal

In the BizTalk360 BAM Portal, the Related Documents are only downloaded from the archive folder. From this version of BizTalk360 v8.8, the Related documents will get downloaded either from archive folder and from the database. It checks the XML message either in Archive folders or in the database, if it is found, it downloads the XML File in Zip File Format as shown below.

BAM Portal: Related documents

If the file is not available in both archive folders or database, it will throw a “File Not Found exception” as below.

Downloading Related documents: File not Found exception

Filter Improvements

In the previous version, we received a feedback from customers that some filter conditions are not working as expected in the BizTalk360 BAM Portal.

BAM Portal: Filters

In the current version of BizTalk360, we have fixed the below mentioned filter conditions in BizTalk360 BAM.

  1. String Filter
  • Starts with
  • Ends with

BAM Portal: Fixed Filters

2. Date Time Filter

  • Later
  • On or Later

BAM Portal: Fixed Filters

We improved those filters, so it will be working from BizTalk v8.8. We have implemented above functionality in the BAM Portal by keeping our customer feedback and understanding the consequence of users on execution. This implementation will highly optimize the BizTalk360 BAM Portal.

If you are interested, you can read more on our Documentation portal.

BAM Related Activities and Related Documents integration in BizTalk360

BAM Related Activities and Related Documents integration in BizTalk360

BAM Related Activities and Related Documents integration in BizTalk360

Earlier versions of BizTalk360 already had the Business Activity Monitoring (BAM) feature inbuilt, which provides the option for querying the BAM views, perform activity search, check user permissions and to view activities time window. However, the user couldn’t view the detail information on query results in BizTalk360 similar to BAM portal.

bam relationships

Some of our customers requested this functionality to view the related activities and documents of the query result from BizTalk360 application itself. From this release v8.6, we bring the Related activities and documents of each query result to the existing BAM feature. This helps the user to find the solution in a single application rather than switching to multiple applications.

What is a BAM Relationship?

A BAM activity can span to multiple heterogeneous applications (for example, a pipeline, two orchestrations, a line-of-business application, and then another pipeline). The BAM infrastructure can correlate the events from the multiple sources. BAM Relationship is used whenever two activities need to be connected to shared data. The Activity relationship exists when an activity relates to one or more activities in a BAM view.

For Example, for an Order Request, there may be multiple Responses, which may arrive from the end system; the relationship between the request and multiple responses is established using a Common ID which is common between the Inbound and Outbound Order. In our case, it will be the OrderID.

bam view

The user can query the result, by selecting the required activity to sort the data and milestones from the available columns for the query. Next, the user can view the result in the grid view.

bam query results

View the Detailed Information for Request

The user can view the detailed information on each request arrived for the inbound Activity by clicking on the respective row. This action will open a blade where the user can view detailed information such as requested data and the set milestone information.

bam activity status

View the Related Activities for Request

By clicking the Related Activities tab, the user can view all the Responses generated for that request and the user can view the detailed information such as data and milestones for the Responses, by clicking the Eye Icon in each response, as displayed below.

bam activity status inbound order

In the BAM Portal, the user can view above response details, by clicking the Related activities and it will navigate to the response in the Outbound Activity.

Download the Related Documents for the Request

In BizTalk360, the user can also download the Request document by clicking the Related Documents tab. In this screen, the user can view the documents for that request in a grid and the user can also download the file by selecting and clicking the Document link.

related documents

The user can download the related document only when the archive folder is in a shared location.

bam activity status

Functionality for the Response Activity

The Response activity is the Outbound activity; same as in Inbound Activity the user can view the detailed information about the response and view the request for that response in the related Activity tab.  Finally, the detailed information for that request can also be viewed by clicking the Eye icon on the grid.

response activity

Note: From the Above sample, the Common Id should be same.

BAM Operation Using Command Prompt

BAM Operation Using Command Prompt

This blog helps to perform various operations with BAM views using the command prompt such as:

  • deploy BAM Views
  • remove BAM definitions
  • authorize user accounts for BAM View

First, I have created a BAM view named as “BAMOperation” with the Activities Order and Status with items as shown below.

Business Activity Monitoring BAM Operation Using Command Prompt

Business Activity Monitoring BAM Operation Using Command Prompt

The BAM management utility deployment command line tool, bm.exe., allows to apply, modify, and remove BAM definitions. You can use, amongst others, below commands:

  • deploy-all – Deploys a BAM definition
  • remove-all – Removes a BAM definition
  • update-all – Updates a BAM definition
  • add-account – add accounts on BAM definition

Deploy a BAM View and Activities

The BAM Definition file (BAMOperation.xml) becomes deployed using the BAM Command line utility.

Deploying of BAM definition file will create corresponding database tables, views, etc., which will organize the data in a relational view. Follow the below steps to deploy a BAM View:

  1. Open Command Prompt as administrator
  2. Navigate to BizTalk Server Installation path/ Tracking folder and use the below command:

bm.exe deploy-all –DefinitionFile:” <<FileLocation>> BAMOperation.xml”

Business Activity Monitoring BAM Operation Using Command Prompt

which will deploy a “BAMOperation” view in BAM Portal as shown below.

Business Activity Monitoring BAM Operation Using Command Prompt

Remove a BAM Definition

Administrators can use the remove-view / remove-activity command to remove a view and Activity from the BAM Primary Import database. Follow the below steps to remove an existing activity/view:

  1. Open Command Prompt as administrator.
  2. Navigate to BizTalk Server Installation path/ Tracking folder and use the below command.

To Remove a BAM Definition File:

bm.exe deploy-all –DefinitionFile:” <<FileLocation>> BAMOperation.xml”

Business Activity Monitoring BAM Operation Using Command Prompt

Removing a BAM definition file will remove the related tables, views, etc. which were deployed in the database and server.

Remove a BAM Activity

To remove a BAM Activity, the user needs to remove the BAM View first and then the BAM Activity they wanted to remove. Execute the below-shown command to remove the BAM View first and then the BAM Activity.

To remove a BAM View:

bm.exe remove-view -name: {view name}

Business Activity Monitoring BAM Operation Using Command Prompt

To remove a BAM Activity

bm.exe remove-activity -name: {activity name}

Business Activity Monitoring BAM Operation Using Command Prompt

The above command which will remove the specified activity alone form the database.

Update a BAM Activity

The User is also able to update an existing View/Activity. To update an Activity/ View, the user should remove the activity/view first by using the above-mentioned commands and then follow the below steps to update any changes in the existing View /Activity.

For example, I have removed the above “Status” activity and I have updated the activity as shown below.

Business Activity Monitoring BAM Operation Using Command Prompt

We then use the below command to update the changes to the existing view:

bm.exe update-all –DefinitionFile:” <<FileLocation>> BAMOperation.xml”

The above command will add a table, view in the database and it will update the previous activity in the BAM Portal as shown below.

Business Activity Monitoring BAM Operation Using Command Prompt

Add an Account to BAM Definition

Administrators use the add-account command to associate users with BAM views and protect the BAM Excel Spreadsheet views from unauthorized access. When users save BAM views, the views reference a SQL connection string that is hidden within the workbook. The workbook is protected, but you must also ensure that the document is protected.

When you associate users with BAM views, you restrict access to the views to only the users or groups to whom you grant access by using the below command.

bm.exe add-account -AccountName: DomainName UserName -View:(viewname)

Business Activity Monitoring BAM Operation Using Command Prompt