BizTalk Server 2020 Announcement with Hybrid Integration updates

BizTalk Server 2020 Announcement with Hybrid Integration updates

Introduction

Integrate is the premier integration conference for everyone working in the Microsoft Integration Space. Integrate happens annually in London and this year it happened from 3 to 5 June. The 3-day event, with speakers from the Microsoft Product Group and from the Global Integration Community, is the ideal event to listen to leading Integration Specialists and learn what is coming next in Microsoft integration. This year a lot of newer people were joining the event. Initially, the event used to be a BizTalk Server focused event however, from last few years it expanded the scope to general integration technologies in Microsoft space, including Azure Integration capabilities.

The Value of Hybrid Integration

Paul Larsen, a Principal Program Manager for the Microsoft Corporation in Redmond, presented a session on “The Value of Hybrid Integration”. At Microsoft, Paul leads the Host Integration engineering team to deliver and support Azure, BizTalk Server, SQL Server, Office, and Windows connectors to existing IBM systems. At Integrate, he started the session by explaining the legacy of BizTalk Server and explained the meaning of the word Hybrid with some examples.

BizTalk Server 2020 Announcement

After that, Paul made the big announcement of “BizTalk Server 2020” and he also announced it will be released at the end of 2019.

BizTalk Server 2020 Announcement

Important feature updates in BizTalk Server 2020 are:

  • Integrate cloud Services with built-in adapters for cloud, Logic Apps to On-Premise Gateway, Service Bus, Event Hubs, API Management, O365 Mail, O365 Schedule, O365 Contact, Blob Storage (Dev) API management to all the resources (On-Premise and Cloud)
  • Manage BizTalk Server environments by using the new BizTalk Server REST API’s
  • The Blob Storage adapter is still under development. Using File adapters which are published as a UNC Endpoint is not straight forward, as pluggable. Using Blob storage adapter will be a useful feature that will help administrators to, for example, have the Backup BizTalk Server job write the backups to Azure Blob storage
  • Secure the BizTalk Server Infrastructure settings with Read-only administrator view
  • Deployment with VSTS – Application Life Cycle Management to deploy and update BizTalk Server instances using Visual Studio Team System
  • Transport Layer Security 1.2– Securely deploy BizTalk Server using TLS 1.2 authentication and encryption
  • SQL Server Encrypted Data– Read and write to SQL Server with Always encrypted columns using BizTalk Adapter for SQL Server
  • Advanced Scheduling -Set up recurrence on BizTalk Server receive locations with greater precision using time zones, months, weeks, and days
  • Application Insights– Send BizTalk Server tracking data to Application Insights for extensible analytics, to understand performance and usage of BizTalk Server
  • Event Hubs – Send BizTalk Server tracking data to Event Hubs
  • Power BI Integration – Analyze and report BizTalk Server tracking data using Power BI. BizTalk operational data template, for use with Power BI visualizations and dashboards
  • .NET components continue to evolve over the last release of the .NET MQ Client.
  • A new version of Enterprise Single Sign-On

BizTalk Server 2020 Adapters

BizTalk Server 2020 Deployment

BizTalk Server 2020 Advance Scheduling

Host Integration Services

Host Integration Services has improved a lot in this version of BizTalk Server 2020. In short, most of the packages in BizTalk Server are getting updated in this version:

  1. HIS as add on to BizTalk licensing
  2. New Microsoft platforms
  3. New IBM platforms – integrated feature packs

BizTalk Server Migration Tool

Microsoft IT develops the BizTalk Server Migration tool which they use to migrate their own BizTalk environments. Because the architecture of BizTalk Server is kept common between BizTalk Server 2010,2013, 2013 R2 and 2016, the migration does not affect the underlying layer of BizTalk Components. In addition, as Microsoft makes this tool available to BizTalk users, this tool will be helpful for them on dehydrating the configuration on an old BizTalk server and re-hydrate that configuration to new Azure IAAS or VM.

BizTalk Migration Tool References:

Integrate 2019 USA

Logic Apps IBM 3270 Connector

Currently, the Logic Apps connector for IBM 3270 emulation is available in preview. In short, the Microsoft Connector for 3270 integrates Azure applications with resources available via TN 3270 Screen Scraping.

IBM 3270 Connector

Robotic Process Automation (RPA) is an Industrial trend in cloud computing which offers screen scraping capabilities. Host Integration Server extends the Integration to the TN3270 connector with Telnet or TCP protocol to connect to IBM Mainframe systems. However, you require an Integration account to be able to connect the IBM 3270 adapters.

After that, Paul demonstrated the Integration of IBM mainframe programs and Azure Cloud application with 3270 Logic App connector. After the Demo, he made a couple of announcements about CICS, IMS screen clients for 3270/5050 connectors and VSAM, DDM .NET clients for IBM file systems.

Logic Apps IBM Connectors Road Map

In conclusion, Paul touched base on the Logic Apps IBM Connectors Road Map beyond the 2020 version. For instance, BizTalk Server vNext version will include improvement on a number of Logic Apps connectors:

  1. 3270 IBM Connector Improvement
  2. Midrange 5250 AS connector
  3. Mainframe SAM Connectors
  4. DB2 and Informix Support

IBM Connectors Road Map

Conclusion

The well-informed reader might have noticed that several BizTalk Server 2020 features also appear in the so-called Feature Packs for BizTalk Server 2016 which have been released by Microsoft. However, BizTalk Server 2020 also helps in aligning with the latest versions of the platform products. So, this release of BizTalk Server will be of use for all BizTalk Server customers.

Secondly, the release of BizTalk Server 2020 is good news for all existing BizTalk users, as it indicates that Microsoft covers them for the next couple of years. Similarly, we notice that customers who are just starting their integration journey, also choose for Microsoft BizTalk Server. In other words, this new release will be very helpful for them as it will help them with both their on-premise integrations, as well with their hybrid integrations.

The announcement of BizTalk Server 2020 was done during Integrate 2019 in London. However, in case you missed the event, there is no need to worry, because at the end of this month, Integrate will also come to the US! So, if you are still not sure if you should attend Integrate 2019 US, then this blog post, written by our CEO Saravana Kumar, might convince you to participate in the world-wide premier Microsoft Integration event.

Integrate 2019 USA

The post BizTalk Server 2020 Announcement with Hybrid Integration updates appeared first on BizTalk360.

System Alerts and Unmapped Application Artifacts in BizTalk360

System Alerts and Unmapped Application Artifacts in BizTalk360

Introduction

BizTalk360 has powerful monitoring features to manage the BizTalk Application Artifacts, Queues, Infrastructure, Health check tools, etc. Most of the customers are using BizTalk360 for its core monitoring capabilities. BizTalk360 is the single tool to manage operational activities, monitoring and analytics of the mission-critical BizTalk environment. In this transforming phase, it is a tedious process for a dedicated person to monitor all the configurations. To address this scenario, System Alert notification is implemented in BizTalk360 v9.0.

Why System Alerts?

Currently BizTalk360 use Alarms to send out the notifications of Application artifacts, queues, Infrastructure settings health of the BizTalk environment. In the same perspective there is need to know the health of BizTalk360, which will helpful to the administrator who takes care BizTalk360. The BizTalk360 Monitoring service will take care of sending system notifications to the configured admin user email boxes. These email boxes can be configured in the system settings. A new monitoring sub service called ‘System Alert’ has been introduced to send system related notifications. System Alerts can be enabled in the System Alert Settings under Monitoring and Notification tab of system settings. A user can configure multiple admin emails with semicolon separated values.

A System Alert notification can be pushed to the admin users in the two ways:

  • Scheduled
  • Automatically Triggered

The BizTalk360 Monitoring service has a polling interval of 60 seconds and it will check for system alerts to be pushed. Either these alerts will be pushed based on a trigger or on a schedule.

NOTE: Custom Notification channel alerts are not applicable for Systems Alerts in this version (9.0).

System Alert Schedules

System Alert Schedules will send periodic reports of unmapped application artifacts. This will be helpful to the BizTalk Operation Users/ Administrators who can take appropriate actions to configure the artifacts for monitoring.

System Alert configuration can be scheduled as follows:

  1. Daily – Daily schedule will push the notification at the specified time in a day for all the days of the week.
  2. Weekly – Weekly schedule can push the notification on the specified day and time in the week (for example, Wednesday, 12 PM)
  3. Monthly – Monthly schedule can push the notification on the specified day and time in the month (for example, the 15th day of the month, 10 AM)

System Alerts Schedule Configuration

Based on the configuration, the Monitoring Service will alert the Unmapped Application Artifacts in the BizTalk Environment. The Unmapped Application Artifacts list will be available as an attachment in the email notification.

Unmapped BizTalk Application Artifacts

The Unmapped Application Artifacts list feature can be used to manage the artifacts which are not mapped to any of the BizTalk360 Alarms for monitoring.

Scenario

It will be helpful for the administrators who are taking care of the application artifact’s health. When BizTalk Developers or a deployment team deploy new artifacts into the BizTalk Group, the BizTalk administrators might not exactly know the newly deployed artifacts. In such cases, the Unmapped Application Artifacts status will provide a warning indication to the administrators/operators.

The status of Unmapped artifacts is shown in the monitoring dashboard. Unmapped application artifacts status will be healthy when all the application artifacts in an environment are mapped to the alarm. It will show the unhealthy status, when any of the artifacts left unmapped for monitoring. “Unmapped Application Artifacts Link” will be listed with the application artifacts which are still unmapped.

Unmapped Application Artifacts

Unmapped Monitor State

From BizTalk360 version 9.0 on, the new Unmapped monitoring status is introduced. This is the default state for application artifacts. Users can set the expected state and start monitoring the artifacts.

NOTE: Users can use the Do Not Monitor state, if they don’t want to monitor an artifact. In such cases, that artifact will not be listed under Unmapped application artifacts.

Automatic Triggered Alerts

When the events or conditions meet the triggering rules, then the alerts will be triggered automatically. System Alerts can be of triggered based

  • BizTalk360 License Expiration
  • Database Sizes (BizTalk Database & BizTalk360)

In this version of BizTalk360, BizTalk360 License Expiration is implemented. Database Sizes triggered notification will be implemented in a future version of BizTalk360.

License Expiration

When your BizTalk360 license is about to expiry, BizTalk360 System Alerts service will notify the license expiration to the admin users. Notification can be sent on the 30, 15, 7, 2 days before the license expiry date.

System Alert History

Click on the ‘System Alert History’ button to view the historical System Alerts. Alert History is maintained for both alert types(Schedule and Trigger). Users can view the system alerts email notification which has been sent to admin users in HTML Format.

System Alerts History

Data Purging

System Alerts historical records are maintained based on the data purging policy of “Alert & Maintenance History”. BizTalk360 will maintain the system alert notification history up to the configured number of days/months.

Based on your requirement you can set the purge duration.

Conclusion

System Alerts notifications will be helpful to BizTalk Administrators to manage the BizTalk Group health and BizTalk360 environments.

Get started with the free 30 days trial. For any queries/feedback please write to us support@biztalk360.com.

The post System Alerts and Unmapped Application Artifacts in BizTalk360 appeared first on BizTalk360.

Achieving a Consolidated Monitoring View Using Custom Widgets

Achieving a Consolidated Monitoring View Using Custom Widgets

BizTalk360 Monitoring

BizTalk360 comes with out of box capabilities to monitor BizTalk Artifacts, BizTalk Environment, Queues (IBM, Azure Service Bus and MSMQ), File Location (File, FTP and SFTP) and so on. The Monitoring Dashboard represents the Monitoring Status in a nice graphical chart view. Different set of users like BizTalk Administrators, Operators, Technical members and Business users are using the monitoring features for their day to day activities. Users create multiple alarms to suffice their needs to monitor the health of their BizTalk environment.

While users are monitoring the multiple alarms, they must keep an eye on multiple dashboards. It is a cumbersome task if a user takes care of more than two Alarms. Based on this requirement, few customers have requested for consolidate view of alarms. In this article, we are going through the process of how you can create a consolidated view of multiple alarms using Custom Widgets.

Scenarios

BizTalk360 users are configuring the alarms based on their business vertical or alignment to their process. There have been different patterns in alarm configurations. We can see some of the commonly used patterns;

  • Integration-based alarm configuration

Configuring BizTalk Artifacts, Web Endpoints, Queues, File Location etc. of integration in an alarm.

  • Role-based alarm configuration

BizTalk Administrators will look after infrastructure settings like Disk, System Resources, Host Instances and configure in an alarm BizTalk Operators can configure BizTalk Artifacts and other entities in a single alarm

  • Entity-based alarm configuration

For instance, users can configure all their Queues (IBM/Azure Service Bus) in an alarm. Similarly, they can configure Web Endpoints, Infrastructure (Disks, System Resources) in separate alarms.

Based on the above patterns, Role-based and Entity-based alarms can have a single view of multiple alarms in which you can monitor all the entities. When you follow the Integration-based alarm pattern, then entities like Application Artifacts, Azure Service Queues and BizTalk Server health activities are in different alarms.

Many organizations follow the Integration pattern by grouping all the related artefacts in an alarm. In this scenario, users (BizTalk Operators/BizTalk Technician) must monitor multiple alarms. For Instance, BizTalk Operators looking after BizTalk Artifacts or Developers responsible to monitor Azure Service Bus Queues will have to monitor the multiple alarms. Let’s see how to overcome this challenge by using custom widgets to group multiple alarms.

Consolidated Dashboard

The Monitoring Dashboard is one of the most used features among the users to monitor BizTalk Environments and Artifacts in BizTalk360. An enriched monitoring capability while using Custom widgets to group the multiple alarms to get a single view based on business or team members role.

Custom Widgets are a powerful feature in BizTalk360, as such widgets will provide the opportunity to meet their business scenarios. Many customers adopted Custom Widgets to provide solutions to end users.

Following are a few illustrated scenarios of grouping alarms when it is configured based on integrations

  1. Azure Service Bus Queues
  2. BizTalk Application Artifacts
  3. BizTalk Environment Health

In this article we take one of these scenarios, to show how users can achieve the grouping alarms using Custom Widgets

Azure Service Bus Queues

Custom Widgets used BizTalk360 Secure SQL Query API’s to fetch all Azure Service Bus Queues Monitoring Status from BizTalk360 Database. A user can select the filter with different alarms they want to group.

By default, the user can see only the unhealthy queues in an environment. If they set the Enable Healthy Queue option, they will be able to view all queues in the tree nodes. The Azure Service Bus Queues Custom Widget will fetch the information from Secure SQL Query with a refresh interval which is configured in the script of the custom widget.

Follow the below steps to achieve the Custom Widget’s script;

1. Secure SQL Query

Create the Secure SQL Query to fetch the monitoring status of Azure Service Bus Queues in the environment. To know more about how you can execute Secure SQL Queries using custom widgets, refer to this article.

2. Custom Widget Script

Create the initial variables, Place holders and SQL Query to call the BizTalk360 API method in the Custom Widget’s script

// BEGIN User variables
    azureQueueRefresh = 20;
    username = "`username_812414`"; 
    password = "`password_701422`"; 
    environmentId = "`productionenvid_189308`";    
 queryId = "8eea2771-10a7-44c6-8709-a597687434cf"; 
 queryName = "Azure ServiceBus Queues"; 
 sqlInstance = "kovai-bts";     
database = "BizTalk360"; 
    sqlQueryForAzureGraph = "Select AA.Id,AA.[Name],AME.MonitorStatus,AME.ExecutionResult from [dbo].[b360_alert_MonitorExecution] AME Inner Join[dbo].[b360_alert_Alarm] AA ON AME.AlarmId = AA.Id and AME.MonitorGroupType = 'Queues' WHERE AME.LastExecutionTime = (SELECT MAX(LastExecutionTime) from [dbo].[b360_alert_MonitorExecution] WHERE MonitorGroupType = 'Queues' and AA.IsAlertDisabled='false' and AlarmId = AA.Id and LastExecutionTime >= DATEADD(MINUTE,-60,GETUTCDATE())) AND AME.EnvironmentId ='`productionenvid_189308`'"; // The SQL Query which needs to be executed from the custom Widget
    bt360server = "biztalk360.westus.cloudapp.azure.com"; // Name of the BizTalk360 server (needed to do an API call to execute the SQL query
    //Mention the created alarm details and the respective partner name to display in the graph.
    AlarmDetails = [
        { AlarmName: "Threshold-1", PartnerName: "Air India" },
        { AlarmName: "Threshold-2", PartnerName: "British Airways" },
        { AlarmName: "Threshold-3", PartnerName: "DHL" },
		{ AlarmName: "BizTalk360 Default Alarm1", PartnerName: "Jet Airways" }
    ];
    // END User variables

3. GO JS Framework

BizTalk360 uses the GO JS Framework- Organization chart to represent the monitoring hierarchical view in the UI. It will display the organization chart structure as Azure Service Bus Queue node as a root node, Integration friendly name as second level node and Queue threshold violation details as a child node. You can control the nodes expand/collapse capability of through the custom widget code.

4. Filter Options

Users can able to filter the data based on the selected alarms or the option to show healthy queues. By default, users can see only unhealthy Azure Service Bus Queues. Similarly, other scenarios are implemented through custom widgets. To get the full source code of the Custom widgets you can download it from our GitHub Project.

Summary

We hope this article is useful in grouping the multiple alarms into a single view in an environment. This will bring you more control over the custom development you may want to achieve.

Get started with the free 30 days trial. For any queries/feedback please write to us support@biztalk360.com.

The post Achieving a Consolidated Monitoring View Using Custom Widgets appeared first on BizTalk360.

BizTalk360 V8.9.5 Enhanced Features

BizTalk360 V8.9.5 Enhanced Features

Introduction

BizTalk360 has many features that BizTalk Operators and BizTalk Administrators can take advantage to manage their BizTalk Environments. BizTalk360 is constantly improving the existing features based on the customer’s feedback in every release of the product. Similarly, in version 8.9.5 the existing features are improved based on the customer votes for the feature request. In this article, we can see the three different feature improvements like Filter Improvements, ESB Fault Delete Operation, Notification channel reconfiguration that has been implemented.

“The first step in exceeding your customer’s expectations is to know those expectations.” ~ Roy H. Williams, Author & founder of the Wizard Academy Institute.

Filter Improvements

The Filter Component is an important tool to get query results from BizTalk databases like BizTalkMsgBox, BizTalkDTADb and BAMPrimaryImport. In Message Box Queries and Tracking Queries (Graphical Message Flow), there were few filter operators missed in the previous version of BizTalk360. Due this missing filter operators’ customers faced challenges to filter data they expected get results out of it. To overcome this situation, BizTalk360 planned to implement the filter operators to match with BizTalk Admin Console. Please find the consolidated list of filter operators that are implemented in the latest version of BizTalk360.

Message Box Queries:

In Message Box Queries, to the filters Instance Status and Service Class “Not Equals” Operator are implemented in Operation and Data Monitoring Section

  1. All In-Progress Service Instances
  2. Running Service Instances

Similarly, other operators like IS Null, IS NOT NULL and Does Not Contains are implemented. While choosing the filter operators “IS NULL and Is NOT NULL“ there is no need of filter inputs.

Graphical Message Flow:

ESB Filters:

In the ESB Module, “Machine Name and Scope” are implemented in Operation (ESB Exception Data) and Data Monitoring Section (ESB Data) Machine Name filter has operators:

  1. Equals
  2. Not Equals

Scope filter has operators:

  1. Equals
  2. Not Equals
  3. Contains
  4. Does Not Contains

Custom Notification Channel Reconfiguration

Custom Notification channel is a powerful mechanism to notify BizTalk360 monitoring alerts to various channels like Slack, Service Now, Microsoft Teams etc. There is even a possibility that a customer can write their own custom notification channel.

Scenario:

A customer has written their own custom notification channel like PowerShell notification to execute the PowerShell scripts to restart the Windows service when an Error or a Warning occurred. Based on the business case, they want to enhance the custom notification channel implementation.

In this situation, the customer must remove the configured notification channel from the alarms to which it is associated. Then reconfigure custom notification channel to the alarms. If the number of alarms is large, then it is tedious process to remove and reconfigure the notification channel from all the alarms. To Overcome this challenge, in this version 8.9.5 Custom Notification Channel Reconfigure capability has been introduced. By this feature, users don’t need to remove the configured channels from the alarms. Instead the user can click on Reconfigure button and choose the latest DLL to update the existing channel object stream in the database.

The automatically reconfigured notification channel will affect from the next monitoring cycle. The users must reconfigure the notification channel properties value, if they introduce a new alarm or global properties in the updated notification channel. The user can view the list of alarms to which a custom notification channel is associated. This will give insight in the number of alarms the notification channel is associated with.

ESB Fault Delete Operation

From the Feedback portal we understood that customers are interested to have Delete Operation on ESB Fault Exceptions.

Scenario:

A Send Port that you are using in an application fails unexpectedly, therefore both the service instance and the message become suspended and the fault information is written into the ESB exception database. Once the exception is corrected and resubmitted for further go, there may be two scenarios which will come into picture:

  1. Messages which are submitted successfully are residing in the ESB Exception database but are of no further use
  2. Messages are rerouted to the ESB Exception database due to recurrent failure. In this specific case, the original message is also available in the ESB portal

ESB Fault Deletion is introduced in the ESB Exception Management section. The user has the option to select multiple faults, which can be deleted in the one go. The delete operation will remove the data from the following tables in the ESB Exception database.

  1. [MessageData]
  2. [ContextProperty]
  3. [Message]
  4. [Fault]

The Delete operation will delete Multi-Part message or multiple messages and context properties related to the faults from the corresponding tables.

Governance Audit:

BizTalk360 will capture the governance audit about the message delete operation. It will audit the user who performs the delete operation and message Id’s are displayed in governance audit section. Similarly, these operations are notified in Live Feed section.

Summary

We hope that the enhanced features will give you more control over the filters, Custom notification channels and ESB Faults. We always keep track of our Feedback Portal and take up the valid suggestions and feedback. If you would like to know more about how BizTalk360 can help your organization to manage your BizTalk Server middle ware platform, feel free to contact us.

Author: Senthil Kumar Palanisamy

Senthil Kumar Palanisamy is the Technical Lead at BizTalk360 having 14 years of experience in Microsoft Technologies. Worked various products across domains like Health Care, Energy and Retail.

BizTalk Application Deployment Using Azure Pipeline with BizTalk360 API’s

BizTalk Application Deployment Using Azure Pipeline with BizTalk360 API’s

Introduction

The process of BizTalk Application deployment is a complex, time consuming and repetitive task. Microsoft BizTalk Server comes with capabilities to create deployment packages. However, the MSI packages you create with the out-of-the-box features of BizTalk Server have some shortcomings which lead to deployments which are error-prone.

BizTalk Deployment Framework

The BizTalk Deployment Framework (BTDF) covered these deployment issue.

  • Deploy complex BizTalk solutions containing BizTalk artifacts but also for example SSO settings, ESB itineraries and SQL Server scripts, with no human intervention
  • Consolidate all environment-specific configuration and runtime settings into one, easy-to-use Excel spreadsheet
    Maintain a single binding file that works for all deployment environments
  • Make automated deployment a native part of the BizTalk development cycle, then use the same script to deploy to BizTalk servers

To eliminate the manual process, organization’s start to adopt the DevOps practice. In this article, we will see how the BizTalk360 API’s are helpful in a BizTalk environment monitoring while deploying BizTalk Applications using Azure CI/CD Pipelines.

Deploy BizTalk Applications with Azure Pipelines

Microsoft has introduced automated deployment of BizTalk Applications in BizTalk Server 2016 Feature Packs. In BizTalk Server 2016 Feature Pack 1, automatic deployment and application lifecycle management (ALM) experience was introduced. Automatic deployment process has been improved with the release of BizTalk Server 2016 Feature Pack 2.

  1. Within Visual Studio for each project, set the Application Name of your BizTalk application
  2. In addition to using an agent-based deployment, you can also use deployment groups to deploy your BizTalk applications to multiple servers
  3. In the release task, you can install the BizTalk application, and enter the BizTalk management computer, and the path to the deployment package

Azure Pipelines (CICD)

With Automated build/release definitions you can set up the CI or CICD Pipelines based on environment (Development/QA/Staging/Production):

  1. Continuous Integration (CI) : Set up build task definitions rules to the VSTS Source code base
  2. Continuous Integration Continuous Deployment (CICD) : Configure both build task definition and release task definition to automate the build and test process

If you are new to Azure CI/CD Pipeline go through this article Configure automatic deployment with Visual Studio Team Services in BizTalk Server

Deployment Build Definitions

BizTalk Build Pipelines

In the BizTalk Application deployment process, you can create the following build/release definitions:

  1. Visual Studio Build

    The build and release definitions are Visual Studio Team Services tasks, and should be done by a VSTS admin. The build definition builds your project within your git or VSTS repository, and the release definitions deploys it to your BizTalk Server environment

  2. Publish Build Artifacts

    In Publish Build Artifacts stage configure the Path to publish (Application Project Folder)
    Publish Build Task

  3. Deploy BizTalk Server Application

    BizTalkApplicationDeployment
    A BizTalk Server application deployment build definition operation has three options available

    1. Create new BizTalk Application: Deploys a new application, if the application already exists, it uninstalls the current applications (full stop), and installs the new application. If continuous integration is enabled, it automatically redeploys the application when it is updated in the repository
    2. Update an existing BizTalk Application: Appends changes, such as schemas, to an already running application. It does not require a full redeploy of the application
    3. Install BizTalk Server Application: Install the application, enter the BizTalk management computer name and the deployment package path

BizTalk360 API’s in CI/CD

Customer Scenario: In Build pipeline stages the customer wants to utilize the BizTalk360 API to set the Stop Alert for Maintenance in a BizTalk environment. Once after the new build is deployed to BizTalk Server and then user wants to configure newly deployed artifacts to monitor in BizTalk360. As the last step revoke the Stop Alerts in BizTalk360 Application.

We can use the following steps in Build definitions to achieve the above scenario;
BizTalk Depolyment using BizTalk360 API

  1. Visual Studio Build (explained above)
  2. Set Stop Alert for Maintenance

    BizTalk360 customers can take advantage of using the BizTalk360 API’s to set the alert maintenance during the BizTalk Application deployment.
    Create the PowerShell Script stage to call the SetAlertMaintenance API method to set the BizTalk environment under the maintenance. Assign the PowerShell Script full path
    Set Alert Maintenance

    SetAlertMaintenance.ps1

    $Request = '{
      "context": {
        "callerReference": "SAMPLE_REFERENCE",
        "environmentSettings": {
          "id": "DC45E848-E485-41D7-92E5-7C9932D43CB8"
        }
      },
      "alertMaintenance": {
        "comment": "BizTalk Deploy",
        "maintenanceStartTime": "2018-11-22T15:26:37.000",
        "maintenanceTimeUnit": 0,
        "maintenanceTimeLength": 2,
        "isActive": false,
        "expiryDateTime": ""
      }
    }'
    $Request
    $body = ConvertFrom-Json -InputObject $Request
    $body
    $RepsonseSet = Invoke-RestMethod -Uri http://biztalk360webserver/Services.REST/AlertService.svc/SetAlertMaintenance -Method Post -ContentType "application/json" -Body $body -UseDefaultCredentials
    $RepsonseSet 
    
    

    Set Alert Maintenance API returns the alert maintenance object with maintenance Id (out parameter). You can refer this maintenance Id, when revoking the maintenance in the below step.

  3. Publish Build Artifacts (explained Above)
  4. Deploy BizTalk Server Application

    Once after BizTalk Application deployment is completed, next step you can configure the newly artifacts to monitor. For creating BizTalk360 alarms and BizTalk Application mappings during deployments with BTDF, you could consider to BT360Deploy command-line tool. Use this GitHub project to setup pipeline stage.

  5. Remove the Stop alert for Maintenance

    After the deployment of BizTalk Applications, the maintenance mode of BizTalk environment must be revoked in BizTalk360. To achieve this step, create a PowerShell Script stage and assign the PowerShell Script path(RemoveMaintenance.ps1)

    $RequestRem = '{"context":{"callerReference":"SAMPLE_REFERENCE","environmentSettings":{"id":"DC45E848-E485-41D7-92E5-7C9932D43CB8"}},"maintenanceId":"308d8b2c-b3e0-4024-b600-845113b06b80"}' 
    
    Invoke-RestMethod -Uri  http://biztalk360webserver/BizTalk360/Services.REST/AlertService.svc/RemoveAlertMaintenance -Method Post -ContentType "application/json" -Body $RequestRem -UseDefaultCredentials
    
    

    Remove Alert Maintenance

Summary

Automation of Build and/or Deployment will be helpful to reduce the turnaround time. Hopefully, this article has provided insight in how you can setup the BizTalk Application Deployment automation and how you can utilize BizTalk360 API’s to stop alert during deployment and how to set up monitoring of the newly deployed BizTalk Artifacts.

Get started with the free 30 days trial. For any queries/feedback please write to us support@biztalk360.com.

Author: Senthil Kumar Palanisamy

Senthil Kumar Palanisamy is the Technical Lead at BizTalk360 having 14 years of experience in Microsoft Technologies. Worked various products across domains like Health Care, Energy and Retail.

Optimizing the Data Collection in BizTalk360

Optimizing the Data Collection in BizTalk360

Introduction

BizTalk360 is a tool for operating, monitoring and getting analytical information for BizTalk environment(s). BizTalk360 comes with productivity tools like the Advanced Event Viewer, the Throttling Analyzer, the Backup/DR visualizer, the BizTalk Health Monitor etc. These tools are also designed to help support people to perform their activities in a seamless way. BizTalk Server, being middle-ware, high volumes of data passes through the Message Box. Based on the business requirements, Tracking is enabled for important BizTalk Integration transactions.

In similar fashion, BizTalk360 collects important information to perform various activities in operations, monitoring and analytics section. In this article, we show BizTalk360’s data collection features and how you can configure the settings to optimize the data collection.

Advanced Event Viewer

The Event Log is an important component which is used by BizTalk Administrators to investigate, if any configuration or transaction errors have happened in BizTalk Server. The Advanced Event Viewer is a powerful feature in BizTalk360 which in fact is a central repository for collecting the event logs from multiple BizTalk servers and SQL servers in BizTalk environment. It has the capability to filter the event logs by various parameters like Event Id, Event Type, Sources etc.

The BizTalk360 Monitoring service collects the event log information based on Event Logs and Event Sources which are configured in the BizTalk360 Advanced Event viewer settings.

BizTalk360 Monitoring IOPS

BizTalk360 uses Windows Management Instrument (WMI) queries to collect the event log details from BizTalk Servers and SQL Servers in an environment. In a multi-server BizTalk environment, few customers raised concern about BizTalk360 IOPS (input/output operations per second). In this set-up, there is change of high volume of IOPS in BizTalk360 Box (Monitoring Service).

BizTalk360 Recommendation
To avoid large volume of data collection or to improve the performance of the BizTalk360 monitoring services, following event log configuration are suggested
1. Enable the necessary servers to collect Event Logs and Event Sources
• Primary BizTalk Servers in Active – Passive scenario
• Multiple SQL server, enable the event log data collection for important nodes.

2. BizTalk360 provides a default set of Event Logs and Event Sources which are widely used in BizTalk and SQL servers. Configure the necessary Event Sources that are required as per the business requirement:

• BizTalk Event Log Sources(Default):
Bam Event Provider, BamManagementUtility,BAMWebServices,BizTalk DW Reporting, BizTalk Server, BizTalk Server Deployment, BizTalk Server EDI, BizTalk360 Monitor, BusinessRulesEngine, ENTSSO,MSDTC,MSDTC Client, RuleEngineUpdateService, System.ServiceModel 4.0.0.0, System.ServiceModel.Install 3.0.0.0,W3SVC-WP,XLANG/s

Configure required BizTalk Sources as
“BizTalk Server,BizTalk Server Deployment,BizTalk Server EDI,ENTSSO,MSDTC,MSDTC Client,RuleEngineUpdateService”

• SQL Sources(Default):
MSSQLSERVER, MSSQLSERVER$AUDIT, MSSQLServerOLAPService, SQLSERVERAGENT

Configure required SQL Sources as
“MSSQLSERVER, SQLSERVERAGENT”
Note: This configuration will reduce the BizTalk360 Input/Output operations. You can read this article to know more details about Event-Log Data collection configuration.

Analytics Performance Data

In a data-driven business setup, there will be scenarios where critical executive decisions are made based on the performance reports of various components of the listed servers in a BizTalk environment. BizTalk360 aims to offer an out of the box tool with similar capabilities as the Performance Monitor tool in Windows servers.
BizTalk360 Analytics offers visual display of the most important performance counters that are consolidated and arranged on a single screen so that the information can be monitored in a glance. Custom reports can be built in minutes with the metrics that really count for your business, and they are easy to share with other users. In addition, dashboards can give you a summary of many reports on a single page using drag-and-drop widgets for fast and easy customization.

AdvancedEventViewerByServer
Performance metrics data collection can be optimized by selecting the necessary metrics (Windows, SQL, BizTalk and IIS) based on the server type.

• BizTalk Server: Enable Windows, BizTalk by default. Web Application is configured in the server enable IIS and similarly to SQL.
• SQL Server: Enable Windows and SQL performance metrics

Tracking Data

Initial design of the tracking data widgets was simple; it ran the necessary queries directly against the Tracking database and displayed the results in a graphical form on the widget. A larger database degraded the performance of these queries and it had a direct impact on the user experience of the Analytics dashboard, causing slower widget loading and occasional time outs. Since Analytics widgets offered users the option to view data up to 30 days within the widget, an efficient approach was required to improve the performance and user experience.
In BizTalk360 v8.3 to overcome these challenges, we decided to create a service that runs in the background. This service collects the tracking data periodically in small chunks and thereby avoid running expensive queries on the Tracking database. Here, only the resulting metrics are collected rather than pulling all the tracking data to BizTalk360 database. This improves the overall performance of the Tracking widgets as they are not querying the Tracking database directly.
BizTalk360 Analytics needs the tracking data to represent data in custom widgets

• Failure Rate by Port and Schema
• Message Performance by Host, Orchestration, Pipeline, Adapter, Port and Schema

BizTalk360 suggests enabling the data collection for categories and counters needed in custom widgets. If you want custom widgets for Failure Rate and Message Performance of Port and schema, enable only those counters in Tracking data collection.

AnalyticsPerformanceData

Data Purging

Collecting the necessary data is important to be able to represent the data in BizTalk360. However, you don’t want unlimited growth of your BizaTalk360 comes out of box with the “Data Purging” feature to able to manage the size of BizTalk360 database. If you want to know more about Data Purging please read this article.

DataPurgingConfiguration

Constantly manual monitoring the database size is a tedious process. You can utilize the Database Query monitoring when the database size is grown over the threshold limit.

SELECT total_size_mb = CAST(SUM(size) * 8. / 1024 AS INT)
FROM sys.master_files WITH(NOWAIT)
WHERE database_id = DB_ID()

Conclusion

We hope this article is useful to optimize and manage the data collection for various features in BizTalk360.
Get started with the free 30 days trial. For any queries/feedback please write to us support@biztalk360.com.

Author: Senthil Kumar Palanisamy

Senthil Kumar Palanisamy is the Technical Lead at BizTalk360 having 14 years of experience in Microsoft Technologies. Worked various products across domains like Health Care, Energy and Retail.

Best Practices to Create Custom Widgets in BizTalk360

Best Practices to Create Custom Widgets in BizTalk360

Introduction

Custom widgets are powerful feature in BizTalk360, that allow the users to bring third-party integrations into BizTalk360.  It will be useful to bring the custom solutions into the BizTalk environment as it allows to view integrated solutions in one single tool.

Custom widgets can be used for various purposes, below list constitute few important scenarios;

  • Embed third-party portals like Power BI or internal portals
  • Monitor BizTalk Artifact Statuses
  • Display query results using Secure SQL Queries

In this article, we can see important best practices while creating Custom widgets.

Best Practices 

1. Hiding confidential information in Custom Widgets with Placeholders

Custom Widgets are created using API’s (e.g.: BizTalk360 or BizTalk Integration API), which might contain confidential information like the credentials of your BizTalk360 Service Account or API Authenticated credentials. Of course, you don’t want this information to be revealed in plain text in your widgets. To hide such confidential information, you can use Placeholders.

Beside using Placeholders to hide confidential information, you can also use them for providing frequently used data to your Custom Widgets. Think of for example the BizTalk360 “EnvironmentId”, which is a value which often needs to be provided to Custom Widgets. By creating a Placeholder for the “EnvironmentId”, you can easily re-use it and you don’t need to search for such values, each time you are creating a Custom Widget which needs such a value.

Place Holder

Read this article to know more about creating the place holders and how it can be used in widget.

2. Knockout JS Functions

Frequently, utility functions are used while binding the data to view. In these situations, the user can utilize the BizTalk360 referenced utility functions.

BizTalk Integration API methods returns its responses in XML format. To represent the data in client script(JavaScript/jQuery),it needs to be in JSON format.

x2js = new X2JS({ attributePrefix: '' });
bizTalkHostsList = function () 
{
  var _this = this;
 _this.getBizTalkHosts(function (data) 
  {
   _this.bizTalkHosts(x2js.xml_str2json(data.queryResult));
  });
};

When a view expects the API method to return the result as an array, occasionally a single record is produced. In this situation,the API result needs to be converted to an array. 

x2js = new X2JS({attributePrefix: '', arrayAccessForm:"property", 
arrayAccessFormPaths : ["root.records.record"]});
bizTalkSendPortsList = function () {
      var _this = this;                 
      _this.getbizTalkSendPorts(function (data) 
{
      var results = x2js.xml_str2json(data.queryResult);
      if (Array.isArray(results.root.records.record))
         _this.bizTalkSendPorts(results.root.records.record);
      else {
         _this.bizTalkSendPorts([results.root.records.record]);
         }
      });
   };

3. BizTalk360 Styles and Icons

 The styles of BizTalk360 can be used in custom widgets by inspecting the element in the BizTalk360 Application.  You can take advantage of the listed styles and controls while creating the custom widgets.

CSS Styles

Utilize BizTalk360 in-built CSS in custom widget creation

  • List the data in Table: table table-lists
  • Scroll Bar: WidgetScroll
<div id=”WidgetScroll” style=”top:30px;” data-bind=”addScrollBar: WidgetScroll, scrollCallback: ‘false'”> 
  <table class=”table table-lists”>
  </table>
</div>
  • Success/Info Tag: Use the Success or Information tag based the data to be presented in the list
 <span data-bind="text: HostType == 1 ? 'InProcess' : 'Isolated', css: HostType == 1 ? 'success-tag' : 'info-tag'">
 </span>

Boot Strap

Boot Strap styles and tags could be used during widget creation. BizTalk360 uses Boot Strap styles to design the base layouts.

  • Container Fluid
  • row
  • col-md-*
<div class="container-fluid">
  <div class="row">
    <div class="col-md-8"></div>
    <div class="col-md-4"></div>
  </div> 
</div>

Font Awesome Icons

Users can represent status or any information in graphical way, you can use Font Awesome Icons.

  • Success – fa fa-check-circle
  • Error – fa fa-power-off
  • Warning – fa fa-times-circle

KendoUI Controls

BizTalk360 uses KendoUI controls, users can utilize those controls in widget script

  • Drop Down
  • Grid
  • Switch Box
  • DateTime Picker
  • Numeric Text Box
  • Calendar
// View
<div class="form-group margin-t"> //Boot Strap Form Group
  <div class="col-md-10"> // Boot Strap Column
    <div class="col-md-offset-2">
      <div class="alert alert-info" role="alert"> //Boot Strap Information
        <b>Note :</b> Choose the environment and enable the servers for Event Log data to be collected.
      </div>
      <div class="form-horizontal">
        <div class="form-group" data-bind="validationElement: selectedEnvironment">
          <label class="col-md-3 control-label">Select Environment</label>
          <div class="cus-select" style="text-align: left;">
            <label>
              <select data-bind="options: configuredEnvironments, optionsText: 'name', value: selectedEnvironment" 
               class="form-control" style="min-width:300px;"></select> // Kendo Dropdown
             </label>
           </div>
         </div>
       </div>               
     </div>
   </div>
</div>    
        
//ViewModel     
 selectedEnvironment.subscribe(onEnvironmentChanges, this); // change event to the environment

4. Embedding Third-Party Scripts

High Chart Solid Gauge

BizTalk360 uses a component called High Charts to represent data in graphical way. Also, some of our customers are using the custom widgets to represent their BizTalk Integration in graphical chart. BizTalk360 supports High Charts (Basic Widgets) with High Chart JS.  To reference the third-party scripts, use the following JavaScript to reference the external libraries.

function injectScript(src) {
    return new Promise((resolve, reject) => {       
                for (var i = 0; i < src.length; i++) {
    var script = document.createElement('script');
    script.src = src[i];
    script.async = false; // This is required for synchronous execution
    document.head.appendChild(script);
  }        
        script.addEventListener('load', resolve);
        script.addEventListener('error', () => reject('Error loading script.'));
        script.addEventListener('abort', () => reject('Script loading aborted.'));
        document.head.appendChild(script);
    });
}
injectScript(['https://cdnjs.cloudflare.com/ajax/libs/highcharts/5.0.14/highcharts-more.js','https://cdnjs.cloudflare.com/ajax/libs/highcharts/5.0.14/js/modules/solid-gauge.js'])

Customer scenarios

Through our support channel, customers are explaining their scenarios which they require in their day to day activities. Here, we pick few common scenarios that will be useful in BizTalk Integration space.

Monitor Artifacts of a BizTalk application

BizTalk360 has the capability to monitor multiple application artifacts in a single alarm configuration. For instance, a specific group of users wants to monitor the status of specific application artifacts. To achieve this requirement, custom widget creation is one of the possible solutions.  Create the custom widget to monitor an application artifacts status and associate the widget to dashboard. You can download the custom widget of monitoring the application artifacts.

Secure SQL Queries

Secure SQL query is an important feature to fetch data based on the user’s business demand. However, it is hard to create all the possible transaction results out of box from predefined widgets. Custom Widgets will help the BizTalk360 users to address this gap in BizTalk360.

Please follow the article how to create custom widgets using Secure SQL Queries.

Represent the data using Cross Domain API and High Charts

A user wants to represent BizTalk Integration Transaction details in the graphical chart. The Integration API is hosted cross domain. To access the cross-domain API Methods through client script, you need to use the XMLHTTPRequest method

function createCORSRequest(method, url) {
var xhr = new XMLHttpRequest();
 if ("withCredentials" in xhr) {
 // XHR for Chrome/Firefox/Opera/Safari.
 xhr.open(method, url, true);
 } else if (typeof XDomainRequest != "undefined") {
 // XDomainRequest for IE.
 xhr = new XDomainRequest();
 xhr.open(method, url);
 } else {
 // CORS not supported
 xhr = null;
 return xhr;
}
//Make the actual CORS request
function makeCorsRequest(url,method) {
var xhr = createCORSRequest(method, url);
if (!xhr) {
 alert('CORS not supported');
return;
 }
// Response handlers
xhr.onload = function() {
var resultData = xhr.responseText;
expectedResults(x2js.xml_str2json( JSON.parse(resultData).MonitorQueryResult).root.records.record);
 }
 xhr.onerror = function() {
 alert('Woops, there was an error making the request.')
}
 xhr.send();
}
makeCorsRequest(‘https://YOURCROSSDOMAIN/SERVICENAME’,'GET');

In this case, the user wants to represent the data in High Charts solid-gauge pattern . For that, you can use the above mentioned suggestion “Embed Third-Party Script”.

Conclusion

Custom widget is a useful feature to bring the customized solution in a dashboard. Hope this article will be helpful to get start the custom widgets creation. Get started with the free 30 days trial. For any queries/feedback please write to us support@biztalk360.com.

Author: Senthil Kumar Palanisamy

Senthil Kumar Palanisamy is the Technical Lead at BizTalk360 having 14 years of experience in Microsoft Technologies. Worked various products across domains like Health Care, Energy and Retail.