Microsoft Integration Weekly Update: Feb 25, 2019

Microsoft Integration Weekly Update: Feb 25, 2019

Do you feel difficult to keep up to date on all the frequent updates and announcements in the Microsoft Integration platform?

Integration weekly update can be your solution. It’s a weekly update on the topics related to Integration – enterprise integration, robust & scalable messaging capabilities and Citizen Integration capabilities empowered by Microsoft platform to deliver value to the business.

If you want to receive these updates weekly, then don’t forget to Subscribe!

Feedback

Hope this would be helpful. Please feel free to reach out to me with your feedback and questions.

Advertisements

Using BizTalk360 for alarm notification statistics

Using BizTalk360 for alarm notification statistics

It is nothing new that BizTalk360 enables you to set up automated monitoring for BizTalk. This includes the deployed BizTalk application artifacts and different kind of endpoints. You can even track the processing which is taking place in your BizTalk environment.

For each unexpected situation for which you have set up monitoring, you can receive notifications via for example email, SMS/Text message or the Event Log. Besides that, BizTalk360 also provides Notification channels to connect to products like Slack, Microsoft Teams and HP Operations Manager. In case you are using ServiceNow, you can even have BizTalk360 generate incidents into that product! If that still does not meet your requirements, you can use a Web Hook, or the easy-to-use API provided by the product to integrate with your systems.
The BizTalk360 database contains information about the transmitted notifications.

Collecting Alarm statistics

Recently, we had a call with a consultancy company which manages several BizTalk environments for their clients. These clients are using BizTalk360 for operating and monitoring the BizTalk environments.

The consultancy company was interested in knowing whether BizTalk360 provides any statistics about all the alarms which were triggered over time. This kind of statistics serves different goals. On one side, this kind of information gives insight in components which frequently have outage, enabling to take counter measures. On the other side, this can also be used to provide such information to the client of the consultancy company as a kind of reporting.

In this blog post we describe how you can use the information in the BizTalk360 database, to show statistics about which alarms have been transmitting which kind of notifications over time.

Showing SQL data in BizTalk360

As already mentioned, the information about the transmitted notifications is stored in the BizTalk360 database. BizTalk360 provides different capabilities to show data which is stored in SQL Server databases. These capabilities are:

• Secure SQL Queries – see https://docs.biztalk360.com/docs/secure-sql-queries
• Custom Widgets – see https://docs.biztalk360.com/docs/creating-a-custom-widget-for-executing-secure-sql-queries

Depending on the requirements how you want to be able to view that data, you can use either or both features, but the main characteristics are that Secure SQL Queries are mainly used manually, where Custom Widgets can become shown on a dashboard and the output will be refreshed automatically.

The information we will be using in this blog post is stored in a table called b360_alert_notify_ToBeNotified. The content of that table is specific to Threshold monitoring. If you need more detailed information about Data Monitoring statistics, you should have a look at a different table. Based on the b360_alert_notify_ToBeNotified table, you can write a SQL query which looks like below:

SELECT aa.Name AS 'Alarm Name',
an.NotificationRequiredType AS 'Notification Type',
COUNT(an.AlarmId) AS Count,
MAX(CreatedDateTime) AS 'Most recent'
FROM b360_alert_notify_ToBeNotified an
RIGHT OUTER JOIN b360_alert_Alarm aa ON an.AlarmId = aa.Id
WHERE (an.NotificationRequiredType <> 'Regular')
GROUP BY aa.Name, an.NotificationRequiredType
order by [Alarm Name]

In above query, notifications of type ‘Regular’ have been left out. Regular notifications refer to the Health alarms which can be received at a specified time, to show the state of all the mapped artifacts of an alarm. For this scenario, that information is considered as not relevant.

Retention period

It is good to know, that without any filters, the data will be shown according the Purging policy for Alert & Maintenance History. This policy can be viewed and reconfigured in the Settings area, under BizTalk360 Health, in the Purging Policy screen.

Obviously, in case you do want to see a bit more limited data set, you can add WHERE conditions based on the CreatedDateTime field.

Showing Alarm statistics as a Secure SQL Query

When you create a Secure SQL Query for this, the output of the query can look like below.

As you can see, the query shows information about the name of the alarm, which kind of notification is involved, the count of notifications and the most recent date/time a notification has been transmitted, for that alarm/notification type combination.

If you want to know more about how to create Secure SQL Queries, you can check out the Documentation portal about this feature.

Having this kind of information as a Secure SQL Query can be very handy. You have the information easily at hand by accessing that query and run it. The results can also be exported to a .CSV file.

Showing Alarm statistics as a Custom Widget

If you want to have an even better experience, you can consider creating a Custom Widget for this. Doing so, enables you to have this information on a dashboard and because of the behavior of these dashboards, the widget will be automatically refreshed. Besides that, by a simple click, you can export the entire dashboard, which could contain more statistical widgets, to a PDF file and provide it to your clients.

Based on the query we have seen earlier in this article such a widget will look like below.

Once the widget has been added to a dashboard, the query will automatically be executed, and the data will be refreshed each minute. If needed, you can change the refresh interval in the script of the widget.

If you want to read more about creating such Custom Widgets, check the Documentation portal for it. It contains some detailed articles about this topic.

Conclusion

We have shown another way on how BizTalk360 can be useful. So, if you have any questions about this capability, or anything else related to BizTalk360 feel free to contact us. Why not give BizTalk360 a try! You can download a free trial which contains all the features for a limited time.

Author: Lex Hegt

Lex Hegt works in the IT sector for more than 25 years, mainly in roles as developer and administrator. He works with BizTalk since BizTalk Server 2004. Currently he is a Technical Lead at BizTalk360. View all posts by Lex Hegt

Terminating Dehydrated Service instances through BizTalk360 PowerShell Notification channel

Terminating Dehydrated Service instances through BizTalk360 PowerShell Notification channel

BizTalk360 has introduced notification channels from its major version release V8.0. One of the most greeted features from our customers. With the introduction of this capability, it’s easy to send alerts to any external systems like your ticketing system, internal databases, calling REST endpoints or executing PowerShell scripts.

One of the most powerful capabilities is to use an API, to enable you to create notification channels for connecting to your own systems. The PowerShell notification channel allows you to execute a PowerShell script, each time an alarm is triggered when the threshold is crossed.

During business transactions in mission-critical BizTalk Server environments, if any of the service instances are waiting for response messages, they will be turned to the dehydrated state. There might be valid reasons why such service instances are not needed anymore. So, as a BizTalk administrator, you should keep an eye on these instances in their environments to avoid further critical business consequences.

BizTalk360 provides Monitoring capabilities for dehydrated service instances and it sends periodic alerts via different notification channels. One such channel is the PowerShell Notification channel.

This blog post will give you an insight about how to terminate the dehydrated service instances through the PowerShell notification channel in BizTalk360, just by configuring it with the Alarms.

Steps to implement the PowerShell Notification Channel

1. Create the PowerShell script
2. Configure the PowerShell notification channel
3. Configure the PowerShell notification channel with the Alarm

Creating the PowerShell script

To develop the PowerShell script, it is essential to identify the API’s which will take care of these actions.

Step 1:

Retrieve the dehydrated service instances from the Message Box using the below API.
http://localhost/BizTalk360/Services.REST/BizTalkQueryService.svc/ExecuteServiceInstanceQuery

Code Snippet


import-Module Microsoft.PowerShell.Management
$username = "DomainUserName"
$password = "ABC1236890"
$password_base64 = ConvertTo-SecureString $password -AsPlainText -Force
$creds = New-Object System.Management.Automation.PSCredential ($username, $password_base64)
$body_ExecuteServiceInstanceQuery='{
"context":{
"callerReference":"REST-SAMPLE",
"environmentSettings":{
"id":"3beb9328-c435-47ed-8c51-a406117e632b",
"licenseEdition":0
}
},
"query":{
"compositeFilter":{
"filterDescriptorCollection":[
{
"member":"Instance Status",
"filterAvailableMember":[
{
"name":"Application",
"alias":"Application Name",
"dataType": 1,
"isList":true,
"isAutoComplete":false
},
{
"name":"CreationTime",
"alias":"Creation Time",
"dataType":0,
"isList":false,
"isAutoComplete":false
},
{
"name":"GroupResultsBy",
"alias": "Group Results By",
"dataType": 1,
"isList": true,
"isAutoComplete": false
},
{
"name": "HostName",
"alias": "Host Name",
"dataType": 1,
"isList": true,
"isAutoComplete": false
},
{
"name": "InstanceStatus",
"alias": "Instance Status",
"dataType": 1,
"isList": true,
"isAutoComplete": false
},
{
"name": "ServiceClass",
"alias": "Service Class",
"dataType": 1,
"isList": true,
"isAutoComplete": false
},
{
"name": "ServiceInstanceID",
"alias": "Service Instance ID",
"dataType": 1,
"isList": false,
"isAutoComplete": false
},
{
"name": "ServiceName",
"alias": "Service Name",
"dataType": 1,
"isList": true,
"isAutoComplete": false
},
{
"name": "ServiceTypeID",
"alias": "ServiceType ID",
"dataType": 1,
"isList": false,
"isAutoComplete": false
}
],
"memberType": 1,
"filterOperator": 2,
"value": "Dehydrated",
"progressIndicator": false,
"validationError": "",
"isValid": true,
"tempValue": ""
}
]
},
"maxMatches": "10",
"queryType": 0
},
"maxMatches": "10"
}'
$headers=@{"Content-Type"="application/json"}
$uri="http://BT360SUP03/BizTalk360/Services.REST/BizTalkQueryService.svc/ExecuteServiceInstanceQuery"
$response = Invoke-WebRequest -Uri $uri -Headers $headers -Method Post -Body $body_ExecuteServiceInstanceQuery -Credential $creds

Step 2:

Pass the retrieved dehydrated service instances for the termination process to the below API.

http://localhost/BizTalk360//Services.REST/BizTalkQueryService.svc/ExecuteServiceInstanceOperation

Code Snippet


$ExecuteServiceInstanceQueryobj = ConvertFrom-Json $response
if (!$ExecuteServiceInstanceQueryobj.serviceInstances) {write-Host "There are no dehydrated service instances for termination"}

foreach ($serviceinstance in $ExecuteServiceInstanceQueryobj.serviceInstances)
{
$body_ExecuteServiceInstanceOperation='{
"context":{
"callerReference":"REST-SAMPLE",
"environmentSettings":{
"id":"3beb9328-c435-47ed-8c51-a406117e632b",
"licenseEdition":0
}
},
"serviceInstances":[
{
"ServiceName": "'+$($serviceinstance.ServiceName)+'",
"ServiceClass": "Orchestration",
"StatusDisplayText": "Dehydrated",
"Application":"'+$($serviceinstance.application)+'",
"ServiceInstanceID":"'+$($serviceinstance.serviceInstanceID)+'"
}
],
"operation": 1
}'
$headers1=@{"Content-Type"="application/json"}
$uri1="http://localhost/BizTalk360//Services.REST/BizTalkQueryService.svc/ExecuteServiceInstanceOperation"
$response1 = Invoke-WebRequest -Uri $uri1 -Headers $headers1 -Method Post -Body $body_ExecuteServiceInstanceOperation -Credential $creds
$ExecuteServiceInstanceQueryobj = ConvertFrom-Json $response
}

Points to Remember

Following are the parameters need to be changed as per your environment configuration.

  1. Service Account credentials for UserName
  2. Password
  3. Environment Id
  4. URI

As mentioned in the introduction section, BizTalk360 exposes its APIs and can be found in the section Settings -> API documentation.

Terminating Dehydrated Service instances - API Documentation screen

Configuring the PowerShell Notification channel

Once the PowerShell script has been created, the next step is to configure it with the notification channel in BizTalk360.The PowerShell notification channel can be found in Settings -> Monitoring Notification Channels ->B360.Notifier.PowerShellNotification.

Terminating Dehydrated Service instances - Configure PowerShell Notification channel

Select the notification channel and click on the “Configure” button on the top menu bar. In the Configuration Notification Channel section, enter the location of the script which has been created for terminating the dehydrated service instances and click on the Configure button. Now, the PowerShell script has been configured with the BizTalk360 notification channel.

Terminating Dehydrated Service instances - Associate the Powershell Script Path

Associating the PowerShell Notification Channel with Alarms

The next primary step is to Map the notification channel with the Alarms for further execution and configure the Threshold violation settings, as how frequent you want to terminate the dehydrated service instances from your environment. Based on the settings, BizTalk360 will terminate the dehydrated service instances in your BizTalk Server environment.

Terminating Dehydrated Service instances - Alarm Configuration with PowerShell Notification Channel

Terminating Dehydrated Service instances - Threshold Alarm Settings

Conclusion

We strongly believe BizTalk360 has transformed the way you do your work. It is a very common situation where administrators depend on various tools like BizTalk admin console, SQL queries and other custom build tools to monitor their BizTalk Environment. But with this capability “PowerShell Notification Channel”, administrators can create their own scripts and automate their daily activities. In this blog post, we have explained one such scenario we can accommodate with BizTalk360. This capability is not restricted for some functionalities, the component is generically implemented in the product where our customers can achieve a lot more business scenarios by using this powerful capability.

Do try BizTalk360 one-stop monitoring tool for your BizTalk Server.

Author: Mekala Ramesh

Lead QA & Product Support at BizTalk360 – Having around 8 years of experience in software testing & customer support field with the strong knowledge in SDLC and STLC phases. Specialized in various types of testing methodologies. Passionate tester, who always want to deliver the software product with the best quality to the end customers. Possess strong knowledge, to establish the testing process from scratch. Playing a pivotal role in BizTalk360 is making me deliver the product to every customer in a delicious way. View all posts by Mekala Ramesh

Microsoft Integration Weekly Update: Feb 18, 2019

Microsoft Integration Weekly Update: Feb 18, 2019

Do you feel difficult to keep up to date on all the frequent updates and announcements in the Microsoft Integration platform?

Integration weekly update can be your solution. It’s a weekly update on the topics related to Integration – enterprise integration, robust & scalable messaging capabilities and Citizen Integration capabilities empowered by Microsoft platform to deliver value to the business.

If you want to receive these updates weekly, then don’t forget to Subscribe!

Feedback

Hope this would be helpful. Please feel free to reach out to me with your feedback and questions.

Advertisements

Late Valentine Day Gift: Version 4.0.0 of Microsoft Integration and Azure Stencils Pack (BAPI, AI, Office 365 and much more) for Visio is now available on GitHub

Late Valentine Day Gift: Version 4.0.0 of Microsoft Integration and Azure Stencils Pack (BAPI, AI, Office 365 and much more) for Visio is now available on GitHub

Microsoft Integration, Azure, BAPI, Office 365 and much more Stencils Pack it’s a Visio package that contains fully resizable Visio shapes (symbols/icons) that will help you to visually represent On-premise, Cloud or Hybrid Integration and Enterprise architectures scenarios (BizTalk Server, API Management, Logic Apps, Service Bus, Event Hub…), solutions diagrams and features or systems that use Microsoft Azure and related cloud and on-premises technologies in Visio 2016/2013:

  • BizTalk Server
  • Microsoft Azure
    • Azure App Service (API Apps, Web Apps, Mobile Apps, and Logic Apps)
    • Event Hubs, Event Grid, Service Bus, …
    • API Management, IoT, and Docker
    • Machine Learning, Stream Analytics, Data Factory, Data Pipelines
    • and so on
  • Microsoft Flow
  • PowerApps
  • Power BI
  • PowerShell
  • Infrastructure, IaaS
  • Office 365
  • And many more…
  • … and now non-related Microsoft technologies like:
    • SAP Stencils

Visio: Microsoft Integration and Azure Stencils Pack

What’s new in this version?

  • New shapes: new shapes were added to existing modules like Generic, Azure, AI, Developer, Files or Users. But in particular a new module was born:
    • MIS SAP Stencils contains stencils that will represent some SAP services

Visio: Microsoft Integration and Azure Stencils Pack v4.0.0

You can download Microsoft Integration, Azure, BAPI, Office 365 and much more Stencils Pack for Visio from:
Microsoft Integration Azure Stencils Pack VisioMicrosoft Integration, Azure, BAPI, Office 365 and much more Stencils Pack for Visio (18,6 MB)
GitHub

Or from:
Microsoft Integration Azure Stencils Pack VisioMicrosoft Integration and Azure Stencils Pack for Visio 2016/2013 v3.1.0 (18,6 MB)
Microsoft | TechNet Gallery

Author: Sandro Pereira

Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc. He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community. View all posts by Sandro Pereira

Comparing BizTalk360 to DynaTrace

Comparing BizTalk360 to DynaTrace

Introduction

From time to time we hear that organisations are using DynaTrace for operating and monitoring their BizTalk environments. When we talk to such organisations, they ask us about the benefit of using BizTalk360 over Dynatrace for administering their BizTalk environments. In this blog post we try to give a fair comparison between both products.

BizTalk360 and Dynatrace in short

BizTalk360 is a comprehensive product for operations, monitoring and application performance management (APM) of BizTalk Server environments. Unlike other APM products, BizTalk360 is an agent less, non-intrusive software which requires zero configuration on BizTalk servers. This makes it an easy to use and operations-friendly tool. Customers have a lot of influence in the roadmap and the vendor is very receptive to feedback via http://feedback.biztalk360.com. Furthermore, Support is very responsive. This means the product is always up to date to support the latest releases of BizTalk Server.

BizTalk360 is a market leader in BizTalk Server space, has a large customer base and healthy partner ecosystem. The company understands that supporting BizTalk Server applications for business continuity and efficient operations requires a tool which can go beyond being a traditional application performance management tool. Therefore, the product provides BizTalk specific capabilities such as managing BizTalk Applications, proactive monitoring of throttling conditions, role-based user access policies, audit logs, data monitoring etc. In some instances, BizTalk360 has helped customers to meet SOX compliance.

DynaTrace, on the other hand, is a typical Application Performance Management tool with generic capabilities to monitor operating systems, services, network protocols, system metrics, network infrastructure, services, applications etc. There is no out of the box DynaTrace plugin or agent for BizTalk Server. Either the customer needs to create one of their own or use an open source plugin Fastpack. This plugin must be installed on all the BizTalk Servers and some configuration changes must be made to the BizTalk host instance’s configuration file (btsntsvc.exe.config). With an agent it brings the ability to analyse and monitor the performance counters for BizTalk host instances, databases, orchestrations, pipelines etc. The Fastpack plugin seems to have been updated a couple of years back and might not be up to date with the latest BizTalk releases; this is a typical challenge with using open source plugins from individual contributors.

Typical requirements

In this section we will compare the 2 products, BizTalk360 and DynaTrace, for capabilities in achieving BizTalk Server monitoring requirements.

Getting a proactive alarm to warn before a problem occurs

In the BizTalk environment problems can occur in the following situations:

  • Important artefacts such as receive locations, send ports, orchestrations, host instances etc get disabled or stopped, either due to errors or due to unintentionally performed tasks. This leads to outages and stops the flow of mission critical messages.
  • If the SQL Server jobs are stopped, the size of databases increases over time and impacts the messaging performance.
  • Due to several reasons BizTalk can start throttling messages leading to an impact on performance.
  • If suspended messages are not cleaned up, the size of the message box increases resulting in performance issues.

BizTalk360 has features to monitor state-based artefacts, which ensures that all BizTalk application artefacts such as receive locations and orchestrations are up and running and generate notifications in case of aberrations. State based monitoring also ensures that essential SQL jobs and Windows services are running as expected. Features like suspended queue monitoring and throttling monitoring ensure that the BizTalk Server is running in a healthy state.

DynaTrace has no out-of-the box support for BizTalk Server. The Open Source plug-in does not have features such as state-based monitoring, throttling monitoring and suspended queue monitoring. However, it does have monitoring capabilities based on some BizTalk and SQL performance counters. In conclusion, it lags behind BizTalk360 in proactively warning of potential issues.

Problem-solving actions

In some situations, issues do occur and the operations team needs easy to use tooling to correct the faults. For example, if receive locations get disabled, they will have to be re-enabled. If messages get suspended, they will have to be terminated or resumed. If we have a software which auto-corrects these faults and automates the operational tasks then it is a big bonus as it brings the down time to its minimum levels and ensures operational efficiency.

BizTalk360 has a powerful feature called Auto Healing, which allows the users to define the expected state of an artefact and in case there is an aberration, arestores the expected state without any human intervention. With another feature Operational Automation, the day to day monotonous tasks such as resuming/terminating suspended messages can be automated.

DynaTrace on the other hand has no operational capabilities. Auto-healing and Operational Automation are not features you can expect from a typical Application Performance Management system. Hence, DynaTrace will not be the right tool to be used for BizTalk related problem solving actions.

Access to the BizTalk servers

BizTalk Server has a very limited number of user groups. The BizTalk Administrators group has the highest level of privileges and the BizTalk Application Operators group has highly restricted access. There are no access levels to give intermediate level access to BizTalk applications and artefacts. Many customers have ended up providing their users with the unrestricted access to production boxes. It is typical of organisations to provide all the users the access to production servers for business continuity.

One of the widely used features of BizTalk360 is Granular User Access Control. With this feature, user access can be controlled at a very granular level. Users do not need to RDP to production boxes to support BizTalk applications. BizTalk360’s web portal allows them access to all the features without the need for them to log on to production boxes unless they are doing deployments.

Again, access control is the feature which can be expected from an Application Performance Management tool. Hence, DynaTrace clearly has no capabilities which to help to restrict access to BizTalk Servers.

One tool for operations, monitoring and analytics

If a support professional needs to use multiple tools daily, operational efficiency takes a big hit due to all the context switching he must do between the tools. This is one of the main challenges in supporting BizTalk Server applications. Every operations team needs a single tool to perform operations, monitoring and analytics.

BizTalk360 brings all the capabilities of operations, monitoring and analytics into a single web portal. Apart from this, it allows the users to access BAM data, ESB data, EDI reports from the same web portal. This means users do not have to switch to different tools.

DynaTrace being an Application Performance tool, does have the ability to monitor performance counters and detect some anomalies. However, the user will be required to switch to different tools such as admin console, event log, ESB portal, BAM portal etc. to perform various other tasks. In a true sense, it adds another tool to the list.

Tracking

Depending on the performance needs, organisations enable or disable BizTalk tracking. Some organisations enable only event tracking and disable message body and property tracking. Hence each organisation has different tracking needs. Regardless, it is important to have good control to manage the tracking options at various artefacts.

BizTalk360 has a feature called Tracking Manager, which brings great control to manage tracking options at various artefacts. This helps customers to ensure they are following the tracking standards set at the organisation. BizTalk360 also has a feature called Graphical Flow, which is an analytics feature providing a visualisation of message flow across various artefacts such as receive ports, orchestrations and send ports. Enabling pipeline tracking is a minimum requirement for the feature. Not many of our customers use it due to performance requirements. This is the only feature which is dependent on tracking. Most of the earlier discussed features of BizTalk360 are not dependent on the tracking.

DynaTrace, like BizTalk360, will not be able to provide analytics related to ports and orchestrations if the tracking is not enabled. It surely is not a tracking management tool either.

Summary

Comparing the capabilities of BizTalk360 and Dynatrace against the day-to-day requirements necessary to work with BizTalk Server, BizTalk360 comes out as a clear winner. Dynatrace is a generic APM tool and will not match most of the capabilities of BizTalk360. The features such as State-Based Monitoring, Auto Healing, Operational Automation, Throttling Monitoring etc. are essential to bring the operational efficiency and ensure the health of BizTalk environments and applications.

Why not give BizTalk360 a try! A trial version of BizTalk360 can be requested here.

Author: Lex Hegt

Lex Hegt works in the IT sector for more than 25 years, mainly in roles as developer and administrator. He works with BizTalk since BizTalk Server 2004. Currently he is a Technical Lead at BizTalk360. View all posts by Lex Hegt

Integration Down Under | February 14, 2019 | How we are using Microsoft Integration features and related Azure technologies to improve our processes

Integration Down Under | February 14, 2019 | How we are using Microsoft Integration features and related Azure technologies to improve our processes

Finally, I will be in Australia and New Zealand but unfortunately not in person, maybe next time, but instead remotely. Nevertheless, it will be a pleasure to present in this amazing community for the first time and it’s tomorrow already on a session about How we at DevScope are using Microsoft Integration features and related Azure technologies to improve our processes and by doing so engaging customers to also adopt some of these Microsoft Integration features.

Integration Down Under Sandro Pereira: Microsoft Integration features Azure

Session Name: “How we are using Microsoft Integration features and related Azure technologies to improve our processes

Session Overview: In this session, I will show you real live scenarios on how we at DevScope are using Microsoft Integration features (Logic Apps, API Management, API’s) and related Azure technologies like PowerApps, Flows and Power BI to:

  • First, improve our internal processes like expenses reports, time reports and so on;
  • And, secondly, how the first step helps us out to extend our product and our business by exporting these same approaches and concepts to our clients

This will be a lightweight talk addressing some real scenarios and show them in action.

I invite you all to join us tomorrow morning or in the afternoon depending on where you are from. Grab your “seat” by registering in this session here.

Author: Sandro Pereira

Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc. He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community. View all posts by Sandro Pereira

Interesting support cases 2018

Interesting support cases 2018

The year 2018 passed away just like few months have passed phew…! As a support engineer, we receive different cases on daily basis. Every support case is unique because the problem will be faced by different customers in different environment architecture. But some of the support cases will be interesting by the root cause of the problem and the way of troubleshooting the case.

We have already written a series of blogs about the interesting support tickets we often receive.

In a similar way, this blog post highlights 5 interesting cases we have received and solved in the past year. We are sure this consolidated blog will be useful for our customers if they come across any of them in the near future.

Case 1: Custom port number

A customer tried to upgrade BizTalk360 to the latest version 8.9.3153.0310. The installer ran the upgrade process till the end, however, for some reason, it failed without a failure warning or exception message shown in the installer.

Troubleshooting

During the troubleshooting, the customer informed that while installing the BizTalk360 default URL has been used.

Say for example:

http://localhost/biztalk360

Later they have changed the URL from HTTP (80) to HTTPS (443). Once after changing to the new URL, they have deleted the default port (HTTP: 80) in IIS.

In the log file, we found that the installer is still trying to find the default port.

Solution

Most of the customers will not delete or remove the default port from IIS. During the upgrade process, the installer will look for the registry values for the default port information. If there is any mismatch in the stored port information and available in IIS, the installer will be forcefully stopped, which leads to unsuccessful installation.

As a best practice, we recommend before deleting the default port, the new values (Custom Port number) required to be updated in the registry to experience a seamless installation.

Case 2: Unable to view the BizTalk360 API documentation section

Web APIs are becoming increasingly vital to the organization across all sectors. We exposed our APIs specifically to facilitate the creation of their own scripts to ease the use of BizTalk Server integration space much better. To access the BizTalk360 API documentation you must contact license@biztalk360.com as we are providing this feature as an Add-on pack.

In this case, a customer can access all the features except the API documentation menu, though they have a valid license to access it.  After the investigation, we have observed an error “Access is denied” in Internet Explorer browser.

Troubleshooting

Most of the customers wanted to access the BizTalk360 application in a secure way (HTTPS) based on their Organisation policy. As we progressed on our investigation, we had identified the root cause is because of deleted/removed the default port configuration in the IIS.

We suggested them to add the HTTP port into IIS and check the API. Yes, it helped after adding the HTTP and they were able to access the API documentation.

Solution

The interesting part was the fix to the problem. We are using Swagger for the API documentation. Since they removed the HTTP, Swagger kept on checking the HTTP as per mentioned in the configuration in the schemes. As a workaround, we recommended them to change the schemes upside down, it helped them to access the API documentation. Later we handled the part in the code for the permanent fix.

It took few months to solve this case because of the restricted access at a client side, we would like to appreciate the customer for their patience in this case as it took a long time to identify the root cause.

Case 3: Widgets moving around in the Dashboard

Sometimes we don’t believe our own eyes as the support ticket makes us think beyond the limit. One such interesting support case we have received is about the Analytics section. A customer added few widgets in the dashboard, but the changes were not persisted when they moved them around to different places and removed a few of the widgets.  We suspect it’s a ghost activity, absolutely it’s an old joke, have a smile! 😛

Troubleshooting

As it is difficult to identify the root cause, we went for a web sharing session and found that a few widgets don’t have any data. We presumed that the issue is related to “Event Log data count” widget.

There were 4 Event log data count widgets added with different server names, out of that 2 were with data and 2 were empty.

In the generated widget, we were able to see two problems.

  • Related to permission
  • And with custom sources

Solution

The Event Log widget will collect information from all the servers (if permission is provided). When we check for the permissions in all the enabled servers, we were able to see some server hadn’t had local admin permission. After providing the necessary permission, they were able to remove the event log data count widget from the dashboard.
The second widget is with custom sources, this was identified as an issue at our code and we provided the fix in the related stored procedure to resolve the issue.

Case 4: NT Service monitoring

Monitoring few NT services leads to an exception “no longer exist in the server” in the alert email.

Troubleshooting

We monitor the NT services by using the service name. Every time while logging off the user, the monitored services were removed from the server and when logging in to the machine or once after a reboot, a new NT service was created with the addition of some random number (at prefix for both service name and display name).

This led to the “Orphaned” status as the originally mapped service was not found by the BizTalk360 monitoring service in the actual server

Example: If the service name is CDPUserSvc_159f93a17 once after the logout and login or after a reboot, the name will be changed to CDPUserSvc_29174f

We investigated the usage of the services where those services are structured with user data like contact info, calendars, messages, and other content in Windows 10 and Windows Server 2016.

Solution

The Fix is available in our latest version, where it handles the new services created with random numbers.

Case 5: Cross-domain file monitoring

An issue is that file monitoring is getting Orphaned once after configuring it with an alarm.

Troubleshooting

We started with the basic troubleshooting steps and understood that the configured File path in BizTalk Server is present in a different domain configuration than the domain of the actual BizTalk server. Technically, BizTalk360 will monitor the specific file location without any issues because BizTalk360’s monitoring service account is being used to fetch the configuration information by providing the necessary permissions

Solution

To find the exact root cause of the issue, we have developed a console application (with logs enabled) and provided it to the customer. At last, we have found that it is an issue with authentication for the service account in cross-domain architecture. We have found this as an issue at our end and fixed the same in our latest version.

This case helps me to explore more in cross-domain architecture.

For more details on this case, please refer here.

Conclusion

Above are 5 interesting cases of 2018 which helped me to learn and explore new areas and technically learn new concepts. Wait for another blog post series where I would like to share another 5 more interesting cases.

Author: Sivaramakrishnan Arumugam

Sivaramakrishnan is our Support Engineer with quite a few certifications under his belt. He has been instrumental in handling the customer support area. He believes Travelling makes happy of anyone. View all posts by Sivaramakrishnan Arumugam

Microsoft Integration Weekly Update: Feb 11, 2019

Microsoft Integration Weekly Update: Feb 11, 2019

Do you feel difficult to keep up to date on all the frequent updates and announcements in the Microsoft Integration platform?

Integration weekly update can be your solution. It’s a weekly update on the topics related to Integration – enterprise integration, robust & scalable messaging capabilities and Citizen Integration capabilities empowered by Microsoft platform to deliver value to the business.

If you want to receive these updates weekly, then don’t forget to Subscribe!

Feedback

Hope this would be helpful. Please feel free to reach out to me with your feedback and questions.

Advertisements

How I’m automating my Porto Airbnb host with Microsoft Forms and Microsoft Flow

How I’m automating my Porto Airbnb host with Microsoft Forms and Microsoft Flow

Yes, if you want to visit Porto, my city and a destination that has been fashionable due to the various awards, nominations, and mentions all across the world like:

  • Elected as the most romantic city in the world;
  • Elected the best destination in the world;
  • I think 3 years in a row as best European destination;
  • Many travelers believe the most beautiful McDonald’s in the world is in Porto;
  • One of the most beautiful libraries in the world;
  • and many other crazy nominations…

Personally, I believe that today we living in a fanatical environment of everyone or everything wanting to be the best in the world, the honest reality is that Porto is a beautiful place like many others in the world and I recommend you to visit… and I have just a perfect romantic and traditional place for you!

Casa Sr. José (Mr. Joseph’s house) is my AIRBNB host, as traditional and recently renovated house that is at a walking distance from the Douro River (1 min – 78 m) in the beautiful and typical Porto civil parish of Massarelos. Located in the heart of Porto, next to the Douro River, is a perfect spot that is near to almost everything: Porto´s Unesco World Heritage historical center (10 mins); Foz do Douro (10 min) were Douro river and the Atlantic Ocean meet (beach); and near of 1 most well-known landmarks of Porto: the Clérigos Tower (10 min) and it is available for all you that once again, want to visit Porto.

Casa do Sr- José: The perfect AIRBNB house in Porto

This is my non-technology pet project and I love it but it also gives me some challenges, challenges that I’m trying to solve with technology.

One of these challenges is that in Portugal, as well as in most European countries, all paid accommodation establishments need to record the stay and collect identification details from all foreign citizens, whether they are from the Member States of the European Union or Third Country Nationals, in order to communicate them within three days to the Immigration and Borders Service. At the moment, I have a very rudimentary process in place:

  • A form paper (Ficha de hóspede / Guest card / Carte d’invité) that the guests need to fill once they arrive;
  • I have to go to the house to get the form;
    • The guests don’t need to be there if they allow me to enter the house to grab the form;
    • Otherwise, I need to arrange a date hour with them to grab the form;
  • I then need to access to the SEF (Serviço de Estrangeiros e Fronteiras – Foreign Service and borders) online portal, authenticate with my account and manually fill an online form (SIBA – Sistema de Informação de Boletins de Alojamento – Accommodation Bulletin) with the information of all guests like:
    • Full name
    • Date of birth
    • Nationality
    • Country of residence
    • And so on
  • and submit the accommodation bulletin to SEF.

This is a very time-consuming task and inconvenient for both me and the guests:

  • I need to “annoy” the guests to access “their house for a week” or arrange a date/hour for them to meet with me to give me the form;
  • I then need to understand the hand write and type everything to an only form;
  • And so on

Again all of these are time-consuming tasks.

This week, and I’m being an integration guy, my mind automatically started to think:

  • How can I easily automate these tasks?
  • But it needs to be user-friendly to my guests;
  • And it needs to be easier for my wife to understand, to share with the guest and to control and… she is not a technical person;
    • If I started to speak in terms of Flow, Azure or anything else… she will say no! I don’t know what you are talking about.

In the end, I elaborated a process with two parts:

  • Part 1: the first approach to address and expedite our interaction with our guests;
  • Part 2: my end goal, fully integrated;

In this post, I will address Part 1, the one that is at the moment in-place.

How Microsoft Forms and Microsoft Flow allows me to better interact and automate my Porto Airbnb host processes?

The first thing I decide to do was an Online Form for the guest to fill, instead of using a paper. This could be done with several options online: Google Spreadsheet and Form; Formstack; Cognito Forms and so on. I choose to use Microsoft Forms, to do that you need to

  • Access in your web browser to forms.office.com.
  • Sign in with your Office 365 school credentials, Office 365 work credentials, or Microsoft account (Hotmail, Live, or Outlook.com).
  • Under “My Forms”, click “New Form” to begin creating your form.

Microsoft Form: Create New Form

  • You should enter a name for your form, and you can also enter an optional subtitle and picture.
    • I decide to call my form name: “Ficha de hóspede / Guest card / Carte d’invité”
    • Add a picture and a subtitle with an explanation of why guest need to fill this form

Microsoft Form: Header

Then, you need to start creating your questions. To do that you need:

  • Click “Add Question” to add a new question to the form.

Microsoft Form: add question

  • By default you have the options “Choice”, “Text”, “Rating”, or “Date” questions, but you can always click to see more option under “…” button, and you can select from “Ranking”, “Likert”, or “Net Promoter Score®” questions.

Microsoft Form: add question types

  • In my case I added:
    • Two Date questions:
      • “Data de entrada / Check-in date / Date d’arrivée”
      • “Data de saída / Check-out date / Date de départ”

Microsoft Form: add date questions

    • 8 text questions:
      • “Nome completo /Full name / Nom complet (Person 1)”
      • “Data de Nascimento / Date of Birth / Date de Naissance (Person 1): dd/MM/yyyy”
      • “Nacionalidade / Nacionality / Nationalité (Person 1)”
      • “Local de Nascimento / Birthplace / Lieu de résidence (Person 1)”
      • “Local de Residência / Country of residence / Pays de résidence (Person 1)”
      • “País de residência / Country of residence / Pays de résidence (Person 1)”
      • “Numero de documento / Document number / Numéro de document (Person 1)”
      • “País emissor do documento / Country issuing the document / Pays délivrant le document (Person 1)”

Microsoft Form: add Text questions

    • 1 choose question
      • “Tipo de documento / Document type (Person 1)”

Microsoft Form: add choose question

    • And finally, a true/false question, that is basically another optional question
      • “Têm mais hóspedes? / Do you have more guests? / Avez-vous plus d’invités?”

Microsoft Form: add yes or no question

And here is where the fun begins. Why this last question?

The first reason is that I need to deliver to SEF the information of all the guests and my house can host a maximum of 6 guests. But there is more… Because my house can host a maximum of 6 guests, I have to duplicate these questions 6 times, with the exception of the first two:

  • “Data de entrada / Check-in date / Date d’arrivée”
  • “Data de saída / Check-out date / Date de départ”

Because all of them are from the same booking. That’s why in my final form you will see the same question duplicated with the suffix: Person 2, Person 3… Person 6

Microsoft Form: add guest 2Microsoft Form: add guest 3

But my first form draft didn’t have this last question. I add it becaus, without this question, the guests had to scroll until the bottom of the form to submit it, for example, if I was hosting a couple, they had to fill Person 1 and Person 2 information but them they had to scroll down passing all questions of person 3, 4, 5 and 6 to finally submit the form.

Again, to improve and provide the best user experience to my guest I decide to create this question and apply some logic inside allowing me to create like an array of guest information data – this doesn’t exist in Microsoft Forms by the way.

To accomplish that at the end of this list of questions for each guest I’m adding this last question:

  • “Têm mais hóspedes? / Do you have more guests? / Avez-vous plus d’invités?”

And add a branching logic to my form so that it changes according to the responses to that question. This way, If the guest says:

  • “Sim / Yes” it will show the list of questions for the second guest… and so on.
  • If “Não / No”, it will hide the rest of the questions and “go” directly to the end – “Submit”

Here is how the experience will look like if you say “No” or the default behavior:

Microsoft Form: branching experience

And here is the behavior if you select or change to “Yes”:

Microsoft Form: branching experience

For example, despite all questions in the form are mandatory, if I select or refer that I only have two guests, the form is smart to know that this is an optional pack of questions:

  • If you select that have a new guest you need to fill the questions for that specific guest, otherwise, it allows you to submit the previous questions, without having to fill dummy information for guests 3 to 6 for example.

This improved a lot the Form user experience with the guests.

To accomplish this, you need to:

  • Click the ellipses button (…) on the main menu, and then select Branching.

Microsoft Form: add branching

  • On the Branching options page, there are dropdown menus that appear with each answer under every question. Click the arrow to see a dropdown menu.

Microsoft Form: Branching rules

    • In this case, if I select the “Sim / Yes” option this should “Go” to question 13 that is the beginning of a new guest form
      • Basically, it shows the “second part” of the form
    • If you select “Não / No” it should go to the end of the form
      • Basically, it hides the rest of the form

And of course, I can share the form thru a URL, QR Code, thru an iframe on a webpage and by email.

Microsoft Form: Share

Being the Microsoft Form

Microsoft Form is great to help to improve the interaction with my guests, but it allows me to go to a certain point but not solve all my problems/concerns.

Now that I have retrieved the data from my guests, the question is how I can automate my process my SEF and how can I notify the persons that are managing the house (me, my wife, my brother-in-law, and sister-in-law).

For that, I choose to use a Microsoft Flow that is triggered each time a new form Is submitted and automated these tasks. Unfortunately, do some restrictions I still cannot integrate directly to SEF thru API. I had to request access and I’m still waiting for their approval.

That is the reason why this is PART 1… end goal is to be fully automated without any manual process (PART 2).

So, to minimize the manual work, I decide that is this phase the Flow will go thru each question and notify us by email that a new form was submitted but also give us on the body email all the information submitted for us to be better to copy that information and fill the online SEF form and submit them – basically it would be a copy and past operation, instead of adding to manually type all the data which can lead to many mistakes. This way we are:

  • Minimizing mistakes and misspellings;
  • Minimize the time I need to fill these SEF forms;

To accomplish that you need to:

  • Access the Flow portal: https://flow.microsoft.com/ and sign in with your account
  • In flow.microsoft.com, select “My flows” in the top navigation bar, and then select “+ Create from blank” to create a Flow from scratch.
  • On the “Create a flow from blank”, select “Create from blank” to start the process of creating an empty Flow
  • On the Flow designer, on the Trigger event enter “Microsoft Forms” and select the option “When a new response is submitted” and pick the form you like from the dropdown list

Microsoft Flow: When a new response is submitted

  • Add the next step by clicking the “+New step” button and then choose the “Add an action” option
  • On the “Choose an action” window, enter “Variables” and select the action “Variables – Initialize variable”
    • On the Variable action configuration:
      • On the “Name” property, type “EmailBody”
      • On the “Type” property, specify the type as “String”
      • On the “Value” property leave it blank

Microsoft Flow: Initialize variable

  • Add the next step by clicking the “+New step” button and then choose the “Add an action” option
  • On the “Choose an action” window, enter “Control” and select the action “Apply to each”
    • On the Apply to each action configuration:
      • On the “Select an output from previous step” property, select from the list of tokens the “Key” token “List of response notifications” from the “When a new response is submitted” trigger

Microsoft Flow: for each form response

  • Inside the loop, add the next step by clicking the “+New step” button and then choose the “Add an action” option
  • On the “Choose an action” window, enter “Microsoft Forms” and select the action “Get response details”

Microsoft Flow: Get response details

    • On the action configuration:
      • On the “Form Id” property, pick once again the Form from the dropdown list
      • On the “Response Id” property, type the following expression
@{items('Apply_to_each')['resourceData']['responseId']}

Microsoft Flow: for each form response

  • Add the next step by clicking the “+New step” button and then choose the “Add an action” option
  • On the “Choose an action” window, enter “Variables” and select the action “Variables – Append to string variable”
    • On the Variable action configuration:
      • On the “Name” property, select “EmailBody” variable
      • On the “Type” property leave it blank
Ficha de hóspedes: 

<b>Data de entrada: </b> @{body('Get_response_details')['rfb0478a71c77463eb855db4a6617b596']}
<b>Data de saída: </b> @{body('Get_response_details')['rbad828de65664088a332fbbad18ba4c3']}


Hóspede 1:
<b>Nome completo: </b> @{body('Get_response_details')['r9bde314fe5ea4c74883f46750183bd0f']}

<b>Data de nascimento: </b>@{body('Get_response_details')['rf3e762220a604c578cca006c83d98756']}
<b>Nacionalidade: </b>@{body('Get_response_details')['r8b7428bfbda94eb684c0e43edbff0a87']}

<b>Local de nascimento: </b>@{body('Get_response_details')['r2c926c630529444e86ba53359b6adaf4']}

<b>Local de residência: </b>@{body('Get_response_details')['rddbf3f41f16e402b9dba65f49631c27c']}
<b>País de residência: </b>@{body('Get_response_details')['r52c69618890540e392dca6fd4f8f5fc0']}
<b>N. documento: </b>@{body('Get_response_details')['r9b9ee64cf84d42c89bd7770389ea94d9']}
<b>Tipo documento: </b>@{body('Get_response_details')['r48f5e4ecf66a48328269b656b034ddd1']}

<b>País emissor: </b>@{body('Get_response_details')['raa2bcdec2c6b4b5d8b3d17cd6f946a8a']}

Basically, all the tokens were added from the tokens list

Microsoft Flow: First Guest info extracted

And what we are doing is creating a string, king of a string builder, with the HTML email body with the guest information

Then we need to check if another guest was added and append that information by:

  • And then add a new condition by selecting “Add new action”, enter “Control” and select the action “Condition”
    • On the “Condition” configuration add the following condition:
      • If the first “Têm mais hóspedes? / Do you have more guests? / Avez-vous plus d’invités?” question value is equal to “Sim / Yes”

Microsoft Flow: Second Guest check condition

  • On the “If yes” branch
    • Choose the “Add an action” option and on the “Choose an action” window, enter “Variables” and select the trigger “Variables – Append to string variable”
    • And add the same previous code but this time with the data of the second guest

Microsoft Flow: Second Guest check condition, yes branch

  • On the “If no” branch
    • Leave it without any action

Now we just need to do the same for the other 6 guests:

Microsoft Flow: all guest conditions

Finally, on the end send an email with that data to our host email address, to do that we need to:

  • Add the next step by clicking the “+New step” button and then choose the “Add an action” option
  • On the “Choose an action” window, enter “Mail” and select the action “Send an email notification”
    • On the action configuration:
      • On the “to” property, type your email
      • On the “Subject” property, type “Guest notification”
      • On the “Email Body” property, select from the list of tokens the “Key” token variable “EmailBody”

Microsoft Flow: Email notification

Give a proper name to the flow and save it

The end result

Each time a guest submits an Accommodation Bulletin I will be receiving an email with this format:

Microsoft Flow: Email notification result

That will help my task to manually fulfill the SEF online form…. Until I completely automate this task.

And by the way… if you planning to visit Porto, have a chat with me, stay in our house and I may get you a discount: https://www.airbnb.pt/rooms/18272087?guests=1&adults=1

Author: Sandro Pereira

Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc. He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community. View all posts by Sandro Pereira