Sessions, sessions, and more sessions this has been my frantic start to the year in terms of contributions to the community, four different presentations delivered in January where Logic Apps was the only common factor:
Logic Apps: Development experiences at Azure User Group Portugal with Pedro Almeida (video not yet available)
How to create robust monitor solutions with PowerShell, Azure Functions and Logic Apps at Azure Lowlands (video not yet available)
and Logic Apps: Anywhere, Everywhere at Microsoft Integrate Conference DACH
Today I’m happy to share with you the slides and the video of this last session are now available online for those who want to see them.
About the session
Session name: Logic App: Anywhere, Everywhere.
Abstract: A walk-thru session on how and where we can or should use Logic Apps and start building fantastic business processes. We will be addressing topics like: what tools should you use: Azure Portal, Visual Studio, or Visual Studio Code. What kind of solutions you can create, cloud integration, hybrid integration, or on-premises integration. Along with some best practices and what are the advantages and drawbacks of each approach.
For any reason, you could not be present at this online event, or if you want to review it again, you can now do it here:
It was a pleasure to deliver a presentation on this event, and most important, by doing that, I was able to raise and give away 1000€ to a non-profit organization.
Last January 14, I had the pleasure to be the first guest of 101 Talk Arena chat with Nino Crudele on a talk about integration without any filters, and it was amusing!
Now I’m happy to announce that the record is online and available for all of you to watch on youtube:
I’ve known Nino for many years, he is like a brother to me, and for that reason, I can’t say no to him and his crazy ideas. When he invited (challenged) me to have a 101 talk about Integration, obviously, I said yes!
It is already next Wednesday, January 14, 2021, that I will have an open conversation about Enterprise Integration and Azure:
A conversation about Enterprise Integration like BizTalk Server, Logic App, API Management, Service Bus, Azure Functions…
Is still worth using BizTalk Server?
What about the new Integration stack offered by Microsoft?
Azure RAD vs BizTalk Server, when, where, and why?
What about migration from BizTalk Server to Azure?
And other topics with Nino Crudele, and I will invite all of you to join us it is free. This is a one to one talk without filters, no marketing, nothing planned, and where people can also jump in with any question.
You can see more about the event on Nino Crudele’s website: 101 Talk Arena with Sandro Pereira – What about integration now?
I know, BAM Portal is deprecated in BizTalk Server 2020 but it is still available and until there is no better alternative – you can always use BizTalk360 – it is better than nothing.
Yesterday while I was once again playing with BAM in BizTalk Server 2020 I ended up having the following error message while trying to access to BAM Portal:
HTTP Error 401.2 – Unauthorized
You are not authorized to view this page due to invalid authentication headers.
That was a bit strange since normally Everyone has access to BAM Portal, what normally can happen is that they don’t see any BAM View to explore.
Cause
For me, it was clear that this error was somehow related to the user-authentication methods/protocols installed on IIS. So I went to check the available user-authentication methods on IIS and realized that was missing the most critical method for BAM Portal to work: Windows Authentication.
BAM Portal requires Windows Authentication to work correctly.
This error is not related
Solution
This issue is very simple to fix. Windows Authentication was not available as an option, so we have to install it, and to do that you need to:
Start by running the Server Manager, if it is not already open, on the Windows Start menu, click the Server Manager option.
Under the Manage menu, select Add Roles and Features (or press Add roles and features under the Dashboard panel).
On the Before You Begin screen, click Next.
On the Installation Type screen, select Role-based or feature-based installation and click Next.
On the Server Selection screen, select the appropriate server, leave the default options, and click Next.
On the Server Roles screen, expand the Web Server (IIS) > Web Server > Security options.
Make sure that you select Windows Authentication option
Click Next.
On the Features screen, click Next.
On the Confirmation screen, click Install
Now if you try you will be able to successfuly access to BAM Portal
When migrating your BizTalk Server environment or deploy to a new or different environment, there are many different resources or configurations that you need to take into considerations like:
local queue creations
cloud queue creations
folder creations
and so on.
One of the common resources that we use on our integration solutions is local folders, whether for archiving or routing messages. I do not mean that it is a good practice or not. That you should do it or not, this is not the goal, only that this is common to happen.
In a first analysis, we could think that the quickest and most effective solution would be to copy and paste the folder structure from one environment to another. Still, it may not be the best solution in many cases since it may contain thousands of documents, which are unnecessary to copy/migrate.
This blog post will address how we can easily recreate a folder structure in a different environment/server using PowerShell.
PowerShell script to recreate a local folder structure
With this PowerShell sample, we will be able to recreate an existing local folder structure on a different BizTalk Server environment.
The output of this PowerShell script will be a creation of a different PS script containing the instructions that you can use to recreate all the folder structures in different environments.
When migrating your BizTalk Server environment or deploy to a new or different environment, there are many different resources or configurations that you need to take into considerations like:
local queue creations
cloud queue creations
folder creations
and so on.
In this blog post, I will address how we can easily “export” a list of existing local private Message Queues (MSMQ) and recreate them, and set proper permissions in a different environment/server.
PowerShell script to extract a list of all private Message Queues (MSMQ) names
With this PowerShell sample, we will be able to extract a list of all private Message Queues (MSMQ) names to a CSV file to be used on a different script to deployed these resources on a different BizTalk Server environment.
set or update the URI (address) or part of the URI on a list of BizTalk Server Receive Locations deployed in your BizTalk Server environment.
PowerShell script to create private Message Queues (MSMQ)
With this PowerShell sample, we will be able to create local Private Message Queues (MSMQ) and set proper permissions on a different BizTalk Server environment.
Starting with BizTalk Server 2020 and newer, administrators can configure BizTalk Server to generate an audit trail for management operation on application artifacts, such as to send ports, receive ports, receive locations, orchestrations, and resources. Auditing of suspend/resume/terminate operations on service instances is also possible.
This feature may not be perfect yet, and it has space to grow in the future, but it is a good start, and I hope the BizTalk product team will improve these capabilities in the future.
How to configure the Audit Logs?
The auditing feature isn’t enabled by default. To enable auditing you need to:
Open BizTalk Server Administration console, right-click the BizTalk Group, and select Settings
On the Group panel, under the Tracking and Reporting section:
Enable the Audit management operations option.
And set a Maximum number of audit entries property according to your needs. By default, BizTalk stores 10000 most recent entries.
Select OK to save your changes.
Now every time you make the following operations:
On Ports (Receive or Send): Create, Update and Delete
On Service Instances: Suspend, Resume and Terminate
On Application resource: Add, Update and Remove
On Bindings: Import bindings files
One or more audit entries are logged. All these correlation entries have the same BatchId. The audit log has the following information:
Property
Description
ID
Id of type Guid, unique per entry. Example: 3bf539a3-4b59-4148-b589-d22c83f32d25
BatchId
Same for all audited operations performed in a single SQL transaction. Insightful in correlating user operations with lower level details Example: a4ffa64b-8064-4e54-9d99-2e7d60797100
UserPrincipal
User who performed the operation. Example: BTS2020LAB01Administrator
Machine
Machine name from which operation was performed. Example: BTS2020LAB01
ArtifactId
Unique id of the artifact. Example: 1
ParentArtifactId
If an artifact is child of another artifact, then this field will have artifact id of the parent. Example: 1
ArtifactType
Type of artifact on which operation was performed. Example: SendPort, ReceiveLocation, etc.
ArtifactName
User configured name of the artifact. Example: BatchControlMessageRecvLoc
OperationName
Action performed on the artifact. Example: Ports: Create/Update/DeleteService Instances: Suspend/Resume/TerminateApplication resources: Add/Update/RemoveBinding file: Import
Payload
Contains information about what is changed in JSON structure. Example: { “Disabled”: 0 }
CreatedDate
Timestamp when the operation was performed Example: 2020-11-27T09:21:48.55Z
How to view the audit logs?
The easier way to access the BizTalk Server Audit logs is to access the Audit Logs REST service using your browser by using the following URL:
GET http://localhost/BizTalkOperationalDataService/AuditLogs
This will provide a JSON response containing all the latest logs.
For a better user experience, you could use a tool like postman to interact with this service:
You can also define a date range to retrieve the desire logs. For that you should use the following query parameters:
fromDate: beginning date, for example, 2020-12-01T01:00:00
toDate: end date, for example, 2020-12-03T01:00:00
GET request example:
GET http://localhost/BizTalkOperationalDataService/AuditLogs?fromDate=2020-12-01T01:00:00&toDate=2020-12-03T01:00:00
Supported date formats are: yyyy-MM-dd or yyyy-MM-ddThh:mm:ss.
Azure Logic Apps adapter is used by BizTalk Server to communicate with the Azure Logic Apps. This could also be possible using the HTTP adapter, but the Logic App adapter provides a better and straightforwardly experience.
You may be thinking that I’m out of my mind. Logic App Adapter was already available in BizTalk Server 2016. Indeed you are right. However, the Logic App adapter was an optional feature and a separate download in BizTalk Server 2016. You can know more about how to install it on this white paper I wrote some time ago: Step-by-Step Logic App Adapter Installation Guide. But it is now installed with the BizTalk Server 2020 default installation process, so you do not need to install it and configure it manually. Everything or almost everything will be already configured for you automatically.
The Logic App Adapter is one of the adapters that support two-way communications:
The Logic App Receive Adapter that is responsible for receiving messages from Logic Apps and delivering them to BizTalk is, in fact, a WCF Service that runs inside Internet Information Services (IIS).
Install and Configure an On-premises Data Gateway;
Click the Test Settings button to verify the application pool identity and pass the authentication and authorization tests.
Click the OK button to save the changes.
Open BizTalk Server Administration, expand BizTalk Server Administration à BizTalk Group > Applications, and expand our application.
Right-select Receive Ports, select New, and select One-way Receive Port.
In the Receive Port properties window, enter the following configurations:
Name: Enter a name for the receive port.
Select Receive Locations, and select New
In the Receive Location properties window, enter the following configurations:
Enter a Name for the receive location
For the Type, select LogicApp from the list, and select the Configure button.
In the General tab, configure the endpoint address for your logic app:
Address: Required. Enter the BizTalk ReceiveService IIS application URL. In our case: /LogicAppTestService/Service1.svc
Public Address: Required. This is the public full URL of the service. In our case: http:///LogicAppTestService/Service1.svc
In the Binding tab, you can configure any timeout and encoding-related properties of the underlying WCF-WebHttp binding. These properties are helpful when dealing with large messages.
In the Security tab, configure any security properties.
Click the OK button, to save your configurations.
For the Receive handler, select PassThruReceive from the list and select OK to save your configurations.
And finally, create our Logic App to send messages to BizTalk Server
Sign in to the Azure portal. Create a blank logic app.
After Logic App Designer opens, in the search box, enter Request as your filter, and from the triggers list, select the When a HTTP request is received trigger
Select + New step
In the search box, enter BizTalk Server as your filter
From the connector list, select the Send message action form the BizTalk Server connector
Once the action is added to your logic app, you need to setup the connections, perform the following actions:
Select the option Connect via on-premises data gateway and on the gateway properties, select the Subscription and the desired Connection Gateway.
On the Connection Name property, provide a proper name for your connector
On the BizTalk Server URL property, provide a public UTL for the Management Application on the BizTalk Server IIS
On the Authentication Type property, set Windows and provide a proper Username and Password to access the above service (BizTalkManagementService)
Select Create
Once you create the connector, the Send message action will appear on the Logic App designer
From the Receive Location list, select the receive location we just create above
On the Input Message, specify the Body token of the When a HTTP request is received trigger
The Logic App Send Adapter is responsible for sending messages from BizTalk Server to Logic Apps.
We need to first create a Logic App in our Azure Subscription has as a trigger the When a HTTP request is received present in the Request built-in Connector.
Note: The Logic App adapter doesn’t support the new Logic App (preview) that are hosted on Azure Function runtime.
We need to configure TLS 1.2 as the default security protocol on BizTalk Server.
And finally, create a BizTalk Server Application to send message to Logic App
Right-select Send Ports, select New, and select Static One-way Send Port
In the Send Port properties, enter the following:
Name for the send port. For example, enter POC_SEND_MSG_LOGIC_APP.
For the Type, select LogicApp from the list, and select the Configure button
In the General tab, configure the Callback URI of your logic app trigger by selecting Configure…
On the Logic App Details Page, select Sign-in to Azure and authenticate with an Azure account
After you authenticate, you can be able to access your Azure and select the Subscription, the Resource Group that contains the Logic Apps, and finally the Logic App and the Trigger
The Trigger will be always manual
Select OK to save your configurations.
In the Messages tab, set the content-type header as:
Content-Type:application/json
Setting up the BizTalk Server 2020 Logic App adapter
I told above that almost everything will be already configured. However, there is a critical bug in the default installation of the Logic App adapter that will affect the process of receiving messages from Logic Apps using the BizTalk Server Connector.
The Logic App Receive handler, or what we normally call the Logic App Receive adapter is by default configured to use the default In-Process Host, normally the BizTalkServerApplication, in this case as you saw in the picture bellow BizTalkServerReceiveHost.
If you leave this configuration, you will end up having errors when trying to activate a Receive Location that uses the Logic App adapter:
The receive location “Receive Location name” with URL “/iis-application-name/Service1.svc” is shutting down. Details
This happens assigned because the Receive handler is associated with the In-process Host and it should be bound to the Isolated Host.
To fix this bug we need to:
Remove the adapter from all assigned send ports and receive locations in my applications
Therefore, is important to do this immediately after the installation and configuration of your BizTalk Server environment. Otherwise, it will affect your existing application that uses the Logic App adapter to send messages to Logic Apps.
Until now, I usually have used the Logic App adapter to send messages to Azure Logic Apps and extend the BizTalk Server capabilities with the Azure Services. Yesterday, once I was trying the inverse capabilities, i.e., receiving a message from Logic App into BizTalk Server using the Logic App Adapter and, of course, the BizTalk Server Connector available on Logic App. I was surprised with the following error while I was trying to access the exposed service to receive messages from Logic App:
Receive location for address “/LogicAppTestServoce/Service1.svc” not found. (The BizTalk receive location may be disabled.)
This is a common error. It means that the Receive Location doesn’t exist or it is disabled. So I went to the BizTalk Server Administration Console and Enabled the Receive Location, but it automatically disabled again.
Once I check the Event Viewer for errors I found the 3 following errors:
The Messaging Engine encountered an error when creating the receive adapter “LogicApp”. The Assembly is: “Microsoft.BizTalk.Adapter.LogicApp.Runtime.LogicAppReceiver, Microsoft.BizTalk.Adapter.LogicApp.Runtime”. The error occurred because the component does not implement the mandatory interface “IBTTransportControl”.
The Messaging Engine failed to add a receive location “POC_LOGICAPP_TO_BIZTALK_LA” with URL “/LogicAppTestService/Service1.svc” to the adapter “LogicApp”. Reason: “80070057”.
The receive location “POC_LOGICAPP_TO_BIZTALK_LA” with URL “/LogicAppTestService/Service1.svc” is shutting down. Details:”The Messaging Engine failed while notifying an adapter of its configuration. “.
Cause
There is a critical bug in the default installation of the Logic App adapter that will affect the process of receiving messages from Logic Apps using the BizTalk Server Connector.
The Logic App Receive handler, or what we usually call the Logic App Receive adapter, is, by default, configured to use the default In-Process Host, normally the BizTalkServerApplication, in this case, as you saw in the picture bellow BizTalkServerReceiveHost.
Like what happens with the HTTP adapter, the Logic App Adapter is one of the adapters that support two-way communications. Still, unlike other adapters, this adapter has two characteristics that define it:
The Logic App Receive Adapter that is responsible for delivering messages to BizTalk is, in fact, a WCF Service that runs inside Internet Information Services (IIS).
And for that reason, it must be configured in IIS – it is not there out-of-the-box.
This means that when we create and configure a receive location that uses the Logic App adapter inside the BizTalk Server Administration Console, this receive location uses an application within IIS.
So, if you leave this Logic App Adapter default configuration, you will end up having the above errors when trying to activate a Receive Location. This happens assigned because the Receive handler is associated with the In-process Host and it should be bound to the Isolated Host.
Solution
To fix this bug we need to:
Remove the adapter from all assigned send ports and receive locations in my applications
Therefore, it is essential to do this immediately after the installation and configuration of your BizTalk Server environment. Otherwise, it will affect your existing application that uses the Logic App adapter to send messages to Logic Apps.
Last weekend I delivered a session about BizTalk Server 2020 in action on Global Integration Bootcamp 2020 Madrid, where I spoke about the new features available in the most recent version of the product.
Today I will start this series of blog posts about what’s new in BizTalk Server 2020 by beginning to speak about one of the most expected and log time requested features: XSLT 3.0 Support.
XSLT 3.0 represents a significant upgrade of the XSLT 1.0 (and even XSLT 2.0) standards to become a general-purpose transformation language for the most common data storage and messaging formats.
There are a lot of advantages to using XSLT 3.0 compare with the XSLT 1.0 or, in fact, XSLT 2.0, and the goal here is not to address every single of them, but we can highlight the following one:
Extended Function Set, Sequences, Arrays, and Maps: The initial function set for XSLT1.0 was the same as XPath 1.0 functions and was very limited. Minimal math support, no regular expression support, minimal string manipulation capabilities, no support for set (sequence) operations, no support for dates.
Grouping:
xsl:for-each-group: a set of items can be selected and arranged into groups based on specific criteria (for example, common values); then each group can be processed in turn
special XPath functions within for-each-group: current-grouping-key(), currentgroup()
Variants of <xsl:for-each-group>:
group-adjacent: adjacent items are allocated to the same group if they have common values for the grouping key
group-starting-with: whenever an item matches the pattern, a new group is started with this item
group-by: whenever an item matches the pattern, a new group is started after this item
Sorting:
xsl:sort: can be used inside of <xsl:for-each>, <xsl:for-each-group>, <xsl:applytemplates> and <xsl:perform-sort>. It defines the order in which the data is processed by the instruction.
Several subsequent sort keys can be defined
BizTalk’s default XSL transform engine implementation is based on .Net Framework XSLT Transformations. This support is limited to XSLT 1.0.
Starting with BizTalk Server 2020, users can choose Saxon:registered: 9 (Saxon 9 HE) as the XSLT transform engine. But most importantly, it will be possible to plug-in your own custom XSLT transform engine.
YYou can implement a custom XSLT transform engine by defining XSLT transform implementation derived from abstract class Microsoft.XLANGs.BaseTypes.ITransform2 in assembly Microsoft.XLANGs.BaseTypes.dll.
This makes it possible for BizTalk server maps to support newer versions of XSLT. Using Saxon 9 HE, we can readily use XSLT3.0.
Use XSLT Transform Engine property to specify the XSLT transform engine you wish to use:
This was one of the most expected features, but be careful, this can be a poisoned gift because Saxon 9 HE doesn’t support embedded scripting. As a result, functoids shipped as part of BizTalk may not function well. And for many cases, this can be a showstopper.
The advantages of using XSLT 3.0 with Saxon 9 HE:
Development: The XSLT file can be developed separately and hosted in a BizTalk map. It will simplify specific scenarios, especially scenarios that grouping and sorting are required. As we mentioned above, XSLT 3.0 provides an extended set of functions, which was very limited within XSLT 1.0.
Disadvantages when using XSLT 3.0 with Saxon 9 HD:
Level of effort: Not quite as intuitive, Functoids are easier to read visually on the map grid and therefore Requires “geeky” coding skills.
Overview: You lose the visual map representation. You can conjugate Saxon 9 HE with the BizTalk Mapper, but it is harder to do it.
BizTalk Mapper Editor limitations: except for a few built-in Functoids like Scripting Functoid (not all script types will work properly) or advance Functoids, BizTalk Functoids will not work while using the Saxon 9 HE XSLT Transformation Engine.
BizTalk Development Tool Integration: it is a brand new feature, and it seems that doesn’t have, at least for now, the same level of integration capabilities with the BizTalk Server development tools that we are usual to have and use if we compared with the default transformation engine
