Do you feel difficult to keep up to date on all the frequent updates and announcements in the Microsoft Integration platform and Azure iPaaS?
Integration weekly updates can be your solution. It’s a weekly update on the topics related to Integration – enterprise integration, robust & scalable messaging capabilities and Citizen Integration capabilities empowered by Microsoft platform to deliver value to the business.
We usually see two implementation cases when dealing with unwanted or obsolete messages:
They get suspended on BizTalk Server – not that common, at least in the long run. We may see this earlier when these types of messages are encountered.
or a common way to solve these situations is to create a send port to filter these types of messages and send them to a support folder in BizTalk Server hard drive or a shared location.
Of course, this last option should be considered as a backup plan that can work as intended if you have good control and clean best practices like creating a scheduling task that cleans these messages from the hard drive from time to time. Otherwise, your hard drive becomes full at some point, and more critical issues will arise.
BizTalk Server Void Adapter
The BizTalk Server Void adapter is a simple way to easily discard all unwanted messages from your BizTalk Server environments instead of getting suspended or sent to a hard drive support folder.
This custom send adapter for BizTalk Server discards outgoing messages – sends messages to nowhere – into the void! Without the need for external jobs (like scheduling tasks) or added cleaning concerns.
How to install it?
If you want to install the BizTalk Server FILE-RADITZ Adapter, you need to:
Place the folder BizTalkVoidAdapter on any location on your BizTalk Server box. I will recommend to put it in:
C:Program Files (x86)
Note: if you put in any other path you need to modify the VoidAdapter.reg file
Access the folder and double-click on the VoidAdapter.reg file
This will register the adapter to run under 32-bit host instances. To make it available also for 64-bits, you need to:
Click Start
Type %windir%SysWoW64cmd.exe in the search box and press enter
Run the same adapter registry (.reg) file from this command prompt
Now you just need to add this adapter to the BizTalk Server Administration Console:
Open BizTalk Administration Console by pressing the Windows key to switch to the Start menu, type BizTalk Server Administration or BizTalk, click the BizTalk Server Administration option from the Search window.
In the console left tree, expand BizTalk Server Administration > BizTalk Group > Platform Settings and then Adapters.
In the Adapter Properties
In the Name box, type a descriptive name for this adapter.
Void
In the Adapter combo box, select the adapter from the drop-down that you want to add.
Void
In the Description box, type a description for the adapter (this is optional).
The BizTalk Server Void adapter is a simple way to easily discard all unwanted messages from your BizTalk Server environments instead of getting suspended or sent to a hard drive support folder.
Click OK to complete the process of adding the adapter.
Note: If you want to use the Event Viewer to track/log the discarded messages you should first create the Log Name and Log Source. This way the tracking data will not be registered in the Application Log but instead in a custom log. To do that you can use the PowerShell provided on the runtime folder: mngt-Create-Event-Source.ps1.
Where can you use it?
This version available on GitHub is currently compiled on .NET 4.6 and optimized for BizTalk Server 2020. Nevertheless, you can take this code and compile it in other versions of .NET and BizTalk Server. It will be 100% compatible.
You may already know my BizTalk Pipeline Components Extensions Utility Pack project available on GitHub for those who follow me. The project is a set of custom pipeline components (libraries) with several custom pipeline components that can be used in received and sent pipelines, which will extend BizTalk’s out-of-the-box pipeline capabilities.
This month my team and I update this project with another new component: ODBC File Decoder Pipeline Component.
ODBC File Decoder Pipeline Component
ODBC File Decoder Pipeline Component is, as the name mentioned, a decode component that you can use in a receive pipeline to process DBF or Excel files. Still, it can be possible to process other ODBC types (maybe requiring minor adjustments). The component uses basic ADO.NET to parse the incoming DBF or Excel files into an XML document.
If consuming DBF files is not a typical scenario, we can’t say the same for Excel files. Yet, we often find these requirements, and there isn’t any out-of-the-box way to process these files.
Honestly, I don’t know the original creator of this custom component. I came across this old project that I found interesting while organizing my hard drives. However, when I tested it in BizTalk Server 2020, it wasn’t working correctly, so my team and I improved and organized the structure of the code of this component to work as expected.
How does this component work?
If we take has an example and Excel File (.xls) that has a table with:
FirstName
LastName
Address
Position
We can use the ODBC File Decoder Pipeline Component to process these documents. First, we need to create a custom pipeline component and add this component to the decode stage. Once we publish this pipeline, we can configure it as follows to be able to process these types of Excel documents:
ConnectionString: ODBC Connection String
For Excel documents: Provider=Microsoft.Jet.OLEDB.4.0;Extended Properties=Excel 8.0;
For DBF: Provider=Microsoft.Jet.OLEDB.4.0;Extended Properties=dBASE IV;
DataNodeName: Rows node name for the generated XML message
For example: Line
Filter: Filter for Select Statement
Leave it empty
NameSpace: Namespace for the generated XML message
For example: http://ODBCTest.com
RootNodeName: Root node name for the generated XML message
For example: TesteXMLResult
SqlStatement: Select Statement to Read ODBC Files
For example: SELECT * FROM [Sheet1$]
TempDropFolderLocation: Support temp folder for processing the ODBC Files
For example: c:Tempodbcfiles
TypeToProcess: Type of file being Processed
0 to process Excel
1 to process DBF
The outcome result will be an Excel similar to this:
<?xml version="1.0" encoding="utf-8"?><ns0:TesteXMLResult xmlns:ns0="http://ODBCTest.com">
<Line>
<FirstName>Fred</FirstName>
<LastName>Black</LastName>
<Address>187 Main Street</Address>
<Position>Sales Lead</Position>
</Line>
<Line>
<FirstName>John</FirstName>
<LastName>Smith</LastName>
<Address>182 Front Street</Address>
<Position>Marketing</Position>
</Line>
<Line>
<FirstName>Sally</FirstName>
<LastName>White</LastName>
<Address>183 Main Street</Address>
<Position>Marketing</Position>
</Line>
</ns0:TesteXMLResult>
Does it work with Xlsx files?
Honestly, I didn’t try it yet. I didn’t have that requirement, and I only remember this scenario now that I’m writing this post, but it should be able to process it. The only thing I know is that we need to use a different connection string, something similar to this:
Administrators of Business Activity Monitoring (BAM) definitions use the out-of-the-box BAM Management utility tool (BM.exe) to manage and maintain all aspects of the BAM infrastructure.
You can use the BAM utility to perform the following tasks:
Consume BAM definition and BAM configuration XML as input. The BAM definition XML or XLS files define the data to track and aggregate and the business end user’s view on the tracking data. The BAM configuration XML mandates where to deploy the infrastructure, such as the server name, database name, and other database settings.
Deploy the run-time infrastructure on the server, including the BAM Primary Import database, BAM Star Schema database, BAM Analysis database, and corresponding Data Transformation Services (DTS) packages. The following items are created during this step:
BAM Primary Import database
BAM Star Schema database
BAM Archive database
BAM Analysis database
You can learn more about this command-line tool in one of my previous posts: BAM Management Utility Commands – bm.exe. Nevertheless, despite being a powerful tool that works well, it is still an old command-line tool.
BizTalk Business Activity Monitoring Management Utility Tool is a Windows Application tool that works on top of the BM.exe to modernize and simplify its use. It is the equivalent of the BAM Management utility (BM.exe) tool with a GUI.
In this new version we:
Tried to modernize the layout of the tool by adding a picture and some other minor adjustments.
Fix some minor issues getting the correct version of the product – we removed the drop-down box, and now we are getting this info directly from the environment itself.
Tested in BizTalk Server 2020.
Credits
Rikard Alard, was the original creator of this tool.
Diogo Formosinho | Linkedin | Member of my team, and that help me migrate this tool and add a more modern look.
Download
THIS TOOL IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND.
You can download the BizTalk Business Activity Monitoring Management Utility Tool from GitHub here:
The full 7.3 version is complete with the release of v.7.3.1. This was a massive work of adding new shapes of new services that appear on Azure and changing the existing one with the new version of the shapes. Work is done and I hope you enjoy it!
What’s new in this version?
This is the list of changes and additions present in this release:
New shapes on MIS Azure Stencils, MIS Azure Additional or Support Stencils, MIS Developer Stencils, and MIS Security and Governance packages: add a considerable amount of new shapes of new services that appear on Azure has both changing the existing one with their new layout.
Move old versions of the shape layout to MIS Azure Old Versions package.
New shapes on MIS AI and Machine Learning Stencils: several new shapes add it to this package with several Cognitive Services.
New shapes on Microsoft Integration Stencils: some new shapes add it to this package describing Schemas, Maps, Aggrements, Partners, Assemblies and so on.
Lock the aspect ratio of the new stencil icons: This was a requested made that can be very handly to protects against accidental resizing with another shape aspect.
SVG files: new SVG files added.
Microsoft Integration, Azure, Power Platform, Office 365 and much more Stencils Pack
Microsoft Integration, Azure, Power Platform, Office 365 and much more Stencils Pack it’s a Visio package that contains fully resizable Visio shapes (symbols/icons) that will help you to visually represent On-premise, Cloud or Hybrid Integration and Enterprise architectures scenarios (BizTalk Server, API Management, Logic Apps, Service Bus, Event Hub…), solutions diagrams and features or systems that use Microsoft Azure and related cloud and on-premises technologies in Visio 2016/2013:
BizTalk Server
Microsoft Azure
Integration
Integration Service Environments (ISE)
Logic Apps and Azure App Service in general (API Apps, Web Apps, and Mobile Apps)
Azure API Management
Messaging: Event Hubs, Event Grid, Service Bus, …
Azure IoT and Docker
AI, Machine Learning, Stream Analytics, Data Factory, Data Pipelines
SQL Server, DocumentDB, CosmosDB, MySQL, …
and so on
Microsoft Power Platform
Microsoft Flow
PowerApps
Power BI
Office365, SharePoint,…
DevOps and PowerShell
Security and Governance
And much more…
… and now non-related Microsoft technologies like:
SAP Stencils
The Microsoft Integration Stencils Pack is composed of 28 files:
Microsoft Integration Stencils
MIS Additional or Support Stencils
MIS AI and Machine Learning Stencils
MIS Apps and Systems Logo Stencils
MIS Azure Additional or Support Stencils
MIS Azure Black and Gray
MIS Azure Old Versions
MIS Azure Stencils
MIS Black and Cyan
MIS Buildings Stencils
MIS Databases and Analytics Stencils
MIS Deprecated Stencils
MIS Developer Stencils
MIS Devices Stencils
MIS Files and Message Types Stencils
MIS Generic Stencils
MIS Infrastructure and Networking Stencils
MIS Integration Fun
MIS Integration Patterns Stencils
MIS IoT Stencils
MIS Office, Office 365 and Dynamics 365
MIS Power BI Stencils
MIS Power Platform Stencils
MIS SAP Stencils
MIS Security and Governance
MIS Servers (Hexagonal) Stencils
MIS Users and Roles Stencils
MIS API Connectors
Organisational Stencils
That you can use and resize without losing quality, in particular, the new shapes.
Download
You can download Microsoft Integration, Azure, BAPI, Office 365 and much more Stencils Pack for Visio from GitHub Here:
Do you feel difficult to keep up to date on all the frequent updates and announcements in the Microsoft Integration platform and Azure iPaaS?
Integration weekly updates can be your solution. It’s a weekly update on the topics related to Integration – enterprise integration, robust & scalable messaging capabilities and Citizen Integration capabilities empowered by Microsoft platform to deliver value to the business.
The full 7.3.0 version is not yet complete, but I decided to release it in small pieces instead of taking a long time to make all the planned changes and being the Azure part that will require the most work.
What’s new in this version?
This is the list of changes and additions present in this release:
New shapes on MIS: Office, Office 365 and Dynamics 365 package: add some new Dynamic 365 shapes like Dataverse, SCM Warehousing, Project Timesheet, Return To School orReturn To Work, and several new Office/Office 365 shapes.
Remove API Connectors shapes from MIS: Power Platform package: there goal was to simplify this package and migrate that stencils to a dedicated package, since there are more then 600 connectors. Also these connectores are common to Power Automate, Power Apps and Logic Apps.
Create a new package MIS: API Connectors: This package will provide stencils to all connectors currently provided for Microsoft Power Automate, Microsoft Power Apps, and Azure Logic Apps.
Lock the aspect ratio of the new stencil icons: This was a requested made that can be very handly to protects against accidental resizing with another shape aspect.
SVG files: new SVG files added.
Microsoft Integration, Azure, Power Platform, Office 365 and much more Stencils Pack
Microsoft Integration, Azure, Power Platform, Office 365 and much more Stencils Pack it’s a Visio package that contains fully resizable Visio shapes (symbols/icons) that will help you to visually represent On-premise, Cloud or Hybrid Integration and Enterprise architectures scenarios (BizTalk Server, API Management, Logic Apps, Service Bus, Event Hub…), solutions diagrams and features or systems that use Microsoft Azure and related cloud and on-premises technologies in Visio 2016/2013:
BizTalk Server
Microsoft Azure
Integration
Integration Service Environments (ISE)
Logic Apps and Azure App Service in general (API Apps, Web Apps, and Mobile Apps)
Azure API Management
Messaging: Event Hubs, Event Grid, Service Bus, …
Azure IoT and Docker
AI, Machine Learning, Stream Analytics, Data Factory, Data Pipelines
SQL Server, DocumentDB, CosmosDB, MySQL, …
and so on
Microsoft Power Platform
Microsoft Flow
PowerApps
Power BI
Office365, SharePoint,…
DevOps and PowerShell
Security and Governance
And much more…
… and now non-related Microsoft technologies like:
SAP Stencils
The Microsoft Integration Stencils Pack is composed of 27 files:
Microsoft Integration Stencils
MIS Additional or Support Stencils
MIS AI and Machine Learning Stencils
MIS Apps and Systems Logo Stencils
MIS Azure Additional or Support Stencils
MIS Azure Black and Gray
MIS Azure Old Versions
MIS Azure Stencils
MIS Black and Cyan
MIS Buildings Stencils
MIS Databases and Analytics Stencils
MIS Deprecated Stencils
MIS Developer Stencils
MIS Devices Stencils
MIS Files and Message Types Stencils
MIS Generic Stencils
MIS Infrastructure and Networking Stencils
MIS Integration Fun
MIS Integration Patterns Stencils
MIS IoT Stencils
MIS Office, Office 365 and Dynamics 365
MIS Power BI Stencils
MIS Power Platform Stencils
MIS SAP Stencils
MIS Security and Governance
MIS Servers (Hexagonal) Stencils
MIS Users and Roles Stencils
Organisational Stencils
That you can use and resize without losing quality, in particular, the new shapes.
Download
You can download Microsoft Integration, Azure, BAPI, Office 365 and much more Stencils Pack for Visio from GitHub Here:
This post was an old post that I published on BizTalkAdminsBlogging.com that is no longer available, so I decided to republish it here on my blog.
In one of my previous posts, I demonstrated how you could change the startup type for BizTalk Server services automatically using PowerShell. This time I will demonstrate how you can create a script that allows us to automatically restart all your BizTalk services:
No matter what you do, issues will happen at some point, and we need to deal with them. For example, on BizTalk Server, messages can get suspended for many reasons. Sometimes, we can resume these suspended messages. But, in other situations, we can’t, or no water what we do, they cannot be processed because they contain some errors that need to be fixed.
Regardless of the scenario or the reasons for this to happen, there are situations that we endup having several messages suspended on BizTalk Server. Using the out-of-the-box BizTalk Server Administration tool, we are only allowed to save one message at a time, which means that we need to go one by one and, after several clicks, save the message on the hard drive. This is a huge time-consuming operation.
To expedite these operations, we can use PowerShell. It’s true that we will need to spend a little time creating these PowerShell scripts, but the gain that we get is massive, and once made, we can use it anytime.
With this script, we can retrieve the body/payload of all suspended BizTalk Server messages and save them to a specific folder – no filters required.
The result will be a list of messages created on our hard drive:
Download
THIS POWERSHELL IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND.
You can download Save/archive all BizTalk Server suspended messages with PowerShell from GitHub here:
As part of the Intergalactic Automation Summit 2022 online event organized by the Power Community that is taking place between 4-6th February 2022:
4th Feb- Power Automate Bootcamp
5th Feb- Azure Integration Bootcamp
6th Feb- Power Platform ALM DevOps
All of these events are free! And you can register here.
I choose to submit a session to the Azure Integration Bootcamp, and I’m honored to be accepted as a guest speaker on a session about Logic Apps: A new set of Best practices, tips and tricks. My session will take place at 11:00 am according to GMT/UTC.
Logic Apps: A new set of Best practices, tips and tricks
As I mentioned before, my session will be all about best practices and small tips and tricks that we can apply to our Logic Apps flows. For those reasons, I would like to invite you to join me at the Azure Integration Bootcamp virtual event on Saturday, February 5, 2022.
Session name: Logic Apps: A new set of Best practices, tips and tricks
Abstract: A brand new set of tips and tricks and best practices that you should know for being more productive and building more reliable and effective Logic Apps. This is not an introduction session anymore. In this session, we will go through a list of 10 new best practices, tips, and tricks addressing advanced topics like deployment, dynamic connectors configurations, etc.
Join us and reserve your presence at the Azure Integration Bootcamp virtual event on Saturday, February 5, 2022, it is free!