Surprisingly enough, or not, legacy systems can still be found across many organizations. IBM MQ is a family of message-oriented middleware products that IBM launched in December 1993. It was called MQSeries initially and was renamed WebSphere MQ in 2002 to join the suite of WebSphere products. In April 2014, it was renamed to IBM MQ.
We can mainly see the use and the need for integration with these products in banks systems. BizTalk Server provides two ways to communicate with IBM MQ:
By using the MQSeries built-in adapter that will act as a bridge between Microsoft BizTalk Server and IBM MQSeries servers:
The MQSeries adapter is available with the BizTalk Server installation. It supports transactional and non-transactional messaging by setting the Transaction Supported property in the MQSeries adapter transport properties
The MQSeries adapter uses the MQSAgent2 COM+ application to put/get messages into the IBM WebSphere Queue Manager
The MQSeries adapter requires IBM’s MQSeries Server for Windows to reside on the client system to communicate with an IBM WebSphere Queue Manager running on Windows or non-Windows. MQSeries Server for Windows is not a Microsoft product and must be obtained from your IBM WebSphere MQ program
By using the Host Integration Server (HIS) that provides a WCF Channel for MQ and BizTalk Adapter for MQ Series Client (MQSC) — depending on the version of BizTalk Server you are using that will support a variety of IBM WebSphere versions:
The MQSC adapter is a light-weight component of MQ that does not require the MQSeries Queue Manager run-time code to reside on a client system. The MQSC adapter is available in the BizTalk Adapters for Host Systems installation part of Host Integration Server (HIS). This adapter can connect to an IBM WebSphere Queue Manager on Windows or non-Windows. It also supports transactional and non-transactional messaging
This whitepaper’s goal is not how to install and configure MQSeries, neither compare the advantages or disadvantages of using this adapter compared to MQSC. Instead, this whitepaper’s focus is on improving the performance of the MQSeries adapter to better support low latency scenarios that most of the time are required in financial transactions.
Where I can download it
You can download the whitepaper here:
You can also find, download and enjoy several other free whitepapers of my own here:
Last week was Microsoft MVP renewal time – as you’ll probably have seen – and usually, I share this news firsthand, but this year my wife caught me off guard and in a week of hard work and was faster than me!
I’m delighted to share with you that on July 1st, I was renewed as a Microsoft Azure MVP (Microsoft Most Valuable Professional) for one more year. This is my 11th straight year on the MVP Program, a fantastic journey that started in 2011, back them as a BizTalk Server MVP. It looks like it was yesterday! And even though it’s already been 11 years, I still feel the same joy, excitement, and privilege of belonging to this group as on the first day!
It is an honor and privilege to be among great minds and community leaders! I want to send a big thanks to Cristina González Herrero, Irene Otero Perez for all the fantastic work managing the program in my region. And to all my fellow MVPs, my beautiful family, my coworkers, and to my team at DevScope, and in special all my blog readers, friends, members of Microsoft Enterprise Integration Community – THANKS! Thanks for your support during these years.
Historically, deploying BizTalk Server solutions across environments is or can be a complicated process depending how complex is your solution. There are many ways to deploy BizTalk artifacts for example:
Importing them as part of an application by using the Deployment Wizard (from an .msi file), importing them using BTSTask.exe – this is the default way to deploy across environments.
You can replace and use allow BTSTask, PowerShell scripts.
Or deploy them from Visual Studio – this is the default way to deploy to your development environment.
During the years, the BizTalk Server Community created an open-source deployment framework called Deployment Framework for BizTalk (BTDF) – https://github.com/BTDF/DeploymentFramework. The Deployment Framework for BizTalk is an easy-to-use toolkit for deploying and configuring your BizTalk solutions. In reality, BTDF is an MSBuild project with custom MSBuild tasks, and it can be customizable according to customer BizTalk project needs, and it is extensible. This framework brings new capabilities and advantages to deploying BizTalk Server solutions, but it also has limitations or disadvantages.
Azure DevOps and Azure Pipelines
Microsoft has introduced automated deployment of BizTalk Applications in BizTalk Server 2016 Feature Packs using Azure DevOps (previously called Visual Studio Team Services – VSTS). In BizTalk Server 2016 Feature Pack 1, automatic deployment and application lifecycle management (ALM) experience was introduced. The automatic deployment process has been improved with the release of BizTalk Server 2016 Feature Pack 2. These features were only available on the Enterprise edition of BizTalk Server 2016.
BizTalk Server 2020 brings all these functionalities out-of-the-box across all editions: Enterprise, Standard, Development, or Branch.
To accomplish this, we need basically 3 steps:
BizTalk Server: Add a BizTalk Server Application project in your Visual Studio solution.
We will not address this topic today.
DevOps: Create a build agent.
DevOps: Create a Build and release Azure Pipeline.
Today we will talk about starting to configure your Azure Pipeline to create a BizTalk Server Build Agent.
Create a Personal Access Token
A personal access token (PAT) is created in DevOps. This token is your password and is used by the DevOps build agent to authenticate. The token is only shown when you create it. After that, it isn’t shown anymore. Once you create it, you should save it to another file in a rememberable location.
If you do not have an account, select Create new account, and enter a name. To manage your code, choose your personal preference between Git or Team Foundation Version Control. When finished, your new account is created, you will be able to access Azure DevOps Portal.
Select your DevOps organization and then click the top second right-side corner icon – User settings – and select User settings > Personal access tokens.
The Personal Access Tokens page will be presented a list of all existing personal access tokens.
If you don’t have an existing PAT for your agent, select Add, and on the Create a new personal access token page, enter the following configuration:
On the Name property, enter a name for your PAT, for example, BizTalk Build Agent.
On the Organization property, leave the default organization.
On the Expiration (UTC) property, set an expiration date, for example, 90 days.
In Scopes, select Show all scopes, and then select Agent Pools – Read & manage option and Connected server – Connected server.
Select Create to finish the PAT creation.
Important Note: You need to save the token value. You need it in future steps. If you don’t know the access token value and didn’t take note of it anywhere, it cannot be retrieved. In this case, you need to create a new PAT.
Install the Build Agent
The build agent is installed on the BizTalk development computer. If using deployment groups, the build agent is installed on all the BizTalk servers you want to deploy to. Also, use these same steps to add a build computer, which might be different than the BizTalk development computer.
The following steps show you how to install the build agent on a single computer:
Open your Azure DevOps organization and then select the Organization settings icon and then Agent Pools.
Optional you can choose a Project inside your Organization and then select the Project settings icon and select Agent Pools.
Open Agent pools page, select the Default (Azure Pipelines) agent.
On the Default agent page, select New agent.
On the Get the agent pop-up window, select your SO, and on the Download the agent section, select Download.
It is important for you to save the file to your Downloads folder – on your BizTalk Server Development machine since the scripts will be referencing that folder.
Depending on your SO, this will download a zip file, for example, vsts-agent-win-x64-2.188.3.zip, that you will need to create the agent on the BizTalk Server Development machine.
The first step is to create the agent on your BizTalk Server Development machine. To do that open Windows PowerShell as Administrator and type the following command:
Note: The vsts-agent file version changes. Make sure the zip file name is the correct one.
The second step, as you also see in the picture is to configure the agent. To do that type the following command:
PS C:agent> .config.cmd
Enter the following details:
Server URL: Type https://dev.azure.com/{your-organization}.
Authentication Type: Enter PAT.
Personal access token: Paste your Azure DevOps token.
Agent pool: Click Enter for assuming the default value.
Agent name: Click Enter for assuming the default value.
Replace: Only displays if you have an existing agent.
Work folder: Click Enter for assuming the default value.
Run agent as a service: Enter Y.
User account: This value is up to you, but you may run into a permissions issue. Consider entering your current logged-on account, which is a local admin.
To validate if the agent was properly installed, Open services.msc to see the new service called Azure Pipelines Agent (<organization>.<agent pool>.<server>). The job should be running, otherwise type the following command:
PS C:agent> .run.cmd
Now, if we go back to our DevOps organization > Organization settings > Agent pools > Default (Azure Pipelines) > Agents, you will see your BizTalk Server Development server on the list:
I’m not a Dynamics 265 expert, and Dynamics 365 is not my focus area. Nevertheless, I couldn’t ignore the flood of requests to add the new Dynamics 365 logos, especially the App icons. It took a while, but they are finally here.
What’s new in this version?
These are the list of changes and additions present in this major release:
New shapes: There are new shapes on the following Visio Stencils files (.vssx):
MIS Office, Office 365 and Dynamics 365: add the new Dynamic 265 logo, Dynamics 365 App Icons, and Dynamics 365 Mixed Reality Icons.
MIS Azure Stencils and MIS Azure Additional or Support Stencils: there were a few new icons add to the stencils, most of them related to new preview features and integration services like the new Logic App icon.
SVG files: new SVG files added.
Microsoft Integration, Azure, Power Platform, Office 365 and much more Stencils Pack
Microsoft Integration, Azure, Power Platform, Office 365 and much more Stencils Pack it’s a Visio package that contains fully resizable Visio shapes (symbols/icons) that will help you to visually represent On-premise, Cloud or Hybrid Integration and Enterprise architectures scenarios (BizTalk Server, API Management, Logic Apps, Service Bus, Event Hub…), solutions diagrams and features or systems that use Microsoft Azure and related cloud and on-premises technologies in Visio 2016/2013:
BizTalk Server
Microsoft Azure
Integration
Integration Service Environments (ISE)
Logic Apps and Azure App Service in general (API Apps, Web Apps, and Mobile Apps)
Azure API Management
Messaging: Event Hubs, Event Grid, Service Bus, …
Azure IoT and Docker
AI, Machine Learning, Stream Analytics, Data Factory, Data Pipelines
SQL Server, DocumentDB, CosmosDB, MySQL, …
and so on
Microsoft Power Platform
Microsoft Flow
PowerApps
Power BI
Office365, SharePoint,…
DevOps and PowerShell
Security and Governance
And much more…
… and now non-related Microsoft technologies like:
SAP Stencils
The Microsoft Integration Stencils Pack is composed of 27 files:
Microsoft Integration Stencils
MIS Additional or Support Stencils
MIS AI and Machine Learning Stencils
MIS Apps and Systems Logo Stencils
MIS Azure Additional or Support Stencils
MIS Azure Black and Gray
MIS Azure Old Versions
MIS Azure Stencils
MIS Black and Cyan
MIS Buildings Stencils
MIS Databases and Analytics Stencils
MIS Deprecated Stencils
MIS Developer Stencils
MIS Devices Stencils
MIS Files and Message Types Stencils
MIS Generic Stencils
MIS Infrastructure and Networking Stencils
MIS Integration Fun
MIS Integration Patterns Stencils
MIS IoT Stencils
MIS Office, Office 365 and Dynamics 365
MIS Power BI Stencils
MIS Power Platform Stencils
MIS SAP Stencils
MIS Security and Governance
MIS Servers (Hexagonal) Stencils
MIS Users and Roles Stencils
Organisational Stencils
That you can use and resize without losing quality, in particular, the new shapes.
Download
You can download Microsoft Integration, Azure, BAPI, Office 365 and much more Stencils Pack for Visio from GitHub Here:
I’m just playing with BizTalk Server doing a small proof-of-concept using BizTalk Server and Azure Service Bus, and I was surprised with a few bunches of errors while I tried to compile my simple project:
Severity Code Description Project File Line Suppression State Error CS0426 The type name ‘SerializableAttributeAttribute’ does not exist in the type ‘System’ POC.BizTalk.AzureServiceBus C:DEVPOCPOC.BizTalk.AzureServiceBusPOC.BizTalk.AzureServiceBusSchemasASBPropertySchema.xsd.cs 9 Active Severity Code Description Project File Line Suppression State Error CS0426 The type name ‘Xml’ does not exist in the type ‘System’ POC.BizTalk.AzureServiceBus C:DEVPOCPOC.BizTalk.AzureServiceBusPOC.BizTalk.AzureServiceBusSchemasASBPropertySchema.xsd.cs 85 Active
Were is the full error list:
The type name ‘SerializableAttributeAttribute’ does not exist in the type ‘System’
The type name ‘SerializableAttribute’ does not exist in the type ‘System’
The type name ‘NonSerializedAttributeAttribute’ does not exist in the type ‘System’
The type name ‘NonSerializedAttribute’ does not exist in the type ‘System’
The type name ‘NonSerializedAttributeAttribute’ does not exist in the type ‘System’
The type name ‘NonSerializedAttribute’ does not exist in the type ‘System’
The type name ‘SerializableAttributeAttribute’ does not exist in the type ‘System’
The type name ‘SerializableAttribute’ does not exist in the type ‘System’
The type name ‘NonSerializedAttributeAttribute’ does not exist in the type ‘System’
The type name ‘NonSerializedAttribute’ does not exist in the type ‘System’
The type name ‘Xml’ does not exist in the type ‘System’
The type name ‘Xml’ does not exist in the type ‘System’
The type name ‘Type’ does not exist in the type ‘System’
The type name ‘SerializableAttributeAttribute’ does not exist in the type ‘System’
The type name ‘SerializableAttribute’ does not exist in the type ‘System’
The type name ‘NonSerializedAttributeAttribute’ does not exist in the type ‘System’
The type name ‘NonSerializedAttribute’ does not exist in the type ‘System’
The type name ‘Xml’ does not exist in the type ‘System’
The type name ‘Xml’ does not exist in the type ‘System’
The type name ‘Type’ does not exist in the type ‘System’
The type name ‘SerializableAttributeAttribute’ does not exist in the type ‘System’
The type name ‘SerializableAttribute’ does not exist in the type ‘System’
The type name ‘NonSerializedAttributeAttribute’ does not exist in the type ‘System’
The type name ‘NonSerializedAttribute’ does not exist in the type ‘System’
The type name ‘Xml’ does not exist in the type ‘System’
The type name ‘Xml’ does not exist in the type ‘System’
The type name ‘Type’ does not exist in the type ‘System’
Cause
Initially, I have to be honest, I was not realizing why this error was happening, mainly because the main error description may elude us and point us to DLL reference problems: Xml, System, SerializableAttributeAttribute, and so on. But looking carefully to the error message details, all of the errors will point us to the PropertySchema file.
After realizing that, it was not difficult to realize that I had on my Property Schema an element call System. System is a reserved word that you CANNOT use inside the property schemas.
Solution
The solution to this issue is quite simple, you need to:
Rename your System element inside your Property Schema to another word, for example ExtSystem
Of course, fix all the dependencias inside your project, if you were already using this element iside your solution, for example Message Assigment shape
There was a big buzz this weekend regarding the release of a new Azure Logo by Microsoft. The new Azure icon represents the unity of Azure within the larger Microsoft family of product icons. It’s part of Microsoft’s Fluent Design System, carefully crafted to produce icons that look familiar to what customers know and love, while representing the agile future of our business.
I usually don’t update my stencils only because of a new icon. Still, this time I made an exception, and I updated my package with this new Azure icon (thanks, Tiago Costa, for providing me the resource). I also toked this opportunity to add also a previous request: the new Dataverse icon.
What’s new in this version?
These are the list of changes and additions present in this major release:
New shapes: There are new shapes on the following Visio Stencils files (.vssx):
MIS Power Platform Stencils: the new logo of Dataverse was added.
MIS Azure Stencils: the new logo of Azure was added.
SVG files: new SVG files added.
Microsoft Integration, Azure, Power Platform, Office 365 and much more Stencils Pack
Microsoft Integration, Azure, Power Platform, Office 365 and much more Stencils Pack it’s a Visio package that contains fully resizable Visio shapes (symbols/icons) that will help you to visually represent On-premise, Cloud or Hybrid Integration and Enterprise architectures scenarios (BizTalk Server, API Management, Logic Apps, Service Bus, Event Hub…), solutions diagrams and features or systems that use Microsoft Azure and related cloud and on-premises technologies in Visio 2016/2013:
BizTalk Server
Microsoft Azure
Integration
Integration Service Environments (ISE)
Logic Apps and Azure App Service in general (API Apps, Web Apps, and Mobile Apps)
Azure API Management
Messaging: Event Hubs, Event Grid, Service Bus, …
Azure IoT and Docker
AI, Machine Learning, Stream Analytics, Data Factory, Data Pipelines
SQL Server, DocumentDB, CosmosDB, MySQL, …
and so on
Microsoft Power Platform
Microsoft Flow
PowerApps
Power BI
Office365, SharePoint,…
DevOps and PowerShell
Security and Governance
And much more…
… and now non-related Microsoft technologies like:
SAP Stencils
The Microsoft Integration Stencils Pack is composed of 27 files:
Microsoft Integration Stencils
MIS Additional or Support Stencils
MIS AI and Machine Learning Stencils
MIS Apps and Systems Logo Stencils
MIS Azure Additional or Support Stencils
MIS Azure Black and Gray
MIS Azure Old Versions
MIS Azure Stencils
MIS Black and Cyan
MIS Buildings Stencils
MIS Databases and Analytics Stencils
MIS Deprecated Stencils
MIS Developer Stencils
MIS Devices Stencils
MIS Files and Message Types Stencils
MIS Generic Stencils
MIS Infrastructure and Networking Stencils
MIS Integration Fun
MIS Integration Patterns Stencils
MIS IoT Stencils
MIS Office, Office 365 and Dynamics 365
MIS Power BI Stencils
MIS Power Platform Stencils
MIS SAP Stencils
MIS Security and Governance
MIS Servers (Hexagonal) Stencils
MIS Users and Roles Stencils
Organisational Stencils
That you can use and resize without losing quality, in particular, the new shapes.
A few days ago, I release the latest version of the BizTalk Scheduled Task Adapter optimized for BizTalk Server 2020. Today I merged the changes and improvements made on that version and made it available for BizTalk Server 2016.
What is new?
New schedule capabilities:
Capability to set DateTime you want to use on the Timespan schedule type: DateTime.Now or DateTime.UtcNow
By adding this capability to chose between DateTime.Now or DateTime.UtcNow, it will address and solve the issues reported regarding unlimited messages being triggered if you are using a different timezone than UTC.
The functionality always to trigger the message at 00 seconds and respect the original time set was added again to the Timespan schedule type. The GetNextActivationTime method was improved to fix all these unexpected behaviors reported that indeed was a feature by design.
BizTalk Scheduled Task Adapter
The BizTalk Scheduled Task Adapter is an in-process receive adapter that executes a prescribed task on a daily, weekly or monthly schedule. The adapter is configured entirely within BizTalk, all configurations are stored within the SSODB and can be exported and imported via binding files.
The schedule capabilities are similar to those available with the Windows Scheduled Task Service.
Four simple tasks are included:
XmlStringStreamProvider – generates a BizTalk message from a configured Xml string
FileStreamProvider – generates a BizTalk message from the contents of a file
HttpDownload – generates a BizTalk message from data downloaded from a web site
SQLStreamProvider – generates a BizTalk message from the contents of a SQL Query (similar to the old SQL adapter) – Since version 3.0
Custom tasks can be created. Any .NET class that implements the appropriate interface can be scheduled.
Big thanks to Pedro Almeida, my team member, that took 95% of my work and finalize this version!
Download
You can download this “new” version of the adapter in BizTalk Scheduled Task Adapter from GitHub:
Please feel free to use the adapter, and If you find any problems or bugs, please open an issue on the adapter GitHub site: https://github.com/sandroasp/BizTalk-Scheduled-Task-Adapter and help evolve this community adapter. You can also suggest new features, and I am also open to new ideas.
Recently while I was testing an integration solution on a client that uses WCF-SQL to insert data on a SQL Server database I got the following error:
System.NotSupportedException: The SqlDbType “” is not supported. Modify your table, view, stored procedure, or function definition to avoid having parameters or columns of this type.
This was a surprise since everything was working properly in the development environment.
Cause
In this solution, we were doing bulk insert on a SQL Server database by using a User-Defined Types as the input to the stored procedure, in our case a User-Defined Table Type. There are other ways to do a SQL Server bulk insert or update in BizTalk Server but definitely, this is the best approach. And this is one of the reasons why we are getting on the error message with the keyword: SqlDbType.
This problem occurs because the user account that you used to access the database, in my case the BizTalk Host Instance Account, don’t have permissions on the User-Defined Type associated to the Store procedure that we were invoking.
Solution
To solve this issue, you must give access to the user, in my case BizTalk Host Instance Account to properly execute the stored procedure, you must:
Open SQL Server Management Studio and connect to your server.
In the Object Explorer, select and expand the desired database and expand the Security folder and then the Users.
Right-click on the User, BizTalk Host Instance Account, and choose Properties.
On the Database User windows, choose the Owned Schemas tab, and then on the Schemas owned by this user panel select the schema bind to your User-Defined Table Type.
In our case: Material
Click OK and try again to send the data to your SQL Server. It should work.
It has been a while since I didn’t follow carefully all the comments or issues posted on my blog or GitHub regarding BizTalk Scheduled Task Adapter. It was not by lack of interest but instead due to my lack of free time or just because I was focused on other projects or contributions.
Since I recently (7 days ago) publish I new version of the adapter with some further improvements and optimized it to BizTalk Server 2020. And I have a brand new environment that I need to install this adapter. I decided that it was an excellent opportunity and time to read the existing feedback and address, once and for all, all the reported issues or unexpected behaviors:
Using the Timespan schedule type may trigger unlimited messages;
Using the Timespan schedule type, if the time is set to past time, the task will not be triggered in certain cases.
Using the Timespan schedule type, when stopping the receive locations or stop/restart the host instances, you start to get a time drift from the original time set.
What is new?
New schedule capabilities:
Capability to set DateTime you want to use on the Timespan schedule type: DateTime.Now or DateTime.UtcNow
By adding this capability to chose between DateTime.Now or DateTime.UtcNow, it will address and solve the issues reported regarding unlimited messages being triggered if you are using a different timezone than UTC.
The functionality always to trigger the message at 00 seconds and respect the original time set was added again to the Timespan schedule type. The GetNextActivationTime method was improved to fix all these unexpected behaviors reported that indeed was a feature by design.
BizTalk Scheduled Task Adapter
The BizTalk Scheduled Task Adapter is an in-process receive adapter that executes a prescribed task on a daily, weekly or monthly schedule. The adapter is configured entirely within BizTalk, all configurations are stored within the SSODB and can be exported and imported via binding files.
The schedule capabilities are similar to those available with the Windows Scheduled Task Service.
Four simple tasks are included:
XmlStringStreamProvider – generates a BizTalk message from a configured Xml string
FileStreamProvider – generates a BizTalk message from the contents of a file
HttpDownload – generates a BizTalk message from data downloaded from a web site
SQLStreamProvider – generates a BizTalk message from the contents of a SQL Query (similar to the old SQL adapter) – Since version 3.0
Custom tasks can be created. Any .NET class that implements the appropriate interface can be scheduled.
Big thanks to Pedro Almeida, my team member, that took 95% of my work and finalize this version!
Download
You can download this “new” version of the adapter in BizTalk Scheduled Task Adapter from GitHub:
Please feel free to use the adapter, and If you find any problems or bugs, please open an issue on the adapter GitHub site: https://github.com/sandroasp/BizTalk-Scheduled-Task-Adapter and help evolve this community adapter. You can also suggest new features, and I am also open to new ideas.
Once again, after many requests and many postponements, due to my unavailability and free time to take these tasks, BizTalk Scheduled Task Adapter is finally officially available and optimized for BizTalk Server 2020 on GitHub!
The BizTalk Scheduled Task Adapter is an in-process receive adapter that executes a prescribed task on a daily, weekly or monthly schedule. The adapter is configured entirely within BizTalk, all configurations are stored within the SSODB and can be exported and imported via binding files.
The schedule capabilities are similar to those available with the Windows Scheduled Task Service.
Four simple tasks are included:
XmlStringStreamProvider – generates a BizTalk message from a configured Xml string
FileStreamProvider – generates a BizTalk message from the contents of a file
HttpDownload – generates a BizTalk message from data downloaded from a web site
SQLStreamProvider – generates a BizTalk message from the contents of a SQL Query (similar to the old SQL adapter) – Since version 3.0
Custom tasks can be created. Any .NET class that implements the appropriate interface can be scheduled.
What is new?
New schedule capabilities:
HTTPDownload task was changed and improved to control timeouts better and to avoid the receive locations disabling themselves automatically (Contribution made by Rickard Karlmats)
HTTPDownload task was changed and improved to support REST/JSON (Contribution made by Pedro Almeida)
Improved and optimized to BizTalk Server 2020 but it will also be compatible with BizTalk Server 2016
You can download this “new” version of the adapter in BizTalk Scheduled Task Adapter from GitHub:
Please feel free to use the adapter, and If you find any problems or bugs, please open an issue on the adapter GitHub site: https://github.com/sandroasp/BizTalk-Scheduled-Task-Adapter and help evolve this community adapter. You can also suggest new features, and I am also open to new ideas..
