by Sandro Pereira | Jul 16, 2021 | BizTalk Community Blogs via Syndication
Patrick Wellink, a long-time BizTalk Server consultant from the Netherlands, asked me to broadcast this news. I’m happy to do it because I know that many of you use and like BizTalk Deployment Framework (BTDF) to perform deployment across your environments.
Patrick suppresses a need by developing a Visual Studio 2019 plugin (VSIX) for BizTalk Server 2020. Of course, there was already a new version of The BTDF Framework. But one of the convenient things was the visual studio plugin… that was missing.
Unfortunately, and I understand this feeling entirely because I also have several community projects, we always have to for the owner and developer to upgrade his project, and we usually struggle to find the time. So I take this opportunity to encourage all community members to start contributing to these initiatives as Patrick did. Start performing changes and submit your contributions to the repo. I speak for myself, and from what I know from the community, we accept the changes and are happy that you contribute.
And it wasn’t that difficult, rephrasing Patrick:
With some guidance of this post and some fiddeling I got the VSIX to work for VS2019. After some more fiddeling I got the project wizard to run as well.
I did so with minimal effort.
Where I can download it
You can download the Visual Studio 2019 plugin (VSIX) for BizTalk Server 2020 here:
The post BizTalk Deployment Framework (BTDF) Visual Studio plugin for BizTalk Server 2020 appeared first on SANDRO PEREIRA BIZTALK BLOG.
by Lex Hegt | Feb 22, 2017 | BizTalk Community Blogs via Syndication
Part 2 – Advanced setup
This is an article in a series of 2 about the BizTalk Deployment Framework (BTDF). This open source framework is often used to create installation packages for BizTalk Applications. More information about this software can be found here CodePlex.
With this series of articles, we aim to provide a useful set of hands-on documentation about the framework.
The series consists of the following parts:
- Part 1 – Introduction and basic setup
- Part 2 – Advanced setup (this article)
- Introduction
- Custom Targets
- Conditions
- Host Groups
- Executing third party executables
- Tips and Tricks
- Conclusion
Introduction
In the first part of this series we have seen how you can setup a BizTalk Deployment-project, add artefacts to the project, how you can deploy variable settings for other DTAP environments and how you can generate distributable MSI packages.
In this second article we’ll show somewhat more advanced topics like Custom Targets, Conditions, Host Groups and we’ll see how you can run third party executables from within your deployment. We’ll finish with a couple of Tips and Tricks.
Custom Targets
To be able to customize your deployments, the BizTalk Deployment Framework uses a concept called Custom Targets. These Custom Targets allow you to execute commands during certain phases of the deployment. We will describe a couple of Custom Targets and show how you can use them later on as well. The currently available Custom Targets are shown below.
Adding Custom Targets to your Deployment project
Before describing the Custom Targets, let’s first have a quick look on how to add Custom Targets to your Deployment project.
To keep your Deployment project nicely organized, it is a good idea to keep all the custom targets you will be using grouped together below all the BizTalk artefacts you will deploy. So the Custom Targets will be added at the end of your Deployment project.
Below is a picture which shows where the CustomRedist target is added to the Deployment project. This target was actually already added to the project during creation of the Deployment project.
As you can see, you can simple type the text <Target Name=”…”></Target> and at the ellipsis, provide the name of the Custom Target you will be going to use. Everything you want to perform during the execution of the target, can be entered between the start tag and the end tag. We’ll see examples of the kind of commands which can be performed later in this article.
CustomRedist
This target becomes executed right after BTDF has finished copying files. So this is the ideal moment if you would like for example to add additional files to the MSI package, which will be copied to the installation folder on deployment.
Below you have a sample in which a folder becomes created and some files are copied to that folder:
<MakeDir Directories="$(RedistDir)SQLScripts" />
<CreateItem Include="SQLScripts*.*">
<Output TaskParameter="Include" ItemName="SQLScripts" />
</CreateItem>
<Copy DestinationFolder="$(RedistDir)SQLScripts%(RecursiveDir)" SourceFiles="@(SQLScripts)"/>
CustomDeployTarget and CustomPostDeployTarget
The CustomDeployTarget target runs just before Deployment has started, while the CustomPostDeployTarget target runs directly after Deployment has finished. Either can be used if you want to deploy additional stuff. Later we’ll describe how these targets are used to run third party executables, but you might also use these targets to create for example an EventLog source.
CustomUndeployTarget and CustomPostUndeployTarget
The CustomUndeployTarget target runs just before any undeployment has been performed, while the CustomPostUndeployTarget target runs right after the undeployment has taken place. These targets can be used if you would like to clean up stuff after undeployment. You can think of that as cleaning any created files during deployment. All just to be sure that you keep your environment nice and clean.
Conditions
The concepts of Conditions can be used in both Bindingfiles as in the BTDF project file. With Bindingfiles this might be used to create different ports in different environments.
In the BTDF project file this feature can become used when for example, you want to execute certain targets only in certain environments.
As applying Conditions is almost the same for both kind of files, let’s just have a look at the latter one.
Let’s assume that we want to execute an .exe file only for the Test and Production environment. To be able to execute certain targets just for certain environments, you need to do the following:
– Add an Environment variable to your BTDF Settings file
– Add the variable to your BTDF project file
– Add a filter to the CustomPostDeployTarget
Add an Environment variable to your BTDF Settings file
From Visual Studio open the SettingFileGenerator.xml file in Excel and add a variable called Environment. Each environment will get a unique value. Below picture shows a setting file which is extended with such a setting and contains unique values for all environments, namely DEV, SDEV, TEST and PROD.
Add the variable to your BTDF project file
To be able to use the Environment variable, you need to do the following:
• Open the Deployment.btdfproj file and look for an ItemGroup which contains the value PropsFromEnvSettings. Typically this settings contains the values SSOAppUserGroup and SSOAppAdminGroup.
• Add a semicolon and the new variable Environment. Afterwards the ItemGroup will look like below.
<ItemGroup>
<PropsFromEnvSettings Include="SsoAppUserGroup;SsoAppAdminGroup;Environment" />
</ItemGroup>
Add a filter to the CustomPostDeployTarget
In the final step you’ll arrange that the step which executes the .exe file only gets executed in case you’re deploying on the TEST or PROD environment. Navigate to the target called CustomPostDeployTarget and add the condition: $(Environment)==’TEST’ OR $(Environment)==’PROD’ to it.
When you’ve done that, the target will look as follows:
<Target Name="CustomPostDeployTarget" Condition="$(Environment)=='TEST' OR $(Environment)=='PROD'">
<MakeDir Directories="$(RedistDir)SQLScripts" />
<CreateItem Include="SQLScripts*.*">
<Output TaskParameter="Include" ItemName="SQLScripts" />
</CreateItem>
<Copy DestinationFolder="$(RedistDir)SQLScripts%(RecursiveDir)" SourceFiles="@(SQLScripts)"/>
</Target>
Now, when you save the project file, create an MSI and execute that MSI, the CustomPostDeployTarget will only be executed when you are deploying to the Test or the Production environment.
Host Groups
In most cases a BizTalk application uses only a couple of Host Instances. After a deployment it is a good practice to restart all relevant Host Instances. BTDF enables you to achieve this by using an BizTalkHosts ItemGroup. If you omit this ItemGroup, all Host Instances will be restarted.
You can put such an ItemGroup in the BTDF-project file. To keep the project file nicely organized you can consider to have that ItemGroup below the BizTalk artefacts you are deploying.
<ItemGroup>
<BizTalkHosts Include="SendHost;ReceiveHost;OrchestrationHost" />
</ItemGroup>
As you can see, you can easily add multiple hosts by using semicolons between the separate hosts.
Executing third party executables
The BizTalk Deployment Framework enables you to execute third party executables. Below you find brief examples to show what you can achieve with this capability.
Executing SQL Server scripts
Say you have a custom database which is used by your BizTalk solution and this database needs to be extended with additional tables and stored procedures. Without using the BizTalk Deployment Framework, you would need to run the necessary SQL Server scripts on each environment by hand. This is a time-consuming and error-prone task which can be automated with BTDF.
To be able to execute SQL Server commands, you need to execute an executable called sqlcmd. Below you find an example on how to execute SQL scripts:
<Target Name="CustomPostDeployTarget">
<Exec Command="sqlcmd -S $(SQLServer) -i"SQLScripts<Name of T-SQL deploy script>"" />
</Target>
As you can see, you can simply add an Exec command to the deployment target.
For all the ins and outs on deploying SQL Server scripts, check this article on TechNet Wiki.
Create BizTalk360 alerts
Another example of running third party executables is the creation of BizTalk360 alarms. Normally you’ll have to do this manually after each deployment of a new or changed BizTalk application. This too can be a time-consuming and error prone task. By using BTDF and a tool called BT360Deploy, you can automate the creation of alarms during the deployment of you BizTalk application.
In the picture here above, you have an example of the output of BT360Deploy. More information on BT360Deploy can be found on CodePlex.
Tips and Tricks
We’ll end this article by providing you with a couple of Tips and Tricks
Do not automatically start the Application after deployment
BTDF allows you to automatically start the deployed BizTalk application, i.e. start all ports and orchestrations after deployment. Although that sounds like a convenient feature, especially in Dev and Test environments, your BizTalk administrator probably prefers that you leave the just deployed BizTalk application not started on Production environments. The reason for this is that in this way the BizTalk Administrator has better control over the moment when the BizTalk application must be activated.
Consider (not) restarting Host Instance
We have seen that BTDF can restart the Host Instance(s) after deployment. But in case you use scripting for unattended deployment of multiple BizTalk applications at the same time, you don’t want the Host Instances restarted after each deployment package as this is a waste of time and resources. In that case, you remove the ItemGroup that contains the BizTalkHosts. Instead you set the SkipHostInstancesRestart property in the first PropertyGroup of the project file to True:
<SkipHostInstancesRestart>False</SkipHostInstancesRestart>
ItemGroups per artefact
To keep your BTDF project file organized, it is preferable that you use separate ItemGroups for all the separate types of artefacts you will be deploying.
<ItemGroup>
<Schemas Include="Kovai.BTDF.Schemas.dll">
<LocationPath>..$(ProjectName).Schemasbin$(Configuration)</LocationPath>
</Schemas>
<Schemas Include="Kovai.BTDF.MoreSchemas.dll">
<LocationPath>..$(ProjectName).MoreSchemasbin$(Configuration)</LocationPath>
</Schemas>
</ItemGroup>
<ItemGroup>
<Transforms Include=" Kovai.BTDF.Maps.dll ">
<LocationPath>..$(ProjectName).Mapsbin$(Configuration)</LocationPath>
</Transforms>
<Transforms Include=" Kovai.BTDF.MoreMaps.dll ">
<LocationPath>..$(ProjectName).MoreMapsbin$(Configuration)</LocationPath>
</Transforms>
</ItemGroup>
<ItemGroup>
<Orchestrations Include=" Kovai.BTDF.Orchestrations.dll ">
<LocationPath>..$(ProjectName).Orchestrationsbin$(Configuration)</LocationPath>
</Orchestrations>
<Orchestrations Include=" Kovai.BTDF.MoreOrchestrations.dll ">
<LocationPath>..$(ProjectName).MoreOrchestrationsbin$(Configuration)</LocationPath>
</Orchestrations>
</ItemGroup>
Conclusion
In a series of 2 articles we have seen quite a lot of the capabilities of the BizTalk Deployment Framework. In part 1 we started with pointing out the advantages of using BTDF over the out-of-the-box capabilities of BizTalk Server and we created a simple Deployment project which showed how to setup a deployment with basic artefacts like Schemas, Maps and Orchestrations. We extended this project by creating variables to be used for environment specific settings like URL’s of endpoints.
In part 2 we have seen some more advanced features like Conditions and Host Groups. We have also seen how we can run third party executables to automate your deployment as far as possible and to decrease the amount of manual and error prone labour as much as possible.
Still there are much more possibilities with BTDF which we did not discuss. For example, you can also use it to create virtual directories, deploy SSO settings, create EventLog Event Sources, create Registry keys etc. etc. etc..
It must be clear by now that the BizTalk Deployment Framework is a very powerful framework that can be used with straight forward to complex deployments. So when you are already familiar with the framework and are using it for the deployment of standard BizTalk artefacts, why not expand your experience with BTDF and start deploying less common artefacts like SQL scripts. When you do that properly, your BizTalk Administrator might become your new best friend!
Happy deploying!
Author: Lex Hegt
Lex Hegt works in the IT sector for more than 25 years, mainly in roles as developer and administrator. He works with BizTalk since BizTalk Server 2004. Currently he is a Technical Lead at BizTalk360. View all posts by Lex Hegt
by Lex Hegt | Feb 13, 2017 | BizTalk Community Blogs via Syndication
This is an article in a series of 2 about the BizTalk Deployment Framework (BTDF). This open source framework is often used to create installation packages for BizTalk Applications. More information about this software can be found here CodePlex. With these series of articles, we aim to provide a useful set of hands-on documentation about the framework.
The series consists of the following parts:
- Part 1 – Introduction and basic setup (this article)
- Introduction
- BizTalk out-of-the-box capabilities
- Top reasons to use BTDF
- Setting up a BTDF-project
- Conclusion
- Part 2 – Advanced setup
Introduction
In this first part, we will shortly discuss the shortcomings of the deployment packages which can be created with the out-of-the-box capabilities of BizTalk Server and introduce you to the benefits of BTDF. Further this article will show you how you can create a BTDF-project for the deployment of basic BizTalk artifacts like Schema’s, Maps and Orchestrations and how you can deploy variable settings for other DTAP environments. We will end this article with showing you how you can generate distributable MSI packages.
BizTalk out-of-the-box capabilities
Microsoft BizTalk Server comes with capabilities to create deployment packages. However, the MSI packages you create with the out-of-the-box features of BizTalk Server have some shortcomings which lead to deployments that are time consuming and error-prone. The main reasons for this are:
- The packages can only contain URI’s of Endpoints (Receive Locations and Send Ports) of a single DTAP environment. So DTAP promotion from the same package is not possible.
- Only deployment of BizTalk artifacts is possible. So if you need, for example, to create and populate database tables, this cannot be automated.
Top reasons to use BTDF
As you’ll imagine, BTDF has eliminated these shortcomings, because of which the framework has been widely adopted. Below you’ll find a list which contains a number of reasons why you could use this framework:
- Deploy complex BizTalk solutions containing BizTalk artifacts but also for example SSO settings, ESB itineraries and SQL Server scripts, with no human intervention
- Consolidate all of your environment-specific configuration and runtime settings into one, easy-to-use Excel spreadsheet
- Maintain a single binding file that works for all deployment environments
- Make automated deployment a native part of the BizTalk development cycle, then use the same script to deploy to your servers
Setting up a BTDF-project
After you have downloaded and installed BTDF on your Development box, you can start with setting up your BTDF-project. To get that working, you need to take the following steps:
- Add a BTDF project to your BizTalk solution
- Add BizTalk artifacts to the BTDF project
- Deploy from Visual Studio
- Generate a master PortBinding file
- Apply Environment Specific Settings
- Generate a MSI
Add a BTDF project to your BizTalk solution
Firstly, you need to open a BizTalk solution in Visual Studio and perform the following steps:
- Right-click the Solution Explorer and select Add/New Project…
- In the dialog that opens, in the left hand pane, navigate to BizTalk Projects
- In the right pane, select Deployment Framework for BizTalk Project, give the project the name Deployment and click the OK button to continue the creation of the project.
In the screen that follows you can configure which artifacts you want to deploy. Perform the following:
- Double-click on the text True after Deploy Settings into SSO. It will be set to False
- Uncheck the check mark ‘Write properties to the project file only when…’
- Click the Create Project button
After the project has been created, you’ll get a small dialog box, which states that you need to edit the file Deployment.btdfproj to configure your specific deployment requirements. And that’s exactly what we are going to do in a couple of minutes! For now, just click the OK button.
Although the aforementioned project file is shown in the left part of the screen, the Solution Explorer does not yet show the newly created Deployment project. To have the project shown up there, you need to manually add the project file and some other files to the solution in a Solution folder.
Perform the following actions to add the Deployment project to the solution:
- Right-click on the Solution Explorer, select Add/New Solution Folder, enter Deployment and hit the Enter key
- Right-click on the Deployment folder, select Add/Existing Item… and in the dialog box that appears double-click the folder Deployment
- Select all the files and click on the Add button
The Deployment project is now added to your BizTalk solution and that project in the Solution Explorer looks like below.
Note: Take care that it is important to name that solution folder Deployment, otherwise Visual Studio won’t be able to find the Deployment.btdfproj file.
Add BizTalk artifacts to the BTDF project
Now we have the Deployment project in place, we are going to add some basic BizTalk artifacts to the Deployment project. To start we’ll add the following types of artifacts:
- Schemas
- Maps
- Orchestrations
To keep the Deployment project file nicely organized, we will create an ItemGroup for each type of artifact. As we start with 3 types of artifacts, we’ll end up having 3 ItemGroups. Probably it will be clear by now that ItemGroups can be used to organize the artifacts you want to deploy. An ItemGroup is an element from MSBuild which is used by BTDF. Besides ItemGroups, you will also find PropertyGroups within the BTDF project file. PropertyGroups are used to organize all kind of parameters which can be used during deployment. Think of this for example of the name of the project, but also which kind of artifacts need to be deployed.
The project file already contains an ItemGroup for deploying schemas. We will use this as a template for our own purposes.
Follow below instructions to add schemas, maps and orchestrations to the deployment project.
- In the Deployment project file, search for the ItemGroup that is used for deploying Schemas. It contains the text ‘<Schemas Include=”YourSchemas.dll”>’.
- Replace ‘YourSchemas.dll’ with the name of the DLL which contains your schemas
- The line with the ‘<LocationPath>’ tag, must contain the path where the DLL resides
The ItemGroup will now look like below:
<ItemGroup>
<Schemas Include="Kovai.BTDF.Schemas.dll">
<LocationPath>..$(ProjectName).Schemasbin$(Configuration)</LocationPath>
</Schemas>
</ItemGroup>
Next we’ll prepare deployment of the BizTalk Maps.
- Below the ItemGroup to deploy Schemas, create a new ItemGroup to deploy Maps. Therefore, you can copy/paste below XML:
<ItemGroup>
<Schemas Include="Kovai.BTDF.Schemas.dll">
<LocationPath>..$(ProjectName).Schemasbin$(Configuration)</LocationPath>
</Schemas>
</ItemGroup>
Now we are going to prepare deployment of the BizTalk Orchestrations.
- Below the ItemGroup to deploy Maps, create a new ItemGroup to deploy Orchestrations. Therefore, you can copy/paste this XML:
<ItemGroup>
<Orchestrations Include=" Kovai.BTDF.Orchestrations.dll ">
<LocationPath>..$(ProjectName).Orchestrationsbin$(Configuration)</LocationPath>
</Orchestrations>
</ItemGroup>
If you have multiple projects of for example orchestrations, you can simple copy/paste the <Orchestrations>…</Orchestrations> block and provide it with the correct Include-parameter and location path. Of course, this also works the same for other types of artifacts like schemas, maps, pipelines etc.
Deploy from Visual Studio
We now have deployment for some basic artifacts in place, so let’s deploy the BizTalk solution and see what happens:
- Right-click the solution (in the solution Explorer) and select Build Solution
- When the solution is built without errors, you can deploy the solution, therefore just click the little green arrow in the upper left are of Visual Studio:
If everything runs like expected the Kovai.BTDF application is created and the schemas, the map and the orchestration will be deployed properly, but you’ll end up with the following error:
Could not enlist orchestration ‘Kovai.BTDF.Orchestrations.BizTalk_Orchestration1,Kovai.BTDF.Orchestrations, Version=1.0.0.0, Culture=neutral, PublicKeyToken=72ff5decf00d3ffb’. Could not enlist orchestration ‘Kovai.BTDF.Orchestrations.BizTalk_Orchestration1’. All orchestration ports must be bound and the host must be set.
This is due to the fact that the orchestration Orchestration1 contains ports which will have to be configured and bound in the BizTalk Administration Console. Let’s just do that and afterwards export the Binding file into your Deployment project.
- Open the BizTalk Server Administration Console and navigate to the BizTalk application called BTDF
- Right-click the application and click Configure. The following screen appears:
- Select a Host in the designated dropdown box
- Create a Receive Port and a Receive Location (use the FILE adapter) and bind it to rcvPort
- Create a Send Port (use the FILE adapter) and bind it to sndPort
- Click the OK button
- Right-click the application again and click .. and in the dialog box that appears, click Start again. The application should now be started properly
Generate a master PortBinding file
We now have a running BizTalk application which includes bindings between the deployed orchestration and a couple of ports. In the next step we will make these bindings available to our deployment project. Therefore, we need to export the bindings.
Again Right-click the application Kovai.BTDF and now select Export/Bindings…
- Click on the ellipsis button (…), navigate to your Deployment project, select xml, click the OK button and confirm that you want to overwrite the existing file
- Switch to Visual Studio and notice that you are prompted with the following message:
- Click Yes to reload the PortBindingsMaster.xml file
- Open PortBindingsMaster.xml and notice that it contains the bindings you created earlier
Apply Environment Specific Settings
For just local deployments you now have the basics in place. However, in most scenarios you will have to deploy your BizTalk solution(s) to multiple DTAP environments. In that case, the addresses of your Receive Locations and Send Ports will differ between environments. BTDF facilitates this by providing a XML file which can be edited in Microsoft Excel.
Maintaining DTAP environments
That XML file, named SettingsFileGenerator.xml, can contain columns for each DTAP environment to which you have to deploy your BizTalk solution. Below you find a screen shot of how such a file looks like by default.
As you can see, there are columns for multiple environments; Local Development, Shared Development, Test and Production. You are free to rename the values in the existing columns, or add/remove columns.
Maintaining environment specific settings
In the spreadsheet there currently are 2 settings defined; SsoAppUserGroup and SsoAppAdminGroup. You can add more settings in the first column and give them values for the different environments in the following columns. If the value is the same for two or more environments, you could enter a value in the column Default Values, as this value will be used in case no value is entered for one or more environments.
The screenshot below shows a Settings file that now contains values for the addresses of the Receive Location and the Send Port. As you can see, there are Default values entered and the columns for Local Development and Shared Development are left empty, so while deploying to those environments, the Default values will be used.
To be able to use for example the SP_Address variable from the settings file in the PortBindingsMaster, you need to open the PortBindingsMasterFile.xml file, navigate to the Send Port which should use the addresses and replace the current address with ${SP_Address}.
So before that part of the PortBindingsMaster.xml file looks like this:
<SendPort Name="SendPort2" IsStatic="true" IsTwoWay="false" BindingOption="1">
...
<PrimaryTransport>
<Address>C:BizTalkKovai.BTDFOut%MessageID%.xml</Address>
...
</SendPort>
After the change the XML will look like below:
<SendPort Name="SendPort2" IsStatic="true" IsTwoWay="false" BindingOption="1">
...
<PrimaryTransport>
<Address>${SP_Address}%MessageID%.xml</Address>
...
</SendPort>
Generate a MSI
So far, we have only done deployments through Visual Studio. When we have to deploy our BizTalk solutions to non-development servers, we do that by installing MSI’s. BTDF is also able to generate MSI’s. Besides the BizTalk artifacts which need to be deployed, these MSI’s can also contain custom components, binding files for your entire DTAP environment and all kind of custom scripts, which can also be started during deployment.
Generating a MSI is done from Visual Studio, just choose one of below steps to generate a MSI:
- Click on the below button in the toolbar
- In the Visual Studio menu, select Tools/Deployment Framework for BizTalk/Build Server Deploy MSI
Both options will provide you with a MSI which you can use to deploy the BizTalk application throughout your DTAP environment.
Conclusion
In this first article we have seen how the BizTalk Deployment Framework differs from the out-of-the-box capabilities of BizTalk Server and described the benefits of BTDF. We’ve also setup a simple deployment project in an existing BizTalk solution and shown how you can add some basic BizTalk artifacts to that project. Further we have seen how we can create different settings for, for example, Receive Location addresses and how they become deployed to multiple environments. Finally, we’ve seen deployment from Visual Studio and we’ve generated a MSI based on the BTDF project.
Explore best practices for deploying BizTalk Server 2016 from Sandro Pereira’s Whitepaper.
In the next article, we’ll get familiar with amongst others Custom Targets, Conditions and see how we can run third party executables to automatically deploy non-standard artifacts like SQL Server scripts or even automatically create BizTalk360 alerts from within your deployment project!
Check out the 2nd article of the series here.
Author: Lex Hegt
Lex Hegt works in the IT sector for more than 25 years, mainly in roles as developer and administrator. He works with BizTalk since BizTalk Server 2004. Currently he is a Technical Lead at BizTalk360. View all posts by Lex Hegt
