Comparing Nagios and SCOM to BizTalk360

Comparing Nagios and SCOM to BizTalk360

Application HealthApplication HealthAll products deal with core Application Health scenarios very well. Nagios loses points for less visibility when determining the health of an application.Platform HealthHost Instance AvailabilityAll products deal with Host Instance health scenarios. Good visibility exists when determining the health of a Host instance. The higher rank goes to BizTalk360, because of the Auto-correct feature, for which customization is needed with Nagios and SCOM.Host ThrottlingAll products provide capabilities when it comes to determining Host Throttling, although in case of Nagios, it will be monitoring of PerfMon counters. BizTalk360 however, provides a more comprehensive experience and allows you, besides Host Throttling monitoring, to historically go back to determine exactly when and why BizTalk was throttling.Single Sign On Service availabilityAll products provide suitable functionality in this area. BizTalk360 gets the higher mark as it also has an Auto correct feature, which tries to bring the SSO service in the expected state, after a failureMonitoring SQL Agent JobsAll applications do provide this support. BizTalk360 provides a very friendly user experience, whereas SCOM provide deeper functionality that is found in the SQL Server Management Pack.
Nagios takes the lower ranking as complex customization is needed.Core Infrastructure MonitoringNagios and SCOM provide more Core Infrastructure Monitoring, but since we are keeping this relevant to BizTalk, BizTalk360 provides enough visibility to support a BizTalk environment and earns a top score as well.BizTalk Health Monitor IntegrationBizTalk Health Monitor (BHM) integration is part of BizTalk360’s core offering and is very easy to setup. No integration exists between BizTalk Health Monitor and Nagios and SCOM. Custom development is required in order to provide this functionality.Log/Database Query MonitoringAll products receive partial scores for these features. Currently BizTalk360 provides no capability around parsing log files. It does provide support for SQL Server databases, but not others. Nagios and SCOM both provide support for both Log and Database Monitoring. The downside is that these features often require custom scripts to be written in order to support the requirement. SCOM loses marks on the usability aspects of implementing these functions, while Nagios loses marks as a separate product will be needed for Log monitoring.AnalyticsNagios has no BizTalk oriented analytics.
SCOM uses a data warehouse for reporting and long-term data storage. Because of the extended reporting capabilities, though not easy to use, SCOM gets the higher grade.
BizTalk360 provides a customizable Analytics Dashboard and many widgets give insight in the performance and processing of messages through BizTalk.Operating EnvironmentProcess MonitoringThis is a core feature in BizTalk360 that detects when something is supposed to happen but does not. An example is not receiving a file from a trading partner when you expect to. Your environment can be completely healthy, but if the trading partner does not provide the file, traditional Monitoring techniques used by SCOM will not detect this. BizTalk360 will detect this scenario and send the appropriate notifications.Maintenance Mode/Negative MonitoringNagios supports Scheduling down time, but has no Negative monitoring for BizTalk Server resources.
Both SCOM and BizTalk360 provide functionality in these areas. BizTalk360 provides complete coverage whereas SCOM is only providing partial coverage.
SCOM is able to handle the Maintenance Mode requirements, but not the complete Negative Monitoring requirements. While a SCOM administrator can provide an override when it comes to a Host Instance or Receive Location, if someone starts or enables that BizTalk Service no notifications are generated.Synthetic TransactionsNagios has support for Synthetic Transactions and other types of web site monitoring.
SCOM does provide additional features like running Synthetic Transactions from different servers within the environment. SCOM also provides deeper interrogation of these downstream applications by using .Net Application Performance Monitoring.
BizTalk360 provides good capabilities that will satisfy most requirements. Besides checking for expected HTTP return codes, it is also possible to fire custom requests and check for certain responses/response times.MiscellaneousIntegration with other popular Monitoring PlatformsSCOM provides broad integration with many other Monitoring Platforms and Service Desk applications.
BizTalk360 integrates with HP Operations Manager, Slack, ServiceNow, Microsoft Teams and provides a simple and elegant way of doing so. One could also develop custom Notification Channels to integrate with other Service Desk applications.
Nagios has slightly less features in this area and therefore gets the lower scoreComposite DashboardsAll products provide this capability. BizTalk360 gets a higher grade for the simplicity of the tool.
In order to build these types of Dashboards within SCOM, you need to be an advanced user of the system. Nagios Dashboards are not BizTalk orientedWeb based User InterfacesAll products provide Web based interfaces. BizTalk360 gets the top grade because administrators can perform most of their BizTalk related activities with just BizTalk360, while with Nagios/SCOM they also need access to tools like BizTalk Admin Console, SQL Server, Event Viewer, Performance Monitor, portals, etc.Summary ReportsNagios has all kind of reports, but these are not BizTalk oriented
SCOM does not provide a holistic report out of the box. In order to get this type of reports out of SCOM, you need to custom build a report.
BizTalk360 provides a scheduled Summary Report that will give subscribers an indicator that their environment is Healthy. This gives users a confirmation that everything is operating as expected within the environment.GovernanceNagios and SCOM provide both user management, but the no BizTalk operations can be done from these users.
BizTalk360 provides a much finer grained approach to Governance and Auditing.KnowledgebaseNagios doesn’t have a Knowledge Base, but both SCOM and BizTalk360 include the ability to build up a Company Knowledge base.
SCOM also includes a Product Knowledge base provided by the BizTalk Product Group so it gets the higher score in this area.
BizTalk360 provides the capability to associate KB articles to certain events, like error codes of suspended instances etc. and therefore also deserves the highest rank.
Integrate 2017 – Day 2 session on Bringing Logic Apps into DevOps with Visual Studio

Integrate 2017 – Day 2 session on Bringing Logic Apps into DevOps with Visual Studio

During the second day of Integrate 2017, Jeff and Kevin did a session on DevOps for Logic Apps. They explained the available tooling in Visual Studio for that purpose:

  • Hosted Logic App designer
  • Cloud Explorer
  • Resource Group Projects

Hosted Logic App designer – First of all, since some time it is possible to create your Logic Apps from within Visual Studio. This enables you to for example use Team Foundation Services or Visual Studio Services to check-in your Logic Apps code.

Cloud Explorer – Another feature in Visual Studio is the Cloud Explorer which, besides browse and search capabilities for your Logic Apps, the following features:

  • Edits done in the designer can be published to the Azure portal
  • Enable, disable and delete Logic Apps
  • Run Logics Apps and view the Run History
  • Download Logic App which is handy when started development in the portal

After download the Logic App, you can add the Logic App as an existing item to a Resource Group project, which we will discuss now.

Resource Group projects – Visual Studio comes with the Resource Group project type, which allows you to persist artefacts and configuration for your deployments. Amongst the features are:

  • Storing/using parametrization which supports Azure Vault for safely storing credentials
  • Creation of connections like you can in the Azure portal
  • The configuration of your Integration accounts to easily reference schema’s, maps etc.
  • Source control

After you have created such a project, the Azure Resource template contains the stuff and other stuff you need to be in the deployment. Think of for example:

  • Logic Apps which need to be referenced by Id
  • Parameters, for example – different environments
  • Variables which actually are constants
  • Any post deploy output values like the Request Trigger URL

A good starting point is the Azure Quickstart templates, which contains over 500 templates, including a number of templates for creating Logic Apps. An example of such a template is a Logic App which uses an Azure function. So the fun part of using these templates is that basically, your infrastructure and PaaS solutions can be code and be deployed with these templated.

You’ll find these templates here: https://azure.microsoft.com/en-us/resources/templates/

Related Links – Recap of Day 2 at INTEGRATE 2017

Author: Lex Hegt

Lex Hegt works in the IT sector for more than 25 years, mainly in roles as developer and administrator. He works with BizTalk since BizTalk Server 2004. Currently he is a Technical Lead at BizTalk360.

Understanding the BizTalk Deployment Framework

Understanding the BizTalk Deployment Framework

Part 2 – Advanced setup

This is an article in a series of 2 about the BizTalk Deployment Framework (BTDF). This open source framework is often used to create installation packages for BizTalk Applications. More information about this software can be found here CodePlex.
With this series of articles, we aim to provide a useful set of hands-on documentation about the framework.

The series consists of the following parts:

  • Part 1 – Introduction and basic setup
  • Part 2 – Advanced setup (this article)
    • Introduction
    • Custom Targets
    • Conditions
    • Host Groups
    • Executing third party executables
    • Tips and Tricks
    • Conclusion

Introduction

In the first part  of this series we have seen how you can setup a BizTalk Deployment-project, add artefacts to the project, how you can deploy variable settings for other DTAP environments and how you can generate distributable MSI packages.
In this second article we’ll show somewhat more advanced topics like Custom Targets, Conditions, Host Groups and we’ll see how you can run third party executables from within your deployment. We’ll finish with a couple of Tips and Tricks.

Custom Targets

To be able to customize your deployments, the BizTalk Deployment Framework uses a concept called Custom Targets. These Custom Targets allow you to execute commands during certain phases of the deployment. We will describe a couple of Custom Targets and show how you can use them later on as well. The currently available Custom Targets are shown below.
biztalk deployment custom targets

Adding Custom Targets to your Deployment project

Before describing the Custom Targets, let’s first have a quick look on how to add Custom Targets to your Deployment project.
To keep your Deployment project nicely organized, it is a good idea to keep all the custom targets you will be using grouped together below all the BizTalk artefacts you will deploy. So the Custom Targets will be added at the end of your Deployment project.
Below is a picture which shows where the CustomRedist target is added to the Deployment project. This target was actually already added to the project during creation of the Deployment project.
CustomRedist target in deployment project
As you can see, you can simple type the text <Target Name=”…”></Target> and at the ellipsis, provide the name of the Custom Target you will be going to use. Everything you want to perform during the execution of the target, can be entered between the start tag and the end tag. We’ll see examples of the kind of commands which can be performed later in this article.

CustomRedist

This target becomes executed right after BTDF has finished copying files. So this is the ideal moment if you would like for example to add additional files to the MSI package, which will be copied to the installation folder on deployment.
Below you have a sample in which a folder becomes created and some files are copied to that folder:

    <MakeDir Directories="$(RedistDir)SQLScripts" />
    <CreateItem Include="SQLScripts*.*">
          <Output TaskParameter="Include" ItemName="SQLScripts" />
    </CreateItem>
    <Copy DestinationFolder="$(RedistDir)SQLScripts%(RecursiveDir)" SourceFiles="@(SQLScripts)"/>

CustomDeployTarget and CustomPostDeployTarget

The CustomDeployTarget target runs just before Deployment has started, while the CustomPostDeployTarget target runs directly after Deployment has finished. Either can be used if you want to deploy additional stuff. Later we’ll describe how these targets are used to run third party executables, but you might also use these targets to create for example an EventLog source.
CustomUndeployTarget and CustomPostUndeployTarget
The CustomUndeployTarget target runs just before any undeployment has been performed, while the CustomPostUndeployTarget target runs right after the undeployment has taken place. These targets can be used if you would like to clean up stuff after undeployment. You can think of that as cleaning any created files during deployment. All just to be sure that you keep your environment nice and clean.

Conditions

The concepts of Conditions can be used in both Bindingfiles as in the BTDF project file. With Bindingfiles this might be used to create different ports in different environments.
In the BTDF project file this feature can become used when for example, you want to execute certain targets only in certain environments.
As applying Conditions is almost the same for both kind of files, let’s just have a look at the latter one.
Let’s assume that we want to execute an .exe file only for the Test and Production environment. To be able to execute certain targets just for certain environments, you need to do the following:
– Add an Environment variable to your BTDF Settings file
– Add the variable to your BTDF project file
– Add a filter to the CustomPostDeployTarget

Add an Environment variable to your BTDF Settings file

From Visual Studio open the SettingFileGenerator.xml file in Excel and add a variable called Environment. Each environment will get a unique value. Below picture shows a setting file which is extended with such a setting and contains unique values for all environments, namely DEV, SDEV, TEST and PROD.
adding environment variable to btdf settings

Add the variable to your BTDF project file

To be able to use the Environment variable, you need to do the following:
• Open the Deployment.btdfproj file and look for an ItemGroup which contains the value PropsFromEnvSettings. Typically this settings contains the values SSOAppUserGroup and SSOAppAdminGroup.
• Add a semicolon and the new variable Environment. Afterwards the ItemGroup will look like below.

    <ItemGroup>
      <PropsFromEnvSettings Include="SsoAppUserGroup;SsoAppAdminGroup;Environment" />
    </ItemGroup>

Add a filter to the CustomPostDeployTarget

In the final step you’ll arrange that the step which executes the .exe file only gets executed in case you’re deploying on the TEST or PROD environment. Navigate to the target called CustomPostDeployTarget and add the condition: $(Environment)==’TEST’ OR $(Environment)==’PROD’ to it.
When you’ve done that, the target will look as follows:

    <Target Name="CustomPostDeployTarget" Condition="$(Environment)=='TEST' OR $(Environment)=='PROD'">
        <MakeDir Directories="$(RedistDir)SQLScripts" />
        <CreateItem Include="SQLScripts*.*">
                <Output TaskParameter="Include" ItemName="SQLScripts" />
        </CreateItem>
        <Copy DestinationFolder="$(RedistDir)SQLScripts%(RecursiveDir)" SourceFiles="@(SQLScripts)"/>
    </Target>

Now, when you save the project file, create an MSI and execute that MSI, the CustomPostDeployTarget will only be executed when you are deploying to the Test or the Production environment.

Host Groups

In most cases a BizTalk application uses only a couple of Host Instances. After a deployment it is a good practice to restart all relevant Host Instances. BTDF enables you to achieve this by using an BizTalkHosts ItemGroup. If you omit this ItemGroup, all Host Instances will be restarted.
You can put such an ItemGroup in the BTDF-project file. To keep the project file nicely organized you can consider to have that ItemGroup below the BizTalk artefacts you are deploying.

    <ItemGroup>
        <BizTalkHosts Include="SendHost;ReceiveHost;OrchestrationHost" />
    </ItemGroup>

As you can see, you can easily add multiple hosts by using semicolons between the separate hosts.

Executing third party executables

The BizTalk Deployment Framework enables you to execute third party executables. Below you find brief examples to show what you can achieve with this capability.

Executing SQL Server scripts

Say you have a custom database which is used by your BizTalk solution and this database needs to be extended with additional tables and stored procedures. Without using the BizTalk Deployment Framework, you would need to run the necessary SQL Server scripts on each environment by hand. This is a time-consuming and error-prone task which can be automated with BTDF.
To be able to execute SQL Server commands, you need to execute an executable called sqlcmd. Below you find an example on how to execute SQL scripts:

  <Target Name="CustomPostDeployTarget">
    <Exec Command="sqlcmd -S $(SQLServer) -i&quot;SQLScripts<Name of T-SQL deploy script>&quot;" />
  </Target>

As you can see, you can simply add an Exec command to the deployment target.
For all the ins and outs on deploying SQL Server scripts, check this article on TechNet Wiki.

Create BizTalk360 alerts

Another example of running third party executables is the creation of BizTalk360 alarms. Normally you’ll have to do this manually after each deployment of a new or changed BizTalk application. This too can be a time-consuming and error prone task. By using BTDF and a tool called BT360Deploy, you can automate the creation of alarms during the deployment of you BizTalk application.
creating biztalk360 alerts
In the picture here above, you have an example of the output of BT360Deploy. More information on BT360Deploy can be found on CodePlex.

Tips and Tricks

We’ll end this article by providing you with a couple of Tips and Tricks

Do not automatically start the Application after deployment

BTDF allows you to automatically start the deployed BizTalk application, i.e. start all ports and orchestrations after deployment. Although that sounds like a convenient feature, especially in Dev and Test environments, your BizTalk administrator probably prefers that you leave the just deployed BizTalk application not started on Production environments. The reason for this is that in this way the BizTalk Administrator has better control over the moment when the BizTalk application must be activated.

Consider (not) restarting Host Instance

We have seen that BTDF can restart the Host Instance(s) after deployment. But in case you use scripting for unattended deployment of multiple BizTalk applications at the same time, you don’t want the Host Instances restarted after each deployment package as this is a waste of time and resources. In that case, you remove the ItemGroup that contains the BizTalkHosts. Instead you set the SkipHostInstancesRestart property in the first PropertyGroup of the project file to True:

<SkipHostInstancesRestart>False</SkipHostInstancesRestart>

ItemGroups per artefact

To keep your BTDF project file organized, it is preferable that you use separate ItemGroups for all the separate types of artefacts you will be deploying.

  <ItemGroup>
    <Schemas Include="Kovai.BTDF.Schemas.dll">
      <LocationPath>..$(ProjectName).Schemasbin$(Configuration)</LocationPath>
    </Schemas>
    <Schemas Include="Kovai.BTDF.MoreSchemas.dll">
      <LocationPath>..$(ProjectName).MoreSchemasbin$(Configuration)</LocationPath>
    </Schemas>
  </ItemGroup>

  <ItemGroup>
    <Transforms Include=" Kovai.BTDF.Maps.dll ">
      <LocationPath>..$(ProjectName).Mapsbin$(Configuration)</LocationPath>
    </Transforms>
    <Transforms Include=" Kovai.BTDF.MoreMaps.dll ">
      <LocationPath>..$(ProjectName).MoreMapsbin$(Configuration)</LocationPath>
    </Transforms>
  </ItemGroup>

  <ItemGroup>
    <Orchestrations Include=" Kovai.BTDF.Orchestrations.dll ">
      <LocationPath>..$(ProjectName).Orchestrationsbin$(Configuration)</LocationPath>
    </Orchestrations>
    <Orchestrations Include=" Kovai.BTDF.MoreOrchestrations.dll ">
      <LocationPath>..$(ProjectName).MoreOrchestrationsbin$(Configuration)</LocationPath>
    </Orchestrations>
  </ItemGroup>

Conclusion

In a series of 2 articles we have seen quite a lot of the capabilities of the BizTalk Deployment Framework. In part 1 we started with pointing out the advantages of using BTDF over the out-of-the-box capabilities of BizTalk Server and we created a simple Deployment project which showed how to setup a deployment with basic artefacts like Schemas, Maps and Orchestrations. We extended this project by creating variables to be used for environment specific settings like URL’s of endpoints.
In part 2 we have seen some more advanced features like Conditions and Host Groups. We have also seen how we can run third party executables to automate your deployment as far as possible and to decrease the amount of manual and error prone labour as much as possible.
Still there are much more possibilities with BTDF which we did not discuss. For example, you can also use it to create virtual directories, deploy SSO settings, create EventLog Event Sources, create Registry keys etc. etc. etc..

It must be clear by now that the BizTalk Deployment Framework is a very powerful framework that can be used with straight forward to complex deployments. So when you are already familiar with the framework and are using it for the deployment of standard BizTalk artefacts, why not expand your experience with BTDF and start deploying less common artefacts like SQL scripts. When you do that properly, your BizTalk Administrator might become your new best friend!

Happy deploying!

Author: Lex Hegt

Lex Hegt works in the IT sector for more than 25 years, mainly in roles as developer and administrator. He works with BizTalk since BizTalk Server 2004. Currently he is a Technical Lead at BizTalk360.

Understanding the BizTalk Deployment Framework – Introduction

Understanding the BizTalk Deployment Framework – Introduction

This is an article in a series of 2 about the BizTalk Deployment Framework (BTDF). This open source framework is often used to create installation packages for BizTalk Applications. More information about this software can be found here CodePlex. With these series of articles, we aim to provide a useful set of hands-on documentation about the framework.

The series consists of the following parts:

  • Part 1 – Introduction and basic setup (this article)
    • Introduction
    • BizTalk out-of-the-box capabilities
    • Top reasons to use BTDF
    • Setting up a BTDF-project
    • Conclusion
  • Part 2 – Advanced setup

Introduction

In this first part, we will shortly discuss the shortcomings of the deployment packages which can be created with the out-of-the-box capabilities of BizTalk Server and introduce you to the benefits of BTDF. Further this article will show you how you can create a BTDF-project for the deployment of basic BizTalk artifacts like Schema’s, Maps and Orchestrations and how you can deploy variable settings for other DTAP environments. We will end this article with showing you how you can generate distributable MSI packages.

BizTalk out-of-the-box capabilities

Microsoft BizTalk Server comes with capabilities to create deployment packages. However, the MSI packages you create with the out-of-the-box features of BizTalk Server have some shortcomings which lead to deployments that are time consuming and error-prone. The main reasons for this are:

  • The packages can only contain URI’s of Endpoints (Receive Locations and Send Ports) of a single DTAP environment. So DTAP promotion from the same package is not possible.
  • Only deployment of BizTalk artifacts is possible. So if you need, for example, to create and populate database tables, this cannot be automated.

Top reasons to use BTDF

As you’ll imagine, BTDF has eliminated these shortcomings, because of which the framework has been widely adopted. Below you’ll find a list which contains a number of reasons why you could use this framework:

  • Deploy complex BizTalk solutions containing BizTalk artifacts but also for example SSO settings, ESB itineraries and SQL Server scripts, with no human intervention
  • Consolidate all of your environment-specific configuration and runtime settings into one, easy-to-use Excel spreadsheet
  • Maintain a single binding file that works for all deployment environments
  • Make automated deployment a native part of the BizTalk development cycle, then use the same script to deploy to your servers

Setting up a BTDF-project

After you have downloaded and installed BTDF on your Development box, you can start with setting up your BTDF-project. To get that working, you need to take the following steps:

  • Add a BTDF project to your BizTalk solution
  • Add BizTalk artifacts to the BTDF project
  • Deploy from Visual Studio
  • Generate a master PortBinding file
  • Apply Environment Specific Settings
  • Generate a MSI

Add a BTDF project to your BizTalk solution

Firstly, you need to open a BizTalk solution in Visual Studio and perform the following steps:

  • Right-click the Solution Explorer and select Add/New Project…
  • In the dialog that opens, in the left hand pane, navigate to BizTalk Projects
  • In the right pane, select Deployment Framework for BizTalk Project, give the project the name Deployment and click the OK button to continue the creation of the project.

Add New Project BTDF

In the screen that follows you can configure which artifacts you want to deploy. Perform the following:

  • Double-click on the text True after Deploy Settings into SSO. It will be set to False
  • Uncheck the check mark ‘Write properties to the project file only when…’
  • Click the Create Project button

Create BTDF project

After the project has been created, you’ll get a small dialog box, which states that you need to edit the file Deployment.btdfproj to configure your specific deployment requirements. And that’s exactly what we are going to do in a couple of minutes! For now, just click the OK button.

BTDF Project Created

Although the aforementioned project file is shown in the left part of the screen, the Solution Explorer does not yet show the newly created Deployment project. To have the project shown up there, you need to manually add the project file and some other files to the solution in a Solution folder.

BTDF Project in Solution Explorer

Perform the following actions to add the Deployment project to the solution:

  • Right-click on the Solution Explorer, select Add/New Solution Folder, enter Deployment and hit the Enter key
  • Right-click on the Deployment folder, select Add/Existing Item… and in the dialog box that appears double-click the folder Deployment
  • Select all the files and click on the Add button

The Deployment project is now added to your BizTalk solution and that project in the Solution Explorer looks like below.

BTDF Solution Explorer

Note: Take care that it is important to name that solution folder Deployment, otherwise Visual Studio won’t be able to find the Deployment.btdfproj file.

Add BizTalk artifacts to the BTDF project

Now we have the Deployment project in place, we are going to add some basic BizTalk artifacts to the Deployment project. To start we’ll add the following types of artifacts:

  • Schemas
  • Maps
  • Orchestrations

To keep the Deployment project file nicely organized, we will create an ItemGroup for each type of artifact. As we start with 3 types of artifacts, we’ll end up having 3 ItemGroups. Probably it will be clear by now that ItemGroups can be used to organize the artifacts you want to deploy. An ItemGroup is an element from MSBuild which is used by BTDF. Besides ItemGroups, you will also find PropertyGroups within the BTDF project file. PropertyGroups are used to organize all kind of parameters which can be used during deployment. Think of this for example of the name of the project, but also which kind of artifacts need to be deployed.

The project file already contains an ItemGroup for deploying schemas. We will use this as a template for our own purposes.

Follow below instructions to add schemas, maps and orchestrations to the deployment project.

  • In the Deployment project file, search for the ItemGroup that is used for deploying Schemas. It contains the text ‘<Schemas Include=”YourSchemas.dll”>’.
  • Replace ‘YourSchemas.dll’ with the name of the DLL which contains your schemas
  • The line with the ‘<LocationPath>’ tag, must contain the path where the DLL resides

The ItemGroup will now look like below:

  <ItemGroup>
    <Schemas Include="Kovai.BTDF.Schemas.dll">
      <LocationPath>..$(ProjectName).Schemasbin$(Configuration)</LocationPath>
    </Schemas>
  </ItemGroup>

Next we’ll prepare deployment of the BizTalk Maps.

  • Below the ItemGroup to deploy Schemas, create a new ItemGroup to deploy Maps. Therefore, you can copy/paste below XML:
  <ItemGroup>
    <Schemas Include="Kovai.BTDF.Schemas.dll">
      <LocationPath>..$(ProjectName).Schemasbin$(Configuration)</LocationPath>
    </Schemas>
  </ItemGroup>

Now we are going to prepare deployment of the BizTalk Orchestrations.

  • Below the ItemGroup to deploy Maps, create a new ItemGroup to deploy Orchestrations. Therefore, you can copy/paste this XML:
<ItemGroup>
    <Orchestrations Include=" Kovai.BTDF.Orchestrations.dll ">
      <LocationPath>..$(ProjectName).Orchestrationsbin$(Configuration)</LocationPath>
    </Orchestrations>
  </ItemGroup>

If you have multiple projects of for example orchestrations, you can simple copy/paste the <Orchestrations>…</Orchestrations> block and provide it with the correct Include-parameter and location path. Of course, this also works the same for other types of artifacts like schemas, maps, pipelines etc.

Deploy from Visual Studio

We now have deployment for some basic artifacts in place, so let’s deploy the BizTalk solution and see what happens:

  • Right-click the solution (in the solution Explorer) and select Build Solution
  • When the solution is built without errors, you can deploy the solution, therefore just click the little green arrow in the upper left are of Visual Studio:

Deployment from Visual Studio

If everything runs like expected the Kovai.BTDF application is created and the schemas, the map and the orchestration will be deployed properly, but you’ll end up with the following error:

Could not enlist orchestration ‘Kovai.BTDF.Orchestrations.BizTalk_Orchestration1,Kovai.BTDF.Orchestrations, Version=1.0.0.0, Culture=neutral, PublicKeyToken=72ff5decf00d3ffb’. Could not enlist orchestration ‘Kovai.BTDF.Orchestrations.BizTalk_Orchestration1’. All orchestration ports must be bound and the host must be set.

This is due to the fact that the orchestration Orchestration1 contains ports which will have to be configured and bound in the BizTalk Administration Console. Let’s just do that and afterwards export the Binding file into your Deployment project.

  • Open the BizTalk Server Administration Console and navigate to the BizTalk application called BTDF
  • Right-click the application and click Configure. The following screen appears:

Configure Application

  • Select a Host in the designated dropdown box
  • Create a Receive Port and a Receive Location (use the FILE adapter) and bind it to rcvPort
  • Create a Send Port (use the FILE adapter) and bind it to sndPort
  • Click the OK button
  • Right-click the application again and click .. and in the dialog box that appears, click Start again. The application should now be started properly

Generate a master PortBinding file

We now have a running BizTalk application which includes bindings between the deployed orchestration and a couple of ports. In the next step we will make these bindings available to our deployment project. Therefore, we need to export the bindings.

Again Right-click the application Kovai.BTDF and now select Export/Bindings…

Export Bindings

  • Click on the ellipsis button (…), navigate to your Deployment project, select xml, click the OK button and confirm that you want to overwrite the existing file
  • Switch to Visual Studio and notice that you are prompted with the following message:

Edit PortBindingMaster XML file

  • Click Yes to reload the PortBindingsMaster.xml file
  • Open PortBindingsMaster.xml and notice that it contains the bindings you created earlier

Apply Environment Specific Settings

For just local deployments you now have the basics in place. However, in most scenarios you will have to deploy your BizTalk solution(s) to multiple DTAP environments. In that case, the addresses of your Receive Locations and Send Ports will differ between environments. BTDF facilitates this by providing a XML file which can be edited in Microsoft Excel.

Maintaining DTAP environments

That XML file, named SettingsFileGenerator.xml, can contain columns for each DTAP environment to which you have to deploy your BizTalk solution. Below you find a screen shot of how such a file looks like by default.

BTDF XML file

As you can see, there are columns for multiple environments; Local Development, Shared Development, Test and Production. You are free to rename the values in the existing columns, or add/remove columns.

Maintaining environment specific settings

In the spreadsheet there currently are 2 settings defined; SsoAppUserGroup and SsoAppAdminGroup. You can add more settings in the first column and give them values for the different environments in the following columns. If the value is the same for two or more environments, you could enter a value in the column Default Values, as this value will be used in case no value is entered for one or more environments.

The screenshot below shows a Settings file that now contains values for the addresses of the Receive Location and the Send Port. As you can see, there are Default values entered and the columns for Local Development and Shared Development are left empty, so while deploying to those environments, the Default values will be used.

Maintain Environment Specific Settings

To be able to use for example the SP_Address variable from the settings file in the PortBindingsMaster, you need to open the PortBindingsMasterFile.xml file, navigate to the Send Port which should use the addresses and replace the current address with ${SP_Address}.
So before that part of the PortBindingsMaster.xml file looks like this:

    <SendPort Name="SendPort2" IsStatic="true" IsTwoWay="false" BindingOption="1">
      ...
      <PrimaryTransport>

<Address>C:BizTalkKovai.BTDFOut%MessageID%.xml</Address>

      ...
    </SendPort>

After the change the XML will look like below:

<SendPort Name="SendPort2" IsStatic="true" IsTwoWay="false" BindingOption="1">
      ...
      <PrimaryTransport>

<Address>${SP_Address}%MessageID%.xml</Address>

      ...
    </SendPort>

Generate a MSI

So far, we have only done deployments through Visual Studio. When we have to deploy our BizTalk solutions to non-development servers, we do that by installing MSI’s. BTDF is also able to generate MSI’s. Besides the BizTalk artifacts which need to be deployed, these MSI’s can also contain custom components, binding files for your entire DTAP environment and all kind of custom scripts, which can also be started during deployment.

Generating a MSI is done from Visual Studio, just choose one of below steps to generate a MSI:

  • Click on the below button in the toolbar

BTDF Deployment

  • In the Visual Studio menu, select Tools/Deployment Framework for BizTalk/Build Server Deploy MSI

Both options will provide you with a MSI which you can use to deploy the BizTalk application throughout your DTAP environment.

Conclusion

In this first article we have seen how the BizTalk Deployment Framework differs from the out-of-the-box capabilities of BizTalk Server and described the benefits of BTDF. We’ve also setup a simple deployment project in an existing BizTalk solution and shown how you can add some basic BizTalk artifacts to that project. Further we have seen how we can create different settings for, for example, Receive Location addresses and how they become deployed to multiple environments. Finally, we’ve seen deployment from Visual Studio and we’ve generated a MSI based on the BTDF project.

Explore best practices for deploying BizTalk Server 2016 from Sandro Pereira’s Whitepaper.

In the next article, we’ll get familiar with amongst others Custom Targets, Conditions and see how we can run third party executables to automatically deploy non-standard artifacts like SQL Server scripts or even automatically create BizTalk360 alerts from within your deployment project!

Check out the 2nd article of the series here.

Author: Lex Hegt

Lex Hegt works in the IT sector for more than 25 years, mainly in roles as developer and administrator. He works with BizTalk since BizTalk Server 2004. Currently he is a Technical Lead at BizTalk360.

How to create BizTalk360 Alerts during deployment with BTDF

How to create BizTalk360 Alerts during deployment with BTDF

As you probably know, it is considered a best practice to use the BizTalk Deployment Framework for automatic deployment of your BizTalk solutions. This Framework doesn’t only allow for deployment of BizTalk artifacts, but it also enables you to perform all kind of custom tasks to automate the deployment of your BizTalk application, thereby reducing the risk of error which is introduced when you’re deploying manually. Until recently it wasn’t possible to automate the deployment of BizTalk360 alerts with BTDF, but it is now…

BizTalk360 API

As of BizTalk360 v8, which was released early February 2016, we now have a documented API available, enabling developers to build their own custom solutions by implementing the BizTalk360 API. With this JSON-based API you can operate both BizTalk360 and BizTalk.

The API consists of the following services:

  • ActivityMonitoringService – List of Business Activity Monitoring Services
  • AdminService – List of API’s related to Admin(Super user) activities in BizTalk360
  • AdvancedEventViewerService – List of API’s helps to retrieve details about Advanced Event Viewer
  • AlertService – List of API’s used for Monitoring in BizTalk360
  • BizTalkApplicationService – List of API’s helps to retrieve information about BizTalk Applications
  • BizTalkGroupService – List of API’s helps to get details about the Artifacts in BizTalk Group
  • BizTalkQueryService – List of API’s helps to run queries BizTalk
  • EDIManagementService – List of API’s helps to run EDI queries and to get details about parties and agreements
  • EnvironmentMgmtService – List of API’s related to environment management services in BizTalk360
  • ESBManagementService – List of API’s helps to retrieve details about Enterprise Service Bus

Combined, these services contain many operations, which are used by BizTalk360 itself, but can also be used in custom applications. In this blog post I will describe an example of such a custom application, which automatically creates a BizTalk360 Alert during deployment of a BizTalk application.

If you have BizTalk360 v8 Platinum, Partner, Product Specialist or if you have a Trial license and you want to find out more about the BizTalk360 API, just navigate to Settings / API Documentation. As you’ve probably come to expect from a modern API, you can try each operation directly from the API Documentation.

Note: although you will need one of the above licenses to be able to access the API Documentation, you don’t need a specific license to be able to actually use the API itself!

Deployment of BizTalk applications

In my experience, most organizations that use BizTalk Server as their middleware platform, also use BTDF to deploy their BizTalk solutions. The most important reason to use this open source software over the out-of-the-box capabilities that BizTalk Server offers, is that BTDF enables you to minimize the (sometimes large) number of manual steps required to deploy a full integration solution, thereby significantly reducing or even fully eliminating the risk of human error. Think single-click deployments (your admins will love you for this)!

An important component of BTDF is the SettingsFileGenerator.xml file. In this file, which can be edited with Microsoft Excel, you can define all of the specific settings for each environments in a DTAP street. An example of these type of setting are the URLs of Send Ports, that will be different in a Production environment when compared to a test environment for instance.

Besides being able to deploy all kinds of BizTalk artifacts, the open architecture of BTDF enables you to execute SQL scripts, deploy webservices in IIS and much more. (I actually wrote a blogpost a while ago about how PL/SQL scripts can be executed during the deployment of a BizTalk application).

BT360Deploy

Now that we have a flexible deployment framework with BTDF and a documented BizTalk360 API, it’s only a small step to create a tool that takes advantage of this powerful combination, which will create a BizTalk360 Alert for a BizTalk application during a deployment with BTDF! All this piece of software needs to be capable of doing is reading settings from the BTDF file which contains environment specific settings file and call the BizTalk360 API to create that alert. And the best news is: we’ve already done that for you and we’re offering it for free on CodePlex!

It’s called BT360Deploy, and it works as follows:

BT360Deploy takes 2 arguments, namely:
-a: the name of the BizTalk Application which is deployed
-s: the name and location of the (environment specific) settings file which contains all kind of BizTalk360 parameters

Example:

BT360Deploy -aAxonOlympus.BTDF -sC:DataDevelopmentBizTalkAxonOlympus.BTDFDeploymentEnvironmentSettingsExported_LocalSettings.xml

Below you see the output of the deployment of a BTDF project, in which BT360Deploy was incorporated, with the parameters as shown above:

To be able to create BizTalk360 Alerts during deployment, you need to know which fields in the BizTalk360 User Interface correspond with which fields in the BTDF Settings file. For that purpose we created a document which describes that mapping. Below you will find a sample with a couple of fields. The full document can be downloaded from CodePlex, as part of the BT360Deploy package.

Screen/Field in BizTalk360 User InterfaceField in BTDF Settings file
Alarm – Basic / Basic Details
Alarm NameBizTalk360_alertName
Email IdsBizTalk360_commaSeparatedEmails
Disable Alarm for MaintenanceBizTalk360_isAlertDisabled
Alarm – Threshold / Threshold Alert
Alert on threshold violationBizTalk360_isAlertASAP
If violation persists forBizTalk360_alertASAPWaitDurationInMinutes
Limit number of alerts per violation toBizTalk360_isContinuousErrorRestricted
Notify when things become normal againBizTalk360_isAlertOnCorrection
Set Alerts on set day(s) and time(s) onlyBizTalk360_isThresholdRestricted
DaysBizTalk360_thresholdDaysOfWeek
Start TimeBizTalk360_thresholdRestrictStartTime
End TimeBizTalk360_thresholdRestrictEndTime

Are you interested in creating your BizTalk360 Alerts during deployment, instead of having to create them manually afterwards? Check out the following link on CodePlex to get your hands on it:

http://bt360deploy.codeplex.com

Besides some binaries, the download contains the earlier mentioned mapping document and a ‘How to…’ document which describes how you can incorporate BT360Deploy in your BTDF project.
Enjoy!

How to create BizTalk360 Alerts during deployment with BTDF

How to create BizTalk360 Alerts during deployment with BTDF

As you probably know, it is considered a best practice to use the BizTalk Deployment Framework for automatic deployment of your BizTalk solutions. This Framework doesn’t only allow for deployment of BizTalk artifacts, but it also enables you to perform all kind of custom tasks to automate the deployment of your BizTalk application, thereby reducing the risk of error which is introduced when you’re deploying manually. Until recently it wasn’t possible to automate the deployment of BizTalk360 alerts with BTDF, but it is now…

BizTalk360 API

As of BizTalk360 v8, which was released early February 2016, we now have a documented API available, enabling developers to build their own custom solutions by implementing the BizTalk360 API. With this JSON-based API you can operate both BizTalk360 and BizTalk.

The API consists of the following services:

  • ActivityMonitoringService – List of Business Activity Monitoring Services
  • AdminService – List of API’s related to Admin(Super user) activities in BizTalk360
  • AdvancedEventViewerService – List of API’s helps to retrieve details about Advanced Event Viewer
  • AlertService – List of API’s used for Monitoring in BizTalk360
  • BizTalkApplicationService – List of API’s helps to retrieve information about BizTalk Applications
  • BizTalkGroupService – List of API’s helps to get details about the Artifacts in BizTalk Group
  • BizTalkQueryService – List of API’s helps to run queries BizTalk
  • EDIManagementService – List of API’s helps to run EDI queries and to get details about parties and agreements
  • EnvironmentMgmtService – List of API’s related to environment management services in BizTalk360
  • ESBManagementService – List of API’s helps to retrieve details about Enterprise Service Bus

Combined, these services contain many operations, which are used by BizTalk360 itself, but can also be used in custom applications. In this blog post I will describe an example of such a custom application, which automatically creates a BizTalk360 Alert during deployment of a BizTalk application.

If you have BizTalk360 v8 Platinum, Partner, Product Specialist or if you have a Trial license and you want to find out more about the BizTalk360 API, just navigate to Settings / API Documentation. As you’ve probably come to expect from a modern API, you can try each operation directly from the API Documentation.

Note: although you will need one of the above licenses to be able to access the API Documentation, you don’t need a specific license to be able to actually use the API itself!

Deployment of BizTalk applications

In my experience, most organizations that use BizTalk Server as their middleware platform, also use BTDF to deploy their BizTalk solutions. The most important reason to use this open source software over the out-of-the-box capabilities that BizTalk Server offers, is that BTDF enables you to minimize the (sometimes large) number of manual steps required to deploy a full integration solution, thereby significantly reducing or even fully eliminating the risk of human error. Think single-click deployments (your admins will love you for this)!

An important component of BTDF is the SettingsFileGenerator.xml file. In this file, which can be edited with Microsoft Excel, you can define all of the specific settings for each environments in a DTAP street. An example of these type of setting are the URLs of Send Ports, that will be different in a Production environment when compared to a test environment for instance.

Besides being able to deploy all kinds of BizTalk artifacts, the open architecture of BTDF enables you to execute SQL scripts, deploy webservices in IIS and much more. (I actually wrote a blogpost a while ago about how PL/SQL scripts can be executed during the deployment of a BizTalk application).

BT360Deploy

Now that we have a flexible deployment framework with BTDF and a documented BizTalk360 API, it’s only a small step to create a tool that takes advantage of this powerful combination, which will create a BizTalk360 Alert for a BizTalk application during a deployment with BTDF! All this piece of software needs to be capable of doing is reading settings from the BTDF file which contains environment specific settings file and call the BizTalk360 API to create that alert. And the best news is: we’ve already done that for you and we’re offering it for free on CodePlex!

It’s called BT360Deploy, and it works as follows:

BT360Deploy takes 2 arguments, namely:
-a: the name of the BizTalk Application which is deployed
-s: the name and location of the (environment specific) settings file which contains all kind of BizTalk360 parameters

Example:

BT360Deploy -aAxonOlympus.BTDF -sC:DataDevelopmentBizTalkAxonOlympus.BTDFDeploymentEnvironmentSettingsExported_LocalSettings.xml

Below you see the output of the deployment of a BTDF project, in which BT360Deploy was incorporated, with the parameters as shown above:

To be able to create BizTalk360 Alerts during deployment, you need to know which fields in the BizTalk360 User Interface correspond with which fields in the BTDF Settings file. For that purpose we created a document which describes that mapping. Below you will find a sample with a couple of fields. The full document can be downloaded from CodePlex, as part of the BT360Deploy package.

Screen/Field in BizTalk360 User InterfaceField in BTDF Settings file
Alarm – Basic / Basic Details
Alarm NameBizTalk360_alertName
Email IdsBizTalk360_commaSeparatedEmails
Disable Alarm for MaintenanceBizTalk360_isAlertDisabled
Alarm – Threshold / Threshold Alert
Alert on threshold violationBizTalk360_isAlertASAP
If violation persists forBizTalk360_alertASAPWaitDurationInMinutes
Limit number of alerts per violation toBizTalk360_isContinuousErrorRestricted
Notify when things become normal againBizTalk360_isAlertOnCorrection
Set Alerts on set day(s) and time(s) onlyBizTalk360_isThresholdRestricted
DaysBizTalk360_thresholdDaysOfWeek
Start TimeBizTalk360_thresholdRestrictStartTime
End TimeBizTalk360_thresholdRestrictEndTime

Are you interested in creating your BizTalk360 Alerts during deployment, instead of having to create them manually afterwards? Check out the following link on CodePlex to get your hands on it:

http://bt360deploy.codeplex.com

Besides some binaries, the download contains the earlier mentioned mapping document and a ‘How to…’ document which describes how you can incorporate BT360Deploy in your BTDF project.
Enjoy!

Oracle PL/SQL scripts with BizTalk Deployment Framework

Often the deployment of BizTalk solutions consists of more than deploying just BizTalk Applications. For example, when Oracle databases are involved it might be necessary to create and fill new tables and deploy new stored procedures and/or packages. In our daily practice most of the times the Oracle DBAs perform these deployments. But by incorporating these tasks into your BTDF-project, less coordination with other teams is necessary, and the quality of the deployment is improved.