Wow! In Europe Integration will be on fire in May, not only we will have TUGA IT but also INTEGRATE 2016 (but I leave this last event for another post). Tuga IT 2016 it’s a FREE event (Saturday, 21), that will count with the collaboration of different Portuguese technical communities and Microsoft MVP’s (myself included), […]
Blog Post by: Sandro Pereira
Much has happened in the world of Logic Apps and API Apps since the original announcement back in December of 2014. We have seen the continued development of SaaS connectivity within the product, along with the overall expansion of integration capabilities. We have also seen the team responding to customer feedback actively while maintaining transparency in the process, and even providing a roadmap to give insight into what is coming and when we can expect to see the sweet moment that is GA.
Sometimes, customer feedback causes fairly large shifts in the underlying product. Such is the case seen in the latest updates for the product in the form of a completely overhauled designer, new feature support for triggering flows (i.e., any action can be a trigger), and an API deployment model that is more consistent with the rest of App Service and does not require a dedicated gateway.
One of the most obvious changes that will stick out immediately as you go out to create a Logic App is the new designer that moved over into App Service from Power Apps.
The new designer supports editing workflows build using the updated workflow language (schema version: 2015-08-01-preview), and sports a vertical layout, rather than a horizontal layout, and conditions that appear to wrap around actions instead of being embedded inside actions (though the code view demonstrates that the underlying behavior is similar).
You might also notice that the experience of adding actions is much quicker, as this act no longer provisions a new instance of an API App within your own subscription. Instead in an interesting reversal, Microsoft hosts managed instances of out-of-the-box API Apps. The result is that configuration information is sent as part of the request, instead of stored inside the API App container, and you will have far more simplified ARM deployment templates given that your deployment will no longer need to take into account each API App used by your Logic Apps.
So how do my own custom API Apps end up in the list? Well, you can apply to have them registered in the Azure Marketplace, or you can use the Http + Swagger action in order to point to a custom API App that already exists. Of course that brings us to the question of what it looks like to actually build a custom API App in this refresh of the preview.
In the preview refresh, the process to develop and consume a custom API using the designer is quite a bit different. You still have the ability to use swagger extensions for a clean designer experience – but there are new extensions intended to take advantage of new designer capabilities. These capabilities include things like dynamic schemas for parameters / return types of API (imagine a different object shape depending on the type of entity within a CRM system, or a table within a database), and dynamic values for enumerations.
The biggest change here though is that we no longer have a gateway managing authentication, internal storage, or configuration for our APIs, and get to manage that ourselves, but as a side effect, we’re no longer constrained by where our APIs live – all APIs get the same first class experience.
I would definitely recommend taking some time to read each link within this article before starting out on building a new API. I’m working on building out updates to T-Rex to help with the metadata – while also providing a few example APIs to take advantage of all of the new capabilities – but if you want a head start, the knowledge is out there!
What other changes are under the hood that you should know about? Well, you may have noticed the announcement of the availability of Webhooks for Logic apps for one, or even saw the x-ms-trigger extension called out in the article linked above. The end result of this is that any action within a Logic App can have a polling trigger style behavior, or even an async push style behavior, and the Logic App itself can be triggered manually at an endpoint that isn’t tied to a specific Azure subscription.
We can see some of these changes in action as we look at actions like the Send approval email action from the Office 365 connector/API. The action sends an email, and then notifies the Logic App what the response is when the response is available – without polling.
It even includes the shape of the notification as part of the swagger metadata that is exposed, so that the designer can support using the shape of that async output in later steps. The result is that as a developer, I can use the action to build what looks like a synchronous flow without the complexity of an async flow, and yet I’m benefitting from the performance characteristics of the async implementation (i.e., immediate notification when the event happens rather than polling at a fixed or variable interval).
Reading about all of this might have you wondering what QuickLearn Training is doing about all of this, and/or what you should do about all of this.
Well, I (Nick Hauenstein) am hard at work on an update to QuickLearn’s T-Rex metadata library that takes into account the new way to build API Apps. I’m on target to wrap up the core code by end of week, and hopefully have some decent sample apps out there shortly thereafter.
We’re all busy learning everything we can about the new functionality so that we can rapidly integrate those changes into our Cloud-Based Integration Using Azure App Service course.
In the meantime, keep an eye out for announcements from the BizTalk 360 folks about Integrate 2016 Europe. You might be able to meet up with myself (Nick Hauenstein), Rob Callaway, or John Callaway to talk about BizTalk Server, or any of the things in this post. Also watch for the next release of TRex on NuGet which will include support for all of the new goodies we have available in Logic Apps.
In the meantime, take care, and have fun building great things!
It’s becoming more and more common these days with organisations moving their workload into the cloud. In theBizTalk server scenario, we have noticed a lot of our customers using Microsoft Azure to run their BizTalk environments, especially for development and test purposes. There are alot of advantages in using Microsoft Azure for your BizTalk environment. […]
The post Introducing BizTalk360 Azure Easy Installer appeared first on BizTalk360 Blog.
Blog Post by: Saravana Kumar
I still remember when I first I heard Microsoft was working on Project Siena (which later became PowerApps), the first thing which popped up in my mind was Visual Studio Lightswitch, which allows us to easily create business applications. Anyways I am digressing J … I might blog about Lightswitch vs PowerApps in the near future, otherwise just attend the upcoming Global Azure Bootcamp in Sydney, and ask me in person.
On February 22nd I gave a closet presentation on PowerApps on IntegrationUsergroup.com. Yeah you read it right, a closet presentation J. Just imagine the following, to get a better understanding…
6 am Sydney, as the sun rises the temperature slowly increases. I just got my first cup of Coffee. I hooked up my laptop to the Big Screen TV. While the laptop is booting I take a sip of my Coffee and put on my Bluetooth headset. After logging in, I perform a final sound check and open up my power point presentation. Once everything seems to be working, I connect to the IntegrationUserGroup webinar. There I am, sitting alone in the living with a cup of coffee, ready to present to a virtual audience.
darn, I’ve been digressing again. Ok where was I. O yeah PowerApps
Before I dive a bit more into PowerApps, lets briefly look at why Microsoft has created the PowerApps platform.
In this Mobile First, Cloud First world we are surrounded with thousands if not millions of mobile apps, but only a fraction of these apps are truly mobile business apps.
So what is the reason for these mobile business app type apps to be lacking behind? Well that’s the question Microsoft asked, and they were able to narrow it down for the following reasons:
Mainly because one will have to target devices running in multiple form factors (phone, phablet, tablet, laptop, desktop) across multiple operating systems (iOS, Android, Windows)
Company data is nowadays stored virtually anywhere. Some data might be stored in a SaaS application hosted somewhere in the cloud, while other data might be stored on premise. Accessing this data and integrating these systems is not an easy job.
Once the application(s) have been developed they need to be deployed to a user’s device. Currently in order to be able to do this an application has to be published to an official market-place, and in case we target multiple operating systems (iOS, Android, Windows) we will have to target multiple market-places, with their own processes.
An alternative to the above would of course be side-loading apps, or setting up company market-places. However not all platforms might allow this.
it is pretty damn hard to not only create, integrate and deploy business apps it is most likely takes time, hard work and money to develop and maintain these apps.
Concluding we could say
If you are able to build a business app within minutes, and deploy it to all kinds of devices regardless of their operating systems. You should have a pretty darn valuable business proposition.
Wait…Having said the above…
In short PowerApps is Microsoft’s answer to address the business app gap, it does so by offering a platform which includes tooling to enable employees, developers, and integrators to create and share mobile business apps. These apps work on phones, tablets or desktops and they work across iOS, Windows and soon Android and allow to seamlessly connect to disparate data sources spanning both on-prem and the cloud in a secure way viagra en belgique.
Microsoft PowerApps was announced a few months ago and so far getting access to the preview is on an invitation basis only. You can go to https://powerapps.microsoft.com/en-us/ and request an invite.
Pfff, enough of this blurb stuff, let’s cut to the chase. (click here for some more technical details on what PowerApps are)
Easy answer, why shouldn’t you. No seriously. In my opinion, if you as a business have an active ‘power user’ base who are more than comfortable with Excel and Access and want to quickly leverage ‘intranet’ like business apps which boost productivity or simply allow user to gain quick access, anytime and anywhere to business information, PowerApps is the platform to go for, your imagination is your limitation.
Okay, okay; it would be ideal if the following infrastructure and solutions/platforms are being leveraged by your company: Azure Active Directory Tenant, Dropbox, Dynamics CRM Online, Google Drive, Microsoft Translator, Office 365 Outlook, Office 365 Users, OneDrive, SQL Azure, Salesforce, SharePoint Online, Twitter or any publically facing rest API (preferably with a swagger definition)
In short, if you meet the above requirements, go sign up for PowerApps and start prototyping.
Lacking inspiration, well why not build a PowerApp which allows users to report on ‘Hazardous situations’?
So this app would allow users, if they see a hazardous situation to instantly report this by describing the situation, attaching a picture and the exact location (using GPS). Once reported the responsible business unit can take action and once resolved the one who reported would be notified that the hazardous situation is resolved.
Sounds too good to be true; nah in my following post. I’ll show you how to build and deploy this within an hour; Yeah using the Free or Standard version
In my presentation I identify 2 types of business apps which make prime candidates for PowerApps.
The differences between these two apps are best to be explained by listing a few examples
Intranet like mobile business apps, are typically mobile business apps which offer intranet like functionality and usually contain (if any) a simple workflow (logic flows AKA Logic App) such as
Line of business like mobile business apps, typically would expose and tap into core business processes, have more complex workflows (logic flows AKA Logic App) which could span multiple back-end systems, typically these business apps would want to leverage functionality which is contained within the space of
This latter type of mobile business apps, usually takes some more time to develop and in my opinion requires a good ‘Design’ process and of course special attention needs to be given to the Integration Architecture.
In short, an integration specialist needs to build a decoupled API layer leveraging the full (professional) integration stack which is to one’s disposal. These APIs can then be surfaces as custom business connections such that they can be dragged, dropped and configured by the ‘Power / Business’ user.
If you want to display ‘simple’ information, go ahead and hook directly into the required API’s or leverage the default connections
However, if you want to display composite information, you’d better keep integration practices in the back of your mind. I can only recommend building custom rest APIs with a ‘single’ purpose.
For applications which tap into a business process and mutate / manipulate / insert / update data which affect the process and applications, require ‘guaranteed processing’ are transactional based or require compensation logic I’d recommend to leverage for example Azure Service Bus (light-weight but powerful) maybe in combination with an on premise Middleware platform such as our beloved BizTalk.
My upcoming posts will be covering more hands-on topics, in which I both will guide you through the building process of a simple intranet like business app as well as a LOB business app.
Well I hoped this post was of any use to you. I purposely did not specifically dive into the different components which make up PowerApps (Designer, Logic Flows) nor the difference between the different PowerApps tiers as there are already some good resources available on the world wide web diving into these.
So if you require some more details on PowerApps please have a look at the following link – http://bit.ly/1THnLL8 -, which I find very helpful and will guide you in your discovery path into the wonderful world of PowerApps
Cheers and feel free to add some comments to get the discussion going!
René
RightScale recently announced the results of their annual “State of the Cloud” survey. You can find the report here, and my InfoQ.com story here. A lot of people participated in the survey, and the results showed that a majority of… Read More ›
Blog Post by: Richard Seroter
A while ago I created a post on using the BizTalk Deployment Framework for automated build and deployment. Since then I have worked this out to be more easy and maintainable using PowerShell, which I will show in this post. BizTalk Deployment Framework is one of those pearls for BizTalk developers, allowing complex BizTalk solutions to be deployed easily, having all our artifacts and dependencies together in one MSI. The code with this post can be downloaded from here.
Using PowerShell we will make scripts which will handle all steps of the build and deployment process for us. This will make sure our applications are always deployed in the correct order, using the right versions, and with minimal effort. We have some general helper functions, which will help us clear log files, wait for user input, iterate through directories, etc. We assume you have are using some of the BTDF best practices for these scripts, where it comes to naming conventions and folder structure. Of course, in case anything differs in your environment, you can easily adjust the scripts to meet your requirements.
We will first create the PowerShell scripts which will help us build our applications. To be able to share these scripts along your different developers, where there might be some differences in the environments in how directories are set up, we will make use of a small csv file to hold our build environment settings.
Name;Value projectsBaseDirectory;F:tfs installersOutputDirectory;F:Deployment visualStudioDirectory;F:Program Files (x86)Microsoft Visual Studio 11.0 |
We will load these settings in our script and assign them to specific parameters.
$settings = Import-Csv Settings_BuildEnvironment.csv foreach($setting in $settings) { # The directory where the BizTalk projects are stored if($setting.'Name;Value'.Split(";")[0].Trim() -eq "projectsBaseDirectory") { $projectsBaseDirectory = $setting.'Name;Value'.Split(";")[1].Trim() } # The directory where the MSI's should be saved to if($setting.'Name;Value'.Split(";")[0].Trim() -eq "installersOutputDirectory") { $installersOutputDirectory = $setting.'Name;Value'.Split(";")[1].Trim() } # Directory where Visual Studio resides if($setting.'Name;Value'.Split(";")[0].Trim() -eq "visualStudioDirectory") { $visualStudioDirectory = $setting.'Name;Value'.Split(";")[1].Trim() } } |
Now that we have our environment specific parameters set, we can create a function which will build our BizTalk application. We will assume you have several projects, which are in folders under a common directory ($projectsBaseDirectory), which is probably your source control root directory. Your application’s directories should be under these project’s directories. We will building the application by calling Visual Studio, and using the log to check if the build was successful.
function BuildBizTalkApplication([string]$application, [string]$project) { # Set directory where the BizTalk projects for the current project are stored $projectsDirectory = "$projectsBaseDirectory$project" # Clear log files and old installers ClearLogFiles $application # Build application Write-Host "Building $application" -ForegroundColor Cyan $exitCode = (Start-Process -FilePath "$visualStudioDirectoryCommon7IDEdevenv.exe" -ArgumentList """$projectsDirectory$application$application.sln"" /Build Release /Out $application.log" -PassThru -Wait).ExitCode # Check result if($exitCode -eq 0 -and (Select-String -Path "$application.log" -Pattern "0 failed" -Quiet) -eq "true") { Write-Host "$application built succesfully" -ForegroundColor Green } else { Write-Host "$application not built succesfully" -ForegroundColor Red WaitForKeyPress } } |
Once the applications are built, we will also need to create MSI’s for them, which is where the BTDF comes in. This can be done by calling MSBuild, and passing in the .btdfproj file. Finally we copy the MSI to a folder, so all our MSI’s are together in one location and from there can be copied to the BizTalk server.
function BuildBizTalkMsi([string]$application, [string]$project) { # Set directory where the BizTalk projects for the current project are stored $projectsDirectory = "$projectsBaseDirectory$project" # Build installer $exitCode = (Start-Process -FilePath """$env:windirMicrosoft.NETFrameworkv4.0.30319MSBuild.exe""" -ArgumentList "/t:Installer /p:Configuration=Release ""$projectsDirectory$applicationDeploymentDeployment.btdfproj"" /l:FileLogger,Microsoft.Build.Engine;logfile=$application.msi.log" -PassThru -Wait).ExitCode # Check result if($exitCode -eq 0) { Write-Host "MSI for $application built succesfully" -ForegroundColor Green } else { Write-Host "MSI for $application not built succesfully" -ForegroundColor Red WaitForKeyPress } # Copy installer copy "$projectsDirectory$applicationDeploymentbinRelease*.msi" "$installersOutputDirectory" } |
Once the MSI’s have been created we can copy them to our BizTalk server, and start the deployment process. This process consists of 4 steps, starting with undeploying the old applications, uninstalling the old MSI’s, installing the new MSI’s and deploying the new applications. If your applications have dependencies on other applications, it’s also important to undeploy and deploy them in the correct order. We will want to use one set of scripts for all our OTAP environments, so we will be using another csv file here to keep track of the environment specific settings, like directories and config files to use.
We will start by loading the environment specific parameters.
$settings = Import-Csv Settings_DeploymentEnvironment.csv foreach($setting in $settings) { # Program Files directory where application should be installed if($setting.'Name;Value'.Split(";")[0].Trim() -eq "programFilesDirectory") { $programFilesDirectory = $setting.'Name;Value'.Split(";")[1].Trim() } # Suffix as set in in the ProductName section of the BTDF project file. By default this is " for BizTalk". if($setting.'Name;Value'.Split(";")[0].Trim() -eq "productNameSuffix") { $productNameSuffix = $setting.'Name;Value'.Split(";")[1].TrimEnd() } # Indicator if we should deploy to the BizTalkMgmtDB database from this server. In multi-server environments this should be true on 1 server, and false on the others if($setting.'Name;Value'.Split(";")[0].Trim() -eq "deployBizTalkMgmtDB") { $deployBizTalkMgmtDB = $setting.'Name;Value'.Split(";")[1].Trim() } } |
Now we can write our function for undeploying. We will also be using MSBuild in conjuntion with BTDF here, by passing in the .btdfproj file location with the Undeploy switch. To do so, we will call the following function for each application to be undeployed. Remember to do the undeployment in the correct order.
function UndeployBizTalkApplication([string]$application, [string]$version) { # Execute undeployment $exitCode = (Start-Process -FilePath "$env:windirMicrosoft.NETFrameworkv4.0.30319MSBuild.exe" -ArgumentList """$programFilesDirectory$application$productNameSuffix$versionDeploymentDeployment.btdfproj"" /t:Undeploy /p:DeployBizTalkMgmtDB=$deployBizTalkMgmtDB /p:Configuration=Server" -Wait -Passthru).ExitCode if($exitCode -eq 0) { Write-Host "$application undeployed successfully" -ForegroundColor Green } else { Write-Host "$application not undeployed successfully" -ForegroundColor Red } } |
Once all the applications for our project have been undeployed, we will uninstall the old MSI’s. To do this, we will iterate through the MSI’s in the specified directory, where we will pass in the directory with the last used installers.
function UninstallBizTalkApplications($msiDirectory) { # Get MSI's to be installed $files = GetMsiFiles $msiDirectory # Loop through MSI files foreach($file in $files) { UninstallBizTalkApplication $file } } |
This will call the uninstall command. We will assume our MSI’s are named according to BTDF defaults, which is applicationname-version, so for example MyApplication-1.0.0.msi.
function UninstallBizTalkApplication([System.IO.FileInfo]$fileInfo) { # Get application name $applicationName = $fileInfo.BaseName.Split("-")[0] # Set installer path $msiPath = $fileInfo.FullName # Uninstall application $exitCode = (Start-Process -FilePath "msiexec.exe" -ArgumentList "/x ""$msiPath"" /qn" -Wait -Passthru).ExitCode # Check if uninstalling was successful if($exitCode -eq 0) { Write-Host "$applicationName uninstalled successfully" -ForegroundColor Green } else { Write-Host "$applicationName not uninstalled successfully" -ForegroundColor Red } } |
The next step will be to install all the new MSI’s we have just built. Here we will once again iterate through the specified directory, where we will now pass in the directory with the new installers.
function InstallBizTalkApplications([string]$msiDirectory) { # Clear log files ClearLogFiles $msiDirectory # Get MSI's to be installed $files = GetMsiFiles $msiDirectory # Loop through MSI files foreach($file in $files) { # Install application InstallBizTalkApplication $file } } |
We will also have to load the environment specific parameters here.
$settings = Import-Csv Settings_DeploymentEnvironment.csv foreach($setting in $settings) { # Program Files directory where application should be installed if($setting.'Name;Value'.Split(";")[0].Trim() -eq "programFilesDirectory") { $programFilesDirectory = $setting.'Name;Value'.Split(";")[1].Trim() } # Suffix as set in in the ProductName section of the BTDF project file. By default this is " for BizTalk". if($setting.'Name;Value'.Split(";")[0].Trim() -eq "productNameSuffix") { $productNameSuffix = $setting.'Name;Value'.Split(";")[1].TrimEnd() } } |
And now we can install the MSI. As mentioned before, we will assume our MSI’s are named according to BTDF defaults (applicationname-version.msi).
function InstallBizTalkApplication([System.IO.FileInfo]$fileInfo) { # Get application name and version # We assume msi file name is in the format ApplicationName-Version $application = $fileInfo.BaseName.Split("-")[0] $version = $fileInfo.BaseName.Split("-")[1] # Directory where MSI resides $msiDirectory = $fileInfo.Directory # Set log name $logFileName = "$msiDirectory$application.log" # Set installer path $msiPath = $fileInfo.FullName # Install application Start-Process -FilePath "msiexec.exe" -ArgumentList "/i ""$msiPath"" /passive /log ""$logFileName"" INSTALLDIR=""$programFilesDirectory$application$productNameSuffix$version""" -Wait -Passthru | Out-Null # Check if installation was successful if((Select-String -Path $logFileName -Pattern "success or error status: 0" -Quiet) -eq "true") { Write-Host "$application installed successfully" -ForegroundColor Green } else { Write-Host "$application not installed successfully" -ForegroundColor Red } } |
The last step is to deploy the applications we just installed. First we again have to load the environment specific parameters.
$settings = Import-Csv Settings_DeploymentEnvironment.csv foreach($setting in $settings) { # Program Files directory where application should be installed if($setting.'Name;Value'.Split(";")[0].Trim() -eq "programFilesDirectory") { $programFilesDirectory = $setting.'Name;Value'.Split(";")[1].Trim() } # Suffix as set in in the ProductName section of the BTDF project file. By default this is " for BizTalk". if($setting.'Name;Value'.Split(";")[0].Trim() -eq "productNameSuffix") { $productNameSuffix = $setting.'Name;Value'.Split(";")[1].TrimEnd() } # Indicator if we should deploy to the BizTalkMgmtDB database from this server. In multi-server environments this should be true on 1 server, and false on the others if($setting.'Name;Value'.Split(";")[0].Trim() -eq "deployBizTalkMgmtDB") { $deployBizTalkMgmtDB = $setting.'Name;Value'.Split(";")[1].Trim() } # Name of the BTDF environment settings file for this environment. if($setting.'Name;Value'.Split(";")[0].Trim() -eq "environmentSettingsFileName") { $environmentSettingsFileName = $setting.'Name;Value'.Split(";")[1].Trim() } } |
Deploying is also done by using MSBuild with BTDF, by specifying the Deploy flag. For this we will be calling the following function for each application to be deployed, which of course should be done in the correct order.
function DeployBizTalkApplication([string]$application, [string]$version) { # Set log file $logFileName = "$programFilesDirectory$application$productNameSuffix$versionDeployResultsDeployResults.txt" # Execute deployment $exitCode = (Start-Process -FilePath "$env:windirMicrosoft.NETFrameworkv4.0.30319MSBuild.exe" -ArgumentList "/p:DeployBizTalkMgmtDB=$deployBizTalkMgmtDB;Configuration=Server;SkipUndeploy=true /target:Deploy /l:FileLogger,Microsoft.Build.Engine;logfile=""$programFilesDirectory$application$productNameSuffix$versionDeployResultsDeployResults.txt"" ""$programFilesDirectory$application$productNameSuffix$versionDeploymentDeployment.btdfproj"" /p:ENV_SETTINGS=""$programFilesDirectory$application$productNameSuffix$versionDeploymentEnvironmentSettings$environmentSettingsFileName.xml""" -Wait -Passthru).ExitCode # Check if deployment was successful if($exitCode -eq 0 -and (Select-String -Path $logFileName -Pattern "0 Error(s)" -Quiet) -eq "true") { Write-Host "$application deployed successfully" -ForegroundColor Green } else { Write-Host "$application not deployed successfully" -ForegroundColor Red } } |
From the same location where we call this function, we will also do some additional checks. Sometimes you will want to import some registry files or execute a SQL script, which you might not want to include in your BTDF MSI for any reason. Also, once everything has been deployed, you might want to restart your host instances and IIS, which can also be handled here.
function DeployBizTalkApplications([string[]]$applicationsInOrderOfDeployment, [string[]]$versions, [string]$scriptsDirectory) { # Check which restarts should be done $resetIIS = CheckIfIISShouldBeReset $restartHostInstances = CheckIfHostinstancesShouldBeRestarted # Loop through applications to be deployed for($index = 0; $index -lt $applicationsInOrderOfDeployment.Length; $index++) { # Deploy application DeployBizTalkApplication $applicationsInOrderOfDeployment[$index] $versions[$index] } # Get SQL files to be executed $sqlFiles = GetSQLFiles $scriptsDirectory # Loop through SQL files foreach($sqlFile in $sqlFiles) { # Execute SQL file ExecuteSqlFile $sqlFile } # Get registry files to be imported $registryFiles = GetRegistryFiles $scriptsDirectory # Loop through registry files foreach($registryFile in $registryFiles) { # Import registry file ImportRegistryFile $registryFile } # Do restarts if($resetIIS) { DoIISReset } if($restartHostInstances) { DoHostInstancesRestart } } |
Finally, we have to stitch it all together. When you have downloaded the complete set of functions from this article, you can specify your build scripts as following, where you will only have to change the project name and applications to be built.
# Project specific settings $projectName = "OrderSystem" $applications = @("Contoso.OrderSystem.Orders", "Contoso.OrderSystem.Invoices", "Contoso.OrderSystem.Payments") # Import custom functions . .Functions_Build.ps1 # Build the applications BuildAndCreateBizTalkInstallers $applications $projectName # Wait for user to exit WaitForKeyPress |
As for deployment, all those steps can also be called from one single script as following. Once again, the only thing to change is the project specific settings.
# Project specific settings $oldInstallersDirectory = "F:tmpR9" $newInstallersDirectory = "F:tmpR10" $newApplications = @("Contoso.OrderSystem.Orders", "Contoso.OrderSystem.Invoices", "Contoso.OrderSystem.Payments") $oldApplications = @("Contoso.OrderSystem.Payments", "Contoso.OrderSystem.Invoices", "Contoso.OrderSystem.Orders") $oldVersions = @("1.0.0", "1.0.0", "1.0.0") $newVersions = @("1.0.0", "1.0.1", "1.0.0") # Import custom functions . .Functions_Deploy.ps1 . .Functions_Undeploy.ps1 . .Functions_Install.ps1 . .Functions_Uninstall.ps1 # Undeploy the applications UndeployBizTalkApplications $oldApplications $oldVersions # Wait for user to continue WaitForKeyPress # Uninstall the applications UninstallBizTalkApplications $oldInstallersDirectory # Wait for user to continue WaitForKeyPress # Install the applications InstallBizTalkApplications $newInstallersDirectory # Wait for user to continue WaitForKeyPress # Deploy the applications DeployBizTalkApplications $newApplications $newVersions $newInstallersDirectory # Wait for user to exit WaitForKeyPress |
As you can see, using these PowerShell scripts you can setup scripts for your build and deployment processes very quickly. And by automating all these steps, we will have to spend much less time on builds and deployments, as we will only have to start our scripts, and the rest just goes automatically.
Code
A while ago I created a post on using the BizTalk Deployment Framework for automated build and deployment. Since then I have worked this out to be more easy and maintainable using PowerShell, which I will show in this post. BizTalk Deployment Framework is one of those pearls for BizTalk developers, allowing complex BizTalk solutions to be deployed easily, having all our artifacts and dependencies together in one MSI. The code with this post can be downloaded from here.
Using PowerShell we will make scripts which will handle all steps of the build and deployment process for us. This will make sure our applications are always deployed in the correct order, using the right versions, and with minimal effort. We have some general helper functions, which will help us clear log files, wait for user input, iterate through directories, etc. We assume you have are using some of the BTDF best practices for these scripts, where it comes to naming conventions and folder structure. Of course, in case anything differs in your environment, you can easily adjust the scripts to meet your requirements.
We will first create the PowerShell scripts which will help us build our applications. To be able to share these scripts along your different developers, where there might be some differences in the environments in how directories are set up, we will make use of a small csv file to hold our build environment settings.
Name;Value projectsBaseDirectory;F:tfs installersOutputDirectory;F:Deployment visualStudioDirectory;F:Program Files (x86)Microsoft Visual Studio 11.0 |
We will load these settings in our script and assign them to specific parameters.
$settings = Import-Csv Settings_BuildEnvironment.csv foreach($setting in $settings) { # The directory where the BizTalk projects are stored if($setting.'Name;Value'.Split(";")[0].Trim() -eq "projectsBaseDirectory") { $projectsBaseDirectory = $setting.'Name;Value'.Split(";")[1].Trim() } # The directory where the MSI's should be saved to if($setting.'Name;Value'.Split(";")[0].Trim() -eq "installersOutputDirectory") { $installersOutputDirectory = $setting.'Name;Value'.Split(";")[1].Trim() } # Directory where Visual Studio resides if($setting.'Name;Value'.Split(";")[0].Trim() -eq "visualStudioDirectory") { $visualStudioDirectory = $setting.'Name;Value'.Split(";")[1].Trim() } } |
Now that we have our environment specific parameters set, we can create a function which will build our BizTalk application. We will assume you have several projects, which are in folders under a common directory ($projectsBaseDirectory), which is probably your source control root directory. Your application’s directories should be under these project’s directories. We will building the application by calling Visual Studio, and using the log to check if the build was successful.
function BuildBizTalkApplication([string]$application, [string]$project) { # Set directory where the BizTalk projects for the current project are stored $projectsDirectory = "$projectsBaseDirectory$project" # Clear log files and old installers ClearLogFiles $application # Build application Write-Host "Building $application" -ForegroundColor Cyan $exitCode = (Start-Process -FilePath "$visualStudioDirectoryCommon7IDEdevenv.exe" -ArgumentList """$projectsDirectory$application$application.sln"" /Build Release /Out $application.log" -PassThru -Wait).ExitCode # Check result if($exitCode -eq 0 -and (Select-String -Path "$application.log" -Pattern "0 failed" -Quiet) -eq "true") { Write-Host "$application built succesfully" -ForegroundColor Green } else { Write-Host "$application not built succesfully" -ForegroundColor Red WaitForKeyPress } } |
Once the applications are built, we will also need to create MSI’s for them, which is where the BTDF comes in. This can be done by calling MSBuild, and passing in the .btdfproj file. Finally we copy the MSI to a folder, so all our MSI’s are together in one location and from there can be copied to the BizTalk server.
function BuildBizTalkMsi([string]$application, [string]$project) { # Set directory where the BizTalk projects for the current project are stored $projectsDirectory = "$projectsBaseDirectory$project" # Build installer $exitCode = (Start-Process -FilePath """$env:windirMicrosoft.NETFrameworkv4.0.30319MSBuild.exe""" -ArgumentList "/t:Installer /p:Configuration=Release ""$projectsDirectory$applicationDeploymentDeployment.btdfproj"" /l:FileLogger,Microsoft.Build.Engine;logfile=$application.msi.log" -PassThru -Wait).ExitCode # Check result if($exitCode -eq 0) { Write-Host "MSI for $application built succesfully" -ForegroundColor Green } else { Write-Host "MSI for $application not built succesfully" -ForegroundColor Red WaitForKeyPress } # Copy installer copy "$projectsDirectory$applicationDeploymentbinRelease*.msi" "$installersOutputDirectory" } |
Once the MSI’s have been created we can copy them to our BizTalk server, and start the deployment process. This process consists of 4 steps, starting with undeploying the old applications, uninstalling the old MSI’s, installing the new MSI’s and deploying the new applications. If your applications have dependencies on other applications, it’s also important to undeploy and deploy them in the correct order. We will want to use one set of scripts for all our OTAP environments, so we will be using another csv file here to keep track of the environment specific settings, like directories and config files to use.
We will start by loading the environment specific parameters.
$settings = Import-Csv Settings_DeploymentEnvironment.csv foreach($setting in $settings) { # Program Files directory where application should be installed if($setting.'Name;Value'.Split(";")[0].Trim() -eq "programFilesDirectory") { $programFilesDirectory = $setting.'Name;Value'.Split(";")[1].Trim() } # Suffix as set in in the ProductName section of the BTDF project file. By default this is " for BizTalk". if($setting.'Name;Value'.Split(";")[0].Trim() -eq "productNameSuffix") { $productNameSuffix = $setting.'Name;Value'.Split(";")[1].TrimEnd() } # Indicator if we should deploy to the BizTalkMgmtDB database from this server. In multi-server environments this should be true on 1 server, and false on the others if($setting.'Name;Value'.Split(";")[0].Trim() -eq "deployBizTalkMgmtDB") { $deployBizTalkMgmtDB = $setting.'Name;Value'.Split(";")[1].Trim() } } |
Now we can write our function for undeploying. We will also be using MSBuild in conjuntion with BTDF here, by passing in the .btdfproj file location with the Undeploy switch. To do so, we will call the following function for each application to be undeployed. Remember to do the undeployment in the correct order.
function UndeployBizTalkApplication([string]$application, [string]$version) { # Execute undeployment $exitCode = (Start-Process -FilePath "$env:windirMicrosoft.NETFrameworkv4.0.30319MSBuild.exe" -ArgumentList """$programFilesDirectory$application$productNameSuffix$versionDeploymentDeployment.btdfproj"" /t:Undeploy /p:DeployBizTalkMgmtDB=$deployBizTalkMgmtDB /p:Configuration=Server" -Wait -Passthru).ExitCode if($exitCode -eq 0) { Write-Host "$application undeployed successfully" -ForegroundColor Green } else { Write-Host "$application not undeployed successfully" -ForegroundColor Red } } |
Once all the applications for our project have been undeployed, we will uninstall the old MSI’s. To do this, we will iterate through the MSI’s in the specified directory, where we will pass in the directory with the last used installers.
function UninstallBizTalkApplications($msiDirectory) { # Get MSI's to be installed $files = GetMsiFiles $msiDirectory # Loop through MSI files foreach($file in $files) { UninstallBizTalkApplication $file } } |
This will call the uninstall command. We will assume our MSI’s are named according to BTDF defaults, which is applicationname-version, so for example MyApplication-1.0.0.msi.
function UninstallBizTalkApplication([System.IO.FileInfo]$fileInfo) { # Get application name $applicationName = $fileInfo.BaseName.Split("-")[0] # Set installer path $msiPath = $fileInfo.FullName # Uninstall application $exitCode = (Start-Process -FilePath "msiexec.exe" -ArgumentList "/x ""$msiPath"" /qn" -Wait -Passthru).ExitCode # Check if uninstalling was successful if($exitCode -eq 0) { Write-Host "$applicationName uninstalled successfully" -ForegroundColor Green } else { Write-Host "$applicationName not uninstalled successfully" -ForegroundColor Red } } |
The next step will be to install all the new MSI’s we have just built. Here we will once again iterate through the specified directory, where we will now pass in the directory with the new installers.
function InstallBizTalkApplications([string]$msiDirectory) { # Clear log files ClearLogFiles $msiDirectory # Get MSI's to be installed $files = GetMsiFiles $msiDirectory # Loop through MSI files foreach($file in $files) { # Install application InstallBizTalkApplication $file } } |
We will also have to load the environment specific parameters here.
$settings = Import-Csv Settings_DeploymentEnvironment.csv foreach($setting in $settings) { # Program Files directory where application should be installed if($setting.'Name;Value'.Split(";")[0].Trim() -eq "programFilesDirectory") { $programFilesDirectory = $setting.'Name;Value'.Split(";")[1].Trim() } # Suffix as set in in the ProductName section of the BTDF project file. By default this is " for BizTalk". if($setting.'Name;Value'.Split(";")[0].Trim() -eq "productNameSuffix") { $productNameSuffix = $setting.'Name;Value'.Split(";")[1].TrimEnd() } } |
And now we can install the MSI. As mentioned before, we will assume our MSI’s are named according to BTDF defaults (applicationname-version.msi).
function InstallBizTalkApplication([System.IO.FileInfo]$fileInfo) { # Get application name and version # We assume msi file name is in the format ApplicationName-Version $application = $fileInfo.BaseName.Split("-")[0] $version = $fileInfo.BaseName.Split("-")[1] # Directory where MSI resides $msiDirectory = $fileInfo.Directory # Set log name $logFileName = "$msiDirectory$application.log" # Set installer path $msiPath = $fileInfo.FullName # Install application Start-Process -FilePath "msiexec.exe" -ArgumentList "/i ""$msiPath"" /passive /log ""$logFileName"" INSTALLDIR=""$programFilesDirectory$application$productNameSuffix$version""" -Wait -Passthru | Out-Null # Check if installation was successful if((Select-String -Path $logFileName -Pattern "success or error status: 0" -Quiet) -eq "true") { Write-Host "$application installed successfully" -ForegroundColor Green } else { Write-Host "$application not installed successfully" -ForegroundColor Red } } |
The last step is to deploy the applications we just installed. First we again have to load the environment specific parameters.
$settings = Import-Csv Settings_DeploymentEnvironment.csv foreach($setting in $settings) { # Program Files directory where application should be installed if($setting.'Name;Value'.Split(";")[0].Trim() -eq "programFilesDirectory") { $programFilesDirectory = $setting.'Name;Value'.Split(";")[1].Trim() } # Suffix as set in in the ProductName section of the BTDF project file. By default this is " for BizTalk". if($setting.'Name;Value'.Split(";")[0].Trim() -eq "productNameSuffix") { $productNameSuffix = $setting.'Name;Value'.Split(";")[1].TrimEnd() } # Indicator if we should deploy to the BizTalkMgmtDB database from this server. In multi-server environments this should be true on 1 server, and false on the others if($setting.'Name;Value'.Split(";")[0].Trim() -eq "deployBizTalkMgmtDB") { $deployBizTalkMgmtDB = $setting.'Name;Value'.Split(";")[1].Trim() } # Name of the BTDF environment settings file for this environment. if($setting.'Name;Value'.Split(";")[0].Trim() -eq "environmentSettingsFileName") { $environmentSettingsFileName = $setting.'Name;Value'.Split(";")[1].Trim() } } |
Deploying is also done by using MSBuild with BTDF, by specifying the Deploy flag. For this we will be calling the following function for each application to be deployed, which of course should be done in the correct order.
function DeployBizTalkApplication([string]$application, [string]$version) { # Set log file $logFileName = "$programFilesDirectory$application$productNameSuffix$versionDeployResultsDeployResults.txt" # Execute deployment $exitCode = (Start-Process -FilePath "$env:windirMicrosoft.NETFrameworkv4.0.30319MSBuild.exe" -ArgumentList "/p:DeployBizTalkMgmtDB=$deployBizTalkMgmtDB;Configuration=Server;SkipUndeploy=true /target:Deploy /l:FileLogger,Microsoft.Build.Engine;logfile=""$programFilesDirectory$application$productNameSuffix$versionDeployResultsDeployResults.txt"" ""$programFilesDirectory$application$productNameSuffix$versionDeploymentDeployment.btdfproj"" /p:ENV_SETTINGS=""$programFilesDirectory$application$productNameSuffix$versionDeploymentEnvironmentSettings$environmentSettingsFileName.xml""" -Wait -Passthru).ExitCode # Check if deployment was successful if($exitCode -eq 0 -and (Select-String -Path $logFileName -Pattern "0 Error(s)" -Quiet) -eq "true") { Write-Host "$application deployed successfully" -ForegroundColor Green } else { Write-Host "$application not deployed successfully" -ForegroundColor Red } } |
From the same location where we call this function, we will also do some additional checks. Sometimes you will want to import some registry files or execute a SQL script, which you might not want to include in your BTDF MSI for any reason. Also, once everything has been deployed, you might want to restart your host instances and IIS, which can also be handled here.
function DeployBizTalkApplications([string[]]$applicationsInOrderOfDeployment, [string[]]$versions, [string]$scriptsDirectory) { # Check which restarts should be done $resetIIS = CheckIfIISShouldBeReset $restartHostInstances = CheckIfHostinstancesShouldBeRestarted # Loop through applications to be deployed for($index = 0; $index -lt $applicationsInOrderOfDeployment.Length; $index++) { # Deploy application DeployBizTalkApplication $applicationsInOrderOfDeployment[$index] $versions[$index] } # Get SQL files to be executed $sqlFiles = GetSQLFiles $scriptsDirectory # Loop through SQL files foreach($sqlFile in $sqlFiles) { # Execute SQL file ExecuteSqlFile $sqlFile } # Get registry files to be imported $registryFiles = GetRegistryFiles $scriptsDirectory # Loop through registry files foreach($registryFile in $registryFiles) { # Import registry file ImportRegistryFile $registryFile } # Do restarts if($resetIIS) { DoIISReset } if($restartHostInstances) { DoHostInstancesRestart } } |
Finally, we have to stitch it all together. When you have downloaded the complete set of functions from this article, you can specify your build scripts as following, where you will only have to change the project name and applications to be built.
# Project specific settings $projectName = "OrderSystem" $applications = @("Contoso.OrderSystem.Orders", "Contoso.OrderSystem.Invoices", "Contoso.OrderSystem.Payments") # Import custom functions . .Functions_Build.ps1 # Build the applications BuildAndCreateBizTalkInstallers $applications $projectName # Wait for user to exit WaitForKeyPress |
As for deployment, all those steps can also be called from one single script as following. Once again, the only thing to change is the project specific settings.
# Project specific settings $oldInstallersDirectory = "F:tmpR9" $newInstallersDirectory = "F:tmpR10" $newApplications = @("Contoso.OrderSystem.Orders", "Contoso.OrderSystem.Invoices", "Contoso.OrderSystem.Payments") $oldApplications = @("Contoso.OrderSystem.Payments", "Contoso.OrderSystem.Invoices", "Contoso.OrderSystem.Orders") $oldVersions = @("1.0.0", "1.0.0", "1.0.0") $newVersions = @("1.0.0", "1.0.1", "1.0.0") # Import custom functions . .Functions_Deploy.ps1 . .Functions_Undeploy.ps1 . .Functions_Install.ps1 . .Functions_Uninstall.ps1 # Undeploy the applications UndeployBizTalkApplications $oldApplications $oldVersions # Wait for user to continue WaitForKeyPress # Uninstall the applications UninstallBizTalkApplications $oldInstallersDirectory # Wait for user to continue WaitForKeyPress # Install the applications InstallBizTalkApplications $newInstallersDirectory # Wait for user to continue WaitForKeyPress # Deploy the applications DeployBizTalkApplications $newApplications $newVersions $newInstallersDirectory # Wait for user to exit WaitForKeyPress |
As you can see, using these PowerShell scripts you can setup scripts for your build and deployment processes very quickly. And by automating all these steps, we will have to spend much less time on builds and deployments, as we will only have to start our scripts, and the rest just goes automatically.
Code
What does this setting do? In this post I describe why I found this setting useful and ask what are the disadvantages are of turning this setting to on. I have a BizTalk Server 2013 R2 that uses eight WCF-WebHTTP request response adapters with service bus relays that are hosted in the same isolated […]
Blog Post by: mbrimble
Today while doing a BAM POC to one of my clients, I found a number of problems/errors, some of them unexpected and uncommon, while trying to deploy my BAM definition file. In order to contextualize the problem, for this POC we are using only one BizTalk Server machine with BAM Portal, one dedicated SQL Server […]
Blog Post by: Sandro Pereira
March 7, 2016
Today I encountered an unusual error when executing a pipeline component that utilises the EventStream API to write to BAM. The failure that showed up in the event log looked something like this:
A message received by adapter “WCF-SQL” on receive location “MyReceivePort” with URI “mssql://MyDatabaseInstance/MyDatabase?InboundId=Employee” is suspended.
Error details: There was a failure executing the receive pipeline: “MyReceivePipeline, MyAssembly, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1a2b345c67d89e0f” Source: “Log Message To BAM Receive” Receive Port: “MyReceivePort” URI: “mssql://MyDatabaseInstance/MyDatabase?InboundId=Employee” Reason: Flush failed to run.
A quick Google search pulled up this helpful post by Yossi Dahan, which pointed me in the right direction. I knew that the connection string was all right, and I was using the BufferedEventStream rather than the DirectEventStream that Yossi referred to. (Incidentally, when using the BufferedEventStream your connection string actually points to the BizTalkMsgBoxDb database rather than the BAMPrimaryImport database.)
However, the clue was really in the second part of Yossi’s suggestion (and also in an anonymous comment), “…and related permissions…”. I could see that the BizTalk Application Users domain group had been assigned all of the appropriate roles in SQL Server, and I knew that all the host accounts had been dutifully added to this group when BizTalk was installed.
Er… hang on a moment. I double checked and found that the SQL Adapter was running under a new dedicated host account that had been created specifically for the data warehouse. A simple check on the account using the “net user /domain” command prompt unveiled the culprit. This account had not be granted membership in the BizTalk Application Users domain group.
Once that was accomplished, everything worked smoothly.
It would be nice to actually see an error that hinted towards permission issues. Perhaps the detail was buried somewhere in an inner exception, but the logging does not go past the first level.